Photo by michael schaffler on Unsplash
Lerna: Simplifying Multi-Package JavaScript Projects with Monorepo
Monorepo
A monorepo, short for "monolithic repository", is a software development approach where multiple projects are stored in a single repository. This means that instead of having separate repositories for each project, they are all kept together in a single repository, with version control applied at the repository level.
In a monorepo, all the projects share the same codebase, dependencies, and configuration. This allows for better code reuse, easier cross-project refactoring, and simplified dependency management. Monorepos are commonly used by large software companies such as Google, Facebook, and Microsoft, as well as by many open-source projects.
There are several reasons why one might choose a monorepo for their software development:
- Code sharing: By keeping all code in a single repository, it becomes easier to share code between different projects. This is especially useful when there are common components or libraries that are used across multiple projects.
- Consistency: With a monorepo, all projects share the same codebase and dependencies. This ensures consistency across projects and reduces the risk of incompatibilities.
- Simplified development process: With all code in a single repository, developers can work on multiple projects simultaneously without having to switch between different repositories.
- Easier maintenance: Since all code is in a single repository, it's easier to maintain and manage. This makes it simpler to update dependencies, apply security patches, and perform other maintenance tasks.
- Improved collaboration: By using a monorepo, teams can collaborate more easily since all code is in one place. This can help reduce communication overhead and speed up development.
Overall, a monorepo can simplify the development process and improve code sharing and collaboration, making it a popular choice for many development teams.
Lerna is a popular tool used for managing multi-package JavaScript projects. Some of the most used Lerna commands are:
lerna init
: Initializes a new Lerna repository in the current directory.lerna create
: Creates a new package in the Lerna repository.lerna bootstrap:
Installs package dependencies and links local packages together.lerna add
: Adds a package as a dependency to another package in the Lerna repository.lerna exec
: Runs an arbitrary command in each package of the Lerna repository.lerna changed
: Lists all packages that have changed since the last release.lerna diff
: Shows the difference between the current working tree and the last release of a package.lerna publish
: Publishes new package versions to a package registry, such as npm.lerna run
: Runs an npm script in each package of the Lerna repository.lerna clean
: Removes the node_modules directory from all packages in the Lerna repository.
lerna init
lerna init
is a command used to initialize a new Lerna repository with a basic directory structure and a default package.json
file. It creates a new Git repository and adds an initial commit.
To use lerna init
, you need to have Lerna installed globally on your machine. Once installed, you can navigate to the directory where you want to create the Lerna repository and run the following command:
lerna init
This will create the following directory structure:
my-lerna-repo/
packages/
package.json
lerna.json
The packages
directory is where you will create your individual packages. Each package should have its own directory within the packages
directory. The package.json
file is the main package.json
file for the entire repository. The lerna.json
file contains configuration options for Lerna.
After running lerna init
, you can add packages to the repository using the lerna create
command.
Lerna Create
The lerna create
command is used to create a new package in a Lerna-managed monorepo. It initializes the package with a basic directory structure and a package.json
file.
Here's the basic syntax of the lerna create
command:
lerna create <name> [loc]
<name>
: The name of the package to be created[loc]
: Optional. The location where the package should be created. If not specified, the package will be created in the default packages directory.
For example, to create a new package called my-package
in the default packages directory, you can run the following command:
lerna create my-package
This will create a new directory called packages/my-package
, along with a basic package.json
file. You can then add your source code, tests, and other files to this directory as needed. Once you've added your code, you can use lerna add
to add the new package as a dependency of other packages in your monorepo.
lerna bootstrap
lerna bootstrap
is a command provided by Lerna that enables you to set up a development environment where packages depend on each other within a Lerna monorepo.
When you run lerna bootstrap
, Lerna installs all external dependencies of each package, hoisting dependencies to the root level where possible. It also symlinks packages that are dependencies of each other.
Here's an example of how to use lerna bootstrap
in a Lerna monorepo:
Let's say you have a Lerna monorepo with two packages:
my-monorepo/
โโโ package-1/
โ โโโ package.json
โ โโโ index.js
โโโ package-2/
โ โโโ package.json
โ โโโ index.js
โโโ lerna.json
Inside lerna.json
, you can define which packages should be linked and how:
{
"packages": [
"packages/*"
],
"version": "0.0.0"
}
Then, you can run lerna bootstrap
to install all external dependencies of each package and create symlinks for interdependent packages:
lerna bootstrap
This command will execute the following steps:
- Install all external dependencies for each package
- Symlink packages that depend on each other
After running lerna bootstrap
, you should have the following directory structure:
my-monorepo/
โโโ node_modules/
โโโ package-1/
โ โโโ node_modules/ (symlinked to ../node_modules)
โ โโโ package.json
โ โโโ index.js
โโโ package-2/
โ โโโ node_modules/ (symlinked to ../node_modules)
โ โโโ package.json
โ โโโ index.js
โโโ lerna.json
Now you can develop your packages as usual and Lerna will take care of the dependencies.
lerna add
lerna add
is a command provided by Lerna that allows you to add a dependency to one or more packages in your Lerna-managed monorepo. It automates the process of manually editing the package.json
files of each package to add the dependency.
Here's an example of how to use lerna add
:
Suppose you have a Lerna-managed monorepo with two packages: package-a
and package-b
. You want to add the lodash
library as a dependency to package-a
:
- Open your terminal and navigate to the root of your Lerna-managed monorepo.
Run the following command:
lerna add lodash --scope=package-a
This command adds the
lodash
library as a dependency topackage-a
and updates thepackage.json
file ofpackage-a
. It also installslodash
inpackage-a
.If you want to add
lodash
as a dev dependency, you can use the--dev
flag:lerna add lodash --scope=package-a --dev
This will add
lodash
as a dev dependency topackage-a
.If you want to add the same dependency to multiple packages at once, you can specify a comma-separated list of scopes:
lerna add lodash --scope=package-a,package-b
This will add
lodash
as a dependency to bothpackage-a
andpackage-b
.
lerna exec
lerna exec
is a command provided by Lerna that allows you to execute a command in each of the packages in your Lerna repository. This can be useful for running tests or other commands that need to be executed across all packages.
The basic syntax for lerna exec
is:
lerna exec [command]
This will execute the specified command in each package directory. For example, if you wanted to run the npm test
command in each package, you could use:
lerna exec npm test
You can also use {}
as a placeholder for the package directory name in the command. For example, if you wanted to create a new directory in each package directory, you could use:
lerna exec 'mkdir {}/new-directory'
This would create a new new-directory
directory in each package directory.
lerna exec
also supports various options for controlling the behavior of the command. Some commonly used options include:
-parallel
orP
: Run the command in all packages in parallel.-stream
: Stream output from each package as it completes instead of buffering it.-scope
: Only run the command in packages that match the given scope.
For example, if you wanted to run the npm install
command in all packages in parallel, you could use:
lerna exec --parallel -- npm install
We can use the lerna exec
command with the --scope
flag and a wildcard expression:
lerna exec --scope 'package-*' -- npm run test
This will execute the npm run test
command on all the packages that have a name starting with "package-". The --scope
flag allows us to specify which packages we want to run the command on.
Overall, lerna exec
is a powerful command that can help you automate tasks across multiple packages in your Lerna repository.
lerna changed
The lerna changed
command is used to identify which packages in a Lerna-managed repository have changed since the last tagged release, in preparation for publishing new versions of those packages. It compares the current state of the repository with the latest tagged release and identifies packages that have been modified or added.
Here is an example of using lerna changed
command:
Suppose you have a Lerna-managed repository with two packages named package-1
and package-2
. You have made some changes to package-1
since the last tagged release, and you want to see which packages have changed in the repository. You can use the following command:
lerna changed
This will output a list of packages that have changed since the last tagged release, in this case package-1
.
You can also use the --json
flag to output the result as a JSON object, which can be used by other tools:
lerna changed --json
This will output a JSON object with the list of packages that have changed.
lerna diff
lerna diff
is a Lerna command that shows the difference between the current repository state and the last release of the package(s).
The lerna diff
command outputs a list of the packages that have changed since the last release, along with the type of change that has been made. The types of changes include:
changed
: the package has changed but it doesn't include changes to its dependenciesnew
: a new package has been addedupgraded
: the package has upgraded one of its dependenciesdowngraded
: the package has downgraded one of its dependencies
For example, if you run lerna diff
, and there have been changes made to two packages since the last release, the output might look something like this:
lerna notice cli v3.22.1
lerna info Looking for changed packages since v1.0.0
lerna info found 2 packages to publish
lerna info diffing packages
lerna info comparing to previous version v1.0.0
my-package-1 CHANGED
my-package-2 UPGRADED (from 1.0.0 to 1.1.0)
This indicates that there have been changes made to my-package-1
, and an upgrade made to
my-package-2
since the last release.
lerna publish
lerna publish
is a command in Lerna that helps you publish new versions of packages to the npm registry. It automates many of the tasks associated with publishing multiple packages, including updating version numbers, creating and pushing git tags, and publishing to the npm registry.
Here are the steps involved in using lerna publish
:
- First, run
lerna changed
to see which packages have changed since the last release. - Use
lerna version
to bump the version number of each changed package, update the changelog, create a git tag, and commit the changes. - Finally, run
lerna publish
to publish the new versions of the packages to the npm registry.
During the lerna publish
process, you will be prompted to confirm the versions you want to publish, and you will have the option to choose a custom tag for the release (such as "next" or "beta").
What is conventional-commits ?
Conventional commit messages are a standardized way of structuring commit messages that provide more context and semantic meaning.
The basic structure of a conventional commit message is:
<type>[optional scope]: <description>
[optional body]
[optional footer]
For example, a commit message for a feature implementation might look like this:
feat(login): add forgot password feature
Add a new "forgot password" link to the login page, allowing users to reset their password if they forget it.
By using conventional commits, you can generate a changelog automatically based on the commit messages, making it easier to track changes and releases.
In conventional commits, <type>
represents the nature of the changes made in the commit. This can be one of the following types:
feat
: A new feature.fix
: A bug fix.docs
: Documentation changes.style
: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc).refactor
: A code change that neither fixes a bug nor adds a feature.perf
: A code change that improves performance.test
: Adding missing tests or correcting existing tests.build
: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm).ci
: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs).chore
: Other changes that don't modify src or test files.revert
: Reverts a previous commit.
The [optional scope]
is used to provide additional context about the changes being made, such as the affected component or module. It is enclosed in square brackets and is optional.
There is a awesome library know as commitizen
that will help you generate the conventional commit. Please do check that out !
lerna publish
supports many other options as well, such as publishing only to a specific registry, skipping certain packages, and specifying a custom git commit message.
Here are all the options available for lerna publish
:
-canary
: creates a canary release, which means that it will publish a release with a version suffix such asalpha.X
. This is useful when you want to test a release before publishing it to the public registry.-conventional-commits
: uses the conventional commits specification to determine the new version number based on the commit messages since the last release.-force-publish
: publishes all packages, even if they haven't changed since the last release. You can specify which packages to force publish by providing a comma-separated list of package names.-dist-tag
: the tag that will be used to identify the package when installing it. The default value is "latest".-exact
: forces the package version to be published exactly as it appears in thepackage.json
file, without resolving the version to the latest semver-compatible version.-skip-git
: skips the Git commit and tag creation process, meaning that no Git operations will be performed during the release.-skip-npm
: skips the actual npm publish step, meaning that no packages will be published to the npm registry.-preid
: specifies the pre-release identifier for the version. For example, if you specify-preid alpha
, the version will be something like1.0.0-alpha.0
.-registry
: the URL of the npm registry to use for publishing the packages. By default, it uses the public npm registry at https://registry.npmjs.org/.-message
: the message to use when creating the Git tag for the release. By default, it uses the release type and version number.
Here's an example that uses several of these options:
These options can be used in combination with each other to achieve the desired result. For example, you can use --skip-git --skip-npm --skip-changelog
to quickly generate a new version number without committing any changes or publishing to npm.
lerna publish --canary --conventional-commits --skip-git --skip-npm --registry https://my-registry.com --dist-tag next
This command will create a canary release with a version suffix, using the conventional commits specification to determine the new version number. It will skip the Git commit and tag creation process, as well as the actual npm publish step. It will publish the package to a private registry at https://my-registry.com, and use the "next" dist-tag to identify the package when installing it.
Package release lifecycle ?
The package release lifecycle refers to the various stages that a software package goes through from its initial development to its final release. The lifecycle can be broken down into several phases, each of which serves a specific purpose. The phases typically include:
- Development: This is the phase where the software package is being actively developed. During this phase, developers write code, test it, and make changes as needed.
- Alpha testing: Once the initial development is complete, the package may be released to a limited group of testers for alpha testing. This testing is designed to catch any major bugs or issues before the package is released to a wider audience.
- Beta testing: After alpha testing is complete, the package may be released to a larger group of beta testers. Beta testing is designed to catch any remaining bugs and to gather feedback from users.
- Release candidate: Once beta testing is complete and all major bugs have been fixed, the package may be designated as a release candidate. This version is considered to be stable and is intended to be the final release unless any critical issues are found.
- Final release: After the release candidate has been thoroughly tested and any remaining issues have been fixed, the package is released to the public as a final, stable version.
Throughout the release lifecycle, developers may also release patches or minor updates to the software package to address any bugs or issues that are found. These updates typically follow the same lifecycle process as the initial release, with alpha and beta testing before the final release.
Here are the steps to use Lerna to publish in different release scenarios:
Alpha release: If you want to publish an alpha release, you can use the
--dist-tag
flag to specify aalpha
tag for your package. This will let users install your package with the@your-package-name@alpha
tag.lerna publish --dist-tag alpha
Beta release: Similarly, if you want to publish a beta release, you can use the
--dist-tag
flag to specify abeta
tag for your package. This will let users install your package with the@your-package-name@beta
tag.lerna publish --dist-tag beta
Release candidate (RC) release: To publish an RC release, you can use the
--dist-tag
flag to specify arc
tag for your package. This will let users install your package with the@your-package-name@rc
tag.lerna publish --dist-tag rc
Production (stable) release: Finally, when you are ready to publish a stable release of your package, you can simply run
lerna publish
without any flags. This will publish the latest version of your package to the defaultlatest
tag. This will let users install your package with the@your-package-name@latest
tag or@your-package-name
.lerna publish
lerna run
lerna run
is a command provided by Lerna that allows running npm scripts across packages in the monorepo. It enables running scripts in a specific package, running scripts for all packages, or running scripts in a subset of packages that match a specific glob pattern.
The basic syntax of lerna run
command is:
lerna run <script> [--flags] [packageGlob...]
where <script>
is the name of the npm script to run, --flags
are any flags passed to the npm command, and packageGlob...
is an optional list of glob patterns that filter the packages where the script should be executed.
Here's an example of using lerna run
to run the build
script in all packages:
lerna run build
This command will run the build
script in all packages in the monorepo. You can also pass additional flags to the npm command, for example:
lerna run build -- --prod
This command will pass the --prod
flag to the npm command, which could be used by the build script to create a production build.
You can also use glob patterns to run the script only in certain packages, for example:
lerna run build --scope my-package -- --prod
This will run the build
script for only the my-package
package and pass the --prod
flag to the script.
In addition to the --scope
flag, lerna run
supports other options such as --parallel
to run scripts in parallel, --stream
to stream output from multiple processes, and --no-bail
to continue running scripts even if one fails.
Here's an example of running the test
script in all packages in parallel:
lerna run test --parallel
This command will run the test
script in all packages at the same time, which can speed up the build process if the packages are independent of each other.
lerna clean
lerna clean
is a command provided by Lerna that removes the node_modules
directory from all packages in the Lerna monorepo. It is useful when you want to clean up the dependencies and start fresh.
The lerna clean
command has the following options:
-yes
: skips the confirmation prompt and cleans the packages without confirmation.-ignore
: a comma-separated list of package names that should be ignored during the cleaning process.-nohoist
: prevents Lerna from hoisting the dependencies while cleaning.
Here's an example:
Suppose you have a Lerna monorepo with two packages named package-a
and package-b
, and you want to clean them up.
To clean up all the packages in the monorepo, run the following command:
lerna clean
If you want to clean up only package-a
and ignore package-b
, run the following command:
lerna clean --ignore package-b
If you want to skip the confirmation prompt and clean up all packages in the monorepo, run the following command:
lerna clean --yes
If you want to prevent Lerna from hoisting the dependencies while cleaning, run the following command:
lerna clean --nohoist
Hoisting is a technique used by package managers like npm and Yarn to optimize the installation process by reducing the number of duplicated packages. It involves moving packages that are used by multiple dependencies to a higher level in the dependency tree, so they can be shared by all packages that need them, rather than being duplicated in each package's node_modules folder. This helps to reduce the overall size of the node_modules folder and improve installation times.