70 %
Chris Biscardi

Multi-Package Repos with Lerna

Lerna is a multi-package repo tool similar to Yarn Workspaces. Many projects choose to use Lerna as the UI for interacting with their multi-package repo while Yarn Workspaces is used under the hood to handle linking packages together.

We've previously covered Yarn Workspaces for multi-package repo management on it's own.

Repo Setup

If we take a look at our previous Yarn Workspaces setup, we can see that our packages got hoisted and linked into a root node_modules. The -> represents a symlink which means that when we try to access package-a through the node_modules directory, we get re-directed to ../packages/package-a, where all of our files actually live.

➜ tree .
.
├── node_modules
│   ├── package-a -> ../packages/package-a
│   └── package-b -> ../packages/package-b
├── package.json
├── packages
│   ├── package-a
│   │   ├── index.js
│   │   ├── package.json
│   │   └── yarn-error.log
│   └── package-b
│   ├── index.js
│   └── package.json
└── yarn.lock

Lets install lerna into our root package to prepare to transfer from Yarn Workspaces to exclusively Lerna. (note: we need -W because we've already defined workspaces and Yarn tries to warn us that we're installing into the root.

sh
yarn add lerna -W

Then we can initialize lerna's config.

sh
➜ yarn lerna init
yarn run v1.12.3
$ /workspaces/node_modules/.bin/lerna init
lerna notice cli v3.10.6
lerna info Initializing Git repository
lerna info Updating package.json
lerna info Creating lerna.json
lerna info Creating packages directory
lerna success Initialized Lerna files
✨ Done in 0.86s.

With no arguments, the lerna.json config file that gets spit out defaults to pointing at packages/*, just like our Yarn Workspaces but in a dedicated file.

JS
{
"packages": [
"packages/*"
],
"version": "0.0.0"
}

If we run yarn lerna bootstrap now... nothing interesting will happen because we haven't added a dependency between our packages. Add a dependency on package-b in package-a. We haven't published package-b so we'll use the "any version" version *.

JS
{
"name": "package-a",
"version": "0.0.1",
"main": "index.js",
"author": "Chris Biscardi <chris@christopherbiscardi.com> (@chrisbiscardi)",
"license": "MIT",
"scripts": {
"test": "echo package-a"
},
"dependencies": {
"package-b": "*"
}
}

Now when we bootstrap our multi-package repo, we'll see the packages linked in a slightly different way than our past Yarn Workspaces attempt. The packages are linked into each other rather than hoisted to the root node_modules by default. This is one critical difference in the default behavior of Yarn Workspaces vs Lerna.

.
├── lerna.json
├── package.json
├── packages
│   ├── package-a
│   │   ├── index.js
│   │   ├── node_modules
│   │   │   └── package-b -> ../../package-b
│   │   └── package.json
│   └── package-b
│   ├── index.js
│   └── package.json
└── yarn.lock

Unfortunately, for very large repos, doing this per-package installation and linking is really slow. Luckily, we can speed everything up by using Yarn Workspaces as the underlying package dependency resolution approach. We'll add two new keys to lerna.json: npmClient and useWorkspaces. We can also remove the packages key because Lerna will now look at our package.json workspaces key instead.

JS
{
"version": "0.0.0",
"npmClient": "yarn",
"useWorkspaces": true
}

Now running yarn lerna bootstrap will use Yarn Workspaces and we'll end up with a very similar layout to our original Yarn Workspaces based approach.

.
├── lerna.json
├── node_modules
│   ├── package-a -> ../packages/package-a
│   └── package-b -> ../packages/package-b
├── package.json
├── packages
│   ├── package-a
│   │   ├── index.js
│   │   └── package.json
│   └── package-b
│   ├── index.js
│   └── package.json
└── yarn.lock

But Why?

So that's great, but why would we go through all this effort to use Lerna if we're just going to end up back where we were with Yarn Workspaces on their own? The answer lies in the way Yarn deals with versioning and handles the publishing of packages.

Lets say, for example, that you wanted to publish a new version of all of your changed packages on every merge to master, but also didn't want that release to be the mainline stable release. You can do this with Lerna by specifying a canary release and using a special NPM tag (here we choose ci, but this could be alpha or whatever you want).

lerna publish -y --canary --preid ci --npm-tag=ci

This command gets us versions that look like 0.3.5-ci.263 published to a ci tag on NPM. We can run this on every merge to master, removing a dependency on maintainers to do a release immediately after merging code. Any user of your packages can now pull down the relevant ci version and test it before it moves on to a stable release.

Changesets

Lerna includes a set of features build on the concept of figuring out what's changed since the last release such as lerna changed. If we run it on our project with no releases we see:

➜ yarn lerna changed
yarn run v1.12.3
$ /workspaces/node_modules/.bin/lerna changed
lerna notice cli v3.10.6
lerna info Looking for changed packages since initial commit.
package-a
package-b
lerna success found 2 packages ready to publish
✨ Done in 0.93s.

The full list of commands is found below, along with what they do.

lerna add <pkg> [globs..] Add a single dependency to matched packages
lerna bootstrap Link local packages together and install remaining package dependencies
lerna changed List local packages that have changed since the last tagged release [aliases: updated]
lerna clean Remove the node_modules directory from all packages
lerna create <name> [loc] Create a new lerna-managed package
lerna diff [pkgName] Diff all packages or a single package since the last release
lerna exec [cmd] [args..] Execute an arbitrary command in each package
lerna import <dir> Import a package into the monorepo with commit history
lerna init Create a new Lerna repo or upgrade an existing repo to the current version of Lerna.
lerna link Symlink together all packages that are dependencies of each other
lerna list List local packages [aliases: ls, la, ll]
lerna publish [bump] Publish packages in the current project.
lerna run <script> Run an npm script in each package that contains that script
lerna version [bump] Bump version of packages changed since the last release.

Each command also has a set of filters or other options that we can use to specifically target subsets of our packages. lerna run for example, allows us to filter by scope (package name) or filter so we can operate on all packages that have changed since a specific ref.

➜ yarn lerna run --help
yarn run v1.12.3
$ /Users/biscarch/tmp/workspaces/node_modules/.bin/lerna run --help
lerna run <script>
Run an npm script in each package that contains that script
Positionals:
script The npm script to run. Pass flags to send to the npm client after -- [string] [required]
Command Options:
--npm-client Executable used to run scripts (npm, yarn, pnpm, ...). [string] [default: npm]
--stream Stream output with lines prefixed by package. [boolean]
--parallel Run script with unlimited concurrency, streaming prefixed output. [boolean]
--no-bail Continue running script despite non-zero exit in a given package. [boolean]
--no-prefix Do not prefix streaming output. [boolean]
Filter Options:
--scope Include only packages with names matching the given glob. [string]
--ignore Exclude packages with names matching the given glob. [string]
--no-private Exclude packages with { "private": true } in their package.json. [boolean]
--since Only include packages that have been updated since the specified [ref].
If no ref is passed, it defaults to the most-recent tag. [string]
--include-filtered-dependents Include all transitive dependents when running a command
regardless of --scope, --ignore, or --since. [boolean]
--include-filtered-dependencies Include all transitive dependencies when running a command
regardless of --scope, --ignore, or --since. [boolean]
Global Options:
--loglevel What level of logs to report. [string] [default: info]
--concurrency How many processes to use when lerna parallelizes tasks. [number] [default: 4]
--reject-cycles Fail if a cycle is detected among dependencies. [boolean]
--no-progress Disable progress bars. (Always off in CI) [boolean]
--no-sort Do not sort packages topologically (dependencies before dependents). [boolean]
--max-buffer Set max-buffer (in bytes) for subcommand execution [number]
-h, --help Show help [boolean]
-v, --version Show version number [boolean]
Examples:
lerna run build -- --silent # `npm run build --silent` in all packages with a build script
✨ Done in 0.31s.

In this way, Lerna becomes a powerful front end on top of Yarn Workspaces that can be used to publish packages when they change or operate on subsets of packages locally when developing.

Bonus

With the Yarn Workspaces based setup... we can still use the Yarn commands if we're used to them or find them easier.

➜ yarn workspace package-a test
yarn workspace v1.12.3
yarn run v1.12.3
$ echo package-a
package-a
✨ Done in 0.07s.
✨ Done in 0.49s.