Multi-Package Repos with Yarn

Yarn workspaces are a great option for working on multiple packages at the same time. They replaces npm link, gives you the ability to run a command in all packages (or a specific package), and lets you install all dependencies for all packages at the same time. Popular projects like Jest, Babel, and Gatsby all use the yarn client or yarn workspaces fronted by Lerna to ease the pain of developing large sets of packages (even if they're not interdependent).

Multi-Package Repos

For the purpose of this post, we'll define a multi-package repo as a monorepo with a focus on package-based development instead of allowing importing code as with relative paths. This difference means we focus on NPM packages as our unit of abstraction and when we use one package from another, we use it just as we would use any other package from NPM.

Yarn workspaces and Lerna are two approaches to developing multi-package repos. Lerna can be used with or without Yarn, but is usually much more performant if you use Yarn Workspaces as the underlying implementation for Lerna's interface. We'll dive into Lerna in another post and focus on using Yarn Workspaces standalone for this post.

An NPM Package

An NPM package is defined by a package.json. There are a few fields that are required, such as the name and version. Here's a sample package.json from gatsby-mdx.

{
  "name": "gatsby-mdx",
  "version": "0.3.4",
  "description": "mdx integration for gatsby",
  "main": "index.js",
  "license": "MIT",
  "scripts": {
    "test": "jest"
  },
  "peerDependencies": {
    "@mdx-js/mdx": "^0.16.5",
    "@mdx-js/tag": "^0.16.5"
  },
  "dependencies": {
    "@babel/plugin-proposal-object-rest-spread": "^7.0.0",
    "debug": "^4.0.1",
    "escape-string-regexp": "^1.0.5",
    "fs-extra": "^7.0.0",
    "gray-matter": "^4.0.1",
    "lodash": "^4.17.10",
    "mdast-util-to-string": "^1.0.4",
    "mdast-util-toc": "^2.0.1",
    "mime": "^2.3.1",
    "pretty-bytes": "^5.1.0",
    "remark": "^9.0.0",
    "retext": "^5.0.0",
    "slash": "^2.0.0",
    "static-site-generator-webpack-plugin": "^3.4.2",
    "strip-markdown": "^3.0.1",
    "underscore.string": "^3.3.4",
    "unist-util-map": "^1.0.4",
    "unist-util-remove": "^1.0.1",
    "unist-util-visit": "^1.4.0"
  },
  "devDependencies": {
    "jest": "^23.4.2",
    "js-combinatorics": "^0.5.3"
  },
  "jest": {
    "testEnvironment": "node"
  },
  "keywords": [
    "gatsby",
    "gatsby-plugin",
    "gatsby-transformer-plugin",
    "mdx",
    "markdown",
    "remark",
    "rehype"
  ]
}

The file structure for a small NPM package with no build process might look like this. That is, an index.js file containing the functionality of the module and a package.json defining dependencies, etc.

➜ tree .
.
├── index.js
└── package.json

Multiple Packages

To handle multiple packages we'll put each of our packages in a packages directory in their own folder. We'll have package-a and package-b with their own package.jsons and their own index.js. We haven't added a build process at all so the code in index.js will have to be manually written for the runtime we expect to use it in (ex: the browser or a specific node version).

➜ tree .
.
├── package.json
└── packages
    ├── package-a
    │   ├── index.js
    │   └── package.json
    └── package-b
        ├── index.js
        └── package.json

Finally, we also create a package.json at the root of our workspaces. This defines where our workspaces live. We've defined our workspaces as any package inside of the packages folder, so package-a and package.b both count. Note that our root package.json is also private, which is required to use workspaces. We don't want to publish the entire repo as an NPM package anyway.

{
  "name": "workspaces",
  "version": "0.0.1",
  "main": "index.js",
  "private": true,
  "author": "Chris Biscardi <chris@christopherbiscardi.com> (@chrisbiscardi)",
  "license": "MIT",
  "workspaces": ["packages/*"]
}

Working with Workspaces

So now that we have everything set up, we'll install all dependencies in all of our packages.

yarn

Running yarn not only installs all dependencies for all packages, but it also handles linking the packages between each other if they depend on one another. For example, if package-a had package-b in it's dependencies in it's package.json, running yarn would link package-b into package-a.

This linking means that we can, for example, continue to develop package-b and running tests in package-a. Whenever we make a change to package-b, package-a already knows about it so the tests will use it. Using this linking is very powerful for developing sets of interdependent packages because we can run tests for the entire repo against the entire set of changes. Speaking of, we can run the test script in every workspace with

yarn workspaces run test

This requires that we have a script named test in each of our packages. If we set each of our packages test script to echo the package name like:

{
  "scripts": {
    "test": "echo package-a"
  }
}

Then the output for yarn workspaces run test would look like:

➜ yarn workspaces run test
yarn workspaces v1.12.3
yarn run v1.12.3
$ echo package-a
package-a
✨  Done in 0.08s.
yarn run v1.12.3
$ echo package-b
package-b
✨  Done in 0.08s.
✨  Done in 0.92s.

Working on Individual Packages

We can also target individual packages with the yarn workspace command.

➜ yarn workspace package-a test
yarn workspace v1.12.3
yarn run v1.12.3
$ echo package-a
package-a
✨  Done in 0.07s.
✨  Done in 0.48s.

We can publish each package to the NPM registry individually and use them in other projects or we can also not publish anything and continue using this multi-package repo for all of our package development. Since most NPM projects qualify as NPM packages, we can stick entire applications and other UI surfaces in our multi-package repo, linking our entire package ecosystem (components, sharable logic, custom packages, etc) in when building.

Fin

This does leave a lot of area to explore such as publishing, setting up build steps, installing global tools in the root vs installing them in individual packages. Hopefully the next time you see a yarn workspaces powered repo, you'll understand how to get started, where to find packages, and how to run scripts for the package you care about.