Locking Node.js module versions with Shrinkwrap

Node modules

A lot of Node.js projects seem to start with a debate: what to do with node_modules? The two obvious options are:

Every machine should "npm install" them from scratch
They should be checked-in

As unintuituve as it sounds, #2 used to be the safest option. Npm recently released a tool that changes this, but let's understand the problem first:

It works on my machine

I'll have to admit, until today I was relying on solution #1… and this morning our build machine started failing for no apparent reason. Apparently a javascript function didn't exist anymore.

TypeError: Object #<Object> has no method 'getNextAvailableErrorCode'
[15:21:57][Step 3/3] at Object.<anonymous> (/Users/.../node_modules/file-utils/file-utils.js:1:224)

Let's have a look at what happened. Our project depends on file-utils on version 0.1.x.

"dependencies": {
    "file-utils": "0.1.x"
}

The "x" here means we want to stay on version 0.1, but are happy to install newer patches when available (ex: 0.1.3). This is fine in most cases, because modules that follow semantic versionning assure you that patches only fix bugs while staying backwards compatible. If I run an npm list locally, we can see it pulled down version 0.1.11, which itself pulled down errno-codes version 1.0.0.

my-project@0.0.1
├─┬ file-utils@0.1.11 
│ └── errno-codes@1.0.0

Now let's have a look at the build machine. An npm list on the server shows:

my-project@0.0.1
├─┬ file-utils@0.1.11 
│ └── errno-codes@1.0.1

That's not the same version of errno-codes as me!

The problem

When running npm install, it will always try to match the version specified in the package.json. If you already have that dependency available (like me in version 1.0.0), it will be happy with it. Otherwise it will download the latest version that fits the description.

So let's look at the package.json from file-utils:

    "dependencies": {
      "errno-codes": "*"
    }

No fixed dependencies here, file-utils will always pull down the latest version of errno-codes when needed. And that's the key here:

You can control the exact versions of your top-level dependencies, but not further down the chain.

Since our build machine always checks out the source in a clean folder, when it ran npm install it simply got the latest version of errono-codes (1.0.1). To avoid this, file-utils could lock their dependency to a specific version (1.0.0), and really errno-codes should have backward-compatible patches!

Anyway, that's a problem Ruby devs will be familiar with, because Bundler solved it a while back with the Gemfile.lock!

The solution

Checking-in "node_modules" used to be the answer to this problem, but it came with its own set of problems. This is why NPM released Shrinkwrap, which you can think of as Gemfile.lock for Node.js Smiley

The good news is that it fits really well with the normal NPM workflow. To get started, run npm shrinkwrap in your project folder. It will analyse your local node modules and generate a list of all your dependencies and their version (npm_shrinkwrap.json). The good thing here is that this list is based off your local modules, so as long as you have a working copy everything should be OK. It doesn't matter if the build machine already pulled down a broken dep!

{
    "name": "my-project",
    "version": "0.0.1",
    "dependencies": {
        "file-utils": {
            "version": "0.1.11",
            "dependencies": {
                "errno-codes": {
                    "version": "1.0.0"
                }
            }
        }
    }
}

Note: Shrinkwrap won't run if you have any modules in node_modules that don't come from your package.json. If it complains about "extraneous modules", simply run npm prune to get rid of the offending folders.

As we can see, errno-codes is locked down to version "1.0.0", which is the one we tested against locally. Let's check in the shrinkwrap file. From now on, anyone who does an npm install will get the exact dependencies specified in this file - no more pulling down newer versions that no one tested.

Adding or updating dependencies

That's the one thing that will have to change: as we saw, a simple npm install simply pulls down what the shrinkwrap file specifies. To update dependencies, we need to:

Run npm install moduleName@version
Add it to the package.json
Test your project locally to make sure everything is OK
Finally, run npm shrinkwrap again and check-in the resulting file

Our dependencies are now fixed, and the build's repeatable. One less thing to worry about!

←Async Dev→