you be worried about the number of dependencies?
Keep in mind that this blog post is related to the
blog post that was released a while back. We’ll show a “real-world” example of how
a project’s dependencies can grow from zero to 13K.
Having a large number of dependencies in a project isn’t necessarily a
bad thing, but we’ll get to that later. First, let’s see how a project can
grow from a couple of dependencies to a large number.
Psst. The latest release of AppSignal for Node.js has PostgreSQL Instrumentation baked in. We will be rolling out a whole array of integrations on Node.js one by one in the coming weeks. Check out the docs to see how to set things up.
To best illustrate the growing pains, we are going to create a simple application
that keeps track of things we need to do.
This is how it’s going to look like:
Pretty simple at this point—we don’t want to get wild and fancy at this moment.
You can take a look at the code on just-do-it GitHub repo.
Right now, the only dependency this app needs is a browser that can open HTML
pages. We can accomplish this with pretty much every browser out there.
Let’s see where we are with the number of dependencies at this point. We will
create a table with four columns:
- direct dependencies — the number of dependencies we installed (together with
- inherited dependencies — the number of dependencies that installed with direct dependencies
- total — the total number of the above two
- new dependencies — how many dependencies got installed in the previous installation
||0 (not counting browser)
The project still doesn’t have a
package.json, so we are still not part of
that sweet NPM package ecosystem. This is useful if you don’t want to
complicate things. But right now, we are not here for simple solutions. We
want to simulate how you can go from zero to a bunch of dependencies fast!
So, as any project with front-end out there, we don’t want plain JS.
We want to be cool and use React!
Adding React to Our Project
The React craze is in full swing right now. It may not even reach its peak in
we’ve been using in our project and switch to what the cool kids are using these
days—the almighty React.
We are going to follow the ‘Add React to a Website’ instructions
from the official React website. What this means is that we’ll add a script tag element in our HTML:
<!-- ... other HTML ... -->
<script src="https://unpkg.com/react@16/umd/react.development.js" crossorigin></script>
<script src="https://unpkg.com/react-dom@16/umd/react-dom.development.js" crossorigin></script>
<!-- ... other HTML ... -->
But, since we want to use JSX,
we are going to get a taste of NPM’s ecosystem hands-on. To use JSX, we
have to install Babel. Babel will compile our JSX file
to JS so our browser can render it effortlessly.
First, we need
package.json. We can add it with the following command:
The previous command will set up
package.json and make our project ready for the
installation of Babel:
$ npm install --save-dev email@example.com babel-preset-react-app@3
added 305 packages from 128 contributors and audited 3622 packages in 15.904s
Wow, so we explicitly installed two packages—
babel-preset-react-app, but these brought in more dependencies of their own.
If you’re new to this idea, you might find it strange. I suggest you read the
Installing these two packages gets us from 0 NPM dependencies up to 2 direct and 3620 inherited dependencies.
Great, now we can use JSX while running Babel in the background to watch for our changes:
$ npx babel --watch src --out-dir . --presets react-app/prod
Check out how the repo looks after the changes.
Babel will pre-process our JSX files into JS files. But this sounds a bit
sketchy. I don’t want to have React load in a script tag.
I want a full developer experience, similar to create-react-app.
For this, we’ll have to add a bit more of Babel and a bundler—Webpack.
Adding a Create-React-App-like Setup
To get this right, we will follow the recommended way of
setting up React in a project from scratch.
Let’s remove the old Babel dependencies that we used:
$ npm uninstall babel-cli babel-preset-react-app --save-dev
Nice, we’re back to zero NPM dependencies.
Next, let’s add newer versions of Babel dependencies:
$ npm install --save-dev @babel/core @babel/cli @babel/preset-env @babel/preset-react
added 25 packages from 4 contributors, removed 4 packages, updated 7 packages and audited 4216 packages in 13.394s
babel-core is the main Babel package—it is needed for Babel to do
transformations on our code.
babel-cli allows you to compile files from the
preset-env are both presets that transform
specific flavors of code—in this case, the env preset allows us to transform
ES6+ into more traditional JS and the React preset does the same, but
with JSX instead.
Now that we got that out of the way, we have a total of 4216 dependencies based on the npm
Now, our table of dependencies looks like this:
After this, we need Webpack to bundle everything for us nicely. Also, we will
set up a server that will watch for any changes we make to our app in development.
We can install the necessary dependencies with:
$ npm install --save-dev webpack webpack-cli webpack-dev-server style-loader css-loader babel-loader
added 448 packages from 308 contributors and audited 13484 packages in 23.105s
This added some more packages. Now we’re at a total of 13484.
Well, that escalated quickly! What happened here is that each dependency that we
installed has dependencies of its own. Then those dependencies have their
dependencies. In the total number (13484), all dependencies plus dependencies
of dependencies get included. Even devDependencies! That is why we got an extra
9268 dependencies by installing webpack and other needed dependencies to make it work.
The total number might seem scary and make you wonder what is going on, but
this is how the NPM ecosystem works, heck, this is how a lot of package
managers work. Most of the dependencies are one-liners and should not be
something to worry us. Sometimes, they might be, but we will get to that
For example, if we run
npm audit at this point, we will get this:
$ npm audit
=== npm audit security report ===
│ Manual Review │
│ Some vulnerabilities require your attention to resolve │
│ Visit https://go.npm.me/audit-guide for additional guidance │
│ Low │ Prototype Pollution │
│ Package │ yargs-parser │
│ Patched in │ >=13.1.2 <14.0.0 || >=15.0.1 <16.0.0 || >=18.1.2 │
│ Dependency of │ webpack-dev-server [dev] │
│ Path │ webpack-dev-server > yargs > yargs-parser │
│ More info │ https://npmjs.com/advisories/1500 │
found 1 low severity vulnerability in 13484 scanned packages
1 vulnerability requires manual review. See the full report for details.
We see that a dependency of a
webpack-dev-server has a
has a low severity. It’s a development dependency of
it probably should not worry us. Being a development dependency means that
webpack-dev-server uses this
yargs-parser in development, and it is very
unlikely that this code will end up in our production bundle.
Remember, so far we’ve only added devDependencies. All of these are not going to
end up in our production bundle, so nothing to worry about, right? Let’s
continue and find out.
We now need to add React, since we don’t want to have script tags in the header
to fetch it.
Let’s move React from the head straight into
$ npm install react react-dom
added 5 packages and audited 13506 packages in 6.148s
Now, this is different. We only got 22 extra dependencies from installing
If we run npm audit for production, we’ll get this:
npm audit --production
=== npm audit security report ===
found 0 vulnerabilities
in 22 scanned packages
The audit output means that a total of 22 packages will end up in our production bundle—the
code that we will serve to folks visiting our TODO list application.
That doesn’t sound so bad compared to the staggering total of 13506 dependencies we
have in our development environment.
After we installed all of these dependencies and reworked our code to use
Webpack and Babel, our application is still working:
BTW, you can view all the code we added in the repo on GitHub.
We did not go into details of how we created React components and set up
Webpack. We focused more on the dependencies and what their numbers mean. If
you’re interested in technical details of how the TODO list is working, please
visit the repo mentioned above.
Should You Be Worried About So Many Dependencies?
Yes and no. There’s always risk out there. Let’s go through 2 recent cases of dependency problems that the community encountered.
Slipping Malicious Code Into a Library
A while ago, there was a situation with a widely used NPM package—event-stream.
Someone injected malicious code into the popular package aiming to get
cryptocurrency information from its users. The package did not get hacked. Just
the opposite, the attacker was a maintainer that had write access given by the
package’s creator. If you’re interested in the conversation, check out the GitHub issue about it.
There is also the NPM blog post about it.
The good thing is that this hack got identified and stopped before it reached
the users. The bad thing is that this was an official release of the package.
Breaking Everyone’s Build Process
More recently, another massively used
package broke the whole JS ecosystem. The package itself consists of 2 lines of source code.
It serves to check whether a JS object is a Promise.
The library is used by 3.4 million projects, based on GitHub’s reports. Yes, you read that right—three point four million projects.
The recent 2.2.0 update broke everyone’s build process because the project
didn’t adhere to ES module standards. Stuff like that happens. It’s
not the first time it happened, and it surely will not be the last. If you want
to read more about this, there’s a post mortem about it.
“Trust, but Verify” What You Install
You should know what you’re putting inside your project. You should adopt the
“trust, but verify” mentality. There’s no guarantee that situations that
is-promise won’t happen in the future to some other dependency.
You should not get discouraged from using or contributing to open-source
projects, though. This incident got discovered because of just that—group effort.
Just be mindful of what you put inside of
package.json so you can act properly
when tough times come.
On the other hand, you can’t be aware of every single dependency. Imagine
knowing what happens with all the 13k dependencies that we installed—you would probably
go insane. Luckily, there are tools out there like GitHub security alerts
and GitHub’s whole security initiative.
In the end, there’s not much you can do except be aware of what you add to your
project. Make sure to:
- always upgrade packages with care and look into what they are using/adding
- try to avoid nested dependencies when possible
- pin a package to a specific version and do not auto-update
I hope this blog post added even more insight into what is going on when you add
open-source security and that it will make you more careful when running that
npm install command.
P.P.S. If you’d love an all-in-one APM for Node or you’re already familiar with AppSignal, go and check out the first version of AppSignal for Node.js.