Packages for Mass C  nsumpti  n

an npm story unfolded

Liran Tal

Developer Advocate at 

// NodeGoat Core Team

// Node.js Security WG

Liran Tal

Developer Advocate at 

@liran_tal

github.com/lirantal

source: https://octoverse.github.com

Subtitle

source: https://snyk.io/opensourcesecurity-2019

Some npm History

From the early 2009 days of Node.js v0.x

                for Node.js,            (bower) for the Web

npm, Inc. established in 2014

Created and maintained by @Izs

Some npm History

A deterministic lockfile resolution

Significantly improved package install time

Package checksums

2016 Facebook introduces        (yarn)

The Package Manifest

The package.json

$ npm run test

The Package Manifest

registry metadata

community
metadata

licensing

 Semantic Versioning

Text

1.4.6

 Semantic Versioning

Major Change

Text

1.4.6

1.4.6

Breaking API

 Semantic Versioning

Minor, feature

Text

1.4.6

1.4.6

Backwards Compat.

 Semantic Versioning

Patch, bug fix

Text

1.4.6

1.4.6

Security fix

 Semantic Versioning

✅ 1.4.7

Text

~1.4.6

✅ 1.4.9

❌ 1.5.0

❌ 2.0.0

 Semantic Versioning

✅ 1.4.7

Text

^1.4.6

✅ 1.4.9

 1.5.0

❌ 2.0.0

 Dist-Tags

Text

unique package: package@version

$ npm install package

???

$ npm install package@latest

                        package@1.4.3

$ npm publish --tag next

 Dist-Tags

Text

The Package Manifest

version compat

whitelist packages

executable CLIs

The Package Manifest

installs based on a dep range is not deterministic

package-lock.json

yarn.lock

The Package Manifest

$ npm install lockfile-lint

npm vs bundler

package.json

Gemfile (+gemspec)

package-lock.json

Gemfile.lock

npm install --prod

bundle install --prod

npm install lockfile-lint

Gemfile + bundle install

npm install fastify --save-dev

Gemfile group + bundle install

npm run test

bundle rake exec test

npm show fastify

bundle show fastify

SemVer tokens: ~ ^

SemVer tokens: ~

multi-version of same dep

local project installs

global env installs

The 4 Dimensions
of an npm Dependency 

package.json

#1 Dev vs Prod

#1 Dev vs Prod

install all dependencies for your project

$ npm install --production

$ NODE_ENV=production npm install

#2 Direct vs Indirect

install only prod deps of your deps

package.json

what happens when a dependency breaks?

package.json

breaking changes in the API

why would a package "break"?

package has been compromised

npmjs is unavailable

networking disruptions

maintainer pulled down the package

???

it's open source.
everyone can do as they please.

what's the worst that can happen?

maintainer pulled down the package

1. Azer Koçulu maintains 273 npm modules

2. 3/2016 - Lawyer threatened Azer to wave-off Kik 

3. Azer refused, but npm staff changed ownership 

4. As an act of protest Azer removed all of his modules

5. One of those was a little thing called left-pad

1. Reminder: left-pad is now unpublished

2. Cameron Westland pushed a new left-pad@1.0.0

3. Many builds were still broken as they require ^0.0.3

4. npm re-publishes 0.0.3 from backups

5. total incident time: 2.5 hours

2.1. What if this was a malicious user?

source: https://snyk.io/blog/how-much-do-we-really-know-about-how-packages-behave-on-the-npm-registry

package has been compromised

14%

compromised npm modules

20%

npm total monthly downloads

express

react

debug

moment

request

Compromised Contributors ?

662

users

123456

had their password set to

Compromised Contributors ?

Compromised Contributors ?

124

users

password

had their password set to

Compromised Contributors ?

Compromised Contributors ?

1409

users

had their password set to

their username

Compromised Contributors ?

Compromised Contributors ?

11%

users

had their password set to

previously leaked password

Compromised Contributors ?

#3 Package vs Version

Explicit package name + version

package.json

debug@4.1.0

debug@2.0.0

#3 Package vs Version

Explicit package name + version

debug@4.1.0

debug@2.0.0

#4 Logical and Unique Paths

#4 Logical and Unique Paths

Flat-tree deduplication by default

package.json

$ npm install

#4 Logical and Unique Paths

Unique package paths

package.json

app@1.2.3 -> C@1.0.0 -> B@2.0.0


app@1.2.3 -> B@2.0.0

#4 Logical and Unique Paths

Global packages on disk

$ npm install -g webpack

$ npm install -g snyk

Enterprise Readyness 

Why do you need a local npm?

The official registry might fail

Why do you need a local npm?

Networks disruptions

Bandwidth considerations

Local npm/yarn cache is per dev

Offline and optimal speeds

Why do you need a local npm?

webpack@4.28.3

24 direct dependencies

1,690 total dependencies

1,802 requests to the npm registry

190MB unzipped on disk

Why do you need a local npm?

Packages could suddenly get yanked

Private package hosting cost money

Verdaccio

A Lightweight private npm proxy registry

Pro Patterns for Teams 

npm ci

yarn install --frozen-lockfile

✅ Speed

✅ Surprise-free builds

{

{

solution?

what do I get?

{

my problem is

my CI builds are slow and aren't deterministic with devlopment

use nvm

✅ All team members use the same         Node.js version

✅ npm version can also be set

{

{

solution?

what do I get?

{

my problem is

my developers often can't reproduce the bug in production in their envs

$ npm install xyz --ignore-scripts

✅ Script hooks won't be auto-run

❌ May cause grief with other npm run-scripts

{

{

solution?

what do I get?

{

my problem is

I am concerned installing Node.js modules will run malicious code

$ npm config set ignore-scripts true

$ snyk test

✅ Understand which deps you use

{

{

solution?

what do I get?

{

my problem is

what if I'm using packages with known security vulnerabilities?

$ npm audit or owasp dependency check as fallbacks

✅ Get dependency health stats

✅ Test packages for security issues

use lerna

✅ tooling to track dependencies across projects in a single manifest

{

{

solution?

what do I get?

{

my problem is

I want to use a monorepo for my shared components and project

use yarn workspaces (or Yarn v2)

✅ tooling to track versions and bump releases as needed

$ npm profile enable-2fa auth-and-writes

✅ Every npm publish or login will           require 2FA to execute 

{

{

solution?

what do I get?

{

my problem is

will someone compromise my npm account and publish malicious code?

enable 2fa

add private: true to package.json

✅ Avoid accidentally leaking your secrets (oh no!) or your intellectual property to the public

{

{

solution?

what do I get?

{

my problem is

someone accidentally published a package to the public npm registry

$ use a local npm registry like   verdaccio

use semantic-release

✅ If you are true to semver practices you get it all ✨

{

{

solution?

what do I get?

{

my problem is

I want automation in releasing packages, changelogs and tagging 

use prettier

✅ Deterministic and opinionated formatting across your code-base

{

{

solution?

what do I get?

{

my problem is

code review is a mess because everyone use different formatting

use files: [] in package.json

✅ Avoid accidentally leaking sensitive data to the publish registry

{

{

solution?

what do I get?

{

my problem is

I accidentally leaked some secrets in a config file when I published a pkg

use detect-secrets module in your git hooks

Liran Tal

Developer Advocate at 

@liran_tal

github.com/lirantal

Thank you 

Packages for Mass Consumption - an npm story unfolded

By Liran Tal

Packages for Mass Consumption - an npm story unfolded

Packages for Mass Consumption an npm story unfolded

  • 24
Loading comments...

More from Liran Tal