npm

What is npm?

  • node package manager
  • largest ecosystem of open source libraries
  • makes it easy to share, reuse and update code
  • helps us build complex things with simple parts
  • not just for backend modules any more

Three parts of npm

  • npm the company
    • take open source to new places
    • reduce friction
  • npm the registry (free)
  • npm the command line tool
    • open source
    • ships with Node.js

Module

Anything that can be loaded with require(...) in a Node.js program

Package

  1. directory
    • package.json file
    • something of value (like a module)
  2. gzip of (1)
  3. URL that resolves to (2)
  4. GIT URL that when cloned results in (1)

package.json

A text file in json format that describes your "package" and it's dependencies.

package.json

  • Required properties
    • name
    • version
  • Others properties
    • description
    • keywords
    • homepage
    • author, contributor
    • dependencies & devDependencies
    • main, bugs, license, scripts...

npm registry

  • A website that implements the CommonJS Package Registry specification for reading package info
  • Powered by CouchDB
  • https://www.npmjs.com/
  • Internal registry
    • http://icsnpm.ldschurch.org/

npm registry

  • Search - full text
  • Filtering - human
    • stars
    • version (if they follow semver)
    • downloads (relative)
    • number of releases
    • time since last release (abandoned)?
    • badges
    • author
    • does it have a readme.md and docs?
    • does it solve your problem?
    • does it solve too many problems?
    • is the license acceptable
    • how many open issues? (relative)
    • how many dependencies (relative to complexity)
    • ...

npm init

npm i <pgk>

code & test

npm shrinkwrap

git clone

npm install

Publish

Deploy

?

npm init

npm i <pgk>

code & test

npm shrinkwrap

git clone

npm install

Publish

Deploy

?

Ramp Up

New blank project

Existing project

npm init

npm i <pgk>

code & test

npm shrinkwrap

git clone

npm install

Publish

Deploy

?

Develop

npm init

npm i <pgk>

code & test

npm shrinkwrap

git clone

npm install

Publish

Deploy

?

Publish/Deploy

npm init

npm i <pgk>

code & test

npm shrinkwrap

git clone

npm install

Publish

Deploy

?

Ramp Up - Blank Project

$ npm init --yes
Wrote to my-project/package.json:

{
  "name": "my-project",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "Bruce Campbell <bruce@ldschurch.org>",
  "license": "ISC"
}

$ ls
package.json

$

npm init

npm i <pgk>

code & test

npm shrinkwrap

git clone

npm install

Publish

Deploy

?

Ramp Up - Existing Project or Generated Project

Use npm install to download project dependencies from the registry

  • reads package.json for dependencies
  • uses the dependencies section for modules your app needs to function
  • use the devDependencies for modules your app needs to build
  • downloads dependencies and puts them in the ./node_modules folder within your project

npm init

npm i <pgk>

code & test

npm shrinkwrap

git clone

npm install

Publish

Deploy

?

Develop

  • npm install <package> 
  • npm uninstall <package>
  • Flags
    • --save
    • --save-dev
    • --global

Demo

  • my-app project
  • previously we used npm install without a package name to read dependencies from package.json and install them.
  • Now we want to add jQuery to the list of dependencies and install it.
  • So we add jQuery and --save... npm i --save jquery
    • i is short for install
    • see package.json and ./node_modules for jQuery
    • see src/server/index.js when we import some code by name, without periods or slashes, node knows to look inside the node_modules directory for a folder with a matching name to find the code we want.
    • and there are other ways to consume those packages... webpack, gulp, etc.
    • off we go to write some code using jQuery
  • more...

Demo

  • But then the dev lead on your project points out that all the jQuery functionality you need is built into Javascript now.
    • npm uninstall jQuery
      • see ./node_modules
      • see package.json - why is jQuery still there?
    • npm uninstall --S jQuery   (-S is short for --save) - now it's gone
    • more...

Demo

  • While we're talking about the install command I should probably talk about "global" installs even though it doesn't fit perfectly into this part of the cycle
  • the --global, or -g for short installs a package "globally", or outside your project
    • on a mac that's /usr/local/lib/node_modules
    • on windows that's ... don't know,
      • use npm ls -g --depth 0 to find it
        • the docs say beside the node.exe binary 
      • "ls" or list tells you what packages are installed
      • --depth 0 limits how deep into the dependency tree it will report. Zero lists just the direct dependencies.
      • -g of course is for globally installed modules.
  • there is a trend that less and less packages require that they be installed globally.
  • more...

Demo

  • While we are already on a tangent, we might as well talk about executable packages.
  • Some packages can have a CLI component to them.
  • So instead of "require()"-ing it in to your code you use it as a tool and run it from the command line.
  • Show npm list -g --depth 0
  • notice n and yo... they are both command line tools
  • they were installed onto my laptop as node packages using npm
  • tools that need to be available from any directory on the system should be installed "globally"
  • tools that need only be available in the project directory can be installed "locally" and they can still be executed on the command line
  •  

npm init

npm i <pgk>

code & test

npm shrinkwrap

git clone

npm install

Publish

Deploy

?

Publish/Deploy

semver

1.2.3

http://semver.org/

https://github.com/npm/node-semver

major - breaking changes

minor - new features but compatible

patch - backwards compatible bug fixes

major

minor

patch

version strings

The dependencies sections of package.json specifies the name and a version string for each dependency

"dependencies": {
    "compression": "^1.5.2",
    "cookie-parser": "^1.3.5",
    "dotenv": "^2.0.0",
    "express": "^4.13.3",
    ...
}

npm shrinkwrap

  • Locks down the versions of a package's dependencies so that you can control exactly which versions of each dependency will be used when you install/deploy
  • You could lock down dependencies by specifying exact versions in package.json
  • however npm shrinkwrap is preferred and more flexible

npm shrinkwrap

  • run npm shrinkwrap
  • which produces npm-shrinkwrap.json
  • when npm-shrinkwrap.json is present, the npm install command honors the exact versions found within
  • typically: commit the shrink wrap file to your vcs
  • delete the shrink wrap file when you start your next development cycle
  • npm suggests you shrink wrap your apps, not your modules

Demo

  • run npm shrinkwrap in my-app
  • show new npm-shrinkwrap.json and it's format
  • now if we were to delete the node_modules
  • and run npm i
  • the versions would match the shrink wrap file

more

Updating Dependencies

  • npm outdated
    • to list dependencies that have fallen behind
    • "wanted" column is the highest available version that complies with the package.json range
    • "latest" column is the highest available version within the registry

Updating Dependencies

  • npm update
    • installs new versions of dependencies
    • obeys package.json version ranges
  • to break out of the version range
    • manually modify package.json & run npm update or
    • uninstall and re-install each dependency

Fixing EACCESS error (without sudo)

Do you get errors installing packages globally?

  1. Install Node with Homebrew
  2. Use an alternate global package directory
  3. Change directory permissions

https://docs.npmjs.com/getting-started/fixing-npm-permissions

3. Change Directory Permissions

  • Change the owner of npm's directories to the name of the current user  
    • sudo chown -R $(whoami) $(npm config get prefix)/{lib/node_modules,bin,share}
    • or
    • sudo chown -R $USER $(npm config get prefix)/lib/node_modules

Installing Offline

  • npm keeps a local cache
    • npm config get cache 
  • install offline
    • npm install --cache-min 999999

Troubleshooting Tips

  • delete your ./node_modules directory and re-run npm i after changing node or npm versions
  • check your node and npm versions against the engines property in package.json
  • others tips?

Node Versions

  • Even Node versions will have "Long Term Support"
  • Odd Node versions get the latest features

npm v3

  • progress bar
  • flat flat flat
    • = less files but a little slower

npm Versions

  • Node 4 ships with npm 2
    • run npm i -g npm to have npm upgrade itself
    • remember to delete your node_modules folders
    • upgrade node or npm at the beginning of an iteration, not right before releasing
    • communicate this change to your team

npm Configuration Files

  • per-project config file (/path/to/my/project/.npmrc)
  • per-user config file (~/.npmrc)
  • global config file ($PREFIX/etc/npmrc)
  • npm builtin config file (/path/to/npm/npmrc)

npm hydra worm

npm hydra worm

In theory, malicious code could spread

through the npm ecosystem because

  • npm lifecycle scripts allow arbitrary commands
  • semver allows for changing dependencies
  • persistent auth allows publishing without creds
  • central repository acts as distribution point

Hydra Mitigation

  • Do not stay logged in to npm... npm logout
  • Use npm shrinkwrap to lock down dependencies
  • Disable install scripts  
    • npm i <module> --ignore-scripts or
    • npm config set ignore-scripts true
  • Don't install a package you don't trust
  • Don't sudo your installs

npm's Hydra Mitigations

  • vulnerability scanning
  • two-factor authentication in public registry

 

Both are WIP

Hydra Worm Disclosure: http://tinyurl.com/h6cbwhu

npm response: http://tinyurl.com/hujcz85

npm

By Bruce Campbell