npm
What is npm?
- node package manager
- largest ecosystem of open source libraries
- makes it easy to share, reuse and update code
- helps us build complex things with simple parts
- not just for backend modules any more
Three parts of npm
- npm the company
- take open source to new places
- reduce friction
- npm the registry (free)
- npm the command line tool
- open source
- ships with Node.js
Module
Anything that can be loaded with require(...) in a Node.js program
Package
- directory
- package.json file
- something of value (like a module)
- gzip of (1)
- URL that resolves to (2)
- GIT URL that when cloned results in (1)
package.json
A text file in json format that describes your "package" and it's dependencies.
package.json
- Required properties
- name
- version
- Others properties
- description
- keywords
- homepage
- author, contributor
- dependencies & devDependencies
- main, bugs, license, scripts...
npm registry
- A website that implements the CommonJS Package Registry specification for reading package info
- Powered by CouchDB
- https://www.npmjs.com/
- Internal registry
- http://icsnpm.ldschurch.org/
npm registry
- Search - full text
- Filtering - human
- stars
- version (if they follow semver)
- downloads (relative)
- number of releases
- time since last release (abandoned)?
- badges
- author
- does it have a readme.md and docs?
- does it solve your problem?
- does it solve too many problems?
- is the license acceptable
- how many open issues? (relative)
- how many dependencies (relative to complexity)
- ...
npm init
npm i <pgk>
code & test
npm shrinkwrap
git clone
npm install
Publish
Deploy
?
npm init
npm i <pgk>
code & test
npm shrinkwrap
git clone
npm install
Publish
Deploy
?
Ramp Up
New blank project
Existing project
npm init
npm i <pgk>
code & test
npm shrinkwrap
git clone
npm install
Publish
Deploy
?
Develop
npm init
npm i <pgk>
code & test
npm shrinkwrap
git clone
npm install
Publish
Deploy
?
Publish/Deploy
npm init
npm i <pgk>
code & test
npm shrinkwrap
git clone
npm install
Publish
Deploy
?
Ramp Up - Blank Project
$ npm init --yes
Wrote to my-project/package.json:
{
"name": "my-project",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "Bruce Campbell <bruce@ldschurch.org>",
"license": "ISC"
}
$ ls
package.json
$
npm init
npm i <pgk>
code & test
npm shrinkwrap
git clone
npm install
Publish
Deploy
?
Ramp Up - Existing Project or Generated Project
Use npm install to download project dependencies from the registry
- reads package.json for dependencies
- uses the dependencies section for modules your app needs to function
- use the devDependencies for modules your app needs to build
- downloads dependencies and puts them in the ./node_modules folder within your project
npm init
npm i <pgk>
code & test
npm shrinkwrap
git clone
npm install
Publish
Deploy
?
Develop
- npm install <package>
- npm uninstall <package>
- Flags
- --save
- --save-dev
- --global
Demo
- my-app project
- previously we used npm install without a package name to read dependencies from package.json and install them.
- Now we want to add jQuery to the list of dependencies and install it.
- So we add jQuery and --save... npm i --save jquery
- i is short for install
- see package.json and ./node_modules for jQuery
- see src/server/index.js when we import some code by name, without periods or slashes, node knows to look inside the node_modules directory for a folder with a matching name to find the code we want.
- and there are other ways to consume those packages... webpack, gulp, etc.
- off we go to write some code using jQuery
- more...
Demo
- But then the dev lead on your project points out that all the jQuery functionality you need is built into Javascript now.
- npm uninstall jQuery
- see ./node_modules
- see package.json - why is jQuery still there?
- npm uninstall --S jQuery (-S is short for --save) - now it's gone
- more...
- npm uninstall jQuery
Demo
- While we're talking about the install command I should probably talk about "global" installs even though it doesn't fit perfectly into this part of the cycle
- the --global, or -g for short installs a package "globally", or outside your project
- on a mac that's /usr/local/lib/node_modules
- on windows that's ... don't know,
- use npm ls -g --depth 0 to find it
- the docs say beside the node.exe binary
- "ls" or list tells you what packages are installed
- --depth 0 limits how deep into the dependency tree it will report. Zero lists just the direct dependencies.
- -g of course is for globally installed modules.
- use npm ls -g --depth 0 to find it
- there is a trend that less and less packages require that they be installed globally.
- more...
Demo
- While we are already on a tangent, we might as well talk about executable packages.
- Some packages can have a CLI component to them.
- So instead of "require()"-ing it in to your code you use it as a tool and run it from the command line.
- Show npm list -g --depth 0
- notice n and yo... they are both command line tools
- they were installed onto my laptop as node packages using npm
- tools that need to be available from any directory on the system should be installed "globally"
tools that need only be available in the project directory can be installed "locally" and they can still be executed on the command line
npm init
npm i <pgk>
code & test
npm shrinkwrap
git clone
npm install
Publish
Deploy
?
Publish/Deploy
semver
1.2.3
http://semver.org/
https://github.com/npm/node-semver
major - breaking changes
minor - new features but compatible
patch - backwards compatible bug fixes
major
minor
patch
version strings
The dependencies sections of package.json specifies the name and a version string for each dependency
"dependencies": {
"compression": "^1.5.2",
"cookie-parser": "^1.3.5",
"dotenv": "^2.0.0",
"express": "^4.13.3",
...
}
npm shrinkwrap
- Locks down the versions of a package's dependencies so that you can control exactly which versions of each dependency will be used when you install/deploy
- You could lock down dependencies by specifying exact versions in package.json
- however npm shrinkwrap is preferred and more flexible
npm shrinkwrap
- run npm shrinkwrap
- which produces npm-shrinkwrap.json
- when npm-shrinkwrap.json is present, the npm install command honors the exact versions found within
- typically: commit the shrink wrap file to your vcs
- delete the shrink wrap file when you start your next development cycle
- npm suggests you shrink wrap your apps, not your modules
Demo
- run npm shrinkwrap in my-app
- show new npm-shrinkwrap.json and it's format
- now if we were to delete the node_modules
- and run npm i
- the versions would match the shrink wrap file
more
Updating Dependencies
-
npm outdated
- to list dependencies that have fallen behind
- "wanted" column is the highest available version that complies with the package.json range
- "latest" column is the highest available version within the registry
Updating Dependencies
-
npm update
- installs new versions of dependencies
- obeys package.json version ranges
- to break out of the version range
- manually modify package.json & run npm update or
- uninstall and re-install each dependency
Fixing EACCESS error (without sudo)
Do you get errors installing packages globally?
- Install Node with Homebrew
- Use an alternate global package directory
- Change directory permissions
https://docs.npmjs.com/getting-started/fixing-npm-permissions
3. Change Directory Permissions
-
Change the owner of npm's directories to the name of the current user
- sudo chown -R $(whoami) $(npm config get prefix)/{lib/node_modules,bin,share}
- or
-
sudo chown -R $USER $(npm config get prefix)/lib/node_modules
Installing Offline
- npm keeps a local cache
- npm config get cache
- install offline
- npm install --cache-min 999999
Troubleshooting Tips
- delete your ./node_modules directory and re-run npm i after changing node or npm versions
- check your node and npm versions against the engines property in package.json
- others tips?
Node Versions
- Even Node versions will have "Long Term Support"
- Odd Node versions get the latest features
npm v3
- progress bar
- flat flat flat
- = less files but a little slower
npm Versions
- Node 4 ships with npm 2
- run npm i -g npm to have npm upgrade itself
- remember to delete your node_modules folders
- upgrade node or npm at the beginning of an iteration, not right before releasing
- communicate this change to your team
npm Configuration Files
- per-project config file (/path/to/my/project/.npmrc)
- per-user config file (~/.npmrc)
- global config file ($PREFIX/etc/npmrc)
- npm builtin config file (/path/to/npm/npmrc)
npm hydra worm
npm hydra worm
In theory, malicious code could spread
through the npm ecosystem because
- npm lifecycle scripts allow arbitrary commands
- semver allows for changing dependencies
- persistent auth allows publishing without creds
- central repository acts as distribution point
Hydra Mitigation
- Do not stay logged in to npm... npm logout
- Use npm shrinkwrap to lock down dependencies
- Disable install scripts
- npm i <module> --ignore-scripts or
- npm config set ignore-scripts true
- Don't install a package you don't trust
- Don't sudo your installs
npm's Hydra Mitigations
- vulnerability scanning
- two-factor authentication in public registry
Both are WIP
Hydra Worm Disclosure: http://tinyurl.com/h6cbwhu
npm response: http://tinyurl.com/hujcz85
npm
By Bruce Campbell
npm
- 1,026