There is a better way

Code out of control: red flags

Presentation outline

Build time > 15 seconds

• Large projects are never finished

• Monolithic Node does not work

• Java shop culture

Large projects are bad: Extra reading

Software complexity

= number of interactions

Cognitive research

3 - 7 things at once

"Thinking, Fast and Slow" by Daniel Kahneman

Solution to interaction complexity

function add(a, b) { ... }
var sum = add(2, 3);
console.log('2 + 3 =', sum);

• functions separate variables via scopes

• source files separate code

• teams separate people

• dependencies separate files

• slides separate bullet points

Splitting code via dependencies

Things that are simpler in small projects

• API design
• Testing
• Installation
• Documentation
• Code reviews and standards
• Code and module reuse
• Onboarding

Set clear boundary

// function signature
function add(a, b) { ... }

// package.json
{
    "name": "my-utils",
    "version": "2.0.1",
    "main": "index.js"
    "dependencies": {
        "module-a": "1.0.0",
        "module-b": "0.1.0"
    }
}

Respect the boundary

// good
var a = require('another-module');
// avoid
var a = require(
    './node_modules/another-module/src/something.js');

Each auto part is

1. built separately

2. tested separately

3. has SKU number

4. shipped to the auto plant

5. placed into the car

Assembling apps

• Single repo per app
• Shared code via separate repos (dependencies)
• Each repo has its own version

12 Factor App (from Heroku)

Packaging

Java - Maven

Perl - CPAN

JavaScript - Node Package Manager (NPM)

NPM in action

npm search colors
npm info chalk
npm home chalk
npm install chalk --save

// index.js
require('chalk');
console.log( chalk.blue('Hello world!') );

Node: NPM

{
  "name": "my-module",
  "version": "0.1.0",
  "git": ...,
  "dependencies": {
    "foo": "0.1.*",
    "bar": "~1.2.0",
    "baz": "^2.0.1"
  },
  "devDependencies": {
    "grunt-concat": "0.1.1"
  }
}

Eliminate surprises

Use exact versions

Dependency resolution: Nodejs

// dependencies-resolution uses module-a@2.0.0, module-b@0.6.0
// module-b@0.6.0 uses module-a@1.0.0

$ npm list
dependencies-resolution@0.0.0
├── module-a@2.0.0
└─┬ module-b@0.6.0
  └── module-a@1.0.0

Worry about the top level.

really

major . minor . patch

Semantic versioning

semver.org

• major: I broke it

• minor: I added new feature

• patch: I fixed something

really

Semantic versioning

semver.org

Dependency tree in practice

0.1.0

2.2.0

2.2.0

0.1.0

Dependency tree in practice

0.1.0

2.2.0

2.2.0

0.1.0

0.1.1

0.1.2

Real issue: out of date

// package.json
{
    "dependencies": {
        "module-a": "1.0.0"
    }
}
// npm registry:
module-a: 0.8.0, 0.9.0, 1.0.0, 1.0.1, 2.0.0

david-dm

Q: Can I upgrade?

// package.json
{
    "dependencies": {
        "module-a": "1.0.0"
    }
}
// npm registry:
module-a: 0.8.0, 0.9.0, 1.0.0, 1.0.1, 2.0.0

A: No one knows

// package.json
{
    "dependencies": {
        "module-a": "1.0.0"
    }
}
// npm registry:
module-a: 0.8.0, 0.9.0, 1.0.0, 1.0.1, 2.0.0

Relying on human-supplied semver is like relying on human-typed code comments to be 100% accurate

Q: Is this a mess?

• the mess is manageable.
• the mess maps nicely to the software development:
  • different parts are developed at different speeds.
• using versioned dependencies isolates the true mess: constant merging of commits

Develop + integrate

function add(a, b) { XXX }
...
... add(2, 3); 
    ....
add(-1, 100); 
...
...
console.log(add(0, 0));
...

Develop + integrate

function add(a, b) { XXX }
...
... add(2, 3); 
    ....
add(-1, 100); 
...
...
console.log(add(0, 0));
...

Develop + integrate

function add(a, b) { XXX }
...
... add(2, 3); 
    ....
add(-1, 100); 
...
...
console.log(add(0, 0));
...

Hard to improve "add" AND keep everything working

Develop THEN integrate

// package.json "add": "1.0.0"
var add = require('add');
...
... add(2, 3); 
    ....
add(-1, 100); 
...
...
console.log(add(0, 0));

// module ADD@1.0.0
function add(a, b) { ... }

Develop THEN integrate

// package.json "add": "1.0.0"
var add = require('add');
...
... add(2, 3); 
    ....
add(-1, 100); 
...
...
console.log(add(0, 0));

// module ADD@1.1.0
function add(a, b) { XXX }

Develop THEN integrate

// package.json "add": "1.1.0"
var add = require('add');
...
... add(2, 3); 
    ....
add(-1, 100); 
...
...
console.log(add(0, 0));

// module ADD@1.1.0
function add(a, b) { XXX }

Revert, no harm done

// package.json "add": "1.0.0"
var add = require('add');
...
... add(2, 3); 
    ....
add(-1, 100); 
...
...
console.log(add(0, 0));

// module ADD@1.1.0
function add(a, b) { XXX }

A: next-update

next-update

next-update

next-update

next-update

If you test your software - you will get 3rd party upgrades with zero effort

Q: Is the update from A@x.y.z to A@x.y+1.z

likely to succeed?

next-update-stats

next-update-stats

Semver adherence

Measuring semver adherence

Semver adherence

Measuring semver adherence

Semver adherence

Measuring semver adherence

type      from     to      successful %
patch     0.2.0   0.2.1        100
patch     0.4.0   0.4.1        100
patch     0.6.0   0.6.1        100 
patch     0.6.4   0.6.5        100
minor     0.6.5   0.7.0        100
major     0.8.1   1.0.0          0
minor     1.0.0   1.1.0        100
patch     1.1.0   1.1.1         94
minor     1.1.1   1.2.0        100
patch     1.2.0   1.2.1        100
minor     1.2.1   1.3.0        100
patch     1.3.0   1.3.1        100
patch     1.3.1   1.3.2        100
minor     1.3.2   1.4.0        100

check-types.js

Semver adherence

lodash respects semver

underscore and Ramda do not

Q: Can we keep upgrading ALL projects?

A: flip the switch with next-updater

next-updater

Next-updater

next-updater

Next-updater: scared?

iOS6 vs iOS7 updates

Automatic updates with next-updater are not scary

• Minor / patch updates

• If your project has high code coverage

• If a particular dependency update has a high probability of being successful

We fail to follow semver

We fail to follow SemVer – and why it needn’t matter by Stephen Bonnemann (JSConf Budapest 2015)

We fail to follow semver

- and it does matter

rant

We fail to follow semver

- and it does matter

rant

Automate SemVer

We fail to follow SemVer – and why it needn’t matter by Stephen Bonnemann (JSConf Budapest 2015)

Test each code against its own

previous tests

semantic-release

dont-break from module X

1. Install each dependent project

2. Replace X@x.y.z with X@current

3. Run unit tests 

If we are breaking dependent projects - maybe we should increment MAJOR in semver major.minor.patch

dont-break from module X