Why is CI Broken?

Kelly Selden

@kellyselden

The Problem:

All new PRs are failing for the same cryptic reason. CI was working fine yesterday. Master is broken even though no new commits.

n·p·m de·pend·en·cy hell

/enˈpēəm dəˈpendənsē hel/

noun

  1. somebody, somewhere in the dependency tree **** up.
  2. ¯\_(ツ)_/¯

More specifically:

A dependency three layers deep updated and introduced a bug, violated semver, … or was deleted (left-pad).

"ember-browserify": "^1.0.0"

"ember-browserify": "1.0.0"

"ember-browserify": "1.1.0"

Yesterday

Today

"ember-browserify": "1.0.0"

"ember-browserify": "1.0.0"

Yesterday

Today

"browserify": "13.0.0"

"browserify": "13.0.1"

"ember-browserify": "1.0.0"

"browserify": "^13.0.0"

"ember-browserify": "1.0.0"

"ember-browserify": "1.0.0"

"ember-browserify": "1.0.0"

Yesterday

Today

"browserify": "13.0.0"

"browserify": "13.0.0"

"browserify": "13.0.0"

"glob": "5.0.15"

"glob": "5.0.16"

"glob": "^5.0.15"

And so on...

And so on...

And so on...

And so on...

And so on...

And so on...

And so on...

And so on...

Npm is a Trust Game

You are always at the mercy of your dependencies and how loose their version hints are

...even if you take all the hints out of your app.

When you install a new dependency, do you go through its entire dependency tree and audit every package: decide if they will follow SemVer and if the version hint applied is appropriate?

No

Which means your stuff is going to break.

Started as a pet project to alleviate pain points in Ember-Cli maintenance, but anyone can use. Not just Ember, anything using npm.

It's an Ember app!

  • Uses ember-simple-auth and torii for GitHub auth (more API requests per hour)
  • Uses ember-concurreny to pause and resume crawling (task canceling), and as a request semaphore (enqueue)
  • Uses a request limiter (so I don't get locked out of npm's API) and various request caching
  • Uses ember-exam for random test order and splitting tests across multiple Travis CI jobs (faster!)
  • Uses ember-cli-code-coverage for... code coverage!
  • Has an accompanying node express app that does oauth and proxies npm requests
  • Could use some CSS...

Thank you

Why is CI Broken?

By Kelly Selden

Why is CI Broken?

  • 1,316