All those times
we broke
WaffleJS, 2018-11-07
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5133272/npm-logo-red.png)
Read these slides on your device:
What are we talking about?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5375160/Screenshot_2018-10-15_10.19.45.png)
Warning:
extreme levels of profanity ahead
I stole this talk idea
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458240/slide-2-1024.jpg)
from
#1: The certificate outage
February 2014
We broke the registry a lot in January 2014
SSL in Node.js used to be a lot harder
Migrations are hard
This is going to be a theme
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458392/fuckup-1.png)
DOUBLE FUCK
Rolling back is hard,
even harder when everyone is yelling at you.
What did we learn?
- We are bigger than we thought!
- We can never change cert providers 🙁
- Have a rollback plan
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458425/Screenshot_2018-11-07_07.33.00.png)
#2: The password change outage
April 2014
CouchDB
was not a good idea
Load at 100%
no matter how big a box we buy.
WTF?
Password change errors
Fucked by silent string conversions
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5459370/Screenshot_2018-11-07_17.24.23.png)
Infinite loops will fuck you
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458464/fuckup-2.png)
What did we learn?
- Couch is a terrible fucking database
- Don't roll your own... anything
- Don't ignore "small" errors
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458455/Screenshot_2018-11-07_07.36.15.png)
#3: left-pad
March 2016
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458456/Screenshot_2018-11-07_12.30.54.png)
Kik me
Then we fucked up
by not having a clear policy
404: left-pad not found
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458460/Screenshot_2018-11-07_12.32.43.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458461/fuckup-3.png)
I will feel genuinely bad about left-pad forever
What did we learn?
- Unpublishes are really dangerous
- Unpublishes are impossible after 24 hours
What did we learn?
- We're even bigger now
- Ignore vague legal threats
- Hire a damn lawyer
- Have clear policies
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458477/Screenshot_2018-11-07_07.44.19.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458479/Screenshot_2018-11-07_07.44.54.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5459191/Screenshot_2018-11-07_16.03.12.png)
#4: Cloudflare migration
May 2018
We broke somebody else's registry
Infinite loop AGAIN, motherfucker
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458491/e31.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/136956/images/5458498/fuckup-4.png)
DOUBLE FUCK
Who deploys on a Friday???
What did we learn?
- Don't deploy on a Friday
- Don't have hard deadlines based on early estimates
- We are so big we are responsible for stuff we're not even responsible for
We have fucked up so many more times
I need a longer talk to cover them all
![](https://s3.amazonaws.com/media-p.slid.es/uploads/seldo/images/380749/Twitter_logo_blue.png)
@seldo
These slides are available right now
Now would be a good time to follow me on Twitter
npm ❤️ you
All those times we broke the registry
By seldo
All those times we broke the registry
- 4,571