Your Host Tonight
Senior Developer at MediaSmart
Organizer at MadridJS, Node.js Madrid, JSDayES
What We Will See
What Is DevOps?
Some Case Studies
Build Or Not?
What Is DevOps?
FullStack? Development, 2016
So What Is DevOps?
Amateurs talk about tactics,
but professionals study logistics.
Gen. Robert H. Barrow, USMC
Amateurs worry about programming,
while professionals care about deployment.
Probably the most important practice in DevOps
Continuous Integration: tested on integration env
Continuous Delivery: tested on several envs
Continuous Deployment: go all the way and deploy!
What Else is DevOps?
So, Just a Sysadmin? Not So Fast
Development team deploys,
monitors and orchestrates…
writing code, of course!
Everything is automated
Guess the DevOps Package!
But Node.js is Perfect for DevOps!
Integrate several APIs
Libraries for everything
Automate Your Cloud
Official AWS API:
npm install aws-sdk
Official Azure API:
npm install azure
Official Google Cloud API:
npm install gcloud
(requires C library) (sucks)
Example: Build an Orchestrator
Use "Native" Mechanisms
Run servers using Upstart or Systemd
Run periodic tasks using crontab
Use Bash wherever possible
Connect using SSH
Some Case Studies
We serve mobile advertising
Match bid offers with campaigns
Real time bidding (OpenRTB)
Make bid and compose ad
In near real time (~80 ms)
We help keep the app ecosystem alive
Peaks of 400 krps
Average of 300 krps
30 billion requests per day
34 million daily impressions
A simple bash script to start
Built for (and with) Node.js packages
Held together with some custom Node.js scripts
Recently migrated to StriderCD
Now with a Graphical Interface!
StriderCD In Action
Previously using Amazon's Elastic Load Balancer
Eating up to 20% of our monthly bill
Replaced by an Nginx balancer
DNS balancer in Node.js
Custom Lua logging
Custom Node.js script
Reads stats every minute
Aggregates stats in CloudWatch
Runs using crontab
Can kill instances not responding
Filter out 90% of traffic
Decides which requests are less attractive
Written in Erlang (!)
Rewritten in Golang (!!)
Saved about 80% of costs
Holds the grunt of the 400 krps with about 10 servers
It's an operational miracle
Filter CPU Load
Distribute a command to a set of servers
Custom scripts using AWS libraries
Our next adventure:
migrate to ShipIt
another npm package
Do not Be Afraid to Build
Just going to give you emotional arguments
Do not expect answers! Not from me
At most, some uncomfortable questions
The Joy of Building Things
People build things because they love to
Starting a green field project can be exhilarating
Helps attract and keep elusive talent
Someone had to build first!
Why let them have all the fun?
"Not Invented Here":
Distrust products offered by third parties
If it is a core business function
do it yourself, no matter what.
As easy as identifying core business functions
Core Business Functions
Banks: outsourcing IT
Google: self-driving cars
Facebook: providing Internet access
Apple: music distribution, watches
Reinventing the Wheel
Does the world really need a new testing library?
I don't know or care
If I need it, I will surely build it
$7600 Tesla Wheels
Learning, the Easy Way
The best way to learn is by building
DevOps is not just installing a package
Make infrastructure a first-class citizen
Installing a package can be simple if you know how
The best way to learn infrastructure is by building it
Software devs are artisans practicing their craft
Artisans may get carried away
and build too much
Use modules as standard parts
Maintaining software can be hard
Open Source Software
Middle ground between building and installing
Software as a community
Build custom extensions
and share them!
E.g.: send diffs by email using StriderCD
E.g. use variable instances with ShipIt
Joining the Pieces
Combine several packages into a larger entity
Continuous deployment requires:
- code repos
- test library
- notification tools
- distributed deployment
May require glue code
So, Should I Build or Not?
There is not a universal path for all destinations
- Are you building a state-of-the-art project?
- Which options is more cost-effective?
- Which option is more likely to succeed?
- What is the cost of switching?
- Which option is more fun?
DevOps Is Cool
Not just install a Jenkins instance somewhere
Merge development and sysadmining
Write code to do fun stuff
Build upon your work
Node.js is perfect!
In-browser is also cool
Emerging Node.js packages
Watch operational constraints
Build or Not?
If it gives you a competitive advantage
build it yourself, no matter what.
FullStack DevOps, DevOps Pro Moscow
By Alex Fernández