With A Team Of Three People
Your Host Tonight
Senior Developer at mediasmart
Organizer at MadridJS, Node.js Madrid, JSDayES
What We Will See
Meet The Team
Build Or Not
We serve mobile advertising
Match bid offers with campaigns
Real time bidding (OpenRTB)
Make bid and compose ad
In near real time (~80 ms)
We help keep the app ecosystem alive
Peaks of 800 krps
Average of 500 krps
45 billion requests per day
35 million daily impressions
High level View
Filter out 90% of traffic
Decides which requests are less attractive
Written in Erlang (!)
Rewritten in Go (!!)
Saved about 80% of costs
Holds the grunt of the 300 krps with about 10 servers
It's an operational miracle
Decides the price to pay for each bid
estimates CTR of bid offer
Written in Go (!)
Handles queries in ~500 µs (2 ms)
Use Your Leverage!
Discard traffic fast
Watch what you do online
Take work off hot paths
Optimize hot paths
Make Migrations Fluid
Never commit to anything!
Keep your options open
Isolate external accesses
Meet The Team
Remote By Default
Colleagues all around the world
Constant contact on Slack
Frequent team meetings
Screen sharing essential
Priorities on Mondays
Code Reviews Work!
Increase sense of ownership
Build your team
Brave to try new things
Failure is always an option
Warn operations and sales
Warn customers ASAP
Explain everything, blame nobody
Immediately and thoroughly
Issue recommendations (and follow through)
What Is DevOps?
Amateurs talk about tactics,
but professionals study logistics.
Gen. Robert H. Barrow, USMC
Amateurs worry about programming,
but professionals care about deployment.
Probably the most important practice in DevOps
Continuous Integration: tested on integration env
Continuous Delivery: tested on several envs
Continuous Deployment: go all the way and deploy!
A simple bash script to start
Built for (and with) Node.js packages
Held together with some custom Node.js scripts
Recently migrated to StriderCD
Now with a Graphical Interface!
StriderCD In Action
... But Do not Become This Guy
Build Or Not
Do not Be Afraid to Build
Starting a green field project can be exhilarating
Helps attract and keep elusive talent
Someone had to build first!
Why let them have all the fun?
"Not Invented Here":
Distrust products offered by third parties
If it is a core business function
do it yourself, no matter what.
As easy as identifying core business functions...
which is easy, right?
Core Business Functions
Banks: insourcing IT
Google: self-driving cars
Facebook: providing Internet access
Apple: music distribution, watches
Should I Build or Not?
There is not a universal path for all destinations
- Are you building a state-of-the-art project?
- Which options is more cost-effective?
- Which option is more likely to succeed?
- What is the cost of switching?
- Which option is more fun?
Build Your Own Orchestrator
Previously using Amazon's Elastic Load Balancer
Eating up to 20% of our monthly bill
Replaced by an Nginx balancer
DNS balancer in Node.js
Custom Lua logging
Filter CPU Load
Architecture Is Not Important
You will not get it right the first time
Make sure you can improve it later
Devise strategies to migrate
DevOps Is Cool
Not just install a Jenkins instance somewhere
Merge development and sysadmining
Write code to do fun stuff
Build upon your work
Build or Not?
If it gives you a competitive advantage
build it yourself, no matter what.
Handling 800k Requests/Second With A Three-Person Team
By Alex Fernández