Scalability Lessons
Beyond 10K 40k requests/second
Your Host Tonight
Image source: my 5 yo daughter
Software engineer at MediaSmart Mobile
alexfernandez, @pinchito, alejandrofer
Freelance scalability consultant
What We Do
Serve mobile ads
Performance campaigns
40K+ requests / second
10M+ impressions / day
35+ servers
10+ countries
Guilty!
Image source: Stupid Zombies 2
We help pay for your entertainment
From the particular to the abstract
- Scalability with Node.js
- Technical Challenges
- Effective Engineering in the Cloud
- Messaging and Caching Are your Friends
- Zen of Scalability
Scalability with
What Is Scalability
Serve an increasing numbers of requests
- with increasing resources
- without degradation
- cost-effectively
Is Node.js Fast?
Define ‘fast’
Source: GodTIC (Spanish)
So Why Node.js?
Asynchronous, event-driven, non-blocking
Focus on concurrency, not performance
Node behaves very linearly:
Double the load, double the resources
Smooth learning curve
Few surprises
70K+ Modules to Choose From
Source: modulecounts.com
Ruby down, Maven ~ one month away
Scale Up vs Scale Out
vs
Image source: Rama
Image source: ZDNet
Linearity Rules!
Source: MediaSmart Mobile
Technical Challenges
Image source: usvsth3m
Keep Up with Business
Keep business happy!
And we will be happy as well
Callback Hell and Back
Escape the callback pyramid
Use named functions, not lambdas
Use events as they were meant to be
Would you bet your business on promises?
(Sorry about that)
Waiting for GENERATORs
Image source: Dr Electric
But are generators Really cool?
Perhaps not...
Unpaid Technical Debts Accumulate
Image source: Chelydra.org
Developments go slower
Your code runs slower
Managing Technical Debt
Business Goes First
Do not scale too soon
... and definitely not too late!
Keep lists of what you owe
and repay it as you go along
Don't let performance issues accumulate
Effective Engineering
Image source: Beao
in the Cloud
Load Testing
Gather data instead of opinions
Two kind of tests:
- real (production)
- and fake (load tests)
Measure requests per second
... and then correct by a given factor
Test to See Where It Breaks
Image source: MythBusters
And do not be afraid to work on production if needed
Deployment
DYOD: do your own deployments
Deploy as often as possible!
Monitor after deploying!
Monitor everything!
Send alerts
Avoid False Alarms
Image source: Laughing Squid
Send only necessary alerts
Messaging & Caching
Are Your Friends
Humanity's Greatest Inventions
Image source: Josh Linkner
Beware Swiss Army Knives
Remote Caching
Humanity's best invention
for scalability
Cache everything everywhere!
Locally and remotely
Database-Crowding Architecture
Memcached Architecture
Messaging
Humanity's second best invention
for scalability
Ideal for async processing
Remove work from online processing
and aggregate it
Sample RabbitMQ Architecture
Sample Redis Architecture
Redis has different access patterns
Zen of Scalability
Image source: CloudBees Blog
Locks are Evil
Always
No exceptions
Source:
Haretz
Smash those Bottlenecks
Scalability is about removing bottlenecks
Possible Bottlenecks
- CPU (obviously)
- memory
- disks
- network interfaces
- system buses
- connections
- messaging
- latency
- simultaneous requests
- frontends
- backends
- <insert your own here>: throttle down traffic
Saturation
Is the sign of a bottleneck
Beware the Thundering Herd
... and beware the self-sustaining stampede!
Avoid Unbounded Tasks
First order of business: bound your tasks
Set a target for optimizations
You Found the Cause...
Image source: glee wiki
Now look for the real cause!
Root Cause Analysis
Do root cause analysis immediately — and fully
Farther away from symptom → less effective
Work on symptoms if needed
Why? why? why? why? why?
Improve Your Wheels
Reinventing the wheel is stupid
Image source: Michael Covel
Building improved wheels is great!
$6500 Wheels
Image source: Tesla Motors
Be Humble
The world doesn't care about your feelings
You cannot win an argument with reality
Unverified opinions are useless
(specially your own)
Accept facts and collect data
One More Thing
The great menace to progress is not ignorance
but the illusion of knowledge.
The teacher gets the most out of the lessons,
and the true teacher is a learner.
Thanks!
And now...
Image source: livluvcreate
Questions?
Thanks Again!
http://slid.es/alexfernandez/scalability-lessons
Oh and by the way...
We're hiring!