Scalability Lessons
Beyond 50k 85k requests/second
Your Host Tonight
Image source: my 5 yo daughter
Software engineer at MediaSmart Mobile
Freelance scalability consultant
What We Do
Serve mobile ads
Performance campaigns
85K+ requests / second
10M+ impressions / day
40+ servers
20+ countries
Guilty!
Image source: Stupid Zombies 2
We help pay for your entertainment
From the particular to the abstract
- Scalability with Node.js
- Effective Engineering in the Cloud
- Zen of Scalability
Scalability with
What Is Scalability
Serve an increasing numbers of requests
- with increasing resources
- without degradation
- cost-effectively!
Scale Up vs Scale Out
Learn how to use both, and when to use each
vs
Image source: Rama
Image source: ZDNet
Is Node.js Fast?
Define ‘fast’
Source: GodTIC (Spanish)
So Why Node.js?
Asynchronous, event-driven, non-blocking
Focus on concurrency, not performance
Node behaves very linearly:
Double the load, double the resources
Smooth learning curve
Few surprises
Plus, it's cool...
100K+ Modules to Choose From
Source: modulecounts.com
Far above all other package managers
Linearity Rules!
Source: MediaSmart Mobile
Effective Engineering
Image source: Beao
in the Cloud
Load Testing
Gather data instead of opinions
Two kind of tests:
- real (production)
- and fake (load tests)
Measure requests per second
and convert to visits per day
... with a safety factor
Test to See Where It Breaks
Image source: MythBusters
Do not be afraid to work on production if needed
Keep Up with Business
Keep business happy
And we will all be happy!
Unpaid Technical Debts Accumulate
Image source: BuffaloWorks Photography
Developments go slower
Your code runs slower
Managing Technical Debt
Business Goes First
Do not scale too soon
... and definitely not too late!
Keep lists of what you owe
and repay it as you go along
Don't let performance issues accumulate
Deployment
DYOD: do your own deployments
Deploy as often as possible!
Monitor after deploying!
Monitor everything!
Send alerts
Avoid False Alarms
Image source: DIYLOL
Send only necessary alerts
Remote Caching
Humanity's best invention
for scalability
Cache everything everywhere!
Locally and remotely
Messaging
Humanity's second best invention
for scalability
Ideal for async processing
Remove work from online processing
and aggregate it
memcached + RabbitMQ Architecture
Zen of Scalability
Image source: CloudBees Blog
there is no Perfect Architecture
It's always about trade-offs
Keep your architecture fluid
Locks are Evil
Always
No exceptions
Search for lock-free solutions
Smash those Bottlenecks
Scalability is about removing bottlenecks
Possible Bottlenecks
- CPU (obviously)
- memory
- disks
- network interfaces
- system buses
- connections
- messaging
- latency
- simultaneous requests
- frontends
- backends
- <insert your own here>: throttle down traffic
Saturation
Is the sign of a bottleneck
Beware the Thundering Herd
... and beware the self-sustaining stampede!
You Found the Cause...
Now try to find the real cause
Do root cause analysis immediately — and fully
One More Thing: Be Humble
The world doesn't care about your feelings
You cannot win an argument with reality
Unverified opinions are useless
(specially your own)
Accept facts and collect data
Thanks!
http://slid.es/alexfernandez/scalability-lessons-beyond-85krps