Scalability Lessons

Danny MacAskill's Imaginate

Beyond 50k 85k requests/second

Your Host Tonight


Image source: my 5 yo daughter

Software engineer at MediaSmart Mobile

Freelance scalability consultant

  @pinchito

What We Do


Serve mobile ads


Performance campaigns


85K+ requests / second

10M+ impressions / day

40+ servers

20+ countries

Guilty!

Image source: Stupid Zombies 2

We help pay for your entertainment

From the particular to the abstract



  • Scalability with Node.js


  • Effective Engineering in the Cloud


  • Zen of Scalability

Scalability with


What Is Scalability


Serve an increasing numbers of requests


  • with increasing resources

  • without degradation

  • cost-effectively!



Scale Up vs Scale Out










Learn how to use both, and when to use each

vs

Image source: Rama

Image source: ZDNet

Is Node.js Fast?

Define ‘fast’

Source: GodTIC (Spanish)

So Why Node.js?

Asynchronous, event-driven, non-blocking


Focus on concurrency, not performance


Node behaves very linearly:

Double the load, double the resources


Smooth learning curve

Few surprises


Plus, it's cool...

100K+ Modules to Choose From


Source: modulecounts.com

Far above all other package managers

Linearity Rules!


Source: MediaSmart Mobile

Effective Engineering

    

Image source: Beao

in the Cloud

Load Testing


Gather data instead of opinions


Two kind of tests:


Measure requests per second

and convert to visits per day

... with a safety factor

Test to See Where It Breaks


Image source: MythBusters

Do not be afraid to work on production if needed

Keep Up with Business


Keep business happy

And we will all be happy!

Unpaid Technical Debts Accumulate


Image source: BuffaloWorks Photography

Developments go slower

Your code runs slower

Managing Technical Debt


Business Goes First


Do not scale too soon

... and definitely not too late!


Keep lists of what you owe

and repay it as you go along


Don't let performance issues accumulate

Deployment


DYOD: do your own deployments


Deploy as often as possible!


Monitor after deploying!

Monitor everything!


Send alerts

Avoid False Alarms


Image source: DIYLOL

Send only necessary alerts

Remote Caching


Humanity's best invention

for scalability


Cache everything everywhere!

Locally and remotely


Memcached + node-memcached

  Redis + redis driver

Messaging


Humanity's second best invention

for scalability


Ideal for async processing


Remove work from online processing

and aggregate it


RabbitMQ + node-amqp

memcached + RabbitMQ Architecture



Zen of Scalability

Image source: CloudBees Blog

there is no Perfect Architecture

It's always about trade-offs

Keep your architecture fluid

Locks are Evil

           

Always

No exceptions

Search for lock-free solutions

Smash those Bottlenecks






Scalability is about removing bottlenecks

Possible Bottlenecks

  • CPU (obviously)
  • memory
  • disks
  • network interfaces
  • system buses
  • connections
  • messaging
  • latency
  • simultaneous requests
  • frontends
  • backends
  • <insert your own here>: throttle down traffic

Saturation


Is the sign of a bottleneck

Beware the Thundering Herd

... and beware the self-sustaining stampede!

You Found the Cause...

Now try to find the real cause

Do root cause analysis immediately and fully

One More Thing: Be Humble


The world doesn't care about your feelings


You cannot win an argument with reality


Unverified opinions are useless

(specially your own)


Accept facts and collect data

Thanks!

@pinchito

http://slid.es/alexfernandez/scalability-lessons-beyond-85krps