Scalability Lessons

Danny MacAskill's Imaginate

Beyond 10K 40k requests/second

Your Host Tonight


Image source: my 5 yo daughter

Software engineer at MediaSmart Mobile

alexfernandez, @pinchito, alejandrofer

Freelance scalability consultant

What We Do


Serve mobile ads


Performance campaigns


40K+ requests / second

10M+ impressions / day

35+ servers

10+ countries

Guilty!

Image source: Stupid Zombies 2

We help pay for your entertainment

From the particular to the abstract


  • Scalability with Node.js

  • Technical Challenges

  • Effective Engineering in the Cloud

  • Messaging and Caching Are your Friends

  • Zen of Scalability

Scalability with


What Is Scalability


Serve an increasing numbers of requests


  • with increasing resources

  • without degradation

  • cost-effectively



Is Node.js Fast?

Define ‘fast’

Source: GodTIC (Spanish)

So Why Node.js?


Asynchronous, event-driven, non-blocking


Focus on concurrency, not performance


Node behaves very linearly:

Double the load, double the resources


Smooth learning curve

Few surprises

70K+ Modules to Choose From



Source: modulecounts.com

Ruby down, Maven ~ one month away

Scale Up vs Scale Out

vs

Image source: Rama

Image source: ZDNet

Linearity Rules!


Source: MediaSmart Mobile

Technical Challenges


Image source: usvsth3m

Keep Up with Business


Keep business happy!

And we will be happy as well

Callback Hell and Back


Escape the callback pyramid


Use named functions, not lambdas


Use events as they were meant to be


Would you bet your business on promises?

(Sorry about that)

Waiting for GENERATORs


Image source: Dr Electric

But are generators Really cool?

Perhaps not...

Unpaid Technical Debts Accumulate


Image source: Chelydra.org

Developments go slower

Your code runs slower

Managing Technical Debt


Business Goes First


Do not scale too soon

... and definitely not too late!


Keep lists of what you owe

and repay it as you go along


Don't let performance issues accumulate

Effective Engineering

    

Image source: Beao

in the Cloud

Load Testing


Gather data instead of opinions


Two kind of tests:


Measure requests per second

... and then correct by a given factor

Test to See Where It Breaks


Image source: MythBusters

And do not be afraid to work on production if needed

Deployment


DYOD: do your own deployments


Deploy as often as possible!


Monitor after deploying!

Monitor everything!


Send alerts

Avoid False Alarms


Image source: Laughing Squid

Send only necessary alerts

Messaging & Caching

Are Your Friends

Humanity's Greatest Inventions

Image source: Josh Linkner

Beware Swiss Army Knives

Remote Caching


Humanity's best invention

for scalability


Cache everything everywhere!

Locally and remotely


Memcached + node-memcached

  Redis + redis driver

Database-Crowding Architecture


Memcached Architecture



Messaging

Humanity's second best invention

for scalability


Ideal for async processing


Remove work from online processing

and aggregate it


RabbitMQ + node-amqp

           Redis + redis driver

Sample RabbitMQ Architecture



Sample Redis Architecture




Redis has different access patterns

Zen of Scalability

Image source: CloudBees Blog

Locks are Evil

Always

No exceptions

            Source: Haretz

Search for lock-free solutions

Smash those Bottlenecks

Scalability is about removing bottlenecks

Possible Bottlenecks

  • CPU (obviously)
  • memory
  • disks
  • network interfaces
  • system buses
  • connections
  • messaging
  • latency
  • simultaneous requests
  • frontends
  • backends
  • <insert your own here>: throttle down traffic

Saturation


Is the sign of a bottleneck

Beware the Thundering Herd

... and beware the self-sustaining stampede!

Avoid Unbounded Tasks

First order of business: bound your tasks

Set a target for optimizations

You Found the Cause...


Image source: glee wiki

Now look for the real cause!

Root Cause Analysis


Do root cause analysis immediately and fully


Farther away from symptom → less effective


Work on symptoms if needed


Why? why? why? why? why?

(The five whys)

Improve Your Wheels


Reinventing the wheel is stupid

Image source: Michael Covel

Building improved wheels is great!

$6500 Wheels

Image source: Tesla Motors

Be Humble


The world doesn't care about your feelings


You cannot win an argument with reality


Unverified opinions are useless

(specially your own)


Accept facts and collect data

One More Thing


The great menace to progress is not ignorance

but the illusion of knowledge.

Daniel J. Boorstin


The teacher gets the most out of the lessons,

and the true teacher is a learner.

Elbert Hubbard

Thanks!


And now...


Image source: livluvcreate

Questions?

Thanks Again!




http://slid.es/alexfernandez/scalability-lessons




Oh and by the way...

We're hiring!