Skeepers Influence Workshop

Your HOsts Tonight

Alex Fernández (pinchito)
Director of Eng, Influence
Platform Lead, Influence

Alfredo López

What We Will See

What is Scalability?

Horizontal and Vertical Scaling

Scaling Strategies

🚀 What is  scalability

🪜 Definition of scalability

The capacity to be changed in size or scale.

The ability of a computing process to be used or produced in a range of capabilities.

Lexico (Oxford Dictionary)

⛷️ Scale Up and Down!

📚 Use in Literature

🐧 Example: Linux

From embedded

✋ What makes something Scalable?

The right question is: why doesn't it scale?

  • Scarcity of a resource
  • Growing wait times
  • Unability to answer
  • Blocking on a resource
  • ...

📝 Exercise: non Scalable Service

Install Node.js

Install loadtest
$ npm install -g loadtest 

Run the command
$ loadtest -n 2000 -c 100 --keepalive
Write down the average for requests per second (rps),
average latency and number of errors

📝 Exercise +

Adjust rps from 10 to 100 ±10

$ loadtest -n 2000 -c 100 --rps 30 -k
$ loadtest -n 2000 -c 100 --rps 40 -k
Write down rps, latency and errors

Go up to 100, then 1000
$ loadtest -n 2000 -c 100 --rps 1000 -k

📝 Exercise +

Draw a graph with rps sent and result

Another graph with rps vs latency

📝 Exercise +

Now test against:
loadtest -n 2000 -c 100 -k

What is the difference?

How do latency and rps behave now?

👍 Success!

 🥛 What Resource Run out?

Graph with CPU from AWS:

300 rps
400 rps
1000 rps

🚂 rps vs throughput

📈 Scalability Profiles

Brendan Gregg: Systems Performance

🚒 Latency vs rps

⚖️ Little's Law

Little, 1952 - 1960

The average number of requests in flight L equals:
the rate of requests per second λ
multiplied by the average request time W.

If we increase concurrency L,
the average time per request W grows proportionally.

v and ⇔ h Scaling

🧓 Hard Beginnings

IBM mainframe

💽 Specialized Servers

🗄️ The Usual Cabins

🤖 And then Google Arrived

⇕ vertical Scaling

Buy a bigger machine

And bigger

Until you run out of machines

Hard to go back to a smaller machine 😅

⇕ vertical Scaling

🤫 Sshhh...

For many decades now, supercomputers are just...
clusters of smaller machines
IBM Blue Gene/P: 164k cores, 2007

⇔ Horizontal Scaling

Use many similar machines for a given function


Add or remove machines to scale

When one machine is failing it is removed from service

⇔ horizontal Scaling

📝 Exercise: Storage

Design a corporate storage system with 15 TB

Option 1 ⇕: storage area network (SAN)

Best option as of december 2008

Option 2 ⇔: raw hard drives

Best option as of july 2009


Add controllers

RAID options (Redundant Array of Inexpensive Disks)

Measure the $ difference between option 1 and option 2

Final price?


Consider redundancy strategies

Fault tolerance

Redundancy options: 2x, 3x, ?

Consider scaling strategies

How do they affect the price?

👍 success!

⇔ Horizontal Strategies

🤹 Balancing (server-side)

🕵️ Balancing (client-side)

💝 Affinity

🔱 Independence

🍇 Clustering

🔑 Sharding

🧬 Replication

⌛ Queues

🤹 Server-side Balancing

Example: AWS ELB, Google Cloud Load Balancing

🕵️ Client-Side Balancing

Example: Facebook client
Chooses the API endpoint in the browser

Example: DNS balancing

💝 Affinity ⇔

By cookie or geographical

Needs a sophisticated router
Client-side or server-side

🔱 Independence ⇔

Neutral (or blind) balancing

🍇 Clustering ⇔

Generic term "cluster":
create one machine out of many

In databases usually means having more than one server
all equivalent

🔑 Sharding ⇔

Balancing by key

Needs a sharding algorithm (usually with hashing)

🧬 Replication ⇔

A primary server (read + write) and several replicas (read-only)

Useful when reading > writing

🐫 Active REPLICAtion ⇔

Active-active, multiple primary...

Needs a conciliation algorithm

⌛ Queues ⇔

Production of tasks independent of consumption

Mechanism for polling
(NOT pooling 🙏)

📝 Exercise: Scalable Storage

You work for search engine Fooble in January 2000

You have to store the search index

Design a scaling strategy

Assume index = page sizes


10 KB per page

10 search terms max

Target time of 0.1 seconds per search


50M pages × 10 KB = 500 GB

Cheapest disk drive: Seagate ST317242A, 17.2 GB, $152

32 disks × 16 GB = 512 GB, $4864

8 servers × 4 disks = 32 disks

4M searches × 100 ms = 400k seconds = 4.6 servers
Adding peak time: at least 8 servers


100 ms for ~5 search terms
Average query time to storage < 20 ms


Query time: seek time + 1/2 turn + formatting

Seek time: ~8 ms
7200 rpm disk drive: 8 ms per turn
Query total: >12 ms

Seems doable; better add some caching

👍 Well done!

📚 Bibliography

Skeepers Influence Workshop: Distributed Systems

By Alex Fernández

Skeepers Influence Workshop: Distributed Systems

Workshop for Skeepers Influence: Distributed Systems

  • 369