Scalability
Skeepers Influence Workshop
Your HOsts Tonight
Alex Fernรกndez (pinchito)
Director of Eng, Influence
Platform Lead, Influence
Alfredo Lรณpez
What We Will See
What is Scalability?
Horizontal and Vertical Scaling
Scaling Strategies
๐ช Definition of scalability
The capacity to be changed in size or scale.
The ability of a computing process to be used or produced in a range of capabilities.
โท๏ธ Scale Up and Down!
๐ Use in Literature
โ What makes something Scalable?
The right question is: why doesn't it scale?
- Scarcity of a resource
- Growing wait times
- Unability to answer
- Blocking on a resource
- ...
๐ Exercise: non Scalable Service
Install Node.js
Install loadtest
$ npm install -g loadtest
Run the command
$ loadtest https://gorest.co.in/public/v1/users -n 2000 -c 100 --keepalive
Write down the average for requests per second (rps),
average latency and number of errors
โฎฏ
๐ Exercise +
Adjust rps from 10 to 100 ยฑ10
$ loadtest http://service.pinchito.es:3000/a -n 2000 -c 100 --rps 30 -k
$ loadtest http://service.pinchito.es:3000/a -n 2000 -c 100 --rps 40 -k
...
Write down rps, latency and errors
Go up to 100, then 1000
$ loadtest http://service.pinchito.es:3000/a -n 2000 -c 100 --rps 1000 -k
โฎฏ
๐ Exercise +
Draw a graph with rps sent and result
Another graph with rps vs latency
โฎฏ
๐ Exercise +
Now test against:
loadtest https://www.google.com -n 2000 -c 100 -k
What is the difference?
How do latency and rps behave now?
โฎฏ
๐ฅ What Resource Run out?
Graph with CPU from AWS:
300 rps
400 rps
1000 rps
๐ Scalability Profiles
โ๏ธ Little's Law
Little, 1952 - 1960
The average number of requests in flight L equals:
the rate of requests per second ฮป
multiplied by the average request time W.
If we increase concurrency L,
the average time per request W grows proportionally.
๐ค And then Google Arrived
โ vertical Scaling
Until you run out of machines
Hard to go back to a smaller machine ๐
๐คซ Sshhh...
For many decades now, supercomputers are just...
clusters of smaller machines
IBM Blue Gene/P: 164k cores, 2007
โ Horizontal Scaling
Use many similar machines for a given function
("provisioning")
Add or remove machines to scale
When one machine is failing it is removed from service
๐ Exercise: Storage
Design a corporate storage system with 15 TB
Option 1 โ: storage area network (SAN)
Option 2 โ: raw hard drives
๐ EXERCISE +
Add controllers
RAID options (Redundant Array of Inexpensive Disks)
Measure the $ difference between option 1โ and option 2โ
๐ EXERCISE +
Consider redundancy strategies
Redundancy options: 2x, 3x, ?
Consider scaling strategies
How do they affect the price?
โฎฏ
โ Horizontal Strategies
๐คน Balancing (server-side)
๐ต๏ธ Balancing (client-side)
๐ Affinity
๐ฑ Independence
๐ Clustering
๐งฌ Replication
โ Queues
๐คน Server-side Balancing
Example: AWS ELB, Google Cloud Load Balancing
๐ต๏ธ Client-Side Balancing
Example: Facebook client
Chooses the API endpoint in the browser
Example: DNS balancing
๐ Affinity โ
By cookie or geographical
Needs a sophisticated router
Client-side or server-side
๐ฑ Independence โ
Neutral (or blind) balancing
๐ Clustering โ
Generic term "cluster":
create one machine out of many
In databases usually means having more than one server
all equivalent
๐ Sharding โ
Balancing by key
Needs a sharding algorithm (usually with hashing)
๐งฌ Replication โ
A primary server (read + write) and several replicas (read-only)
Useful when reading > writing
๐ซ Active REPLICAtion โ
Active-active, multiple primary...
Needs a conciliation algorithm
โ Queues โ
Production of tasks independent of consumption
Mechanism for polling
(NOT pooling ๐)
๐ Exercise: Scalable Storage
You work for search engine Fooble in January 2000
You have to store the search index
Design a scaling strategy
Assume index = page sizes
โฎฏ
๐ EXERCISE +
10 KB per page
10 search terms max
Target time of 0.1 seconds per search
โฎฏ
๐ EXERCISE +
50M pages ร 10 KB = 500 GB
Cheapest disk drive: Seagate ST317242A, 17.2 GB, $152
32 disks ร 16 GB = 512 GB, $4864
8 servers ร 4 disks = 32 disks
4M searches ร 100 ms = 400k seconds = 4.6 servers
Adding peak time: at least 8 servers
โฎฏ
๐ EXERCISE +
100 ms for ~5 search terms
Average query time to storage < 20 ms
โฎฏ
๐ EXERCISE +
Query time: seek time + 1/2 turn + formatting
Seek time: ~8 ms
7200 rpm disk drive: 8 ms per turn
Query total: >12 ms
Seems doable; better add some caching
โฎฏ