Scaling APIs with Performance Testing
We skipped the beta,
deployed once,
everybody signed up,
and never had a problem.'
- No one ever
Promise: Scaling API's With Performance Testing
Scaling API's is about:
- Optimizing Code
- Architecture
- Operations
- Sensible Features
Performance is about:
- Measuring performance (testing)
- Identifying the slow bits (profiling)
- Optimizing what matters (optimization)
Promise: Scaling API's With Performance Testing
Scaling API's is about:
- Optimizing Code
- Architecture
- Operations
- Sensible Features
Performance is about:
- Measuring performance (testing) ← This talk!
- Identifying the slow bits (profiling)
- Optimizing what matters (optimization)
Who is Michael Cole?
Fullstack JS, DevOps, Freelancer, Digital Nomad.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/1865983/11425760_10202710857132399_8289717214890549386_o.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/1865993/11406978_10102251678979140_3349581503666242491_n.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/1865994/11136223_10153707726984008_1535954375760784838_o.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/1865999/10661849_10201493254373091_6432087433591113527_o.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2888421/default.png)
bigSmall.io Micro-Consulting Platform
- Micro-consulting platform
- Video chat + invoices + teams
- No job's too small if it's 100% billable
- Sustainable Open Source
- Helping people should be awesome!
Why Performance Testing?
Unit testing = feature confidence for changes
Performance testing = feature confidence for # of users
Performance testing came from Enterprise software (don't google "example test plan")
Agile Performance Testing means leaner, smaller, and iterative
What's your purpose?
-
Can I make this use case faster? In 2 weeks.
-
Can each server to serve 1000 requests/second, without crashing or errors? In 2 weeks.
-
We have a marketing event. App must serve 70,000 requests/second. In 2 weeks.
Measure
Change
Predict
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889485/circular-arrow-clip-art_33546.png)
In 2 weeks.
Perf Testing Menagerie
-
Load testing
How long does it take to process 10,000 requests? -
Endurance/Soak testing
Measure resource leaks over sustained use.
How often will the server "randomly" crash? -
Stress testing
Upper performance limit before failure.
Can it support 10,000 users simultaneously? -
Spike testing
Sudden increased load and measures results.
Slashdot effect testing
Questions?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2879957/6360030805587666061064451943_questions.jpg)
Today!
-
Build a small web app
-
Deploy it
- Performance test it
Today!
-
Build a small web app
https://github.com/MichaelJCole/evil_kittens
-
Deploy it
Locally, then Heroku
- Performance test it
Evil Kittens must REST!
Text
GET /tiny-kittens-in-your-memory/:id -> text/plain: filename
GET /big-kittens-in-your-memory/:id -> image/jpeg: [kitten jpg data]
GET /evil-kittens-in-your-memory/:id -> lock 1 mb of memory for 1 sec
GET /evil-kittens-in-your-cpu/:id -> Calculate fib(30) badly
GET /evil-kittens-in-your-disk/:id -> writes 1 mb to disk
GET /evil-kittens-in-your-network/:id -> sends 512k of random data
Text
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889516/albino-burmese-kitten.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889517/kitten-in-hand.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889523/kitten-met-blauwe-ogen.jpg)
https://github.com/MichaelJCole/evil_kittens
A server to test
# If you'd like to play along on your laptop:
# google `github evil_kittens`
# Get the code
git clone git@github.com:MichaelJCole/evil_kittens.git
cd evil_kittens
# Install dependencies
npm install
# Start server
node single.js
# Make requests
curl http://localhost:8000/evil-kittens-in-your-cpu/hello
wget -qO- http://localhost:8000/evil-kittens-in-your-cpu/hello
Text
https://github.com/MichaelJCole/evil_kittens
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889518/kitten-loves-toy.jpg)
A simple Load Test
# Install ApacheBench
sudo apt-get install apache2-utils
# MEASURE: -n 1000 requests, using -c 10 threads, -l to relax
node single.js
ab -l -n 1000 -c 10 http://localhost:8000/evil-kittens-in-your-cpu/single
# Hmmmm... How could it be faster? Lets turn off logging
node singleNoLog.js
ab -l -n 1000 -c 10 http://localhost:8000/evil-kittens-in-your-cpu/single
# Hmmmn... What if we used the cluster api?
node cluster.js
ab -l -n 1000 -c 10 http://localhost:8000/evil-kittens-in-your-cpu/single
# Yes!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889517/kitten-in-hand.jpg)
A simple Endurance Test
# MEASURE 10000 requests with 1mb memory leak.
node single.js
ab -l -n 10000 -c 10 \
http://localhost:8000/evil-kittens-in-your-memory/single
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889638/cpumem.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889516/albino-burmese-kitten.jpg)
# MEASURE 10000 requests with 1mb memory leak.
node single.js
ab -l -n 10000 -c 10 \
http://localhost:8000/evil-kittens-in-your-memory/single
My laptop has 16 gig of memory.
The "cloud" usually has about 0.5 -> 2 gig.
It's important to test on localhost and in a production-like environment - aka "stage".
If we use `ab` to test staging, all the requests come from localhost. We need some new tools.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889651/fc220x200baby_blue.jpg)
A simple Endurance Test
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889516/albino-burmese-kitten.jpg)
Questions?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2879957/6360030805587666061064451943_questions.jpg)
READER BEWARE:
"evil kittens" could be considered a denial-of-service attack, ESPECIALLY a shared hosting provider like Heroku.
Inappropriate use could cause others, and you, great suffering in downtime, account lock-outs, and/or legal aggression.
Production Like Performance Testing
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889712/kitten-with-monster.jpg)
Production Like Performance Testing
aka. Don't come cryin to me!
READER BEWARE:
"evil kittens" could be considered a denial-of-service attack, ESPECIALLY a shared hosting provider like Heroku.
Inappropriate use could cause others, and you, great suffering in downtime, account lock-outs, and/or legal aggression.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889712/kitten-with-monster.jpg)
A simple Load Test
(and for my final trick of the evening)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889694/kitten-with-monster.jpg)
# Deploy your app.
git push heroku
# Install beeswithmachineguns
pip install git+git://github.com/newsapps/beeswithmachineguns.git
# make a ~/.boto file with your AWS keys.
# Details: https://github.com/newsapps/beeswithmachineguns
# I did this already because it is really slow!
bees up -s 8 -g public -k evilkittens -z us-east-1e
# Wait till bees are no longer "Initailizing" in AWS console.
bees report
# Do the test
bees attack -n 10000 -c 40 \
-u https://evilkittens.herokuapp.com/tiny-kittens-in-your-memory/bees
# forget this and you will get a big bill.
bees down
Questions?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2879957/6360030805587666061064451943_questions.jpg)
Conclusion
- Performance Testing isn't magic beans.
- It's continually finding and fixing the slowest thing.
- Test locally, then globally.
- Remember: Cloud Computing = Cheap Hardware
Measure
Change
Predict
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2889485/circular-arrow-clip-art_33546.png)
In 2 weeks.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/361097/images/2888421/default.png)
- Put a button on your site
- Get paid to help people
- New paying clients
- Premium Support
- Sustainable Open Source
- Helping people should be awesome!
bigSmall.io Micro-Consulting Platform
Thanks for coming!
Scaling APIs with Performance Testing
By michaelcole
Scaling APIs with Performance Testing
- 1,336