Scaling APIs with Performance Testing

We skipped the beta,

deployed once,

everybody signed up,

and never had a problem.'

- No one ever

Promise:  Scaling API's With Performance Testing

Scaling API's is about:

  • Optimizing Code
  • Architecture
  • Operations
  • Sensible Features

 

Performance is about:

  • Measuring performance (testing)
  • Identifying the slow bits (profiling)
  • Optimizing what matters (optimization)

 

Promise:  Scaling API's With Performance Testing

Scaling API's is about:

  • Optimizing Code
  • Architecture
  • Operations
  • Sensible Features

 

Performance is about:

  • Measuring performance (testing)  ← This talk!
  • Identifying the slow bits (profiling)
  • Optimizing what matters (optimization)

 

Who is Michael Cole?

Fullstack JS,  DevOps, Freelancer, Digital Nomad.

bigSmall.io Micro-Consulting Platform

  • Micro-consulting platform
  • Video chat + invoices + teams
  • No job's too small if it's 100% billable
  • Sustainable Open Source
  • Helping people should be awesome!

Why Performance Testing?

Unit testing = feature confidence for changes

 

Performance testing = feature confidence for # of users

 

Performance testing came from Enterprise software (don't google "example test plan")

 

Agile Performance Testing means leaner, smaller, and iterative

What's your purpose?

  • Can I make this use case faster? In 2 weeks.
     

  • Can each server to serve 1000 requests/second, without crashing or errors? In 2 weeks.
     

  • We have a marketing event. App must serve 70,000 requests/second. In 2 weeks.

Measure

Change

Predict

In 2 weeks.

Perf Testing Menagerie

  • Load testing  
    How long does it take to process 10,000 requests?

  • Endurance/Soak testing 
    Measure resource leaks over sustained use.
    How often will the server "randomly" crash?

  • Stress testing 
    Upper performance limit before failure.
    Can it support 10,000 users simultaneously?

  • Spike testing 
    Sudden increased load and measures results.
    Slashdot effect testing

Questions?

Today!

  • Build a small web app

     
     

  • Deploy it
     
     
  • Performance test it

 

Today!

  • Build a small web app

    https://github.com/MichaelJCole/evil_kittens
     

  • Deploy it
    Locally, then Heroku
     
  • Performance test it

Evil Kittens must REST!

Text

GET /tiny-kittens-in-your-memory/:id   -> text/plain: filename 

GET /big-kittens-in-your-memory/:id    -> image/jpeg: [kitten jpg data]

GET /evil-kittens-in-your-memory/:id   -> lock 1 mb of memory for 1 sec

GET /evil-kittens-in-your-cpu/:id      -> Calculate fib(30) badly

GET /evil-kittens-in-your-disk/:id     -> writes 1 mb to disk

GET /evil-kittens-in-your-network/:id  -> sends 512k of random data

Text

https://github.com/MichaelJCole/evil_kittens
 

A server to test

# If you'd like to play along on your laptop: 
# google `github evil_kittens`

# Get the code
git clone git@github.com:MichaelJCole/evil_kittens.git
cd evil_kittens

# Install dependencies
npm install

# Start server
node single.js

# Make requests
curl http://localhost:8000/evil-kittens-in-your-cpu/hello 
wget -qO- http://localhost:8000/evil-kittens-in-your-cpu/hello 

Text

https://github.com/MichaelJCole/evil_kittens
 

 A simple Load Test

# Install ApacheBench
sudo apt-get install apache2-utils

# MEASURE:  -n 1000 requests, using -c 10 threads, -l to relax
node single.js
ab -l -n 1000 -c 10 http://localhost:8000/evil-kittens-in-your-cpu/single


# Hmmmm... How could it be faster?  Lets turn off logging
node singleNoLog.js
ab -l -n 1000 -c 10 http://localhost:8000/evil-kittens-in-your-cpu/single


# Hmmmn... What if we used the cluster api?
node cluster.js
ab -l -n 1000 -c 10 http://localhost:8000/evil-kittens-in-your-cpu/single

# Yes!

 A simple Endurance Test

# MEASURE 10000 requests with 1mb memory leak.
node single.js
ab -l -n 10000 -c 10 \
   http://localhost:8000/evil-kittens-in-your-memory/single
# MEASURE 10000 requests with 1mb memory leak.
node single.js
ab -l -n 10000 -c 10 \
   http://localhost:8000/evil-kittens-in-your-memory/single

My laptop has 16 gig of memory.

The "cloud" usually has about 0.5 -> 2 gig.

 

It's important to test on localhost and in a production-like environment - aka "stage".

 

If we use `ab` to test staging, all the requests come from localhost.  We need some new tools.

 A simple Endurance Test

Questions?

READER BEWARE:

"evil kittens" could be considered a denial-of-service attack, ESPECIALLY a shared hosting provider like Heroku.

 

Inappropriate use could cause others, and you, great suffering in downtime, account lock-outs, and/or legal aggression.

Production Like Performance Testing

Production Like Performance Testing

aka. Don't come cryin to me!

READER BEWARE:

"evil kittens" could be considered a denial-of-service attack, ESPECIALLY a shared hosting provider like Heroku.

 

Inappropriate use could cause others, and you, great suffering in downtime, account lock-outs, and/or legal aggression.

 A simple Load Test

(and for my final trick of the evening)

# Deploy your app.
git push heroku

# Install beeswithmachineguns
pip install git+git://github.com/newsapps/beeswithmachineguns.git

# make a ~/.boto file with your AWS keys.
# Details: https://github.com/newsapps/beeswithmachineguns

# I did this already because it is really slow!  
bees up -s 8 -g public -k evilkittens -z us-east-1e 

#  Wait till bees are no longer "Initailizing" in AWS console.
bees report

# Do the test
bees attack -n 10000 -c 40 \
  -u https://evilkittens.herokuapp.com/tiny-kittens-in-your-memory/bees

# forget this and you will get a big bill.
bees down

Questions?

Conclusion

  • Performance Testing isn't magic beans.  
  • It's continually finding and fixing the slowest thing.
     
  • Test locally, then globally.
  • Remember: Cloud Computing = Cheap Hardware

Measure

Change

Predict

In 2 weeks.

  • Put a button on your site
  • Get paid to help people
  • New paying clients 
  • Premium Support
  • Sustainable Open Source
  • Helping people should be awesome!

bigSmall.io Micro-Consulting Platform

Thanks for coming!

Scaling APIs with Performance Testing

By michaelcole

Scaling APIs with Performance Testing

  • 1,265