Continuous Performance Testing using locust.io

Andrea Janes

topics

this is how we will spend our/your time:

q&a

how does it work?

why do we need load testing?

how we use it

What is load testing?

• Testing: we want to make sure something works as expected
• Load: we want to check if it is able to handle a specific load
• Typically, we set up one or more tests defining:
  • Load (1, 10, 100, ... users)
  • Context (the hardware, database, external systems)
• After running the test, we look at:
  • Response time
  • Throughput
  • CPU/Memory/Net/Disk usage

Why do you need a load testing tool?

• It is difficult to "guarantee" that software is capable to handle a certain load.
• Since all (useful) software grows over time (laws of Lehman of the '80), inevitably, its performance degrades. Therefore it "must be continually adapted or it becomes progressively less satisfactory"
  • We need to be aware of when its time to refactor
  • We need to understand where to refactor
  • We need to understand when it is enough
• We might want to compare different solutions, based on their performance
• We might want to test auto-scaling capabilities

How is load testing usually performed? (1/2)

1. Either you define a goal (e.g., "I want to be able to support 5000 users in parallel with a FID of < 100ms, as suggested by Google Core Web Vitals) or
2. You want to measure your software and identify bottlenecks.
   • ​You need something to compare with, you could at least check that your software does not degrade.
   • You use general usability metrics (0.1s/1s/10s [1])

How is load testing usually performed? (2/2)

3. You decide which software to test.
4. You decide where to install the software to test. You need some kind of realistic environment to test your software.
   1. The laptop of your developers
   2. A dedicated machine with a similar environment as your target machine 
   3. Your target machine during the weekend.
   4. An exact copy of the target environment.
5. You run one or more tests and collect the results.
6. You interpret the results to improve your software.

topics

this is how we will spend our/your time:

q&a

how does it work?

why do we need load testing?

how we use it

How does it work? (1/2)

• Installation:

  • pip3 install locust

• Run:

  • locust

How does it work? (2/2)

The locust file

topics

this is how we will spend our/your time:

q&a

how does it work?

why do we need load testing?

how we use it

Running locust manually

• How does it look like to use locust.io?
  • You prepare the steps mentioned before
  • You run

    • locust -f my_locust_file.py

  • Locust runs, you can connect to

    •  http://127.0.0.1:8089​

How we use locust.io at unibz

• We want to deploy our SUT using Docker
  • That means that for every software we have:
    • locust.py
    • docker-compose.yml
    • configuration.ini
• ​We use two machines:
  • Driver: contains locust
  • Testbed: empty, clean machine
    • Both machines are on a Docker swarm
  • We run locust just from the command line

How we use locust.io within PPTAM

• PPTAM (Production and Performance Testing Based Application Monitoring)
  • ​Open Source: https://github.com/pptam/pptam-tool
  • ​Executes locust.io experiments and saves the results in separate folders.

topics

this is how we will spend our/your time:

q&a

how does it work?

why do we need load testing?

how we use it