Load Testing Serverless Architectures

bene@theodo.co.uk
Ben Ellerby

@EllerbyBen

Using Serverless

Ben Ellerby

@EllerbyBen

http://serverless-transformation.com/

serverless-transformation

https://www.theodo.co.uk/experts/serverless

@EllerbyBen

Serverless

What is this Serverless thing?

  • Architectural movement
    • “allows you to build and run applications and services without thinking about servers” — AWS
    • Developers send application code which is run by the cloud provider in isolated containers abstracted from the developer.
    • Use 3rd party services used to manage backend logic and state (e.g. Firebase, Cognito)
  • A framework with the same name

 

@EllerbyBen

Why Serverless?

💰 Cost reduction

👷‍♂️ #NoOps... well LessOps

💻 Developers focus on delivering                   business value

📈 More scalable

🌳 Greener

@EllerbyBen

Not just Lambda (FaaS)

Lambda

S3

Dynamo

API Gateway

Compute

Storage

Data

API Proxy

Cognito

Auth

SQS

Queue

Step Functions

Workflows

EventBridge

Bus

Power and Flexibility to build...

@EllerbyBen

Microservices

@EllerbyBen

How does Serverless Scale Differently?

  • Pay-per-use => leading to denial of wallet
  • AWS Service Limits (both hard and soft)
  • Outpace non-serverless components / third parties
  • Cold start impacts for sudden spikes
  • Combination of multiple services in a distributed system makes bottlenecks harder to spot 
  • Regional distribution of traffic

@EllerbyBen

@EllerbyBen

Side Note: Lambda Scaling

  • When a function is invoked AWS Lambda creates an instance of the function, runs the handler and returns the response.
  • The function remains active, waiting to process more events.
  • If invoked again during the processing of another request Lambda initialises another instance and process the two events concurrently

@EllerbyBen

Side Note: Lambda Scaling

  • Concurrency: "Number of instances that serve requests at a given time"
  • When a traffic burst occurs, cumulative concurrency is limited by a "Burst concurrency quota"
  • After that the concurrency continues to increase at a rate of 500 / min
  • There is also an overall Concurrency limit
  • See Provisioned and Reserved Concurrency

What is Load Testing?

  • Different things to different people.

@EllerbyBen

Simulating different concurrent traffic levels on an application to validate its scalability.

 

Protocol vs Browser

  • 2 Types:
    • Protocol Based: Simulating at the API level
    • Browser Based: Spinning up browsers and simulating interactions with browser elements to trigger realistic protocol-level requests.
  • Typically, Serverless Architectures are best tested at the Protocol level as the scale of testing is usually high and browser simulated testing at this level would be expensive a slow.
  • Protocol could be HTTP API requests, or more custom triggering of the AWS SDK Directly

@EllerbyBen

Components of a good load test

  • Exact replica of production infrastructure
  • Observability tooling
  • Repeatable scenarios
  • Ability to simulate high load
  • Realistic user flows
  • Geographic distribution

@EllerbyBen

Example Application: Gamercraft

@EllerbyBen

@EllerbyBen

The Gamercraft platform needed the ability to support a massive volume of users and accommodate traffic spikes during large-scale tournaments and low usage periods

Gamercraft

@EllerbyBen

Gamercraft

@EllerbyBen

What we want test?

  • Validate our cost estimates as load increases
  • Identify AWS Service Limits that need raising
  • Identify AWS Service Limits that can't be raised
  • Verify 3rd party and non-serverless components are protected from spikes
  • Identify hidden bottlenecks
  • Verify impact of regional distribution

@EllerbyBen

How do I start?

@EllerbyBen

🤷‍♂️

Environment to Test Against

@EllerbyBen

🌎

Isomorphic Ephemeral Load Testing Environments 

@EllerbyBen

  • 100% serverless architectures can be deployed to short lived environments.
  • In "Serverless Flow" we spin up an environment per PR to run integration testing.
  • There is 0 mocking, and the architecture is isomorphic to production
  • This same approach can be taken for isolated load testing.

* Non-serverless components and 3rd parties add complexity

Metrics

@EllerbyBen

📊

Basic Metrics

@EllerbyBen

  • Response times
  • Error rates
  • Throttles

Know what’s happening

@EllerbyBen

  • The flexibility, distribution and granularity of Serverless architectures makes logging hard.

  • Cloudwatch & XRay are the minimum.

@EllerbyBen

CloudWatch Lambda Insights

@EllerbyBen

Dedicated Observability Service

Load Testing Toolkit

@EllerbyBen

🛠

Artillery

@EllerbyBen

Artillery is a load testing and smoke testing solution for SREs, developers and QA engineers

Artillery - Test Definition

@EllerbyBen

config:
  target: "https://shopping.service.staging"
  phases:
    - duration: 60
      arrivalRate: 5
      name: Warm up
    - duration: 120
      arrivalRate: 5
      rampTo: 50
      name: Ramp up load
    - duration: 600
      arrivalRate: 50
      name: Sustained load
  payload:
    # Load search keywords from an external CSV file and make them available
    # to virtual user scenarios as variable "keywords":
    path: "keywords.csv"
    fields:
      - "keywords"
scenarios:
  # We define one scenario:
  - name: "Search and buy"
    flow:
      - post:
          url: "/search"
          body: "kw={{ keywords }}"
          # The endpoint responds with JSON, which we parse and extract a field from
          # to use in the next request:
          capture:
            json: "$.results[0].id"
            as: "id"
      # Get the details of the product:
      - get:
          url: "/product/{{ id }}/details"
      # Pause for 3 seconds:
      - think: 3
      # Add product to cart:
      - post:
          url: "/cart"
          json:
            productId: "{{ id }}"
artillery run search-and-add-to-cart.yml

But where would we run this from?

A server... 🤮

@EllerbyBen

What if there was another way?
 

A service that can run code (without us having to managing servers) with support for massive parallel scale?

@EllerbyBen

Serverless-Artillery (slsart)

@EllerbyBen

Combine serverless with artillery and you get serverless-artillery for instant, cheap, and easy performance testing at scale.

 

Serverless-Artillery (slsart)

@EllerbyBen

Running From Different AWS Account

@EllerbyBen

  • We are running our load test using AWS Services. (i.e. Lambda)
  • We don't want the load-testing infra to impact limits on our infra under test
  • More realistic traffic paths

Committing Experiments

@EllerbyBen

  • All tests should be repeatable experiments.
  • The context for the test, scenario templates and results should all be committed to the repo.
  • Allows future analysis and repeating of experiments.

Conclusion

@EllerbyBen

🌎

📊

🛠

Components of a good load test

  • Exact replica of production infrastructure
  • Observability tooling
  • Repeatable scenarios
  • Ability to simulate high load
  • Realistic user flows
  • Geographic distribution
  • ​Committed repeatable tests

@EllerbyBen

https://slides.com/benellerby

@EllerbyBen

serverless-transformation

https://www.theodo.co.uk/experts/serverless

Made with Slides.com