Load Testing Serverless Architectures

bene@theodo.co.uk
Ben Ellerby

@EllerbyBen

Using Serverless

Ben Ellerby

@EllerbyBen

http://serverless-transformation.com/

serverless-transformation

https://www.theodo.co.uk/experts/serverless

@EllerbyBen

Serverless

What is this Serverless thing?

  • Architectural movement
    • “allows you to build and run applications and services without thinking about servers” — AWS
    • Developers send application code which is run by the cloud provider in isolated containers abstracted from the developer.
    • Use 3rd party services used to manage backend logic and state (e.g. Firebase, Cognito)
  • A framework with the same name

 

@EllerbyBen

Why Serverless?

💰 Cost reduction

👷‍♂️ #NoOps... well LessOps

💻 Developers focus on delivering                   business value

📈 More scalable

🌳 Greener

@EllerbyBen

Not just Lambda (FaaS)

Lambda

S3

Dynamo

API Gateway

Compute

Storage

Data

API Proxy

Cognito

Auth

SQS

Queue

Step Functions

Workflows

EventBridge

Bus

Power and Flexibility to build...

@EllerbyBen

Microservices

@EllerbyBen

How does Serverless Scale Differently?

  • Pay-per-use => leading to denial of wallet
  • AWS Service Limits (both hard and soft)
  • Outpace non-serverless components / third parties
  • Cold start impacts for sudden spikes
  • Combination of multiple services in a distributed system makes bottlenecks harder to spot 
  • Regional distribution of traffic

@EllerbyBen

@EllerbyBen

Side Note: Lambda Scaling

  • When a function is invoked AWS Lambda creates an instance of the function, runs the handler and returns the response.
  • The function remains active, waiting to process more events.
  • If invoked again during the processing of another request Lambda initialises another instance and process the two events concurrently

@EllerbyBen

Side Note: Lambda Scaling

  • Concurrency: "Number of instances that serve requests at a given time"
  • When a traffic burst occurs, cumulative concurrency is limited by a "Burst concurrency quota"
  • After that the concurrency continues to increase at a rate of 500 / min
  • There is also an overall Concurrency limit
  • See Provisioned and Reserved Concurrency

What is Load Testing?

  • Different things to different people.

@EllerbyBen

Simulating different concurrent traffic levels on an application to validate its scalability.

 

Protocol vs Browser

  • 2 Types:
    • Protocol Based: Simulating at the API level
    • Browser Based: Spinning up browsers and simulating interactions with browser elements to trigger realistic protocol-level requests.
  • Typically, Serverless Architectures are best tested at the Protocol level as the scale of testing is usually high and browser simulated testing at this level would be expensive a slow.
  • Protocol could be HTTP API requests, or more custom triggering of the AWS SDK Directly

@EllerbyBen

Components of a good load test

  • Exact replica of production infrastructure
  • Observability tooling
  • Repeatable scenarios
  • Ability to simulate high load
  • Realistic user flows
  • Geographic distribution

@EllerbyBen

Example Application: Gamercraft

@EllerbyBen

@EllerbyBen

The Gamercraft platform needed the ability to support a massive volume of users and accommodate traffic spikes during large-scale tournaments and low usage periods

Gamercraft

@EllerbyBen

Gamercraft

@EllerbyBen

What we want test?

  • Validate our cost estimates as load increases
  • Identify AWS Service Limits that need raising
  • Identify AWS Service Limits that can't be raised
  • Verify 3rd party and non-serverless components are protected from spikes
  • Identify hidden bottlenecks
  • Verify impact of regional distribution

@EllerbyBen

How do I start?

@EllerbyBen

🤷‍♂️

Environment to Test Against

@EllerbyBen

🌎

Isomorphic Ephemeral Load Testing Environments 

@EllerbyBen

  • 100% serverless architectures can be deployed to short lived environments.
  • In "Serverless Flow" we spin up an environment per PR to run integration testing.
  • There is 0 mocking, and the architecture is isomorphic to production
  • This same approach can be taken for isolated load testing.

* Non-serverless components and 3rd parties add complexity

Metrics

@EllerbyBen

📊

Basic Metrics

@EllerbyBen

  • Response times
  • Error rates
  • Throttles

Know what’s happening

@EllerbyBen

  • The flexibility, distribution and granularity of Serverless architectures makes logging hard.

  • Cloudwatch & XRay are the minimum.

@EllerbyBen

CloudWatch Lambda Insights

@EllerbyBen

Dedicated Observability Service

Load Testing Toolkit

@EllerbyBen

🛠

Artillery

@EllerbyBen

Artillery is a load testing and smoke testing solution for SREs, developers and QA engineers

Artillery - Test Definition

@EllerbyBen

config:
  target: "https://shopping.service.staging"
  phases:
    - duration: 60
      arrivalRate: 5
      name: Warm up
    - duration: 120
      arrivalRate: 5
      rampTo: 50
      name: Ramp up load
    - duration: 600
      arrivalRate: 50
      name: Sustained load
  payload:
    # Load search keywords from an external CSV file and make them available
    # to virtual user scenarios as variable "keywords":
    path: "keywords.csv"
    fields:
      - "keywords"
scenarios:
  # We define one scenario:
  - name: "Search and buy"
    flow:
      - post:
          url: "/search"
          body: "kw={{ keywords }}"
          # The endpoint responds with JSON, which we parse and extract a field from
          # to use in the next request:
          capture:
            json: "$.results[0].id"
            as: "id"
      # Get the details of the product:
      - get:
          url: "/product/{{ id }}/details"
      # Pause for 3 seconds:
      - think: 3
      # Add product to cart:
      - post:
          url: "/cart"
          json:
            productId: "{{ id }}"
artillery run search-and-add-to-cart.yml

But where would we run this from?

A server... 🤮

@EllerbyBen

What if there was another way?
 

A service that can run code (without us having to managing servers) with support for massive parallel scale?

@EllerbyBen

Serverless-Artillery (slsart)

@EllerbyBen

Combine serverless with artillery and you get serverless-artillery for instant, cheap, and easy performance testing at scale.

 

Serverless-Artillery (slsart)

@EllerbyBen

Running From Different AWS Account

@EllerbyBen

  • We are running our load test using AWS Services. (i.e. Lambda)
  • We don't want the load-testing infra to impact limits on our infra under test
  • More realistic traffic paths

Committing Experiments

@EllerbyBen

  • All tests should be repeatable experiments.
  • The context for the test, scenario templates and results should all be committed to the repo.
  • Allows future analysis and repeating of experiments.

Conclusion

@EllerbyBen

🌎

📊

🛠

Components of a good load test

  • Exact replica of production infrastructure
  • Observability tooling
  • Repeatable scenarios
  • Ability to simulate high load
  • Realistic user flows
  • Geographic distribution
  • ​Committed repeatable tests

@EllerbyBen

https://slides.com/benellerby

@EllerbyBen

serverless-transformation

https://www.theodo.co.uk/experts/serverless

Load Testing AWS Serverless Architectures Using Serverless

By Ben Ellerby

Made with Slides.com

Load Testing AWS Serverless Architectures Using Serverless

Serverless architectures on AWS, involving services like AWS Lambda, DynamoDB, Cognito, Step Functions, API Gateway, bring instant scalability when built and configured in the correct way. We’ll look at how AWS Serverless architectures need to be treated differently to ensure optimal scalability and how Serverless tools (like Serverless Artillery) can be used to verify scalability. Not only will we look at achieving scalability, we’ll also look at the tools and techniques to predict and limit the cost of scaling. To bring these topics to life we’ll look at the architecture of 2 live Serverless applications built on AWS for scale and discuss how they were architected, how costs were monitored and kept in line and how serverless load testing was used to verify scalability and catch edge cases.

  • 1,358

More from Ben Ellerby

  • Minimum Viable Migrations (MVM), A Path to Cloud Modernization (Python Web Conf)
    Ben Ellerby
    653
  • DRAFT: Building Event-Driven Serverless Architectures on AWS (Melbourne AWS Meetup 2021)
    Ben Ellerby
    966