Load Testing Serverless Architectures
bene@theodo.co.uk
Ben Ellerby
@EllerbyBen
Using Serverless
Ben Ellerby
@EllerbyBen
http://serverless-transformation.com/
serverless-transformation
https://www.theodo.co.uk/experts/serverless
@EllerbyBen
Serverless
What is this Serverless thing?
-
Architectural movement
- “allows you to build and run applications and services without thinking about servers” — AWS
- Developers send application code which is run by the cloud provider in isolated containers abstracted from the developer.
- Use 3rd party services used to manage backend logic and state (e.g. Firebase, Cognito)
- A framework with the same name
@EllerbyBen
Why Serverless?
💰 Cost reduction
👷♂️ #NoOps... well LessOps
💻 Developers focus on delivering business value
📈 More scalable
🌳 Greener
@EllerbyBen
Not just Lambda (FaaS)
Lambda
S3
Dynamo
API Gateway
Compute
Storage
Data
API Proxy
Cognito
Auth
SQS
Queue
Step Functions
Workflows
EventBridge
Bus
Power and Flexibility to build...
@EllerbyBen
Microservices
@EllerbyBen
How does Serverless Scale Differently?
- Pay-per-use => leading to denial of wallet
- AWS Service Limits (both hard and soft)
- Outpace non-serverless components / third parties
- Cold start impacts for sudden spikes
- Combination of multiple services in a distributed system makes bottlenecks harder to spot
- Regional distribution of traffic
@EllerbyBen
@EllerbyBen
Side Note: Lambda Scaling
- When a function is invoked AWS Lambda creates an instance of the function, runs the handler and returns the response.
- The function remains active, waiting to process more events.
- If invoked again during the processing of another request Lambda initialises another instance and process the two events concurrently
@EllerbyBen
Side Note: Lambda Scaling
- Concurrency: "Number of instances that serve requests at a given time"
- When a traffic burst occurs, cumulative concurrency is limited by a "Burst concurrency quota"
- After that the concurrency continues to increase at a rate of 500 / min
- There is also an overall Concurrency limit
- See Provisioned and Reserved Concurrency
What is Load Testing?
- Different things to different people.
@EllerbyBen
Simulating different concurrent traffic levels on an application to validate its scalability.
Protocol vs Browser
-
2 Types:
- Protocol Based: Simulating at the API level
- Browser Based: Spinning up browsers and simulating interactions with browser elements to trigger realistic protocol-level requests.
- Typically, Serverless Architectures are best tested at the Protocol level as the scale of testing is usually high and browser simulated testing at this level would be expensive a slow.
- Protocol could be HTTP API requests, or more custom triggering of the AWS SDK Directly
@EllerbyBen
Components of a good load test
- Exact replica of production infrastructure
- Observability tooling
- Repeatable scenarios
- Ability to simulate high load
- Realistic user flows
- Geographic distribution
@EllerbyBen
Example Application: Gamercraft
@EllerbyBen
@EllerbyBen
The Gamercraft platform needed the ability to support a massive volume of users and accommodate traffic spikes during large-scale tournaments and low usage periods
Gamercraft
@EllerbyBen
Gamercraft
@EllerbyBen
What we want test?
- Validate our cost estimates as load increases
- Identify AWS Service Limits that need raising
- Identify AWS Service Limits that can't be raised
- Verify 3rd party and non-serverless components are protected from spikes
- Identify hidden bottlenecks
- Verify impact of regional distribution
@EllerbyBen
How do I start?
@EllerbyBen
🤷♂️
Environment to Test Against
@EllerbyBen
🌎
Isomorphic Ephemeral Load Testing Environments
@EllerbyBen
- 100% serverless architectures can be deployed to short lived environments.
- In "Serverless Flow" we spin up an environment per PR to run integration testing.
- There is 0 mocking, and the architecture is isomorphic to production
- This same approach can be taken for isolated load testing.
* Non-serverless components and 3rd parties add complexity
Metrics
@EllerbyBen
📊
Basic Metrics
@EllerbyBen
- Response times
- Error rates
- Throttles
Know what’s happening
@EllerbyBen
-
The flexibility, distribution and granularity of Serverless architectures makes logging hard.
-
Cloudwatch & XRay are the minimum.
@EllerbyBen
CloudWatch Lambda Insights
@EllerbyBen
Dedicated Observability Service
Load Testing Toolkit
@EllerbyBen
🛠
Artillery
@EllerbyBen
Artillery is a load testing and smoke testing solution for SREs, developers and QA engineers
Artillery - Test Definition
@EllerbyBen
config:
target: "https://shopping.service.staging"
phases:
- duration: 60
arrivalRate: 5
name: Warm up
- duration: 120
arrivalRate: 5
rampTo: 50
name: Ramp up load
- duration: 600
arrivalRate: 50
name: Sustained load
payload:
# Load search keywords from an external CSV file and make them available
# to virtual user scenarios as variable "keywords":
path: "keywords.csv"
fields:
- "keywords"
scenarios:
# We define one scenario:
- name: "Search and buy"
flow:
- post:
url: "/search"
body: "kw={{ keywords }}"
# The endpoint responds with JSON, which we parse and extract a field from
# to use in the next request:
capture:
json: "$.results[0].id"
as: "id"
# Get the details of the product:
- get:
url: "/product/{{ id }}/details"
# Pause for 3 seconds:
- think: 3
# Add product to cart:
- post:
url: "/cart"
json:
productId: "{{ id }}"artillery run search-and-add-to-cart.ymlBut where would we run this from?
A server... 🤮
@EllerbyBen
What if there was another way?
A service that can run code (without us having to managing servers) with support for massive parallel scale?
@EllerbyBen
Serverless-Artillery (slsart)
@EllerbyBen
Combine serverless with artillery and you get serverless-artillery for instant, cheap, and easy performance testing at scale.
Serverless-Artillery (slsart)
@EllerbyBen
Running From Different AWS Account
@EllerbyBen
- We are running our load test using AWS Services. (i.e. Lambda)
- We don't want the load-testing infra to impact limits on our infra under test
- More realistic traffic paths
Committing Experiments
@EllerbyBen
- All tests should be repeatable experiments.
- The context for the test, scenario templates and results should all be committed to the repo.
- Allows future analysis and repeating of experiments.
Conclusion
@EllerbyBen
🌎
📊
🛠
Components of a good load test
- Exact replica of production infrastructure
- Observability tooling
- Repeatable scenarios
- Ability to simulate high load
- Realistic user flows
- Geographic distribution
- Committed repeatable tests
@EllerbyBen
@EllerbyBen
serverless-transformation
Load Testing AWS Serverless Architectures Using Serverless
By Ben Ellerby
Load Testing AWS Serverless Architectures Using Serverless
Serverless architectures on AWS, involving services like AWS Lambda, DynamoDB, Cognito, Step Functions, API Gateway, bring instant scalability when built and configured in the correct way. We’ll look at how AWS Serverless architectures need to be treated differently to ensure optimal scalability and how Serverless tools (like Serverless Artillery) can be used to verify scalability. Not only will we look at achieving scalability, we’ll also look at the tools and techniques to predict and limit the cost of scaling. To bring these topics to life we’ll look at the architecture of 2 live Serverless applications built on AWS for scale and discuss how they were architected, how costs were monitored and kept in line and how serverless load testing was used to verify scalability and catch edge cases.
