Cloud Parallelism
with Serverless
bene@theodo.co.uk
Ben Ellerby
@EllerbyBen
Ben Ellerby
@EllerbyBen
serverless-transformation
@EllerbyBen
Serverless
What is this Serverless thing?
-
Architectural movement
- “allows you to build and run applications and services without thinking about servers” — AWS
- Developers send application code which is run by the cloud provider in isolated containers abstracted from the developer.
- Use 3rd party services used to manage backend logic and state (e.g. Firebase, Cognito)
- A framework with the same name
@EllerbyBen
Why Serverless?
💰 Cost reduction
👷♂️ #NoOps... well LessOps
💻 Developers focus on delivering business value
📈 More scalable
🌳 Greener
@EllerbyBen
@EllerbyBen
Lambda
Runtimes
@EllerbyBen
Runtimes
@EllerbyBen
@EllerbyBen
3.8 3.7, 3.6, 2.7
A Simple Function
@EllerbyBen
def handler_name(event, context):
...
return some_valueEvent: The event body
dict, list, str, int, float, or NoneType
Context: Meta Data
dict
A Simple Function
@EllerbyBen
def my_handler(event, context):
message = 'Hello {} {}!'.format(event['first_name'],
event['last_name'])
return {
'message' : message
} Handler Function
@EllerbyBen
A handler function is a function of your code triggered by AWS when your lambda is invoked
Serverless... Framework
@EllerbyBen
Serverless... Framework
@EllerbyBen
npm install -g serverless
serverless create
--template aws-python3
--path myService@EllerbyBen
serverless.yml
handler.py
Handler
@EllerbyBen
import json
def hello(event, context):
body = {
"message": "Go Serverless v1.0! Your function executed successfully!",
"input": event
}
response = {
"statusCode": 200,
"body": json.dumps(body)
}
return response
serverless.yml
@EllerbyBen
service: myService
provider:
name: aws
runtime: python3.7
functions:
hello:
handler: handler.hello
local
@EllerbyBen
➜ sls invoke local -f hello
{
"statusCode": 200,
"body": "{\"message\": \"Go Serverless v1.0! Your function executed successfully!\", \"input\": {}}"
}
deploy
@EllerbyBen
➜ sls deploy
➜ sls invoke -f hello
{
"statusCode": 200,
"body": "{\"message\": \"Go Serverless v1.0! Your function executed successfully!\", \"input\": {}}"
}Lambda Triggers
@EllerbyBen
Not just Lambda (FaaS)
Lambda
S3
Dynamo
API Gateway
Compute
Storage
Data
API Proxy
Cognito
Auth
SQS
Queue
Step Functions
Workflows
EventBridge
Bus
Microservices
@EllerbyBen
@EllerbyBen
@EllerbyBen
@EllerbyBen
Lambda Characteristics
| Memory | 128MB - 10,240 MB |
| Timeout | 900 seconds |
| Burst Concurrency | 500-3000 |
| /tmp dir storage | 512 MB |
| Concurrent Executions | 1,000 ⏫ |
@EllerbyBen
What if my task takes longer than 900 seconds?
@EllerbyBen
Example: PDF Generation
- Reports written as web application using React and JS Graphing library.
- Users have ability to export as PDF for compliance reasons.
- Reports can be extremely long
@EllerbyBen
Basic Architecture
@EllerbyBen
Side Note: Lambda Scaling
- When a function is invoked AWS Lambda creates an instance of the function, runs the handler and returns the response.
- The function remains active, waiting to process more events.
- If invoked again during the processing of another request Lambda initialises another instance and process the two events concurrently
@EllerbyBen
Side Note: Lambda Scaling
- Concurrency: "Number of instances that serve requests at a given time"
- When a traffic burst occurs, cumulative concurrency is limited by a "Burst concurrency quota"
- After that the concurrency continues to increase at a rate of 500 / min
- There is also an overall Concurrency limit
- See Provisioned and Reserved Concurrency
@EllerbyBen
Timing Breakdown
We can use parallel Lambda invocations to break down the large PDF into smaller tasks
@EllerbyBen
Example: PDF Generation
@EllerbyBen
Step Functions
@EllerbyBen
Step Functions
AWS Step Functions is a serverless function orchestrator that makes it easy to sequence AWS Lambda functions and multiple AWS services into business-critical applications.
@EllerbyBen
Step Functions
@EllerbyBen
CloudFormation
AWSTemplateFormatVersion: '2010-09-09'
Description: An example template for a Step Functions state machine.
Resources:
MyStateMachine:
Type: AWS::StepFunctions::StateMachine
Properties:
StateMachineName: HelloWorld-StateMachine
DefinitionString: |-
{
"StartAt": "HelloWorld",
"States": {
"HelloWorld": {
"Type": "Task",
"Resource": "ARN of Function.....",
"End": true
}
}
}
RoleArn: arn-of-role
Tags:
-
Key: "keyname1"
Value: "value1"
-
Key: "keyname2"
Value: "value2"@EllerbyBen
serverless-step-functions
@EllerbyBen
serverless-step-functions
stepFunctions:
stateMachines:
MyStateMachine:
name: MyMachine
definition:
Comment: "Does something cool"
States:
HelloWorld:
Type: Task
Resource: "HelloLambdaARN"
End: true@EllerbyBen
Use Cases
Function Orchestration
Branching
Error Handling
Human Approval
Parallel Processing
Dynamic Parallelism
@EllerbyBen
Map State
- Runs a set of steps for each element of an input array.
- Iterator: Object defining a state machine that will process each element of the array.
- Item Path: Location of the array in the input
- Max Concurrency: Upper bound on how many invocations of the iterator can run in parallel.
@EllerbyBen
@EllerbyBen
Another Example: Load Testing
Example Application: Gamercraft
@EllerbyBen
Gamercraft
@EllerbyBen
Gamercraft
@EllerbyBen
Artillery
@EllerbyBen
Artillery is a load testing and smoke testing solution for SREs, developers and QA engineers
Artillery - Test Definition
@EllerbyBen
config:
target: "https://shopping.service.staging"
phases:
- duration: 60
arrivalRate: 5
name: Warm up
- duration: 120
arrivalRate: 5
rampTo: 50
name: Ramp up load
- duration: 600
arrivalRate: 50
name: Sustained load
payload:
# Load search keywords from an external CSV file and make them available
# to virtual user scenarios as variable "keywords":
path: "keywords.csv"
fields:
- "keywords"
scenarios:
# We define one scenario:
- name: "Search and buy"
flow:
- post:
url: "/search"
body: "kw={{ keywords }}"
# The endpoint responds with JSON, which we parse and extract a field from
# to use in the next request:
capture:
json: "$.results[0].id"
as: "id"
# Get the details of the product:
- get:
url: "/product/{{ id }}/details"
# Pause for 3 seconds:
- think: 3
# Add product to cart:
- post:
url: "/cart"
json:
productId: "{{ id }}"artillery run search-and-add-to-cart.ymlBut where would we run this from?
A server... 🤮
@EllerbyBen
What if there was another way?
A service that can run code (without us having to manage servers) with support for massive parallel scale?
@EllerbyBen
Serverless-Artillery (slsart)
@EllerbyBen
Combine serverless with artillery and you get serverless-artillery for instant, cheap, and easy performance testing at scale.
Serverless-Artillery (slsart)
@EllerbyBen
Further Real World Examples
@EllerbyBen
UAE Mars Mission
@EllerbyBen
"The mission Science Data Center will use AWS Step Function to orchestrate various AWS Lambda functions to check, index, and move files into a master database, and Amazon Simple Storage Service (Amazon S3) for shared storage."
https://aws.amazon.com/blogs/publicsector/uae-mars-mission-uses-aws-advance-scientific-discoveries/
NeuronBridge
@EllerbyBen
- EM-LM Image search
- Step Function checks DynamoDB once per second.
- When ready triggers combination and sort.
- Scaled to 3,000 concurrent Lambdas
https://aws.amazon.com/blogs/architecture/scaling-neuroscience-research-on-aws/
Conclusion
@EllerbyBen
- Serverless is an architectural movement allowing us to abstract complexities in building and running applications.
- Serverless in not just Lambda
- Serverless services like AWS Lambda have very elastic scaling - with the ability to have high concurrency
- Large problems can be broken down into chunks orchestrated by services like AWS Step Functions
@EllerbyBen
serverless-transformation