Cloud Parallelism
with Serverless

bene@theodo.co.uk
Ben Ellerby

@EllerbyBen

Ben Ellerby

@EllerbyBen

serverless-transformation

https://www.theodo.co.uk/experts/serverless

@EllerbyBen

Serverless

What is this Serverless thing?

  • Architectural movement
    • “allows you to build and run applications and services without thinking about servers” — AWS
    • Developers send application code which is run by the cloud provider in isolated containers abstracted from the developer.
    • Use 3rd party services used to manage backend logic and state (e.g. Firebase, Cognito)
  • A framework with the same name

 

@EllerbyBen

Why Serverless?

💰 Cost reduction

👷‍♂️ #NoOps... well LessOps

💻 Developers focus on delivering                   business value

📈 More scalable

🌳 Greener

@EllerbyBen

@EllerbyBen

Lambda

Runtimes

@EllerbyBen

Runtimes

@EllerbyBen

@EllerbyBen

3.8 3.7, 3.6, 2.7

A Simple Function

@EllerbyBen

def handler_name(event, context): 
    ...
    return some_value

Event:  The event body

dict, list, str, int, float, or NoneType

 

Context:  Meta Data

dict

 

 

A Simple Function

@EllerbyBen

def my_handler(event, context):
    message = 'Hello {} {}!'.format(event['first_name'], 
                                    event['last_name'])  
    return { 
        'message' : message
    }

Handler Function

@EllerbyBen

A handler function is a function of your code triggered by AWS when your lambda is invoked
 

Serverless... Framework

@EllerbyBen

Serverless... Framework

@EllerbyBen

npm install -g serverless  

serverless create 
    --template aws-python3
    --path myService

@EllerbyBen

serverless.yml

handler.py

Handler

@EllerbyBen

import json


def hello(event, context):
    body = {
        "message": "Go Serverless v1.0! Your function executed successfully!",
        "input": event
    }

    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }

    return response

serverless.yml

@EllerbyBen


service: myService

provider:
  name: aws
  runtime: python3.7

functions:
  hello:
    handler: handler.hello

local

@EllerbyBen

➜  sls invoke local -f hello

{
    "statusCode": 200,
    "body": "{\"message\": \"Go Serverless v1.0! Your function executed successfully!\", \"input\": {}}"
}

deploy

@EllerbyBen

➜  sls deploy

➜  sls invoke -f hello

{
    "statusCode": 200,
    "body": "{\"message\": \"Go Serverless v1.0! Your function executed successfully!\", \"input\": {}}"
}

Lambda Triggers

@EllerbyBen

Not just Lambda (FaaS)

Lambda

S3

Dynamo

API Gateway

Compute

Storage

Data

API Proxy

Cognito

Auth

SQS

Queue

Step Functions

Workflows

EventBridge

Bus

Microservices

@EllerbyBen

@EllerbyBen

https://medium.com/serverless-transformation/what-a-typical-100-serverless-architecture-looks-like-in-aws-40f252cd0ecb

@EllerbyBen

@EllerbyBen

Lambda Characteristics 

Memory 128MB - 10,240 MB
Timeout 900 seconds
Burst Concurrency 500-3000
/tmp dir storage 512 MB
Concurrent Executions 1,000 ⏫

@EllerbyBen

What if my task takes longer than 900 seconds?

@EllerbyBen

Example: PDF Generation

  • Reports written as web application using React and JS Graphing library.
  • Users have ability to export as PDF for compliance reasons.
  • Reports can be extremely long 

@EllerbyBen

Basic Architecture

@EllerbyBen

Side Note: Lambda Scaling

  • When a function is invoked AWS Lambda creates an instance of the function, runs the handler and returns the response.
  • The function remains active, waiting to process more events.
  • If invoked again during the processing of another request Lambda initialises another instance and process the two events concurrently

@EllerbyBen

Side Note: Lambda Scaling

  • Concurrency: "Number of instances that serve requests at a given time"
  • When a traffic burst occurs, cumulative concurrency is limited by a "Burst concurrency quota"
  • After that the concurrency continues to increase at a rate of 500 / min
  • There is also an overall Concurrency limit
  • See Provisioned and Reserved Concurrency

@EllerbyBen

Timing Breakdown 

We can use parallel Lambda invocations to break down the large PDF into smaller tasks

@EllerbyBen

Example: PDF Generation

@EllerbyBen

Step Functions

@EllerbyBen

Step Functions

 AWS Step Functions is a serverless function orchestrator that makes it easy to sequence AWS Lambda functions and multiple AWS services into business-critical applications.
 
 

@EllerbyBen

Step Functions

@EllerbyBen

CloudFormation

AWSTemplateFormatVersion: '2010-09-09'
Description: An example template for a Step Functions state machine.
Resources:
  MyStateMachine:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      StateMachineName: HelloWorld-StateMachine
      DefinitionString: |-
        {
          "StartAt": "HelloWorld",
          "States": {
            "HelloWorld": {
              "Type": "Task",
              "Resource": "ARN of Function.....",
              "End": true
            }
          }
        }
      RoleArn: arn-of-role
      Tags:
        -
          Key: "keyname1"
          Value: "value1"
        -
          Key: "keyname2"
          Value: "value2"

@EllerbyBen

serverless-step-functions

@EllerbyBen

serverless-step-functions

stepFunctions:
  stateMachines:
    MyStateMachine:
      name: MyMachine
      definition:
        Comment: "Does something cool"
        States:
          HelloWorld:
            Type: Task
            Resource: "HelloLambdaARN"
            End: true

@EllerbyBen

Use Cases

Function Orchestration

Branching

Error Handling

Human Approval

Parallel Processing

Dynamic Parallelism

@EllerbyBen

Map State

  • Runs a set of steps for each element of an input array.
  • Iterator: Object defining a state machine that will process each element of the array.
  • Item Path: Location of the array in the input
  • Max Concurrency: Upper bound on how many invocations of the iterator can run in parallel.

@EllerbyBen

@EllerbyBen

Another Example: Load Testing

Example Application: Gamercraft

@EllerbyBen

Gamercraft

@EllerbyBen

Gamercraft

@EllerbyBen

Artillery

@EllerbyBen

Artillery is a load testing and smoke testing solution for SREs, developers and QA engineers

Artillery - Test Definition

@EllerbyBen

config:
  target: "https://shopping.service.staging"
  phases:
    - duration: 60
      arrivalRate: 5
      name: Warm up
    - duration: 120
      arrivalRate: 5
      rampTo: 50
      name: Ramp up load
    - duration: 600
      arrivalRate: 50
      name: Sustained load
  payload:
    # Load search keywords from an external CSV file and make them available
    # to virtual user scenarios as variable "keywords":
    path: "keywords.csv"
    fields:
      - "keywords"
scenarios:
  # We define one scenario:
  - name: "Search and buy"
    flow:
      - post:
          url: "/search"
          body: "kw={{ keywords }}"
          # The endpoint responds with JSON, which we parse and extract a field from
          # to use in the next request:
          capture:
            json: "$.results[0].id"
            as: "id"
      # Get the details of the product:
      - get:
          url: "/product/{{ id }}/details"
      # Pause for 3 seconds:
      - think: 3
      # Add product to cart:
      - post:
          url: "/cart"
          json:
            productId: "{{ id }}"
artillery run search-and-add-to-cart.yml

But where would we run this from?

A server... 🤮

@EllerbyBen

What if there was another way?
 

A service that can run code (without us having to manage servers) with support for massive parallel scale?

@EllerbyBen

Serverless-Artillery (slsart)

@EllerbyBen

Combine serverless with artillery and you get serverless-artillery for instant, cheap, and easy performance testing at scale.

 

Serverless-Artillery (slsart)

@EllerbyBen

Further Real World Examples

@EllerbyBen

UAE Mars Mission

@EllerbyBen

"The mission Science Data Center will use AWS Step Function to orchestrate various AWS Lambda functions to check, index, and move files into a master database, and Amazon Simple Storage Service (Amazon S3) for shared storage."

https://aws.amazon.com/blogs/publicsector/uae-mars-mission-uses-aws-advance-scientific-discoveries/

NeuronBridge

@EllerbyBen

  • EM-LM Image search
  • Step Function checks DynamoDB once per second.
  • When ready triggers combination and sort.
  • Scaled to 3,000 concurrent Lambdas
https://aws.amazon.com/blogs/architecture/scaling-neuroscience-research-on-aws/

Conclusion

@EllerbyBen

  • Serverless is an architectural movement allowing us to abstract complexities in building and running applications.
  • Serverless in not just Lambda
  • Serverless services like AWS Lambda have very elastic scaling - with the ability to have high concurrency
  • Large problems can be broken down into chunks orchestrated by services like AWS Step Functions

https://slides.com/benellerby

@EllerbyBen

serverless-transformation

https://www.theodo.co.uk/experts/serverless

DRAFT WIP - Cloud Parallelism With Serverless On AWS - PythonWebConf

By Ben Ellerby

Made with Slides.com

DRAFT WIP - Cloud Parallelism With Serverless On AWS - PythonWebConf

Serverless offers scalability and speed with a nice abstraction layer to many web and mobile applications, as well as massive concurrency which can be run large workloads in parallel. This talk will discuss what is possible with Serverless for parallel algorithms and services to use. We will cover: - What Serverless is (AWS Lambda and beyond) - The Serverless Services on AWS and other cloud providers - How to write code for Serverless - How parallelism works on Serverless - How to orchestrate parallel workloads with Serverless. Services like AWS Step Functions

  • 1,014

More from Ben Ellerby

  • Minimum Viable Migrations (MVM), A Path to Cloud Modernization (Python Web Conf)
    Ben Ellerby
    756
  • DRAFT: Building Event-Driven Serverless Architectures on AWS (Melbourne AWS Meetup 2021)
    Ben Ellerby
    1085