Introdução a Serverless para Pythonistas

Lucas Costa

TRILHA PYTHON

Serverless?

Serverless Architecture Principles

Use a compute service to execute code on demand (FaaS)
Write single-purpose stateless functions
Design push-based, event-driven pipelines
Create thicker, more powerful front-ends
Embrace third-party services

Peter Sbarski

“Serverless Architectures on AWS: With examples using AWS Lambda”

FaaS - Function as a Service

On Premises

IaaS

FaaS

PaaS

AWS EC2/ECS, Azure VMs, Google Compute Engine, OpenStack, VMware

Heroku, OpenShift, GC App Engine, AWS Beanstalk, Azure App Service

AWS Lambda, Azure Functions, Webtask
Google Cloud Functions, IBM OpenWhisk

Datacenter

Event-driven Pipelines

Powerful Front-ends / Third-party Services

"Serverless" by Badri Janakiraman on martinfowler.com

Serverless Architectures

Compute as Back-end

Serverless Architectures

Compute as Back-end - A Cloud Guru

Serverless Architectures

Legacy API Wrapper

Serverless Architectures

Hybrid

Serverless Architectures

GraphQL

Serverless Architectures

Compute as Glue

Serverless Architectures

Real-time Processing

Cost
Scalability
Productivity
Experimentation
Time to Market

Limits
Debbuging/Testing
Vendor Issues

Serverless Python

Main Providers

*

AWS Lambda

Azure
Functions

Cloud
Functions

OpenWhisk

( JS only )

( experimental )

Serverless Python

Weapon of Choice

AWS Lambda

Serverless Python

AWS Lambda Pricing

1 million requests per month are free

$0.20 per 1 million requests thereafter ($0.0000002 per request)

400,000 GB-seconds of compute time per month free

Serverless Python

AWS Lambda Programming Model

import json

def hello(event, context):
    body = {
        "message": "Your function executed successfully!",
        "input": event
    }

    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }

    return response

handler.py

Serverless Python

AWS Lambda Context

import time
def get_my_log_stream(event, context):       
    print("Log stream name:", context.log_stream_name)
    print("Log group name:",  context.log_group_name)
    print("Request ID:",context.aws_request_id)
    print("Mem. limits(MB):", context.memory_limit_in_mb)
    time.sleep(1) 
    print("Time remaining (MS):", context.get_remaining_time_in_millis())

Serverless Python

AWS Lambda Logging

import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def my_logging_handler(event, context):
    logger.info('got event{}'.format(event))
    logger.error('something went wrong')
    return 'Hello World!'

CloudWatch

Serverless Python

AWS Lambda Function Errors

def always_failed_handler(event, context):
    raise Exception('I failed!')

{
  "errorMessage": "I failed!",
  "stackTrace": [
    [
      "/var/task/lambda_function.py",
      3,
      "my_always_fails_handler",
      "raise Exception('I failed!')"
    ]
  ],
  "errorType": "Exception"
}

Dead Letter Queue (SQS/SNS)
State Machine (custom error handling)

Handling

Serverless Python

AWS Step Functions

Serverless Python

AWS Step Functions

Serverless Python

Chalice

Frameworks

Serverless Python

The Framework

Open Source
Language Agnostic
Multi-Platform
$3M Funding

CLI
Beyond FaaS
Pluggable
Extensive docs
Great community

Service

Organization unit, project file (multiple services for a single application).

Function

Independent unit of deployment, code deployed in the cloud.

Events

Anything that triggers a Function (HTTP request, timers, uploads, streams, etc.)

Resources

Infrastructure components (DB Table, Storage, etc.)

Concepts

Stages

Deployment environments (dev, staging, production, etc.)

Getting Started

> pip install --upgrade --user awscli
> aws configure

Install and configure AWS CLI

> npm install -g serverless

Install Serverless

> serverless create --template aws-python3

Create Python service

> serverless deploy

Deploy! 😎

serverless.yml

service: service-name

provider:
  name: aws
  region: us-east-1
  runtime: python3.6
  timeout: 300
  memorySize: 128
  environment:
    SERVICE_NAME: ${self:service}
    STAGE: ${opt:stage}
    MY_ENV_VAR: ${self:custom.env.myEnvVar}

custom:
  env: ${file(env.yml)}

package:
  individually: true

functions:
  hello:
    handler: handler.hello
  events:
    - http:
      path: /
      method: get
  package:
    include:
      - handler.py
      - include_me_dir/**
    exclude:
      - include_me_dir/not_this_one.py

serverless.yml ➭ CloudFormation template

WSGI Plugin

from flask import Flask
app = Flask(__name__)


@app.route("/cats")
def cats():
    return "Cats"


@app.route("/dogs/<id>")
def dog(id):
    return "Dog"

service: flask-example

provider:
  name: aws
  runtime: python3.6

plugins:
  - serverless-wsgi

functions:
  api:
    handler: wsgi.handler
    events:
      - http: ANY {proxy+}

custom:
  wsgi:
    app: api.app

Flask==0.11.1

api.py

serverless.yml

requirements.txt

github.com/logandk/serverless-wsgi

Serverless Python

Dependencies

project
├── package1
|   ├── .requirements
|   ├── handler1.py
|   └── requirements.txt
├── package2
|   ├── .requirements
|   ├── handler2.py
|   └── requirements.txt
├── utils
|   ├── .requirements
|   ├── helper.py
|   └── requirements.txt
└── serverless.yml

import json
import os
import sys

import pkg1dependency
from utils import helper

# Add requirements to PATH
file_path = os.path.realpath(__file__)
file_dir = os.path.dirname(file_path)
requirements_path = os.path.join(file_dir, '.requirements')
sys.path.append(requirements_path)

def handler1(event, context):
    data = helper.function(event)
    response = pkg1dependency(data)
    return response

handler1.py

Serverless Python

Dependencies

project
├── package1
|   ├── .requirements
|   ├── __init__.py
|   ├── handler1.py
|   └── requirements.txt
├── package2
|   ├── .requirements
|   ├── __init__.py
|   ├── handler2.py
|   └── requirements.txt
├── utils
|   ├── .requirements
|   ├── __init__.py
|   ├── helper.py
|   └── requirements.txt
└── serverless.yml

import json

import package1  # noqa
import pkg1dependency
from utils import helper

def handler1(event, context):
    data = helper.function(event)
    response = pkg1dependency(data)
    return response

import os
import sys

# Add requirements to PATH
file_path = os.path.realpath(__file__)
file_dir = os.path.dirname(file_path)
requirements_path = os.path.join(file_dir, '.requirements')

__init__.py

handler1.py

Serverless Python

Dependencies

install:
   find . -name "*requirements.txt" -not -path "*node_modules*"
     -exec bash -c 'pip3 install -r {} -t $$(dirname {})/.requirements' \;

uninstall:
   find . -name "*requirements.txt" -not -path "*node_modules*"
     -exec bash -c 'rm -rf $$(dirname {})/.requirements' \;

devinstall: install
   pip3 install -r requirements-dev.txt

Makefile