Get Your Lambda...

Damian Wysocki

Mirumee Commerce Lab

For over 15 years, we have been shaping the future of technology through open source, transparency, and
a sustainable approach to product development.

A. Who We Are

  • We combine technology, design, and data.

  • We build global
    e-commerce platforms using a modern stack.

  • Open source at our core.

  • Headless architecture for ambitious brands.

A. What We Do

Django

Django

Lambda

HTTP REQUEST

Lambda

Web

The Developer’s Dilemma

debugging Lambdas locally is painful

The Developer’s Dilemma

debugging Lambdas locally is painful

Slow feedback.

       Hard to debug.

 

Traditional Lambda Workflow

pytest

  • easy
  • fast
  • deterministic

NO HTTP

serverless

  • somewhat involved
  • lack of Python 3.12 support
  • exposes the lambda over HTTP

NO PDB

localstack

  • mimics the real thing closely
  • supports API GW v1
    (in the community version)

CODE REFRESH TAKES ~2 MINUTES

... same goes for the real thing really

I just want the service to reload on save...

... just like it happens on every other web service I'm developing

asd
def http_handler(event, context):
    breakpoint()
    
    return {
        "statusCode": 200,
        "body": "Hello World"
    }

Minimal setup

[tool.smyth]
host = "0.0.0.0"
port = 8080

[tool.smyth.handlers.http_handler]
handler_path = "handler.http_handler"
url_path = "/api/{path:path}"
> python -m smyth
❯ uv run python -m smyth
Uvicorn:Worker[4664]    [12:41:39] INFO     Will watch for changes in these directories: ['/Users/pkucmus/Development/drops']
Uvicorn:Worker[4664]               INFO     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Uvicorn:Worker[4664]               INFO     Started reloader process [4664] using StatReload
Uvicorn:Worker[4666]    [12:41:39] INFO     Started server process [4666]
Uvicorn:Worker[4666]               INFO     Waiting for application startup.
Smyth:SpawnProcess-1               INFO     Started process http_handler:0
Uvicorn:Worker[4666]               INFO     Application startup complete.
Uvicorn:Worker[4664]    [12:41:46] WARNING  StatReload detected changes in 'handler.py'. Reloading...
Uvicorn:Worker[4666]    [12:41:46] INFO     Shutting down
Uvicorn:Worker[4666]               INFO     Waiting for application shutdown.
Smyth:SpawnProcess-1               INFO     Stopping process http_handler:0
Smyth:http_handler:0    [12:41:46] DEBUG    Received message: type='smyth.stop' event=None context=None
Smyth:http_handler:0               DEBUG    Stopping process
Uvicorn:Worker[4666]               INFO     Application shutdown complete.
Uvicorn:Worker[4666]               INFO     Finished server process [4666]
Uvicorn:Worker[4682]    [12:41:47] INFO     Started server process [4682]
Uvicorn:Worker[4682]               INFO     Waiting for application startup.
Smyth:SpawnProcess-2               INFO     Started process http_handler:0
Uvicorn:Worker[4682]               INFO     Application startup complete.

Reload

COUNTER = 0


def http_handler(event, context):
    global COUNTER
    COUNTER += 1
    
    return {
        "statusCode": 200,
        "body": f"Hello World: {COUNTER}"
    }

the module is imported once, per a Smyth Process lifetime

Cold start simualtion

[tool.smyth]
host = "0.0.0.0"
port = 8080

[tool.smyth.handlers.http_handler]
handler_path = "handler.http_handler"
url_path = "/api/{path:path}"
concurrency = 2
❯ uv run python -m smyth
Uvicorn:Worker[4851]    [12:53:54] INFO     Will watch for changes in these directories: ['/Users/pkucmus/Development/drops']
Uvicorn:Worker[4851]               INFO     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Uvicorn:Worker[4851]               INFO     Started reloader process [4851] using StatReload
Uvicorn:Worker[4854]    [12:53:54] INFO     Started server process [4854]
Uvicorn:Worker[4854]               INFO     Waiting for application startup.
Smyth:SpawnProcess-1               INFO     Started process http_handler:0
Smyth:SpawnProcess-1               INFO     Started process http_handler:1
Uvicorn:Worker[4854]               INFO     Application startup complete.

Concurrency

HTTP REQUEST

EVENT

Lambda

Web

The real thing

HTTP REQUEST

EVENT

Lambda

Web

Smyth

Lambda

Web

import boto3

lambda_client = boto3.client(
    "lambda", 
    endpoint_url="http://localhost:8080"  
)  

def order_handler(event, context):
    lambda_client.invoke(
        FunctionName="email_handler",  
        InvocationType="Event",  # or RequestResponse
        Payload=b'{"to": "hello@mirumee.com", "subject": "Order made"}',
    )
    return {"statusCode": 200, "body": f"Orders requests: {COUNT}"}

def email_handler(event, context):
    print(event)  
    return {"statusCode": 200, "body": f"Products requests: {COUNT}"}

add it to the TOML

Lambda invoke endpoint

from smyth.types import EventData, RunnerProcessProtocol, SmythHandler


async def generate_api_gw_v1_event_data(
  request: Request,
  smyth_handler: SmythHandler,
  process: RunnerProcessProtocol
) -> EventData:
    source_ip = None
    if request.client:
        source_ip = request.client.host

    return {
        "resource": request.url.path,
        "path": request.url.path,
        "httpMethod": request.method,
        "headers": dict(request.headers),
        "queryStringParameters": dict(request.query_params),
        "pathParameters": {},  # You may need to populate this based on your routing
        "stageVariables": None,
        "requestContext": {
            "resourceId": "offlineContext_resourceId",
            "resourcePath": request.url.path,
            "httpMethod": request.method,
            "extendedRequestId": "offlineContext_extendedRequestId",
            "requestTime": "21/Nov/2020:20:13:27 +0000",
            "path": request.url.path,
            "accountId": "offlineContext_accountId",
            "protocol": request.url.scheme,
            "stage": "dev",
            "domainPrefix": "offlineContext_domainPrefix",
            "requestTimeEpoch": int(request.timestamp().timestamp() * 1000),
            "requestId": "offlineContext_requestId",
            "identity": {
                ...
            },
            "domainName": "offlineContext_domainName",
            "apiId": "offlineContext_apiId"
        },
        "body": (await request.body()).decode("utf-8"),
        "isBase64Encoded": False
    }

[tool.smyth]
host = "0.0.0.0"
port = 8080

[tool.smyth.handlers.http_handler]
handler_path = "handler.http_handler"
url_path = "/api/{path:path}"
event_data_function_path = "handler.generate_api_gw_v1_event_data"

Event generation

There's more...

Dispatch Strategies

Environment Variable Injection

Custom Entrypoint for complex usecases

Non-HTTP Invocation

https://mirumee.github.io/smyth/user_guide/

There's another
problem

you still need to sell the whole serverless thing to the team

There's another
problem

you still need to sell the whole serverless thing to the team

asd

Web

ASGI

The gist of it

import asyncio
from lynara import Lynara, APIGatewayProxyEventV2Interface
from fastapi import FastAPI

app = FastAPI()
lynara = Lynara(app=app)

def lambda_handler(event, context):
    return asyncio.run(
        lynara.run(event, context, APIGatewayProxyEventV2Interface)
    )

Setup

import asyncio
from lynara import LifespanMode, Lynara, APIGatewayProxyEventV2Interface
from fastapi import FastAPI

app = FastAPI()
lynara = Lynara(app=app, lifespan_mode=LifespanMode.AUTO)

def lambda_handler(event, context):
    return asyncio.run(
        lynara.run(event, context, APIGatewayProxyEventV2Interface)
    )

Lifespan support

import asyncio
from lynara import (
  Lynara,
  APIGatewayProxyEventV1Interface,
  APIGatewayProxyEventV2Interface
)
from fastapi import FastAPI

app = FastAPI()
lynara = Lynara(app=app)

# AWS API Gateway Proxy V2 or Lambda function URL
def http_handler(event, context):
    return asyncio.run(
        lynara.run(event, context, APIGatewayProxyEventV2Interface)
    )
  
# AWS API Gateway Proxy V1
def http_handler(event, context):
    return asyncio.run(
        lynara.run(event, context, APIGatewayProxyEventV1Interface)
    )

Interface support

Sure:

https://mirumee.github.io/lynara/interfaces/interfaces/

if you are stubborn enough and can translate an event into a ASGI request you could use FastAPI to handle DynamoDB, SQS and other events

Custom Interfaces

Benchmarks

Cold start performance

def lambda_handler_pure(event, context):
    return {"data": "Hello It's me Damian!"}
def lambda_handler_pure(event, context):
    return {"data": "Hello It's me Damian!"}

0.49 sec of cold start

from fastapi import FastAPI


def lambda_handler_pure(event, context):
    return {"data": "Hello It's me Damian!"}
from fastapi import FastAPI


def lambda_handler_pure(event, context):
    return {"data": "Hello It's me Damian!"}

1.39 sec of cold start

Lambda Layers

Lambda Layers

Reduce cold start by 1 second

serverless

Fueled by your favourite ASGI framework

Great

power

Great

power

With

comes great responsibility

https://www.aws-geek.com/

No rewrite needed. Your ASGI app is portable!

It's your choice...

Dev/Staging on Lambda but prod on EC2

Early stage development on lambda

Private environment for QAs

 

Smyth GitHub 

Lynara GitHub

Thank You.

deck

By Damian Wysocki

Made with Slides.com

deck

  • 102

More from Damian Wysocki