Writing Lambda functions in Rust!

Luciano Mammino (@loige)

loige.link/lambda-rust

AWS User Group Dublin 2023-09-13

👋 I'm Luciano (🇮🇹🍕🍝🤌)

👨‍💻 Senior Architect @ fourTheorem

nodejsdp.link

📔 Co-Author of Node.js Design Patterns  👉

Let's connect!

linktr.ee/loige

loige.link/lambda-rust

Grab the slides

$ ~ whoami

Always re-imagining

We are a pioneering technology consultancy focused on AWS and serverless

loige

✉️ Reach out to us at  hello@fourTheorem.com

😇 We are always looking for talent: fth.link/careers

We can help with:

Cloud Migrations

Training & Cloud enablement

Building high-performance serverless applications

Cutting cloud costs

awsbites.com

loige

What is Rust? 🦀

  • A (relatively) new programming language

  • Most loved... for 7 years in a row!

  • Low-level, but general-purpose

  • Performant & memory-safe

loige

Why do I like it ❤️

  • A lovely mascot

  • Strongly typed with a really good type-system

  • Takes inspiration from Haskell, C++, OCaml, JavaScript, Ruby

  • Great (built-in) package manager (Cargo)

  • Good ecosystem of libraries

  • Pattern matching

  • No null, Option & Result types

loige

use std::env;

fn main() {
    let region = env::var("AWS_REGION");
}

loige

Result<String, VarError>
use std::env;

fn main() {
    let region = env::var("AWS_REGION");
    
    match region {
        Ok(region) => println!("Selected region: {}", region),
        Err(_) => println!("Error: AWS_REGION not set"),
    }
}

loige

use std::env;

fn main() {
    let region = env::var("AWS_REGION")
        .expect("AWS_REGION environment variable not set");
}

loige

String
use std::env;

fn main() {
    let region = env::var("AWS_REGION")
        .unwrap_or_else(|_| "eu-west-1".to_string());
}

loige

Rust makes it very hard for you to ignore possible errors or the absence of values.

Serverless, in a nutshell 🥜

  • A way of running applications in the cloud

  • Of course, there are servers... we just don't have to manage them

  • We pay (only) for what we use

  • Small units of compute (functions), triggered by events

loige

Serverless... with benefits 🎁

  • More focus on the business logic (generally)

  • Increased team agility (mostly)

  • Automatic scalability (sorta)

  • Not a universal solution, but it can work well in many situations!

loige

AWS Lambda

Serverless FaaS offering in AWS

Can be triggered by different kinds of events

  • HTTP Requests
  • New files in S3
  • Jobs in a Queue
  • Orchestrated by Step Functions
  • On a schedule
  • Manually invoked

loige

Some use cases 🛠️

  • HTTP APIs backend with API Gateway
  • Custom logic step in a Step Function
  • Process new S3 objects (Create picture thumbnails)
  • Scrape/Synchronise data on a schedule (Import data from an FTP)
  • Process jobs from a queue (Generate PDF invoices and send them)
  • Execute code when a business event happens (Send welcome email)
  • Define the logic to rotate secrets
  • Analyse logs and react to suspicious activities
  • etc ...

loige

(some) Limitations 😖

  • Maximum execution time is 15 minutes...
  • Payload size (request/response) is limited
  • Doesn't have a GPU option (yet)

loige

... so again, it's not a silver bullet for all your compute problems! 🔫

AWS Lambda Pricing 💸

Cost = Allocated Memory 𝒙 time

loige

aws.amazon.com/lambda/pricing/

AWS Lambda Pricing 💸

Cost = Allocated Memory 𝒙 time

loige

aws.amazon.com/lambda/pricing/

512 MB = $0.0000000083/ms

Executing a lambda for 15 mins...

0.0000000083 * 900000 = 0.007 $

AWS Lambda... what about CPU? 🙄

You don't explicitly configure it:

CPU scales based on memory

loige

loige.link/lambda-cpu

AWS Lambda... what about CPU? 🙄

You don't explicitly configure it:

CPU scales based on memory

loige

Memory vCPUs
128 - 3008 MB 2
3009 - 5307 MB 3
5308 - 7076 MB 4
7077 - 8845 MB 5
8846+ MB 6

loige.link/lambda-cpu

🏃‍♂️ Lambda execution model

  • It's serverless: it should run only when needed
  • Lambda code is stored in S3
  • event-based: an event can trigger a lambda execution
  • if no instance is available, one is created on the fly (cold-start)
  • if an instance is available and ready, use that one
  • if an instance is inactive for a while, it gets destroyed

loige

🏃‍♂️ Lambda execution model

loige

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

event (JSON)

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

response or
error

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

response or
error

response (JSON)
or error

in detail

Why Rust + Lambda = ❤️

  • Performance + Efficient memory-wise = COST SAVING 🤑
  • Very fast cold starts! (proof) ⚡️
  • Multi-thread safety 💪
  • No null types + Great error primitives = fewer bugs 🐞

loige

Supported Lambda runtimes

  • Node.js

  • Python

  • Java

  • .NET

  • Go

  • Ruby

  • Custom

loige

Supported Lambda runtimes

  • Node.js

  • Python

  • Java

  • .NET

  • Go

  • Ruby

  • Custom

loige

RUST?!

[deprecated]

Rust Runtime for Lambda

loige

github.com/awslabs/aws-lambda-rust-runtime

OK, Where do we start?

loige

install cargo-lambda

Cargo Lambda

  • A third-party command for Cargo that makes it easier to author, test and deploy Lambdas in Rust
  • Mostly built by an AWS employee (@calavera)
  • It can cross-compile for Linux ARM (on Win/Mac/Linux)
  • Integrates well with SAM and CDK for IaC

loige

cargo-lambda.info

loige

use aws_lambda_events::event::s3::S3Event;
use lambda_runtime::{run, service_fn, Error, LambdaEvent};

async fn function_handler(event: LambdaEvent<S3Event>) -> Result<(), Error> {
    for record in event.payload.records {
        tracing::info!(
            "[{}] Bucket={} Key={}",
            record.event_name.unwrap_or_default(),
            record.s3.bucket.name.unwrap_or_default(),
            record.s3.object.key.unwrap_or_default()
        );
    }
    Ok(())
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing_subscriber::fmt()
        .with_max_level(tracing::Level::INFO)
        .with_target(false)
        .without_time()
        .init();

    run(service_fn(function_handler)).await
}

loige

loige

Event & Context

loige

async fn function_handler(event: LambdaEvent<S3Event>)
  -> Result<(), Error> {
    // let event = event.payload;
    let (event, ctx) = event.into_parts();
    println!(
        "This execution will expire at {}", 
        ctx.deadline
    );

    for record in event.records {
        // ...
    }
    Ok(())
}

loige

Request & Response types

loige

async fn function_handler(event: LambdaEvent<S3Event>) 
  -> Result<(), Error> {
    // ...
    Ok(())
}

loige

Request

Response

What if we want to use different types? 🤨

Option 1

use type definitions in the aws_lambda_events crate

loige

loige

Processing jobs from SQS

loige

Example

Jobs

Event source mapper

# Cargo.toml

[dependencies]
aws_lambda_events = { 
  version = "0.10.0",
  default-features = false,
  features = [
    "sqs",
  ]
}

loige

use aws_lambda_events::event::sqs::{BatchItemFailure, SqsBatchResponse, SqsEvent};
// ...

async fn function_handler(event: LambdaEvent<SqsEvent>)
  -> Result<SqsBatchResponse, Error> {
    let mut failed_jobs = Vec::with_capacity(event.payload.records.len());

    for record in event.payload.records {
        // process the job
        // ...
        // if the job failed, add it to the failed_jobs list
        failed_jobs.push(BatchItemFailure {
            item_identifier: record.message_id.unwrap_or_default(),
        });
    }

    Ok(SqsBatchResponse {
        batch_item_failures: failed_jobs,
    })
}

// ...

loige

Option 2

Create custom request and response types

loige

loige

Custom logic in Step Function

Example

loige.link/fsb-refactor

# Cargo.toml

[dependencies]
serde = "1.0.183"
serde_json = "1.0.104"

loige

// ...

#[derive(serde::Deserialize)]
struct Request {
    url: String,
}

#[derive(serde::Serialize)]
struct Response {
    issue_number: u32,
}

async fn function_handler(event: LambdaEvent<Request>) 
  -> Result<Response, Error> {
    println!("I am going to scrape {}", event.payload.url);
    // TODO: actual scraping logic here
    Ok(Response { issue_number: 333 })
}

// ...

loige

loige

Option 3

Use arbitrary JSON values!

loige

// ...

async fn function_handler(
    event: LambdaEvent<serde_json::Value>,
) -> Result<serde_json::Value, Error> {
    let url = event
        .payload
        .as_object()
        .unwrap()
        .get("url")
        .unwrap()
        .as_str()
        .unwrap(); // 🤮
    println!("I am going to scrape {}", url);
    // TODO: actual scraping logic here
    Ok(serde_json::json!({ "issue_number": 333 }))
}

// ...

loige

HTTP-based lambdas

loige

loige

use lambda_http::{run, service_fn, Body, Error, Request, RequestExt, Response};

async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
    // Extract some useful information from the request
    let who = event
        .query_string_parameters_ref()
        .and_then(|params| params.first("name"))
        .unwrap_or("world");
    
    let message = format!("Hello {who}, this is an AWS Lambda HTTP request");

    // Return something that implements IntoResponse.
    // It will be serialized to the right response event 
    // automatically by the runtime
    let resp = Response::builder()
        .status(200)
        .header("content-type", "text/html")
        .body(message.into())
        .map_err(Box::new)?;
    Ok(resp)
}

loige

use lambda_http::{run, service_fn, Body, Error, Request, RequestExt, Response};

async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
    // Extract some useful information from the request
    let who = event
        .query_string_parameters_ref()
        .and_then(|params| params.first("name"))
        .unwrap_or("world");
    
    let message = format!("Hello {who}, this is an AWS Lambda HTTP request");

    // Return something that implements IntoResponse.
    // It will be serialized to the right response event 
    // automatically by the runtime
    let resp = Response::builder()
        .status(200)
        .header("content-type", "text/html")
        .body(message.into())
        .map_err(Box::new)?;
    Ok(resp)
}

loige

These are just abstractions! 🧐

Lambda is still using JSON behind the scenes.

For HTTP you generally use the Lambda-Proxy integration.

Building & Deploying

cargo lambda build --release && cargo lambda deploy

loige

loige

loige

NO TRIGGER CONFIGURED! 🙄

SAM

loige

Serverless Application Model

IaC with...

AWS SAM

  • YAML-based Infrastructure as code (IaC) tool focused on serverless apps
  • Great when you have to go beyond just one lambda
  • ... or when you need more advanced integrations
    (e.g. API Gateway)
  • It supports everything that is natively supported with CloudFormation, but with a slightly simpler syntax
  • Deploys through CloudFormation!

loige

aws.amazon.com/serverless/sam

# template.yaml

AWSTemplateFormatVersion : '2010-09-09'
Transform:
  - AWS::Serverless-2016-10-31

Description: |
  A sample Serverless project triggered from S3 CreateObject events
Resources:
  ExampleFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs18.x
      Handler: index.handler
      Events:
        S3CreateObject:
          Type: S3
          Properties:
            Bucket: !Ref MyPhotoBucket
            Events: s3:ObjectCreated:*

  MyPhotoBucket:
    Type: AWS::S3::Bucket

loige

AWS SAM + Cargo Lambda

SAM Works with Cargo Lambda (beta feature):

  • Define IaC with the full power of SAM
  • Build and run your Rust lambdas with Cargo Lambda
  • Can simulate API Gateway locally!

loige

Note: Cargo Lambda also works with CDK
(github.com/cargo-lambda/cargo-lambda-cdk)

# template.yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform:
  - AWS::Serverless-2016-10-31

Resources:
  ExampleHttpLambda:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: rust-cargolambda
    Properties:
      CodeUri: .
      Handler: bootstrap
      Runtime: provided.al2
      Architectures:
        - arm64
      Events:
        HttpPost:
          Type: Api
          Properties:
            Path: /
            Method: get

loige

# samconfig.toml

version = 0.1

[default]
[default.global]
[default.global.parameters]
stack_name = "rust-http-lambda"

[default.build.parameters]
beta_features = true
[default.sync.parameters]
beta_features = true

Tells SAM to build using Cargo Lambda

Selects a "custom runtime"

Defines an HTTP trigger
(API Gateway)

Enables SAM beta features

Building, Local testing & Deploying

 

sam build
sam local start-api
sam deploy

loige

loige

loige

Closing notes

  • Lambda is great (most of the time)
  • Writing Lambdas in Rust is fun and it can be very cost-efficient
  • Still not very common to write Lambdas in Rust, but the tooling is already quite good (Cargo Lambda + SAM)
  • Go, have fun, share your learnings!

loige

BONUS: SAM + Cargo Lambda
a complete example

loige

github.com/lmammino/earthquake-notifier

Cover photo by Ahmed Sobah on Unsplash

Thanks to @gbinside, @conzy_m, @eoins, and @micktwomey for kindly reviewing this talk!

THANKS!

nodejsdp.link

loige

loige.link/lambda-rust

Grab these slides!

Writing Lambda Functions in Rust, AWS User Group Dublin

By Luciano Mammino

Writing Lambda Functions in Rust, AWS User Group Dublin

Rust is taking the software engineering world by storm, but how does it affect serverless? In AWS it’s not even a supported runtime, so how can we even use it… and should we even try to do that? Spoiler: yes we should and it’s actually quite easy to get started with it!

  • 1,868

More from Luciano Mammino