Writing Lambda functions in Rust!

Luciano Mammino (@loige)

AWS User Group Dublin 2023-09-13

👋 I'm Luciano (🇮🇹🍕🍝🤌)

👨‍💻 Senior Architect @ fourTheorem

📔 Co-Author of Node.js Design Patterns  👉

Let's connect!

linktr.ee/loige

Grab the slides

$ ~ whoami

Always re-imagining

We are a pioneering technology consultancy focused on AWS and serverless

✉️ Reach out to us at  hello@fourTheorem.com

😇 We are always looking for talent: fth.link/careers

We can help with:

Cloud Migrations

Training & Cloud enablement

Building high-performance serverless applications

Cutting cloud costs

What is Rust? 🦀

  • A (relatively) new programming language

  • Most loved... for 7 years in a row!

  • Low-level, but general-purpose

  • Performant & memory-safe

Why do I like it ❤️

  • A lovely mascot

  • Strongly typed with a really good type-system

  • Takes inspiration from Haskell, C++, OCaml, JavaScript, Ruby

  • Great (built-in) package manager (Cargo)

  • Good ecosystem of libraries

  • Pattern matching

  • No null, Option & Result types

use std::env;

fn main() {
    let region = env::var("AWS_REGION");
}





Result<String, VarError>
use std::env;

fn main() {
    let region = env::var("AWS_REGION");
    
    match region {
        Ok(region) => println!("Selected region: {}", region),
        Err(_) => println!("Error: AWS_REGION not set"),
    }
}
use std::env;

fn main() {
    let region = env::var("AWS_REGION")
        .expect("AWS_REGION environment variable not set");
}




String
use std::env;

fn main() {
    let region = env::var("AWS_REGION")
        .unwrap_or_else(|_| "eu-west-1".to_string());
}




Rust makes it very hard for you to ignore possible errors or the absence of values.

Serverless, in a nutshell 🥜

  • A way of running applications in the cloud

  • Of course, there are servers... we just don't have to manage them

  • We pay (only) for what we use

  • Small units of compute (functions), triggered by events

Serverless... with benefits 🎁

  • More focus on the business logic (generally)

  • Increased team agility (mostly)

  • Automatic scalability (sorta)

  • Not a universal solution, but it can work well in many situations!

AWS Lambda

Serverless FaaS offering in AWS

Can be triggered by different kinds of events

  • HTTP Requests
  • New files in S3
  • Jobs in a Queue
  • Orchestrated by Step Functions
  • On a schedule
  • Manually invoked

Some use cases 🛠️

  • HTTP APIs backend with API Gateway
  • Custom logic step in a Step Function
  • Process new S3 objects (Create picture thumbnails)
  • Scrape/Synchronise data on a schedule (Import data from an FTP)
  • Process jobs from a queue (Generate PDF invoices and send them)
  • Execute code when a business event happens (Send welcome email)
  • Define the logic to rotate secrets
  • Analyse logs and react to suspicious activities
  • etc ...

(some) Limitations 😖

  • Maximum execution time is 15 minutes...
  • Payload size (request/response) is limited
  • Doesn't have a GPU option (yet)

... so again, it's not a silver bullet for all your compute problems! 🔫

AWS Lambda Pricing 💸

Cost = Allocated Memory 𝒙 time

AWS Lambda Pricing 💸

Cost = Allocated Memory 𝒙 time

512 MB = $0.0000000083/ms

Executing a lambda for 15 mins...

0.0000000083 * 900000 = 0.007 $

AWS Lambda... what about CPU? 🙄

You don't explicitly configure it:

CPU scales based on memory

AWS Lambda... what about CPU? 🙄

You don't explicitly configure it:

CPU scales based on memory

Memory vCPUs
128 - 3008 MB 2
3009 - 5307 MB 3
5308 - 7076 MB 4
7077 - 8845 MB 5
8846+ MB 6

🏃‍♂️ Lambda execution model

  • It's serverless: it should run only when needed
  • Lambda code is stored in S3
  • event-based: an event can trigger a lambda execution
  • if no instance is available, one is created on the fly (cold-start)
  • if an instance is available and ready, use that one
  • if an instance is inactive for a while, it gets destroyed

🏃‍♂️ Lambda execution model

in detail

🏃‍♂️ Lambda execution model

Runtime

Handler (logic)

in detail

🏃‍♂️ Lambda execution model

Runtime

Handler (logic)

Poll for events

in detail

🏃‍♂️ Lambda execution model

Runtime

Handler (logic)

Poll for events

event (JSON)

in detail

🏃‍♂️ Lambda execution model

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

in detail

🏃‍♂️ Lambda execution model

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

response or
error

in detail

🏃‍♂️ Lambda execution model

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

response or
error

response (JSON)
or error

in detail

Why Rust + Lambda = ❤️

  • Performance + Efficient memory-wise = COST SAVING 🤑
  • Very fast cold starts! (proof) ⚡️
  • Multi-thread safety 💪
  • No null types + Great error primitives = fewer bugs 🐞

Supported Lambda runtimes

  • Node.js

  • Python

  • Java

  • .NET

  • Go

  • Ruby

  • Custom

Supported Lambda runtimes

  • Node.js

  • Python

  • Java

  • .NET

  • Go

  • Ruby

  • Custom

RUST?!

Rust Runtime for Lambda

OK, Where do we start?

install cargo-lambda

Cargo Lambda

  • A third-party command for Cargo that makes it easier to author, test and deploy Lambdas in Rust
  • Mostly built by an AWS employee (@calavera)
  • It can cross-compile for Linux ARM (on Win/Mac/Linux)
  • Integrates well with SAM and CDK for IaC
use aws_lambda_events::event::s3::S3Event;
use lambda_runtime::{run, service_fn, Error, LambdaEvent};

async fn function_handler(event: LambdaEvent<S3Event>) -> Result<(), Error> {
    for record in event.payload.records {
        tracing::info!(
            "[{}] Bucket={} Key={}",
            record.event_name.unwrap_or_default(),
            record.s3.bucket.name.unwrap_or_default(),
            record.s3.object.key.unwrap_or_default()
        );
    }
    Ok(())
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing_subscriber::fmt()
        .with_max_level(tracing::Level::INFO)
        .with_target(false)
        .without_time()
        .init();

    run(service_fn(function_handler)).await
}

Event & Context

async fn function_handler(event: LambdaEvent<S3Event>)
  -> Result<(), Error> {
    // let event = event.payload;
    let (event, ctx) = event.into_parts();
    println!(
        "This execution will expire at {}", 
        ctx.deadline
    );

    for record in event.records {
        // ...
    }
    Ok(())
}

Request & Response types

async fn function_handler(event: LambdaEvent<S3Event>) 
  -> Result<(), Error> {
    // ...
    Ok(())
}

Request

Response

What if we want to use different types? 🤨

Option 1

use type definitions in the aws_lambda_events crate

Processing jobs from SQS

Example

Jobs

# Cargo.toml

[dependencies]
aws_lambda_events = { 
  version = "0.10.0",
  default-features = false,
  features = [
    "sqs",
  ]
}
use aws_lambda_events::event::sqs::{BatchItemFailure, SqsBatchResponse, SqsEvent};
// ...

async fn function_handler(event: LambdaEvent<SqsEvent>)
  -> Result<SqsBatchResponse, Error> {
    let mut failed_jobs = Vec::with_capacity(event.payload.records.len());

    for record in event.payload.records {
        // process the job
        // ...
        // if the job failed, add it to the failed_jobs list
        failed_jobs.push(BatchItemFailure {
            item_identifier: record.message_id.unwrap_or_default(),
        });
    }

    Ok(SqsBatchResponse {
        batch_item_failures: failed_jobs,
    })
}

// ...

Option 2

Create custom request and response types

Custom logic in Step Function

Example

# Cargo.toml

[dependencies]
serde = "1.0.183"
serde_json = "1.0.104"
// ...

#[derive(serde::Deserialize)]
struct Request {
    url: String,
}

#[derive(serde::Serialize)]
struct Response {
    issue_number: u32,
}

async fn function_handler(event: LambdaEvent<Request>) 
  -> Result<Response, Error> {
    println!("I am going to scrape {}", event.payload.url);
    // TODO: actual scraping logic here
    Ok(Response { issue_number: 333 })
}

// ...

Option 3

Use arbitrary JSON values!

// ...

async fn function_handler(
    event: LambdaEvent<serde_json::Value>,
) -> Result<serde_json::Value, Error> {
    let url = event
        .payload
        .as_object()
        .unwrap()
        .get("url")
        .unwrap()
        .as_str()
        .unwrap(); // 🤮
    println!("I am going to scrape {}", url);
    // TODO: actual scraping logic here
    Ok(serde_json::json!({ "issue_number": 333 }))
}

// ...

HTTP-based lambdas

use lambda_http::{run, service_fn, Body, Error, Request, RequestExt, Response};

async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
    // Extract some useful information from the request
    let who = event
        .query_string_parameters_ref()
        .and_then(|params| params.first("name"))
        .unwrap_or("world");
    
    let message = format!("Hello {who}, this is an AWS Lambda HTTP request");

    // Return something that implements IntoResponse.
    // It will be serialized to the right response event 
    // automatically by the runtime
    let resp = Response::builder()
        .status(200)
        .header("content-type", "text/html")
        .body(message.into())
        .map_err(Box::new)?;
    Ok(resp)
}
use lambda_http::{run, service_fn, Body, Error, Request, RequestExt, Response};

async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
    // Extract some useful information from the request
    let who = event
        .query_string_parameters_ref()
        .and_then(|params| params.first("name"))
        .unwrap_or("world");
    
    let message = format!("Hello {who}, this is an AWS Lambda HTTP request");

    // Return something that implements IntoResponse.
    // It will be serialized to the right response event 
    // automatically by the runtime
    let resp = Response::builder()
        .status(200)
        .header("content-type", "text/html")
        .body(message.into())
        .map_err(Box::new)?;
    Ok(resp)
}

These are just abstractions! 🧐

Lambda is still using JSON behind the scenes.

For HTTP you generally use the
Lambda-Proxy integration.

Building & Deploying

cargo lambda build --release && cargo lambda deploy

NO TRIGGER CONFIGURED! 🙄

SAM

Serverless Application Model

IaC with...

AWS SAM

  • YAML-based Infrastructure as code (IaC) tool focused on serverless apps
  • Great when you have to go beyond just one lambda
  • ... or when you need more advanced integrations
    (e.g. API Gateway)
  • It supports everything that is natively supported with CloudFormation, but with a slightly simpler syntax
  • Deploys through CloudFormation!
# template.yaml

AWSTemplateFormatVersion : '2010-09-09'
Transform:
  - AWS::Serverless-2016-10-31

Description: |
  A sample Serverless project triggered from S3 CreateObject events
Resources:
  ExampleFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs18.x
      Handler: index.handler
      Events:
        S3CreateObject:
          Type: S3
          Properties:
            Bucket: !Ref MyPhotoBucket
            Events: s3:ObjectCreated:*

  MyPhotoBucket:
    Type: AWS::S3::Bucket

AWS SAM + Cargo Lambda

SAM Works with Cargo Lambda (beta feature):

  • Define IaC with the full power of SAM
  • Build and run your Rust lambdas with Cargo Lambda
  • Can simulate API Gateway locally!

Note: Cargo Lambda also works with CDK
(github.com/cargo-lambda/cargo-lambda-cdk)

# template.yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform:
  - AWS::Serverless-2016-10-31

Resources:
  ExampleHttpLambda:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: rust-cargolambda
    Properties:
      CodeUri: .
      Handler: bootstrap
      Runtime: provided.al2
      Architectures:
        - arm64
      Events:
        HttpPost:
          Type: Api
          Properties:
            Path: /
            Method: get
# samconfig.toml

version = 0.1

[default]
[default.global]
[default.global.parameters]
stack_name = "rust-http-lambda"

[default.build.parameters]
beta_features = true
[default.sync.parameters]
beta_features = true

Tells SAM to build using Cargo Lambda

Selects a "custom runtime"

Defines an HTTP trigger
(API Gateway)

Enables SAM beta features

Building, Local testing & Deploying

 

sam build
sam local start-api
sam deploy

Closing notes

  • Lambda is great (most of the time)
  • Writing Lambdas in Rust is fun and it can be very cost-efficient
  • Still not very common to write Lambdas in Rust, but the tooling is already quite good (Cargo Lambda + SAM)
  • Go, have fun, share your learnings!

BONUS: SAM + Cargo Lambda
a complete example

Cover photo by Ahmed Sobah on Unsplash

Thanks to @gbinside, @conzy_m, @eoins, and @micktwomey for kindly reviewing this talk!

THANKS!

Grab these slides!

Writing Lambda Functions in Rust, AWS User Group Dublin

By Luciano Mammino

Writing Lambda Functions in Rust, AWS User Group Dublin

Rust is taking the software engineering world by storm, but how does it affect serverless? In AWS it’s not even a supported runtime, so how can we even use it… and should we even try to do that? Spoiler: yes we should and it’s actually quite easy to get started with it!

  • 738