Easy Serverless Golang

Ahead of the Curve Meetup

May 2017

Why SLS
Why Golang
SLS + Golang

Yun Zhi Lin

  • Microservices Architect and Strategic Delivery Lead @amaysim
  • @yunspace @yunzhilin
  • Powered by AWS, Serverless, Docker & Rancher

Why Serverless Framework?

Event Driven Architecture

  • Can get a little confusing
  • Project Convention / Source Control
  • Low coupling but what about High cohesion
  • Environments and Configuration
  • Build Artifact = CloudFormation scripts?
  • Function is easy, Infrastructure Plumbing is hard

Elegant Microservice Grouping

functions:
  create:
    handler: todos/create.create
    events:
      - http:
          path: todos
          method: post
          cors: true
  get:
    handler: todos/get.get
    events:
      - http:
          path: todos/{id}
          method: get
          cors: true
  update:
    handler: todos/update.update
    events:
      - http:
          path: todos/{id}
          method: put
          cors: true
  delete:
    handler: todos/delete.delete
    events:
      - http:
          path: todos/{id}
          method: delete
          cors: true

Equivalent CloudFormation

aws-node-rest-api-with-dynamodb/.serverless> wc -l *.json

   15 cloudformation-template-create-stack.json
  900 cloudformation-template-update-stack.json
 1148 serverless-state.json
 2063 total

aws-node-rest-api-with-dynamodb/.serverless> more cloudformation-template-create-stack.json

    "CreateLambdaFunction": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        "Code": {
          "S3Bucket": {
            "Ref": "ServerlessDeploymentBucket"
          },
          "S3Key": "serverless/serverless-rest-api-with-dynamodb/dev/1495030572700-2017-05-17T14:16:12.700Z/serverless-rest-api-with-dynamodb.zip"
        },
        "FunctionName": "serverless-rest-api-with-dynamodb-dev-create",
        "Handler": "todos/create.create",
        "MemorySize": 1024,
        "Role": {
          "Fn::GetAtt": [
            "IamRoleLambdaExecution",
            "Arn"
          ]
        },
        "Runtime": "nodejs6.10",
        "Timeout": 6
      },
      "DependsOn": [
        "CreateLogGroup",
        "IamRoleLambdaExecution"
      ]
    },

Out of Box 12 Factor and

Event Sourcing

service: sls-messages-${env:ENV}
provider:
  name: aws
  runtime: nodejs6.10
  region: ${env:AWS_REGION}

  # separate deployment stage concept
  stage: ${env:ENV}
  profile: ${env:AWS_PROFILE}
functions:
  messagesHandler:
    handler: src/handler.messagesHandler
    environment:

      # configuration using Environment variables
      DB_HOST: ${env:DATABASE}
      DB_USER: ${env:DB_USER}
      DB_PASS: ${env:DB_PASS}

    # event sourcing from both API Gateway and Kinesis
    events
      - http: POST messages
      - stream:
          type: kinesis
          arn:
            Fn::ImportValue: ${env:MESSAGE_KINESIS_ARN}

Infrastructure Management

# compatible with CloudFormation yml for 
# for more complex Resource dependencies
resources:
  Resources:
    TodosDynamoDbTable:
      Type: 'AWS::DynamoDB::Table'
      DeletionPolicy: Retain
      Properties:
        AttributeDefinitions:
          -
            AttributeName: id
            AttributeType: S
        KeySchema:
          -
            AttributeName: id
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        TableName: ${self:provider.environment.DYNAMODB_TABLE}
  • Lambda on its own is sad (and useless)
  • Easy Infrastructure Management adds more Friends!
# Automatic API Gateway provisioning
functions:
    events
      - http: POST 

CD Pipeline Friendly

Code and Infrastructure all packaged into Immutable Artifact

Configured by Env Vars

Step Lambda Compatibilty

service: sls-step-up

provider:
  name: aws
  runtime: nodejs6.10

plugins:
  - serverless-step-functions

stepFunctions:
  stateMachines:
    Function1:
        ...
    Function2:
        ...
  activities:
    - myTask
    - yourTask

More FaaS than AWS

provider:
  name: google
plugins:
  - serverless-google-cloudfunctions
provider:
  name: aws
provider:
  name: azure
plugins:
  - serverless-azure-functions
provider:
  name: openwhisk

Comparable Frameworks

Framework Provider Language Infrastructure Management
Apex AWS Node
Golang
None - Terraform Lambda only
GoSparta AWS Golang Poor - Raw Cloudformation
Serverless AWS
Azure
Google
Openwhisk
Python
Node
Java
.Net
(Golang*)
Great!
- CF compatible custom yml
- Neatly defines big picture EDA

Amaysim/Trineo compared several frameworks for Function + API Gateway + Events. Easy Infrastructure Management is crucial

Use SLS

  • Short Stateless Request Response
  • Queue based Async
  • Real time Data Processing
  • Wrapper Service or Vendor Gateway

Use Docker

  • Anything Long lived: HTTP2/Websocket, Daemons
  • Thread based Async
  • Batch Data Processing
  • Complex logic or DB
  • Performance critical to the millisecond

Why Golang?

Perfect for Serverless

  • Static Typed
  • Single Static Binary
  • First class Functions
  • Elegant Concurrency 

The People have Spoken

And the People got .Net support instead

Serverless Golang

Serverless Golang Event

# already have node, golang docker and make
>
> npm install -g serverless
>
> serverless install -u https://github.com/yunspace/serverless-golang/tree/master/aws/event -n sls-golang-event
>
> cd sls-golang-event
> make package
> serverless deploy

Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
......................
Serverless: Stack update finished...
Service Information
service: sls-golang-event
stage: dev
region: ap-southeast-2
api keys:
  None
endpoints:
  POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/raw
  POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/http
functions:
  raw: sls-golang-event-dev-raw
  http: sls-golang-event-dev-http

Golang Event Code

import (
    "encoding/json"
    "github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
)

func HandleRaw(evt *json.RawMessage, ctx *runtime.Context) (interface{}, error) {
    ...
}
import (
    "github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
    "github.com/eawsy/aws-lambda-go-event/service/lambda/runtime/event/apigatewayproxyevt"
)

func HandleHTTP(evt *apigatewayproxyevt.Event, ctx *runtime.Context) (interface{}, error) {
    ...
}
import (
    "github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
    "github.com/eawsy/aws-lambda-go-event/service/lambda/runtime/event/kinesisstreamsevt"
)

func HandleKinesis(evt kinesisstreamsevt.Event, ctx *runtime.Context) (interface{}, error) {
    ...
}

Serverless Golang Net

# already have node, golang docker and make
>
> npm install -g serverless
>
> serverless install -u https://github.com/yunspace/serverless-golang/tree/master/aws/net -n sls-golang-net
>
> cd sls-golang-net
> make package
> serverless deploy

Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..................
Serverless: Stack update finished...
Service Information
service: sls-golang-net
stage: dev
region: ap-southeast-2
api keys:
  None
endpoints:
  POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos
  GET - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos
  GET - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
  PUT - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
  DELETE - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
functions:
  crud: sls-golang-net-dev-crud
Serverless: Removing old service versions...

Golang Net Code

// Set up the Handler to be exported to AWS Lambda
var Handler apigatewayproxy.Handler

func init() {
	Handler = NewHandler()
}

func NewHandler() apigatewayproxy.Handler {
	ln := net.Listen()

	// Amazon API Gateway Binary support out of the box.
	handle := apigatewayproxy.New(ln, nil).Handle

	// Any Go framework complying with http.Handler can be used. I.e. Gorilla Mux
	r := mux.NewRouter()
	r.HandleFunc("/todos", list).Methods(http.MethodGet)

	go http.Serve(ln, r)

	return handle
}

// Handle the Request as per standard go/net HTTP code
func list(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("X-Powered-By", "serverless-golang")
	fmt.Fprintf(w, "%d - Listing All", http.StatusOK)
}

How it works

  1. Secret Sauce: github.com/eawsy/aws-lambda-go-shim
  2. Build using Docker container to ensure dependencies:
    1. Go code is build using plugin mode into .so file
    2. Python shim and .so file are packaged as ZIP artifact
  3. Serverless deploys the package ZIP
  4. Runtime python shim calls the Go plugin

Shim Performance

Production Numbers

  • Productivity
    • 14/03/17 - First commit by new team member
    • 17/03/17 - Deployed to Production with full CD
  • Costs:
    • 1M calls per month free tier
    • 20c per 1M calls after
  • Duration: Acceptable average of 100~200ms
    • Minimum Warm duration: 40ms
    • Maximum Cold duration: 500ms
  • Maintenance Effort:
    • Dev Team: Minimal BAU monitoring
    • Ops Team: Setup initial permissions

Questions and Links

Easy Serverless Golang

By Yun Zhi Lin

Easy Serverless Golang

  • 1,988
Loading comments...

More from Yun Zhi Lin