Easy Serverless Golang

Ahead of the Curve Meetup

May 2017

Why SLS
Why Golang
SLS + Golang

Yun Zhi Lin

Microservices Architect and Strategic Delivery Lead @amaysim
@yunspace @yunzhilin
Powered by AWS, Serverless, Docker & Rancher

Why Serverless Framework?

Event Driven Architecture

Can get a little confusing
Project Convention / Source Control
Low coupling but what about High cohesion
Environments and Configuration
Build Artifact = CloudFormation scripts?
Function is easy, Infrastructure Plumbing is hard

Elegant Microservice Grouping

functions:
  create:
    handler: todos/create.create
    events:
      - http:
          path: todos
          method: post
          cors: true
  get:
    handler: todos/get.get
    events:
      - http:
          path: todos/{id}
          method: get
          cors: true
  update:
    handler: todos/update.update
    events:
      - http:
          path: todos/{id}
          method: put
          cors: true
  delete:
    handler: todos/delete.delete
    events:
      - http:
          path: todos/{id}
          method: delete
          cors: true

github.com/serverless/examples/aws-node-rest-api-with-dynamodb

Equivalent CloudFormation

aws-node-rest-api-with-dynamodb/.serverless> wc -l *.json

   15 cloudformation-template-create-stack.json
  900 cloudformation-template-update-stack.json
 1148 serverless-state.json
 2063 total

aws-node-rest-api-with-dynamodb/.serverless> more cloudformation-template-create-stack.json

    "CreateLambdaFunction": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        "Code": {
          "S3Bucket": {
            "Ref": "ServerlessDeploymentBucket"
          },
          "S3Key": "serverless/serverless-rest-api-with-dynamodb/dev/1495030572700-2017-05-17T14:16:12.700Z/serverless-rest-api-with-dynamodb.zip"
        },
        "FunctionName": "serverless-rest-api-with-dynamodb-dev-create",
        "Handler": "todos/create.create",
        "MemorySize": 1024,
        "Role": {
          "Fn::GetAtt": [
            "IamRoleLambdaExecution",
            "Arn"
          ]
        },
        "Runtime": "nodejs6.10",
        "Timeout": 6
      },
      "DependsOn": [
        "CreateLogGroup",
        "IamRoleLambdaExecution"
      ]
    },

Out of Box 12 Factor and

Event Sourcing

service: sls-messages-${env:ENV}
provider:
  name: aws
  runtime: nodejs6.10
  region: ${env:AWS_REGION}

  # separate deployment stage concept
  stage: ${env:ENV}
  profile: ${env:AWS_PROFILE}
functions:
  messagesHandler:
    handler: src/handler.messagesHandler
    environment:

      # configuration using Environment variables
      DB_HOST: ${env:DATABASE}
      DB_USER: ${env:DB_USER}
      DB_PASS: ${env:DB_PASS}

    # event sourcing from both API Gateway and Kinesis
    events
      - http: POST messages
      - stream:
          type: kinesis
          arn:
            Fn::ImportValue: ${env:MESSAGE_KINESIS_ARN}

Infrastructure Management

# compatible with CloudFormation yml for 
# for more complex Resource dependencies
resources:
  Resources:
    TodosDynamoDbTable:
      Type: 'AWS::DynamoDB::Table'
      DeletionPolicy: Retain
      Properties:
        AttributeDefinitions:
          -
            AttributeName: id
            AttributeType: S
        KeySchema:
          -
            AttributeName: id
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        TableName: ${self:provider.environment.DYNAMODB_TABLE}

Lambda on its own is sad (and useless)
Easy Infrastructure Management adds more Friends!

# Automatic API Gateway provisioning
functions:
    events
      - http: POST

CD Pipeline Friendly

Code and Infrastructure all packaged into Immutable Artifact

Configured by Env Vars

Step Lambda Compatibilty

github.com/horike37/serverless-step-functions

service: sls-step-up

provider:
  name: aws
  runtime: nodejs6.10

plugins:
  - serverless-step-functions

stepFunctions:
  stateMachines:
    Function1:
        ...
    Function2:
        ...
  activities:
    - myTask
    - yourTask

More FaaS than AWS

provider:
  name: google
plugins:
  - serverless-google-cloudfunctions

provider:
  name: aws

provider:
  name: azure
plugins:
  - serverless-azure-functions

provider:
  name: openwhisk

Comparable Frameworks

Framework	Provider	Language	Infrastructure Management
Apex	AWS	Node Golang	None - Terraform Lambda only
GoSparta	AWS	Golang	Poor - Raw Cloudformation
Serverless	AWS Azure Google Openwhisk	Python Node Java .Net (Golang*)	Great! - CF compatible custom yml - Neatly defines big picture EDA

Amaysim/Trineo compared several frameworks for Function + API Gateway + Events. Easy Infrastructure Management is crucial

Use SLS

Short Stateless Request Response
Queue based Async
Real time Data Processing
Wrapper Service or Vendor Gateway

Use Docker

Anything Long lived: HTTP2/Websocket, Daemons
Thread based Async
Batch Data Processing
Complex logic or DB
Performance critical to the millisecond

Why Golang?

Perfect for Serverless

Static Typed
Single Static Binary
First class Functions
Elegant Concurrency

The People have Spoken

And the People got .Net support instead

Serverless Golang

Serverless Golang Event

# already have node, golang docker and make
>
> npm install -g serverless
>
> serverless install -u https://github.com/yunspace/serverless-golang/tree/master/aws/event -n sls-golang-event
>
> cd sls-golang-event
> make package
> serverless deploy

Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
......................
Serverless: Stack update finished...
Service Information
service: sls-golang-event
stage: dev
region: ap-southeast-2
api keys:
  None
endpoints:
  POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/raw
  POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/http
functions:
  raw: sls-golang-event-dev-raw
  http: sls-golang-event-dev-http

Golang Event Code

import (
    "encoding/json"
    "github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
)

func HandleRaw(evt *json.RawMessage, ctx *runtime.Context) (interface{}, error) {
    ...
}

import (
    "github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
    "github.com/eawsy/aws-lambda-go-event/service/lambda/runtime/event/apigatewayproxyevt"
)

func HandleHTTP(evt *apigatewayproxyevt.Event, ctx *runtime.Context) (interface{}, error) {
    ...
}

import (
    "github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
    "github.com/eawsy/aws-lambda-go-event/service/lambda/runtime/event/kinesisstreamsevt"
)

func HandleKinesis(evt kinesisstreamsevt.Event, ctx *runtime.Context) (interface{}, error) {
    ...
}

Serverless Golang Net

# already have node, golang docker and make
>
> npm install -g serverless
>
> serverless install -u https://github.com/yunspace/serverless-golang/tree/master/aws/net -n sls-golang-net
>
> cd sls-golang-net
> make package
> serverless deploy

Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..................
Serverless: Stack update finished...
Service Information
service: sls-golang-net
stage: dev
region: ap-southeast-2
api keys:
  None
endpoints:
  POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos
  GET - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos
  GET - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
  PUT - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
  DELETE - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
functions:
  crud: sls-golang-net-dev-crud
Serverless: Removing old service versions...

Golang Net Code

// Set up the Handler to be exported to AWS Lambda
var Handler apigatewayproxy.Handler

func init() {
	Handler = NewHandler()
}

func NewHandler() apigatewayproxy.Handler {
	ln := net.Listen()

	// Amazon API Gateway Binary support out of the box.
	handle := apigatewayproxy.New(ln, nil).Handle

	// Any Go framework complying with http.Handler can be used. I.e. Gorilla Mux
	r := mux.NewRouter()
	r.HandleFunc("/todos", list).Methods(http.MethodGet)

	go http.Serve(ln, r)

	return handle
}

// Handle the Request as per standard go/net HTTP code
func list(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("X-Powered-By", "serverless-golang")
	fmt.Fprintf(w, "%d - Listing All", http.StatusOK)
}

How it works

Secret Sauce: github.com/eawsy/aws-lambda-go-shim
Build using Docker container to ensure dependencies:
1. Go code is build using plugin mode into .so file
2. Python shim and .so file are packaged as ZIP artifact
Serverless deploys the package ZIP
Runtime python shim calls the Go plugin

Shim Performance

Production Numbers

Productivity
- 14/03/17 - First commit by new team member
- 17/03/17 - Deployed to Production with full CD
Costs:
- 1M calls per month free tier
- 20c per 1M calls after
Duration: Acceptable average of 100~200ms
- Minimum Warm duration: 40ms
- Maximum Cold duration: 500ms
Maintenance Effort:
- Dev Team: Minimal BAU monitoring
- Ops Team: Setup initial permissions

Questions and Links

github.com/yunspace/serverless-golang
github.com/eawsy/aws-lambda-go-shim
serverless.com
Credit goes to @cristim - co-author who proposed the original idea via #2712 and made awesome code contributions