Easy Serverless Golang
Ahead of the Curve Meetup
May 2017
Why SLS
Why Golang
SLS + Golang
Yun Zhi Lin
- Microservices Architect and Strategic Delivery Lead @amaysim
- @yunspace @yunzhilin
- Powered by AWS, Serverless, Docker & Rancher
Why Serverless Framework?
Event Driven Architecture
- Can get a little confusing
- Project Convention / Source Control
- Low coupling but what about High cohesion
- Environments and Configuration
- Build Artifact = CloudFormation scripts?
- Function is easy, Infrastructure Plumbing is hard
Elegant Microservice Grouping
functions:
create:
handler: todos/create.create
events:
- http:
path: todos
method: post
cors: true
get:
handler: todos/get.get
events:
- http:
path: todos/{id}
method: get
cors: true
update:
handler: todos/update.update
events:
- http:
path: todos/{id}
method: put
cors: true
delete:
handler: todos/delete.delete
events:
- http:
path: todos/{id}
method: delete
cors: true
Equivalent CloudFormation
aws-node-rest-api-with-dynamodb/.serverless> wc -l *.json
15 cloudformation-template-create-stack.json
900 cloudformation-template-update-stack.json
1148 serverless-state.json
2063 total
aws-node-rest-api-with-dynamodb/.serverless> more cloudformation-template-create-stack.json
"CreateLambdaFunction": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": {
"Ref": "ServerlessDeploymentBucket"
},
"S3Key": "serverless/serverless-rest-api-with-dynamodb/dev/1495030572700-2017-05-17T14:16:12.700Z/serverless-rest-api-with-dynamodb.zip"
},
"FunctionName": "serverless-rest-api-with-dynamodb-dev-create",
"Handler": "todos/create.create",
"MemorySize": 1024,
"Role": {
"Fn::GetAtt": [
"IamRoleLambdaExecution",
"Arn"
]
},
"Runtime": "nodejs6.10",
"Timeout": 6
},
"DependsOn": [
"CreateLogGroup",
"IamRoleLambdaExecution"
]
},
Out of Box 12 Factor and
Event Sourcing
service: sls-messages-${env:ENV}
provider:
name: aws
runtime: nodejs6.10
region: ${env:AWS_REGION}
# separate deployment stage concept
stage: ${env:ENV}
profile: ${env:AWS_PROFILE}
functions:
messagesHandler:
handler: src/handler.messagesHandler
environment:
# configuration using Environment variables
DB_HOST: ${env:DATABASE}
DB_USER: ${env:DB_USER}
DB_PASS: ${env:DB_PASS}
# event sourcing from both API Gateway and Kinesis
events
- http: POST messages
- stream:
type: kinesis
arn:
Fn::ImportValue: ${env:MESSAGE_KINESIS_ARN}
Infrastructure Management
# compatible with CloudFormation yml for
# for more complex Resource dependencies
resources:
Resources:
TodosDynamoDbTable:
Type: 'AWS::DynamoDB::Table'
DeletionPolicy: Retain
Properties:
AttributeDefinitions:
-
AttributeName: id
AttributeType: S
KeySchema:
-
AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:provider.environment.DYNAMODB_TABLE}
- Lambda on its own is sad (and useless)
- Easy Infrastructure Management adds more Friends!
# Automatic API Gateway provisioning
functions:
events
- http: POST
CD Pipeline Friendly
Code and Infrastructure all packaged into Immutable Artifact
Configured by Env Vars
Step Lambda Compatibilty
service: sls-step-up
provider:
name: aws
runtime: nodejs6.10
plugins:
- serverless-step-functions
stepFunctions:
stateMachines:
Function1:
...
Function2:
...
activities:
- myTask
- yourTask
More FaaS than AWS
provider:
name: google
plugins:
- serverless-google-cloudfunctions
provider:
name: aws
provider:
name: azure
plugins:
- serverless-azure-functions
provider:
name: openwhisk
Comparable Frameworks
Framework | Provider | Language | Infrastructure Management |
---|---|---|---|
Apex | AWS | Node Golang |
None - Terraform Lambda only |
GoSparta | AWS | Golang | Poor - Raw Cloudformation |
Serverless | AWS Azure Openwhisk |
Python Node Java .Net (Golang*) |
Great! - CF compatible custom yml - Neatly defines big picture EDA |
Amaysim/Trineo compared several frameworks for Function + API Gateway + Events. Easy Infrastructure Management is crucial
Use SLS
- Short Stateless Request Response
- Queue based Async
- Real time Data Processing
- Wrapper Service or Vendor Gateway
Use Docker
- Anything Long lived: HTTP2/Websocket, Daemons
- Thread based Async
- Batch Data Processing
- Complex logic or DB
- Performance critical to the millisecond
Why Golang?
Perfect for Serverless
- Static Typed
- Single Static Binary
- First class Functions
- Elegant Concurrency
The People have Spoken
And the People got .Net support instead
Serverless Golang
Serverless Golang Event
# already have node, golang docker and make
>
> npm install -g serverless
>
> serverless install -u https://github.com/yunspace/serverless-golang/tree/master/aws/event -n sls-golang-event
>
> cd sls-golang-event
> make package
> serverless deploy
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
......................
Serverless: Stack update finished...
Service Information
service: sls-golang-event
stage: dev
region: ap-southeast-2
api keys:
None
endpoints:
POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/raw
POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/http
functions:
raw: sls-golang-event-dev-raw
http: sls-golang-event-dev-http
Golang Event Code
import (
"encoding/json"
"github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
)
func HandleRaw(evt *json.RawMessage, ctx *runtime.Context) (interface{}, error) {
...
}
import (
"github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
"github.com/eawsy/aws-lambda-go-event/service/lambda/runtime/event/apigatewayproxyevt"
)
func HandleHTTP(evt *apigatewayproxyevt.Event, ctx *runtime.Context) (interface{}, error) {
...
}
import (
"github.com/eawsy/aws-lambda-go-core/service/lambda/runtime"
"github.com/eawsy/aws-lambda-go-event/service/lambda/runtime/event/kinesisstreamsevt"
)
func HandleKinesis(evt kinesisstreamsevt.Event, ctx *runtime.Context) (interface{}, error) {
...
}
Serverless Golang Net
# already have node, golang docker and make
>
> npm install -g serverless
>
> serverless install -u https://github.com/yunspace/serverless-golang/tree/master/aws/net -n sls-golang-net
>
> cd sls-golang-net
> make package
> serverless deploy
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..................
Serverless: Stack update finished...
Service Information
service: sls-golang-net
stage: dev
region: ap-southeast-2
api keys:
None
endpoints:
POST - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos
GET - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos
GET - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
PUT - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
DELETE - https://XXXXXXX.execute-api.ap-southeast-2.amazonaws.com/dev/todos/{id}
functions:
crud: sls-golang-net-dev-crud
Serverless: Removing old service versions...
Golang Net Code
// Set up the Handler to be exported to AWS Lambda
var Handler apigatewayproxy.Handler
func init() {
Handler = NewHandler()
}
func NewHandler() apigatewayproxy.Handler {
ln := net.Listen()
// Amazon API Gateway Binary support out of the box.
handle := apigatewayproxy.New(ln, nil).Handle
// Any Go framework complying with http.Handler can be used. I.e. Gorilla Mux
r := mux.NewRouter()
r.HandleFunc("/todos", list).Methods(http.MethodGet)
go http.Serve(ln, r)
return handle
}
// Handle the Request as per standard go/net HTTP code
func list(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Powered-By", "serverless-golang")
fmt.Fprintf(w, "%d - Listing All", http.StatusOK)
}
How it works
- Secret Sauce: github.com/eawsy/aws-lambda-go-shim
- Build using Docker container to ensure dependencies:
- Go code is build using plugin mode into .so file
- Python shim and .so file are packaged as ZIP artifact
- Serverless deploys the package ZIP
- Runtime python shim calls the Go plugin
Shim Performance
Production Numbers
- Productivity
- 14/03/17 - First commit by new team member
- 17/03/17 - Deployed to Production with full CD
- Costs:
- 1M calls per month free tier
- 20c per 1M calls after
- Duration: Acceptable average of 100~200ms
- Minimum Warm duration: 40ms
- Maximum Cold duration: 500ms
- Maintenance Effort:
- Dev Team: Minimal BAU monitoring
- Ops Team: Setup initial permissions
Questions and Links
- github.com/yunspace/serverless-golang
- github.com/eawsy/aws-lambda-go-shim
- serverless.com
- Credit goes to @cristim - co-author who proposed the original idea via #2712 and made awesome code contributions
Easy Serverless Golang
By Yun Zhi Lin
Easy Serverless Golang
- 2,645