Serverless
The "Middy" way

—

Workshop
 loige.link/middy-way 

Community Day Dublin
October , 1st 2019

Luciano Mammino
@loige

Wi-Fi:

DC_Conference

October-2019

👋 Hello, I am Luciano!

🇮🇹

🇮🇪

🇺🇸

Cloud Architect

Blog: loige.co

Twitter: @loige

GitHub: @lmammino 

loige.link/middy-way

@loige

💬
Let's chat on:

tlk.io/middy-way

@loige

What is serverless

Compute as functions (FaaS)

Event-based model

@loige

Why serverless is good

Focus on business logic

Scalability

Pay per usage model (compute time * memory)

Managed infrastructure

Forces you to think micro-services

@loige

My Serverless experience

Open source

Enterprise

  • Trading, Billing engine, Market data aggregator solutions (ESB)
  • Big data pipeline to make network metadata searchable (Vectra.ai)
  • Organised various workshops around Europe (Serverlesslab.com)

@loige

Let's get this party started

🎉

@loige

Requirements!
🤷‍♂️

@loige

A nice laptop

@loige

An AWS account

 

aws.amazon.com/free

@loige

👸
Have an admin user
in your AWS account
 

loige.link/aws-admin-user

@loige

💻
install the AWS CLI
 

aws.amazon.com/cli

@loige

💻🤩
configure the AWS CLI
 

loige.link/aws-cli-config

@loige

Verify

aws sts get-caller-identity
{
    "Account": "123456789012",
    "UserId": "ABCDEFGHIJKLMNOPQRSTU",
    "Arn": "arn:aws:iam::123456789012:user/super-mario"
}

@loige


Install Git and Node.js

 

git-scm.com
nodejs.org 

@loige

🤝

You are ready!

@loige

🤔
What are we going to build?

@loige

@loige

A realtime dashboard for public transport!

loige.link/middy-way-starter

  • Read... the README! 😅
  • Clone the project locally
  • Deploy it to your AWS account
  • Play with the APIs

I need staaarz! 🤩

@loige

⏱ 20 mins warm up 🏃‍♀️


1. Use the APIs to create a dashboard that contains:

  • An IrishRail station widget
  • A LUAS stop widget
  • A DublinBus stop widget

(there are utility scripts to list all the stops and stations, check the README)

 

2. Have a look at the code

If you find bugs, please send a PR or report an issue 😇

@loige

AWS Lambda & Middy

❤️

@loige

The problem with Lambdas

@loige

exports.myLambda = function (
    event,
    context,
    callback
) {

  // get input from event and context

  // use callback to return output or errors

}

Anatomy of a Node.js lambda on AWS

@loige

(event, context, callback) => {
  // decrypt environment variables with KMS
  // deserialize the content of the event
  // validate input, authentication, authorization
  
  // REAL BUSINESS LOGIC
  // (process input, generate output)
  
  // validate output
  // serialize response
  // handle errors
}

A typical "REAL" Lambda function

LOTS of BOILERPLATE 😓

@loige

The solution

@loige

npm install @middy/core

Note: using version 1.0.0 preview (alpha)

Give it moar love 😍

@loige

Usage

const middy = require('@middy/core')
const { middleware1, middleware2, middleware3 } = require('some-middlewares')

const originalHandler = (event, context, callback) => {
  /* your business logic */
}

const handler = middy(originalHandler)

handler
  .use(middleware1())
  .use(middleware2())
  .use(middleware3())

module.exports = { handler }

1. define handler

2. "middify"  the handler

3. attach middlewares

4. export "middyfied" handler

@loige

const middy = require('@middy/core')
const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser')
const validator = require('@middy/validator')
const httpErrorHandler = require('@middy/http-error-handler')

const processPaymentHandler = (event, context, callback) => {
  const { 
    creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount
  } = event.body
  
  // do stuff with this data ...
  
  return callback(null,
    { result: 'success', message: 'payment processed correctly'}
  )
}

const inputSchema = {
 // define validation schema here ...
}

const handler = middy(processPaymentHandler)
  .use(urlEncodedBodyParser())
  .use(validator(inputSchema))
  .use(httpErrorHandler())

module.exports = { handler }

Handler

Attach middlewares

Export enhanced handler

@loige

Why?

  • Simplify code
  • Reusability
    • input parsing
    • input & output validation
    • output serialization
    • error handling
    • ...
  • Focus (even) MORE on business logic

@loige

How it works

@loige

Execution order

  1. middleware1 (before)
  2. middleware2 (before)
  3. middleware3 (before)
  4. handler
  5. middleware3 (after)
  6. middleware2 (after)
  7. middleware1 (after)

@loige

When an error happens...

  • Flow is stopped
  • First middleware implementing `onError` gets control
  • It can choose to handle the error, or delegate it to the next handler
  • If the error is handler a response is returned
  • If the error is not handled the execution fails reporting the unhandled error

@loige

Writing a middleware

const myMiddleware = (config) => {
  
  // might set default options in config
  
  return ({
    before: (handler, next) => {
      // might read options from `config`
    },
    after: (handler, next) => {
      // might read options from `config`
    },
    onError: (handler, next) => {
      // might read options from `config`
    }
  })
}

module.exports = myMiddleware

@loige

Inline middlewares

const middy = require('@middy/core')

const handler = middy((event, context, callback) => {
  // do stuff
})

handler.before((handler, next) => {
  // do something in the before phase
  next()
})

handler.after((handler, next) => {
  // do something in the after phase
  next()
})

handler.onError((handler, next) => {
  // do something in the on error phase
  next()
})

module.exports = { handler }

@loige

It supports async handlers!

const middy = require('@middy/core')

const handler = middy(
  async (event, context) => {
    // do stuff
    return { some: 'response' }
  }
)

module.exports = { handler }

@loige

Exercise 0: let's middify! 🛵

In the next exercises we will need the full power of middy, so to get started let's just "middify" all our handlers

—

Note: handlers are defined in "src/handlers", and they get used in "src/handler.js". You can either middify every single handler file or middify them when used in src/handler.js. In the first case, make sure you middify the actual handler function and not the "handler factory".

 

(a solution)

@loige

Exercise 1: body parsing 🤗

We are manually deserializing the JSON input in our APIs.

We can simplify the code a bit by using the http-json-body-parser middleware.

—

 

Install @middy/http-json-body-parser and remove all the instances of JSON.parse() in our handlers.

 

(a solution)

@loige

What happens when one of our handlers crashes? We are actually not even dealing with errors right now... by using the http-error-handler middleware we will be able to handle all HTTP errors and return codes consistently.

—

 

Install @middy/http-error-handler and apply that to all our handlers.

 

(a solution)

Exercise 2: what if we fail? 😓

@loige

We want to make sure that data is validated before being written to the database. We can use the validator middleware for this task!
—

Install @middy/validator and apply that to all our POST API handlers. You can use jsonschema.net to define your schemas or, if you are struggling, you can find some schemas ready-made for you here :)

 

Note: The validator should be added after the json-body-parser, so you can validate individual fields of the input data.

 

(a solution)

Exercise 3: validate! 👩‍🏫

@loige

We have our dashboards fully working, but consuming them in JSON is definitely not the friendliest experience! It would be much better to do in a properly rendered HTML so that we could easily visualize them on our mobile device!

—

You can implement this in (at least) 2 ways :

(a CORS solution - a CodeSandbox React UI)

Exercise 4: mobile rendering 📱

@loige

Did you notice that our APIs are fully open? Knowing the endpoint and the id, anyone can basically change or delete the dashboards! That's unacceptable!

—

Find a way to secure the APIs.

One easy way could be to create a secret random token when a dashboard is created the first time. Then you can create a custom middleware that checks the secret every time someone is trying to modify or delete that dashboard!

 

Note: if you feel fancy you can use JWT tokens, in such case you could take inspiration from the jwt-auth middleware!

Exercise 5: better safe than sorry 🤠

@loige

Sometimes our APIs are very slow. They take literally seconds to answer. This is because of a common serverless issue known as "cold start problem". If you are running for your train every morning, this is unacceptable! We should fix this...

—

 

Use the warmup middleware to mitigate the cold start issue

 

Note: make sure to also setup the proper schedule event for every API. 

2

@loige

Exercise 6: stop the cold! 🥶

Some of your Java friends would love to use these APIs to build a shiny SWING UI... The problem is that they don't really like JSON and they would rather have a good old XML-based API. You might think that this is actually their problem, but also, you don't want to lose your friends...

—

 

Use the http-content-negotiation middleware to provide responses in XML for accept "application/xml".

 

Note: You could use a library like json2xml to automate the conversion of JavaScript objects to XML.

@loige

Exercise 7: legacy friendship 👴

In summary

  • Serverless is cool, it helps you to build apps quickly and with a greater focus on business logic, rather than on infrastructure!

  • Middy helps you to keep focusing on your business logic first

  • You can add extra behaviours with very minimal changes to your core logic by introducing dedicated middlewares

  • You can easily share common functionality through middlewares

If you like Middy

  • Use it 😜

  • Give feedback

  • Contribute (I am looking for co-maintainers)

  • Version 1.0 (stable) coming soon 🤞

Thank you

@loige

Credits

Cover Image by TotumRevolutum from Pixabay

 

A special thank you to all the amazing Middy users and contributors!

 

Thanks to @StefanoAbalsamo for reviewing these slides and to @organicdelight for finding lots of bugs!

@loige