Serverless
The "Middy" way

Workshop
 loige.link/middy-way 

Community Day Dublin
October , 1st 2019

Luciano Mammino
@loige

Wi-Fi:

DC_Conference

October-2019

👋 Hello, I am Luciano!

🇮🇹

🇮🇪

🇺🇸

Cloud Architect

Blog: loige.co

Twitter: @loige

GitHub: @lmammino 

💬
Let's chat on:

tlk.io/middy-way

What is serverless

Compute as functions (FaaS)

Event-based model

Why serverless is good

Focus on business logic

Scalability

Pay per usage model (compute time * memory)

Managed infrastructure

Forces you to think micro-services

My Serverless experience

Open source

Enterprise

  • Trading, Billing engine, Market data aggregator solutions (ESB)
  • Big data pipeline to make network metadata searchable (Vectra.ai)
  • Organised various workshops around Europe (Serverlesslab.com)

Let's get this party started

🎉

Requirements!
🤷‍♂️

A nice laptop

An AWS account

 

aws.amazon.com/free

👸
Have an admin user
in your AWS account
 

loige.link/aws-admin-user

💻
install the AWS CLI
 

aws.amazon.com/cli

💻🤩
configure the AWS CLI
 

loige.link/aws-cli-config

Verify

aws sts get-caller-identity
{
    "Account": "123456789012",
    "UserId": "ABCDEFGHIJKLMNOPQRSTU",
    "Arn": "arn:aws:iam::123456789012:user/super-mario"
}


Install Git and Node.js

 

git-scm.com
nodejs.org
 

🤝

You are ready!

🤔
What are we going to build?

A realtime dashboard for public transport!

  • Read... the README! 😅
  • Clone the project locally
  • Deploy it to your AWS account
  • Play with the APIs

I need staaarz! 🤩

⏱ 20 mins warm up 🏃‍♀️


1. Use the APIs to create a dashboard that contains:

  • An IrishRail station widget
  • A LUAS stop widget
  • A DublinBus stop widget

(there are utility scripts to list all the stops and stations, check the README)

 

2. Have a look at the code

If you find bugs, please send a PR or report an issue 😇

AWS Lambda & Middy

❤️

The problem with Lambdas

exports.myLambda = function (
    event,
    context,
    callback
) {

  // get input from event and context

  // use callback to return output or errors

}

Anatomy of a Node.js lambda on AWS

(event, context, callback) => {
  // decrypt environment variables with KMS
  // deserialize the content of the event
  // validate input, authentication, authorization
  
  // REAL BUSINESS LOGIC
  // (process input, generate output)
  
  // validate output
  // serialize response
  // handle errors
}

A typical "REAL" Lambda function

LOTS of BOILERPLATE 😓

The solution

npm install @middy/core

Note: using version 1.0.0 preview (alpha)

Give it moar love 😍

Usage

const middy = require('@middy/core')
const { middleware1, middleware2, middleware3 } = require('some-middlewares')

const originalHandler = (event, context, callback) => {
  /* your business logic */
}

const handler = middy(originalHandler)

handler
  .use(middleware1())
  .use(middleware2())
  .use(middleware3())

module.exports = { handler }

1. define handler

2. "middify"  the handler

3. attach middlewares

4. export "middyfied" handler

const middy = require('@middy/core')
const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser')
const validator = require('@middy/validator')
const httpErrorHandler = require('@middy/http-error-handler')

const processPaymentHandler = (event, context, callback) => {
  const { 
    creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount
  } = event.body
  
  // do stuff with this data ...
  
  return callback(null,
    { result: 'success', message: 'payment processed correctly'}
  )
}

const inputSchema = {
 // define validation schema here ...
}

const handler = middy(processPaymentHandler)
  .use(urlEncodedBodyParser())
  .use(validator(inputSchema))
  .use(httpErrorHandler())

module.exports = { handler }

Handler

Attach middlewares

Export enhanced handler

Why?

  • Simplify code
  • Reusability
    • input parsing
    • input & output validation
    • output serialization
    • error handling
    • ...
  • Focus (even) MORE on business logic

How it works

Execution order

  1. middleware1 (before)
  2. middleware2 (before)
  3. middleware3 (before)
  4. handler
  5. middleware3 (after)
  6. middleware2 (after)
  7. middleware1 (after)

When an error happens...

  • Flow is stopped
  • First middleware implementing `onError` gets control
  • It can choose to handle the error, or delegate it to the next handler
  • If the error is handler a response is returned
  • If the error is not handled the execution fails reporting the unhandled error

Writing a middleware

const myMiddleware = (config) => {
  
  // might set default options in config
  
  return ({
    before: (handler, next) => {
      // might read options from `config`
    },
    after: (handler, next) => {
      // might read options from `config`
    },
    onError: (handler, next) => {
      // might read options from `config`
    }
  })
}

module.exports = myMiddleware

Inline middlewares

const middy = require('@middy/core')

const handler = middy((event, context, callback) => {
  // do stuff
})

handler.before((handler, next) => {
  // do something in the before phase
  next()
})

handler.after((handler, next) => {
  // do something in the after phase
  next()
})

handler.onError((handler, next) => {
  // do something in the on error phase
  next()
})

module.exports = { handler }

It supports async handlers!

const middy = require('@middy/core')

const handler = middy(
  async (event, context) => {
    // do stuff
    return { some: 'response' }
  }
)

module.exports = { handler }

Exercise 0: let's middify! 🛵

In the next exercises we will need the full power of middy, so to get started let's just "middify" all our handlers

Note: handlers are defined in "src/handlers", and they get used in "src/handler.js". You can either middify every single handler file or middify them when used in src/handler.js. In the first case, make sure you middify the actual handler function and not the "handler factory".

 

(a solution)

Exercise 1: body parsing 🤗

We are manually deserializing the JSON input in our APIs.

We can simplify the code a bit by using the http-json-body-parser middleware.

 

Install @middy/http-json-body-parser and remove all the instances of JSON.parse() in our handlers.

 

(a solution)

What happens when one of our handlers crashes? We are actually not even dealing with errors right now... by using the http-error-handler middleware we will be able to handle all HTTP errors and return codes consistently.

 

Install @middy/http-error-handler and apply that to all our handlers.

 

(a solution)

Exercise 2: what if we fail? 😓

We want to make sure that data is validated before being written to the database. We can use the validator middleware for this task!

Install @middy/validator and apply that to all our POST API handlers. You can use jsonschema.net to define your schemas or, if you are struggling, you can find some schemas ready-made for you here :)

 

Note: The validator should be added after the json-body-parser, so you can validate individual fields of the input data.

 

(a solution)

Exercise 3: validate! 👩‍🏫

We have our dashboards fully working, but consuming them in JSON is definitely not the friendliest experience! It would be much better to do in a properly rendered HTML so that we could easily visualize them on our mobile device!

You can implement this in (at least) 2 ways :

(a CORS solution - a CodeSandbox React UI)

Exercise 4: mobile rendering 📱

Did you notice that our APIs are fully open? Knowing the endpoint and the id, anyone can basically change or delete the dashboards! That's unacceptable!

Find a way to secure the APIs.

One easy way could be to create a secret random token when a dashboard is created the first time. Then you can create a custom middleware that checks the secret every time someone is trying to modify or delete that dashboard!

 

Note: if you feel fancy you can use JWT tokens, in such case you could take inspiration from the jwt-auth middleware!

Exercise 5: better safe than sorry 🤠

Sometimes our APIs are very slow. They take literally seconds to answer. This is because of a common serverless issue known as "cold start problem". If you are running for your train every morning, this is unacceptable! We should fix this...

 

Use the warmup middleware to mitigate the cold start issue

 

Note: make sure to also setup the proper schedule event for every API. 

2

Exercise 6: stop the cold! 🥶

Some of your Java friends would love to use these APIs to build a shiny SWING UI... The problem is that they don't really like JSON and they would rather have a good old XML-based API. You might think that this is actually their problem, but also, you don't want to lose your friends...

 

Use the http-content-negotiation middleware to provide responses in XML for accept "application/xml".

 

Note: You could use a library like json2xml to automate the conversion of JavaScript objects to XML.

Exercise 7: legacy friendship 👴

In summary

  • Serverless is cool, it helps you to build apps quickly and with a greater focus on business logic, rather than on infrastructure!

  • Middy helps you to keep focusing on your business logic first

  • You can add extra behaviours with very minimal changes to your core logic by introducing dedicated middlewares

  • You can easily share common functionality through middlewares

If you like Middy

  • Use it 😜

  • Give feedback

  • Contribute (I am looking for co-maintainers)

  • Version 1.0 (stable) coming soon 🤞

Thank you

Credits

Cover Image by TotumRevolutum from Pixabay

 

A special thank you to all the amazing Middy users and contributors!

 

Thanks to @StefanoAbalsamo for reviewing these slides and to @organicdelight for finding lots of bugs!

Serverless, The Middy Way - Workshop

By Luciano Mammino

Serverless, The Middy Way - Workshop

An introduction workshop about Serverless on AWS using Middy, The stylish Node.js middleware engine for AWS Lambda!

  • 150
Loading comments...

More from Luciano Mammino