Building a Serverless HipChat plugin

How we emulated s3, DynamoDB and API Gateway to create a truly local development environment

http://ape.gs/sls-syd-2

About Me

Elliott Spira - Code @ GorillaStack

 

Twitter: @ElliottSpira | @GorillaStack
GitHub: @em0ney | @GorillaStack
Website: gorillastack.com

 

 

Why This Project

  • Partnership with AWS and Atlassian
  • Released for AtlasCamp in Barcelona
  • GorillaStack's commitment to Open Source

 

Disclaimer!

sls project 0

hipchat project 0

Problems I Hit

  • I want environment specific configuration (beyond environment variables)
  • serverless has no solution for persistence OOB
  • my solution for persistence needed a local environment
  • 'serverless-client' has no dev env

Environment Specific Config

{
  "dev": {
    "logLevel": "debug",
    "host": "http://dfe07aaf.ngrok.io",
    "staticAssetsHost": "http://localhost:8010",
    "useDynamoDBLocal": true,
    "dynamoDBLocalURL": "http://localhost:8000",
    "maxJWTTokenAge": 86400
  },
  "beta": {
    "logLevel": "info",
    "host": "https://3evvaltekb.execute-api.ap-northeast-1.amazonaws.com/beta",
    "staticAssetsHost": "http://serverless-hipchat-connect-client.beta.ap-northeast-1.s3-website-ap-northeast-1.amazonaws.com",
    "maxJWTTokenAge": 86400
  },
  "prod": {
    "host": "https://3evvaltekb.execute-api.ap-northeast-1.amazonaws.com/prod",
    "staticAssetsHost": "http://serverless-hipchat-connect-client.prod.ap-northeast-1.s3-website-ap-northeast-1.amazonaws.com",
    "logLevel": "warn",
    "maxJWTTokenAge": 900
  }
}
const getConfigurationForServerlessStage = (data) => {
  let jsonData = JSON.parse(data);
  return jsonData[process.env.SERVERLESS_STAGE];
};

const readFile = (file) => {
  return fs.readFileSync(file, { encoding: FILE_ENCODING });
};

Persistence

DynamoDB

(Obvious Choice)

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "The AWS CloudFormation template for this Serverless application's resources outside of Lambdas and Api Gateway",
  "Resources": {
    ...
    "DynamoDBAccessTokenTable": {
      "Type": "AWS::DynamoDB::Table",
      "DependsOn": "DynamoDBInstallationTable",
      "Properties": {
        "AttributeDefinitions": [
          {
            "AttributeName": "oauthId",
            "AttributeType": "S"
          }
        ],
        "KeySchema": [
          {
            "AttributeName": "oauthId",
            "KeyType": "HASH"
          }
        ],
        "ProvisionedThroughput": {
          "ReadCapacityUnits": 3,
          "WriteCapacityUnits": 1
        }
      }
    }
  },
  "Outputs": {
    ...
    "InstallationTableName": {
      "Description": "Name of the InstallationTable created by CloudFormation",
      "Value": {
        "Ref": "DynamoDBInstallationTable"
      }
    }
  }
}
  "environment": {
    "SERVERLESS_PROJECT": "${project}",
    "SERVERLESS_STAGE": "${stage}",
    "SERVERLESS_REGION": "${region}",
    "INSTALLATION_TABLE": "${installationTableName}"
  }

Super cool discovery!

CF outputs are available!

DynamoDBLocal

(Obvious Choice?)

Setup DynamoDBLocal

  1. Install Java
  2. Download DynamoDBLocal
  3. `java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar`
  4. When running CLI or API commands, set the --endpoint to be localhost:8000
$ java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar
$ aws dynamodb list-tables --endpoint-url http://localhost:8000 --region <your-region>

create_dynamodb_local_tables.sh

#!/bin/bash

for i in "$@"
do
  case $i in
    -e=*|--region=*)
      REGION="${i#*=}"
      # echo "--region was specified, parameter $REGION" >&2
      shift
      ;;
  esac
done

if [[ -z $REGION ]]; then
  echo "No '--region=...' argument specified.  Exiting..."
  exit 1
fi

REGION_MISSING_DASHES=`echo $REGION | tr -d '-'`
VARIABLE_FILE="./_meta/variables/s-variables-dev-${REGION_MISSING_DASHES}.json"
INSTALLATION_TABLE=`node -pe 'JSON.parse(process.argv[1]).installationTableName' "$(cat ${VARIABLE_FILE})"`
ACCESS_TOKEN_TABLE=`node -pe 'JSON.parse(process.argv[1]).accessTokenTableName' "$(cat ${VARIABLE_FILE})"`

if [[ -z $INSTALLATION_TABLE || -z $ACCESS_TOKEN_TABLE ]]; then
  echo "No INSTALLATION_TABLE or ACCESS_TOKEN_TABLE found in _meta/variables for region $REGION.  Exiting..."
  exit 1
fi

echo INSTALLATION_TABLE=$INSTALLATION_TABLE
echo ACCESS_TOKEN_TABLE=$ACCESS_TOKEN_TABLE

# Create the InstallationTable
aws dynamodb create-table --table-name $INSTALLATION_TABLE --attribute-definitions AttributeName="oauthId",AttributeType="S" --key-schema AttributeName="oauthId",KeyType="HASH" --provisioned-throughput ReadCapacityUnits=3,WriteCapacityUnits=1 --region $REGION --endpoint-url http://localhost:8000

# Create the AccessTokenTable
aws dynamodb create-table --table-name $ACCESS_TOKEN_TABLE --attribute-definitions AttributeName="oauthId",AttributeType="S" --key-schema AttributeName="oauthId",KeyType="HASH" --provisioned-throughput ReadCapacityUnits=3,WriteCapacityUnits=1 --region $REGION --endpoint-url http://localhost:8000

write your code to support a local dev environment

{
  "dev": {
    "logLevel": "debug",
    "host": "http://dfe07aaf.ngrok.io",
    "staticAssetsHost": "http://localhost:8010",
    "useDynamoDBLocal": true,
    "dynamoDBLocalURL": "http://localhost:8000",
    "maxJWTTokenAge": 86400
  },
  ...
}

'serverless-client' has no dev env

what is my local s3?

A simple http server should suffice really

python -m SimpleHTTPServer 8010

But what about environment variables and configuration...

client/

client/dist

client/src

<!doctype html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>${name}</title>
    <link rel="shortcut icon" 
        type="image/png" 
        href="http://abotars.hipch.at/bot/${key}.png">
    ...
  </head>
  ...
</html>
<!doctype html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>HipChat Plugin Dev</title>
    <link rel="shortcut icon" 
        type="image/png" 
        href="http://abotars.hipch.at/bot/12345.png">
    ...
  </head>
  ...
</html>

.gitignore

grunt to the rescue

module.exports = function (grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    copy: {
      dev: {
        expand: true,
        cwd: 'client/src/',
        src: '**',
        dest: 'client/dist/',
        options: {
          process: function (content, srcpath) {
            return substituteConfigAndDescriptorInTemplate(content, 'dev');
          }
        }
      },
      ...
    }
  })
}

grunt to the rescue

module.exports = function (grunt) {
  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    run: {
      babel: {
        exec: 'node node_modules/babel-cli/bin/babel --presets es2015 -d restApi/lib --watch restApi/src'
      },
      'babel-once': {
        exec: 'node node_modules/babel-cli/bin/babel --presets es2015 -d restApi/lib restApi/src'
      },
      's3-local': {
        exec: 'grunt copy:dev && pushd client/dist && python -m SimpleHTTPServer 8010 && popd'
      },
      'create-local-dynamodb-tables': {
        exec: 'bash create_dynamo_db_local_tables.sh',
        options: {
          passArgs: [
            'region'
          ]
        }
      }
    }
  });

Questions?

JOBS

We are looking!

Come speak to me!

Building a Serverless HipChat plugin

By em0ney

Building a Serverless HipChat plugin

  • 1,208