GumGum AMI Bakery

TL;DR: Packer + checksums = 👍

corey@gumgum.com

About GumGum

  • Computer Vision company located in Santa Monica
  • Advertising space
    • Context-aware in-image ads
    • Brand-safety technology
  • New verticals:
    • Sports (working with teams in the NBA and NHL)
    • Dental

Project Goals

  • Use Hashicorp Packer
    • Dependable
    • -debug
  • AMIs defined 100% in code
  • Idempotent builds
    • AMIs are only re-built when their definition changes
  • Solution must be CI-friendly

Problem

  • Packer builds are not idempotent
    • Re-building the same code twice = 2 AMIs
  • Makes integrating Packer into your CI challenging

Solution

  • Introducing the GumGum AMI Bakery Docker image!
  • You tell the AMI Bakery where your AMI config code lives
  • The AMI Bakery will then re-build your AMI only if your AMI config code changes

GumGum AMI Bakery

Packer JSON

AMI config paths

AMI ID

File Checksums

  • Digital signatures for files
    • Our AMI Bakery uses SHA1 checksums
  • Returns a 40 character hexidecimal value for files or strings of any size
  • If any part of a file changes, so will its SHA1 checksum

Our AMI Bakery Algorithm

Get list of all

config files

Sort list

Calculate SHA1 of each file

Calculate SHA1 of list of file SHA1s

= ami_config_checksum

Does an

AMI exist with that

ami_config_checksum

?

Build new AMI

Return existing AMI

No

Yes

Before The AMI Bakery 😥

  • AMIs were hard-coded
    • Required a manual code update
    • ... which humans sometimes forget to do
    • ... which lead to AMIs getting stale

After 😎

  • GumGum AMI Bakery stage inserted
    • If the AMI DNE, it is built
    • If it exists, the AMI's ID is returned
  • Terraform passed the AMI ID via CLI parameter

New stage

Updated stage

Demo

Advantages

  • AMI updates are simple:
    • Update the AMI config code
    • That's it.
  • No external state
  • AMI config checksums remain the same between branch changes = faster builds

Disadvantages

  • A little invasive
    • Need to update existing Terraform code to look up AMIs dynamically
    • But maybe that's a good thing?

GumGum AMI Bakery Code

  • https://github.com/mechtron/ami-bakery
  • Includes an example sample-app

Example Pipeline Integration

docker run --rm -t \
    -e AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION \
    -e AWS_SUBNET_ID=$AWS_SUBNET_ID \
    -e PACKER_JSON_PATH=`pwd`/example-app/ami/packer/ami.json \
    -e AMI_DEFINITION_DIRS=`pwd`/example-app/ami/ \
    --mount type=bind,source=`pwd`,target=`pwd` \
    --mount type=bind,source=$HOME/.aws/,target=/root/.aws \
    docker.io/mechtron/ami-bakery:latest

GumGum AMI Bakery

By Corey Gale

GumGum AMI Bakery

Packer + checksums = win

  • 889