GumGum AMI Bakery
TL;DR: Packer + checksums = 👍
By Corey Gale
meetup.com/West-LA-DevOps
About Me
- Corey Gale
- Senior DevOps Engineer @ GumGum
- Studied Mechatronics Engineering
- 9 years of experience in software
- 6 years in DevOps
- Telecom, Cloud Finance, Education, Travel, Consumer Electronics, Advertising
Project Goals
- Define AMIs 100% in code
- Use Hashicorp Packer
- Dependable
-
-debug
- Idempotent builds
- AMIs are only re-built when their definition changes
- Solution must be CI-friendly
Problem
- Packer builds are not idempotent
- Re-building the same code twice = 2 AMIs
- Makes integrating Packer into your CI challenging
Solution
- Introducing the GumGum AMI Bakery!
- Ships as a Docker image
- You tell the AMI Bakery where your AMI config code lives
- The AMI Bakery will then re-build your AMI only if your AMI config code changes
GumGum AMI Bakery
Packer JSON
AMI config paths
AMI ID
docker pull mechtron/ami-bakery
File Checksums
- Digital signatures for files
- Our AMI Bakery uses SHA1 checksums
- Returns a 40 character hexidecimal value for files or strings of any size
- If a file changes, so will its SHA1 checksum
AMI Bakery Algorithm
Get list of all
config files
Sort list
Calculate SHA1 of each file
Calculate SHA1 of list of file SHA1s
= ami_config_checksum
Does an
AMI exist with that
ami_config_checksum
?
Build new AMI
Return existing AMI
No
Yes
Before The AMI Bakery 😥
- AMI IDs were hard-coded
- Required a manual code update
- ... which humans sometimes forget to do
- ... which lead to AMIs getting stale
WARNING:
Contains manually-pinned AMI IDs
After 😎
- AMI Bakery stage inserted
- If the AMI DNE, it is built
- If it exists, the AMI's ID is returned
- Terraform gets the AMI ID via CLI parameter
New stage
Updated stage
Demo
Advantages
- AMI updates are simple:
- Update the AMI config code
- That's it.
- No external state
- AMI config checksums remain the same between branch changes = faster builds
Disadvantages
- A little invasive
- Need to update existing Terraform code to look up AMIs dynamically
- But maybe that's a good thing?
AMI Bakery Code
-
https://github.com/mechtron/ami-bakery
- Includes an example integration
- sample-app sub-directory
WLAD Tip #0002
coreygale@gmail.com
Question: "what's using all the disk space on this Debian system?"
Answer: apt install ncdu && ncdu /
WLAD Tip #0002
coreygale@gmail.com
Example Pipeline Integration
docker run --rm -t \
-e AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION \
-e AWS_SUBNET_ID=$AWS_SUBNET_ID \
-e PACKER_JSON_PATH=`pwd`/example-app/ami/packer/ami.json \
-e AMI_DEFINITION_DIRS=`pwd`/example-app/ami/ \
--mount type=bind,source=`pwd`,target=`pwd` \
--mount type=bind,source=$HOME/.aws/,target=/root/.aws \
docker.io/mechtron/ami-bakery:latestAMI Bakery: West LA DevOps Edition
By Corey Gale
AMI Bakery: West LA DevOps Edition
Packer + checksums = win
