Building an Empire

Gabor Ratky
CTO at Secret Sauce Partners

Continuous delivery in AWS with Docker

Secret Sauce Partners

http://www.secretsaucepartners.com

empire |ˈemˌpī(ə)r|

noun
1 an extensive group of states or countries
under a single supreme authority

empire |ˈemˌpī(ə)r|

noun
1 an extensive group of apps or services
under a single supreme authority

2 a variety of apple
empire |also ämˈpi(ə)r|
adjective
denoting a style of [...] dress fashionable during the First or (less commonly) the Second Empire in France. 

The Secret Sauce Empire

SSP API

Fit Predictor API

Fit Predictor

Style Finder API

Style Finder

Style Finder Blackbox

Abba

Acme Clothing

Central

Feeds

Luigi's Toolbox

Mario

MongoDB

ZooKeeper

Kafka

Continuous Delivery 101

source: https://twitter.com/raffi/status/371070622685605888

@raffi "build. test. deploy. the circle of life."

Build and test every commit (CI)

Deploy any commit, branch, or tag

Monitor and scale running applications

Rapidly build and deploy new applications

Keep it simple and extensible

You don't need (Docker) to do any of this

DVCS

CI

PaaS

Why use Docker?

Why use Docker?

source: http://www.slideshare.net/shykes/docker-the-road-ahead/13

Build

Deploy

Scale

Monitor

Release

Test

Run

more: https://trello.com/b/lHHKwnSh/goals-principles-practices

CircleCI (Build, Test)

Use existing Docker-based infrastructure

Build Docker image in CircleCI using sspinc/docker-build

Run tests in a container (docker run <image> script/circleci-test)

Push image to Docker Hub if successful (sha1, branch, tag)

If you don't run your tests in a container, you're doing it wrong!

GitHub Flow (Release)

Anyone can cut a release on master

Follow semantic versioning to communicate changes

Tag commit and push release tag to GitHub

more: https://guides.github.com/introduction/flow/

Procfile (Run)

​

Declare commands of your application

web: bundle exec puma -C ./config/puma.rb
worker: bundle exec sidekiq -C ./config/sidekiq.yml
product: bundle exec rake service:product

Empire (Deploy, Scale)

Thin layer on top of ECS, ELB, Route 53

Deploy Docker images from Docker Hub (ECS)

Scale processes (ECS)

Rolling upgrades (ECS)

Rollback releases (ECS)

Load balance internal/external services (ELB, Route 53)

more: https://github.com/remind101/empire

A self-hosted PaaS built on Docker & Amazon ECS

Logspout (Monitor, Log)

​

Log routing for Docker container logs (gliderlabs/logspout)

Runs on all our instances

Write logs to Kafka topics (sspinc/logspout-kafka)

Structured logs and metrics (sspinc/logr)

logger.event('feed_received', partner: 'abercrombie', type: 'product')
      .monitored('Feed received', 'Even more explanation.')
      .metric('total_bytes', 23764237, type: 'counter')
      .info('Received feed')
{
  "logger": "feeds",
  "level": "info",
  "event": { "name": "feed_received", "partner": "abercrombie" },
  "metrics": [{ name: "total_bytes": 23764237 }],
  "message": "Received feed"
}

OMG LIVE DEMO

Thanks!

Questions?

We're hiring!
https://sspinc.workable.com

slides: https://slides.com/sspinc/building-an-empire

Made with Slides.com