Amazon ECS + Beyond
A Vision for Containers
- Cloud providers offer commodity infrastructure
- Platform operators harness the cloud to run anything
- Developers describe how their code should run, and ship to the platform
My Vision for Universe
What is Amazon ECS?
- ECR Registry
- ECS Tasks
- ECS Services
ECR Registry
ECS Tasks
ECS Services
ECS: The Good
- Works quite nicely!
- Supports blue/green deploys
- Fairly simple and flexible API for shipping containers
ECS: The Bad
- Logging is okay, i.e. can ship to CloudWatch
- ECS Instances have IAM Roles, but not Tasks, Services, or Containers
ECS: The Ugly
- ECS Services behind ELB map to the Instance port (not Container port), thus you cannot launch 2+ containers of the same service on the same instance
- To work around this, you have to roll your own solution
- Is this a dealbreaker?
Let's fix that!
The Solution
ECS Instance
- TL;DR: docker-gen to write container ip:port to etcd
docker-register
docker-register sets up a container running docker-gen
docker-gen dynamically generate a python script when containers are started and stopped. This generated script registers the running containers host IP and port in etcd with a TTL. It works in tandem with docker-discover which generates haproxy routes on the host to forward requests to registered containers
etcd
- distributed key/value store
- useful for tracking state of distributed systems
etcd example
HAProxy
- Routes traffic to the appropriate container ports
- Externally, uses subdomain routing
- Interally, uses confd to query etcd and rewrite haproxy.conf + safely reload
confd
- reads from etcd
- writes to haproxy.conf
OMG!
Overwhelmed yet?
Terraform
3 Necessary Modules:
- haproxy_cluster
- ecr_cluster
- ecr_service