Cluster managers, orchestrators and schedulers
Gabor Ratky
CTO at Secret Sauce Partners
Secret Sauce Partners
Glossary
- Cluster manager: software that manages more than one computer [to dispatch work]
- Orchestration: automated arrangement, coordination, and management of complex computer systems, middleware and services
- Scheduling: determine where work should be performed according to some criteria
- Automation: ensure that a task can be repeatedly performed with minimal input
Do I need one?
NO
Do I need one?
GOD NO
Do I need one?
GOD NO
Do I want my system to be more complex?
Complexity monster
Complexity monster
Problem
Complexity
Complexity monster
Problem
Solution
Complexity
Overengineering
Complexity monster
Problem
Solution
Complexity
Essential complexity
Accidental complexity
Complexity monster
Problem
Solution
Complexity
Complexity monster
Problem
Solution
Complexity
Complexity monster
Problem
Solution
Complexity
Technical debt
Complexity monster
Problem
Solution
Complexity
What problem do I have?
Deployment
What problem do I have?
High availability
Deployment
What problem do I have?
Service discovery
High availability
Deployment
What problem do I have?
Utilization
Cost
Service discovery
High availability
Deployment
$$$
SSP: 11%
Industry avg: <10%
Google: 60%
Containers
Binpacking
Docker Swarm
- Docker 1.12: Now With Built-In Orchestration!
- Developed by Docker Inc, the Docker company
- Multi-host, multi-container orchestration
- Same CLI, same API with new concepts (Node & Service)
- Built-in Raft consensus and distributed state store
- Built-in routing mesh and service discovery
- Infrastructure integration in private beta
- Docker for AWS
- Docker for Azure
- Web UI available through Docker Cloud
Nomad
- Developed by Hashicorp
- Provides cluster management and scheduling only
- Operationally simple with single binary agent
- Natively integrates with Consul for service discovery
- Can run virtualized, containerized or standalone workloads
- No built-in load balancing, but fabio is great
- No autoscaling, no persistent volumes
- Missing features are on the roadmap, but only when using Atlas (i.e. Web UI)
Kubernetes
- Initially developed by Google based on Borg, donated to Cloud Native Computing Foundation
- Feature-rich container orchestration consisting of many loosely coupled components and an active community
- More complex than others, solves more problems
- Stores state in etcd
- Conceptually simple networking model
- Supports autoscaling and persistent volumes
- Exceptional documentation and tooling
- Built-in Web UI
Many others
- AWS ECS + Elastic Beanstalk
- AWS ECS + Empire
- AWS ECS + Convox
- CoreOS (fleet)
- Mesos + Marathon
- Mesosphere DC/OS
Every solution solves a different set of problems!
"Everything should be made as simple as possible, but not simpler." -Albert Einstein (supposedly)
Which one to use?
- Wait for Docker Swarm if you want to use an existing, familiar technology that ships with batteries included (watch Dockercon Keynote Day 1)
- Use Nomad if you want an architecturally simple scheduler to run workloads (watch 12-factor apps and the HashiStack by Kelsey Hightower)
- Use Kubernetes if you want a feature-rich and mature platform with a large community and adoption (watch A technical overview of Kubernetes by Brendan Burns)
Gotta watch 'em all, gotta try 'em all!
gabor@secretsaucepartners.com