resource sharing in the data center
@palmerabollo · 2015
Apache Mesos abstracts CPU, memory, storage, and ports away from individual machines.
The cluster looks like one big computer.
A master process that manages slave daemons running on each cluster node.
Frameworks are software systems that run tasks on these slaves
(examples: Hadoop, Spark, Jenkins, Cassandra, ElasticSearch, etc)
pluggable: cgroups, LXC, ...
CPU, memory, network, IO
DEMO. Mesos Portal
Local installation (vagrant): https://github.com/mesosphere/playa-mesos
Datacenter Operating System :|
http://mesosphere.com/
DEMO. Cluster on DigitalOcean
Mesos framework to launch long-running tasks
https://mesosphere.github.io/marathon/
https://github.com/mesosphere/marathon
DEMO. Marathon Portal & API
Simple Task from UI
echo `date`; sleep 5;
curl $MARATON_API/v2/apps
Health check (HTTP)
curl -X POST -H "Content-Type: application/json" $MARATHON_API/v2/apps -d@health-check.json
Docker Container
curl -X POST -H "Content-Type: application/json" $MARATHON_API/v2/apps -d@redis.json docker images, docker ps, ...
Haproxy
cat /etc/haproxy/haproxy.cfg (1 min auto refresh)
Low cost webapps in the cloud
PoC
one-click deploy
no usage = no resources reserved at all
deployments are slow
~90% of webapps are idle
DEMO. Deploy apps with a github hook
Application descriptor (package.json)
"flock": { "instances": 4, "cpus": 0.25, "mem": 32, "ports": 1 }, "checks": [ { "path": "/", "intervalSeconds": 15, "maxConsecutiveFailures": 3 } ]
DEMO. Autoscale apps
frontend http-in bind *:80 acl acl_flock_demo_1_0_1 hdr(host) -i flock_demo_1_0_1.flock.com use_backend flock-demo-1.0.1 if acl_flock_demo_1_0_1 backend flock-demo-1.0.1 balance leastconn http-request set-header X-Flock-App /flock-demo-1.0.1 server server_1 10.132.73.13:31915 check server server_2 10.132.73.14:31677 check server server_backup_1 10.132.73.14:31006 check backup backend flock-backup-1.0.0 balance leastconn server server_1 10.132.73.14:31006 check
flock-scaler: downscale + upscale: haproxy stats
http://$ANY_NODE_IP:9090/stats;csv
flock-backup: upscale (warm up): haproxy backup server
TODO