Mesos(phere)
Sfeir Info Days, March 2016
Christoph Meier, Sfeir Benelux
Change Log:
- 2016-02-09: initial draft
- 2016-03-04: add mesosphere
Apache Mesos
Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.
- Written in C++ (10k lines)
- Runs on Linux (kernel >= 3.10)
- API for C++, Java, Python
→ Create apps (frameworks) running on Mesos
- scheduling
- isolation (LXC, Docker)
- distributed
- fault tolerant (Zookeeper)
Summary
- solid foundation for distributed apps
- resource offers empower frameworks
- frameworks can share resources (data locality)
Chronos
- cron framework for Mesos
- schedule jobs
- native support for Docker
- asynchronous callbacks
- jobs dependency chains
Marathon
- init framework for Mesos
- run services & keep em alive
- native support for Docker
- run apps from resources
- custom scripts
- health checks
- resource constraints
- dependencies
- ...
{
"id": "simple-docker",
"container": {
"docker": {
"image": "busybox"
}
},
"cmd": "echo hello from docker",
"cpus": 0.2,
"mem": 32.0,
"instances": 2
}
{
"id": "/docker/registry",
"instances": 1,
"cpus": 0.5,
"mem": 1024.0,
"disk": 128,
"container": {
"docker": {
"type": "DOCKER",
"image": "registry:latest",
"network": "BRIDGE",
"parameters": [],
"portMappings": [
{ "containerPort": 5000, "hostPort": 0, "protocol": "tcp", "servicePort": 5000 }
]
},
"volumes": [
{
"hostPath": "/local/path/to/store/packages",
"containerPath": "/storage",
"mode": "RW"
}
]
},
"env": {
"SETTINGS_FLAVOR": "local",
"STORAGE_PATH": "/storage"
},
"ports": [ 0 ]
}
{
"id": "toggle",
"cmd": "python toggle.py $PORT0",
"cpus": 0.2,
"disk": 0.0,
"healthChecks": [
{
"protocol": "HTTP",
"path": "/health",
"portIndex": 0,
"gracePeriodSeconds": 5,
"intervalSeconds": 10,
"timeoutSeconds": 10,
"maxConsecutiveFailures": 3
}
],
"instances": 2,
"mem": 32.0,
"ports": [0],
"uris": ["http://downloads.mesosphere.com/misc/toggle.tgz"]
}
Summary
meta-framework to run almost anything on Mesos without having to implement your own framework
Aurora
- alternative to chronos/marathon
- Apache Project (powered by Twitter)
- Python DSL
- more or less same features
import os
hello_world_process = Process(name = 'hello_world', cmdline = 'echo hello world')
hello_world_task = Task(
resources = Resources(cpu = 0.1, ram = 16 * MB, disk = 16 * MB),
processes = [hello_world_process])
hello_world_job = Job(
cluster = 'cluster1',
role = os.getenv('USER'),
task = hello_world_task)
jobs = [hello_world_job]
# copy hello_world.py into the local sandbox
install = Process(
name = 'fetch_package',
cmdline = 'cp %s . && echo %s && chmod +x hello_world.py' % (pkg_path, pkg_checksum))
# run the script
hello_world = Process(
name = 'hello_world',
cmdline = 'python -u hello_world.py')
# describe the task
hello_world_task = SequentialTask(
processes = [install, hello_world],
resources = Resources(cpu = 1, ram = 1*MB, disk=8*MB))
jobs = [
Service(cluster = 'devcluster',
environment = 'devel',
role = 'www-data',
name = 'hello_world',
task = hello_world_task)
]
Mesosphere
- commercial support for Mesos
- owner of Marathon
- contribute to Chronos
- consulting
- training
Datacenter Operating System (DCOS):
- ui & cli for admins
- pre-configured apps
- enterprise features:
- auth (ldap)
DEMO
Zombies Tracker
Thank you! Questions?
Apache Mesos
By Christoph Meier
Apache Mesos
Sfeir Info Days - Apache Mesos
- 144