Mesos(phere)

Sfeir Info Days, March 2016

Christoph Meier, Sfeir Benelux

Change Log:

  • 2016-02-09: initial draft
  • 2016-03-04: add mesosphere

Apache Mesos

Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.

  • Written in C++ (10k lines)
  • Runs on Linux (kernel >= 3.10)
  • API for C++, Java, Python

→ Create apps (frameworks) running on Mesos

  • scheduling
  • isolation (LXC, Docker)
  • distributed
  • fault tolerant (Zookeeper)

Summary

  • solid foundation for distributed apps
  • resource offers empower frameworks
  • frameworks can share resources (data locality)

Chronos

  • cron framework for Mesos
  • schedule jobs
  • native support for Docker
  • asynchronous callbacks
  • jobs dependency chains

Marathon

  • init framework for Mesos
  • run services & keep em alive
  • native support for Docker
  • run apps from resources
  • custom scripts
  • health checks
  • resource constraints
  • dependencies
  • ...
{
    "id": "simple-docker", 
    "container": {
      "docker": {
        "image": "busybox"
      }
    },
    "cmd": "echo hello from docker",
    "cpus": 0.2,
    "mem": 32.0,
    "instances": 2
}
{
  "id": "/docker/registry",
  "instances": 1,
  "cpus": 0.5,
  "mem": 1024.0,
  "disk": 128,
  "container": {
    "docker": {
      "type": "DOCKER",
      "image": "registry:latest",
      "network": "BRIDGE",
      "parameters": [],
      "portMappings": [
        { "containerPort": 5000, "hostPort": 0, "protocol": "tcp", "servicePort": 5000 }
      ]
    },
    "volumes": [
      {
        "hostPath": "/local/path/to/store/packages",
        "containerPath": "/storage",
        "mode": "RW"
      }
    ]
  },
  "env": {
    "SETTINGS_FLAVOR": "local",
    "STORAGE_PATH": "/storage"
  },
  "ports": [ 0 ]
}
{
  "id": "toggle",
  "cmd": "python toggle.py $PORT0",
  "cpus": 0.2,
  "disk": 0.0,
  "healthChecks": [
    {
      "protocol": "HTTP",
      "path": "/health",
      "portIndex": 0,
      "gracePeriodSeconds": 5,
      "intervalSeconds": 10,
      "timeoutSeconds": 10,
      "maxConsecutiveFailures": 3
    }
  ],
  "instances": 2,
  "mem": 32.0,
  "ports": [0],
  "uris": ["http://downloads.mesosphere.com/misc/toggle.tgz"]
}

Summary

meta-framework to run almost anything on Mesos without having to implement your own framework

Aurora

  • alternative to chronos/marathon
  • Apache Project (powered by Twitter)
  • Python DSL
  • more or less same features
import os

hello_world_process = Process(name = 'hello_world', cmdline = 'echo hello world')

hello_world_task = Task(
  resources = Resources(cpu = 0.1, ram = 16 * MB, disk = 16 * MB),
  processes = [hello_world_process])

hello_world_job = Job(
  cluster = 'cluster1',
  role = os.getenv('USER'),
  task = hello_world_task)

jobs = [hello_world_job]
# copy hello_world.py into the local sandbox
install = Process(
  name = 'fetch_package',
  cmdline = 'cp %s . && echo %s && chmod +x hello_world.py' % (pkg_path, pkg_checksum))

# run the script
hello_world = Process(
  name = 'hello_world',
  cmdline = 'python -u hello_world.py')

# describe the task
hello_world_task = SequentialTask(
  processes = [install, hello_world],
  resources = Resources(cpu = 1, ram = 1*MB, disk=8*MB))

jobs = [
  Service(cluster = 'devcluster',
          environment = 'devel',
          role = 'www-data',
          name = 'hello_world',
          task = hello_world_task)
]

Mesosphere

  • commercial support for Mesos
  • owner of Marathon
  • contribute to Chronos
  • consulting
  • training

Datacenter Operating System (DCOS):

  • ui & cli for admins
  • pre-configured apps
  • enterprise features:
    • auth (ldap)

DEMO

Zombies Tracker

Thank you! Questions?

Apache Mesos

By Christoph Meier

Apache Mesos

Sfeir Info Days - Apache Mesos

  • 144