Consul

Alex Giurgiu (alex@giurgiu.io)

using the k/v store and health checks to build a leader election system

Goals for today

  • Introduce Consul and what problems it tries to solve
  • Build(study) a practical example that leverages Consul to implement a leader election system

Start setup

SoftLayer API credentials: https://pastee.org/xjr9v

Code: github.com/nustiueudinastea/consul_workshop_devopsdays

Fill in credentials in bootstrap_machines/bootstrap_softlayer.py

Run bootstrap_softlater.py from your machine or Vagrant

Make sure that your private key is available for Ansible

pip install -r requirements.txt

What is consul?

A distributed and highly available K/V store

(source: consul.io)

About Consul

  • similar to etcd and zookeeper,
  • made by HashiCorp(Vagrant and Packer)
  • uses RAFT for consistency and consensus
  • uses SWIM for scalability

Why Consul?

  • Average application size has increased tremendously; from single machine to an army of machines
  • A trend to split monolithic architectures in service oriented architectures
  • Infrastructure became more dynamic: programability, server density/engineer
  • Docker!

Consul features

  • Service discovery
  • Health checks
  • K/V store

(source: consul.io)

Setup done

You should now have:

  • 3 VM's up and running
  • each VM running a consul server
  • all 3 consul servers joined in a cluster

Quick check:

  • ssh to root@<ip of VM>
  • run "consul members"

Ports closed...PANIC


ssh -L 8555:127.0.0.1:8555 root@<VM ip>

Interacting with Consul

Clone github.com/nustiueudinastea/consul_workshop_devopsdays on all 3 VM's

git clone https://github.com/nustiueudinastea/consul_workshop_devopsdays.git
cd consul_workshop_devopsdays/code/cluster_manager/
pip install -r requirements.txt

Consul health checks

Consul has 3 types of health checks:

- ttl

- script

- port check

python cluster_manager.py --id 2

Check Consul UI

Consul k/v store

python cluster_manager.py --id 2 --election true

Check Consul UI

Storing a key and acquiring a lock

Leader election

Start the other cluster_manager scripts using the election parameter

Consul

By alex giurgiu

Consul

  • 1,074