Consul

Alex Giurgiu (alex@giurgiu.io)

using the k/v store and health checks to build a leader election system

Goals for today

• Introduce Consul and what problems it tries to solve
• Build(study) a practical example that leverages Consul to implement a leader election system

Start setup

SoftLayer API credentials: https://pastee.org/xjr9v

Code: github.com/nustiueudinastea/consul_workshop_devopsdays

Fill in credentials in bootstrap_machines/bootstrap_softlayer.py

Run bootstrap_softlater.py from your machine or Vagrant

Make sure that your private key is available for Ansible

pip install -r requirements.txt

About Consul

• similar to etcd and zookeeper,
• made by HashiCorp(Vagrant and Packer)
• uses RAFT for consistency and consensus
• uses SWIM for scalability

Why Consul?

• Average application size has increased tremendously; from single machine to an army of machines
• A trend to split monolithic architectures in service oriented architectures
• Infrastructure became more dynamic: programability, server density/engineer
• Docker!

Consul features

• Service discovery
• Health checks
• K/V store

Setup done

You should now have:

• 3 VM's up and running
• each VM running a consul server
• all 3 consul servers joined in a cluster

Quick check:

• ssh to root@<ip of VM>
• run "consul members"

Ports closed...PANIC

ssh -L 8555:127.0.0.1:8555 root@<VM ip>

Interacting with Consul

Clone github.com/nustiueudinastea/consul_workshop_devopsdays on all 3 VM's

git clone https://github.com/nustiueudinastea/consul_workshop_devopsdays.git
cd consul_workshop_devopsdays/code/cluster_manager/
pip install -r requirements.txt

Consul health checks

Consul has 3 types of health checks:

- ttl

- script

- port check

python cluster_manager.py --id 2

Check Consul UI

Consul k/v store

python cluster_manager.py --id 2 --election true

Check Consul UI

Storing a key and acquiring a lock

Leader election

Start the other cluster_manager scripts using the election parameter