Iqbal Farabi
System Engineer
Go-Jek Indonesia
We can isolate components using Virtual Machines (VMs). This way, each components can have their own dependencies satisfied without getting in the way of each other.
The problem with VM is that it takes a lot of hardware resources, therefore not ideal for microservice-based app with large number of services.
Containers run as isolated process on host OS instead of running its own guest OS. This is achieved by using Linux namespaces, cgroup, and chroot*.
That way, containers provide isolation without consuming as much resources as VMs. With the same specs, a bare-metal host can run more containers than VMs.
* checkout: Building Containers from Scratch
Greek for pilot, helmsman, governor.
Kubernetes is a software system that allows you to deploy and manage containerized applications on top of it.
Kubernetes enables you to run your software applications on multiple distributed nodes as if all those nodes were a single, enormous resource.
Kubernetes can be thought of as an operating system for the cluster.
Provision:
Visit this page.
gcloud compute instances create istabon --zone asia-east1-a --machine-type n1-standard-1 \
--image ubuntu-1604-xenial-v20181023 --image-project ubuntu-os-cloud
Create a GCP compute instance.
Prepare necessary credentials. These steps will asume you have a service account and it is saved as file named 'gcp-auth.json' and project named 'qblfrb=kubernetes-lab'.
gcloud compute ssh istabon
gcloud compute scp gcp-auth.json istabon:~/gcp-auth.json --zone asia-east1-a
gcloud auth activate-service-account --key-file=gcp-auth.json --project=qblfrb-kubernetes-lab
wget https://apt.puppetlabs.com/puppetlabs-release-pc1-xenial.deb
sudo dpkg -i puppetlabs-release-pc1-xenial.deb
sudo apt-get update
sudo apt-get install puppetserver
Install 'puppetserver'.
Install 'googleauth' and 'google-api-client'
sudo /opt/puppetlabs/puppet/bin/gem install googleauth -v 0.6.7
sudo /opt/puppetlabs/puppet/bin/gem install google-api-client --no-ri --no-rdoc
/opt/puppetlabs/puppet/bin/puppet module install google/cloud
Install Puppet module for Google cloud.
Write the following manifest, for instance 'gcontainer_cluster.pp'.
gauth_credential { 'gauth-credential':
provider => serviceaccount,
path => '/home/iqbalfarabi/gcp-auth.json',
scopes => ['https://www.googleapis.com/auth/cloud-platform'],
}
gcontainer_cluster { 'istabon':
ensure => present,
initial_node_count => 2,
node_config => {
machine_type => 'n1-standard-4',
disk_size_gb => 500,
},
credential => 'gauth-credential',
zone => 'asia-east1-a',
project => 'qblfrb-kubernetes-lab',
}
/opt/puppetlabs/puppet/bin/puppet apply gcontainer_cluster.pp
Apply it.
gcloud container clusters get-credentials istabon
Get cluster credential.
Validate nodes and kube-system pods exist.
kubectl get nodes
kubectl get pods -n kube-system
Your cluster is up and running!
Interesting Ruby-based Kubernetes tools you might want to look at:
require 'sinatra'
enable :run, :show_exceptions
set :environment, :production
set :bind, '0.0.0.0'
set :port, 80
get '/' do
'Hello, world!'
end
Hello world:
FROM ruby:2.5.1
RUN gem install sinatra
WORKDIR /usr/src/app
COPY hello-world.rb .
EXPOSE 80
CMD /usr/local/bin/ruby ./hello-world.rb
Write a Dockerfile:
Build and push to Dockerhub:
docker build -t qblfrb/hello-world-sinatra:0.1.0 .
docker push qblfrb/hello-world-sinatra:0.1.0
apiVersion: apps/v1
kind: Deployment
metadata:
name: sinatra
spec:
selector:
matchLabels:
app: sinatra
replicas: 1
template:
metadata:
labels:
app: sinatra
spec:
containers:
- name: sinatra
image: qblfrb/hello-world-sinatra:0.1.0
ports:
- containerPort: 30080
Deployment:
apiVersion: v1
kind: Service
metadata:
name: sinatra
labels:
app: sinatra
spec:
type: NodePort
ports:
- port: 80
nodePort: 30080
protocol: TCP
name: http
selector:
app: sinatra
Service:
kubectl apply -f sinatra.yaml
Apply:
kubectl get pods -l app=sinatra
kubectl port-forward sinatra-788f79cff4-fpqxw 8080:80
Port-forward:
+
=
+
=
+
Kubeadm
=
The Bootstrapper is a tool that we created to empower developers in managing their own Kubernetes cluster on AWS. This is only one of many Kubernetes related tools we have in Go-Jek.
We implement the reconciler pattern as described in Kris Nova's book Cloud Native Infrastructure. We utilize Terraform and Kops to automate cluster creation, update, and deletion.
cluster_name: istabon.sample-cluster.io
vpc_name: istabon-staging
nodes:
count: 4
size: m5.4xlarge
masters:
count: 3
size: m4.2xlarge
zones:
- ap-southeast-1a
- ap-southeast-1b
- ap-southeast-1c
subnets:
- cidr: 10.14.14.0/24
- cidr: 10.14.15.0/24
- cidr: 10.14.16.0/24
utility_subnets:
- cidr: 10.14.17.0/28
- cidr: 10.14.17.16/28
- cidr: 10.14.17.32/28
topology: private
bucket_region: ap-southeast-1
├── cluster-definition
│ ├── istabon.sample-cluster.io
│ │ └── config.yaml
├── kops
│ ├── istabon.sample-cluster.io
│ │ ├── alertmanager.yaml
│ │ ├── cluster.yaml
│ │ ├── gcr-secret.yaml
│ │ ├── grafana-service.yaml
│ │ ├── instance-group-bastions.yaml
│ │ ├── instance-group-master-1.yaml
│ │ ├── instance-group-master-2.yaml
│ │ ├── instance-group-master-3.yaml
│ │ ├── instance-group-nodes.yaml
│ │ ├── kops-secret.yaml
│ │ ├── kubectl-proxy-secret.yaml
│ │ ├── prometheus-custom-rules.yaml
│ │ ├── prometheus-service.yaml
│ │ └── tiller-rbac-config.yaml
├── scripts
└── terraform
└── istabon.sample-cluster.io
└── services
└── kops-setup
├── backend.tf
├── data.tf
├── main.tf
├── output.tf
├── provider.tf
└── var.tf
Faster Setup Time
Setting up the whole Go-Viet infrastructure only took four days.
Cookie Cutter Model
Repeatable/immutable nature of containerizing helps us to replicate our MVP launch strategy for different geographies.
Scalable
Scaling based on business growth is very easy.
Faster MTTR
In the case of traffic spike, for instance, we can spin up new containers much more quickly than setting up new VMs.
Higher Uptime
High availability setup lead to fewer outage.
Efficiency
System resources like CPU, memory, etc. are more effectively utilized in container world than in VMs.
Easy Configuration
Automatic service discovery allows engineers to not maintain any configuration for multi-data center deployments.
Cost Effective
Save > 60% cost compared to VM per year per country for international expansion projects.
Kubernetes in Action – Mario Luksa
Kubernetes: Up and Running – Joe Beda, Brendan Burns, Kelsey Hightower
Cloud Native Infrastructure – Kris Nova, Justin Garrison
Building Microservices – Sam Newman
Designing Distributed System – Brendan Burns
Giri Kuncoro
Vijay Dhama
Himani Agrawal
Prashant Mittal
Irfan Shah
Sumit Gupta
Willem Pienaar
Arief Hermansyah
Shani Pribadi
Sourabh Gupta