Ansible Operator
超新手 Operator 入門 101
印章
2021/01/26
Agenda
1. Operator Framework Introduction
2. Shell Operator
3. Ansible Operator
4. Compare with Helm3
5. Demo (Ansible Operator 101)
這次分享不會講......
- Kubernetes 基本操作
- Ansible Playbook 撰寫
- 使用 Operator SDK (Golang)
- Operator Hub
Who Am I ?
-
印章 (seal.tw ),
不是海豹 -
本名 吳易璋
-
Gitlab Taiwan社群成員
-
「前」某醫學中心打雜
-
兼任On-Permise Cloud 架構師
-
兼任 Infra 維運
-
兼任 SRE
-
現職 無業遊民
賭神從來不拍照
GitLab 13.0 之後
Login 頁面的標語
Why Operator ?
Kubernetes is good, but ......
- 維護困難 (yaml 缺乏變數、迴圈機制)
- Deployment / DaemonSet
- Service / Ingress / Network Policy
- PV / PVC /ConfigMap / Secret
- RBAC / ServiceAccount
- CRD ?
- Deployment / DaemonSet
Kubernetes is good, but ......
- 缺乏Lifecycle管理、程式相依性管理
- Stateful 機制不足,導致 CrashLoop
- 專業知識要求高,例如:
- MSSQL Always On
- etcd cluster scale up
- Virtual Machine (kubevirt)
- Sidecar Injection (Istio)
Operator Framework
- 此概念由CoreOS 提出,
被大量用在 RedHat OpenShift 中
- 為了管理 CRD 而產生,結合 Lifecycle 的概念
- 復用 kube-controller 的機制
(可視為,管理 Deployment 的一種 Operator)
- Golang為主要的開發語言,提供變數、迴圈的機制
Operator Framework
- Etcd Operator
- Prometheus Operator
- Rook.io (Ceph Operator)
Operator Framework
Operator not Popular?
Pros
- 不用學習 Golang
- Event Hook 機制
- 既有 Shell Script 重複利用
- 不用自行實作「監控CRD」
(不重複造輪子)
- 自帶 metrics 、 logging
Cons
- 變數、迴圈不友善
- 沒有 Template 機制 (envsubst)
- Lifecycle 綁定既定框架
(Auto-scale with custom Metrics)
- 非聲明式 => IaC 難實現
(Infrastructure as Code)
Shell Operator
FROM kubespheredev/shell-operator:v1.0.0-beta.5-alpine3.12
ENV ANSIBLE_ROLES_PATH /kubesphere/installer/roles
WORKDIR /kubesphere
ADD controller/* /hooks/kubesphere/
ADD roles /kubesphere/installer/roles
ADD env /kubesphere/results/env
ADD playbooks /kubesphere/playbooks
USER kubesphere
Ansible / Kubernetes
其實是可以一起使用的......
Compare with......
Ansible / Kubernetes
Same
- IaC 實現
- 豐富的生態系
- 商業支持 (RedHat)
- 腳本為yaml 格式
- 皆沒有 Lifecycle Management
Diff
- Kubernetes:
- Containerilzed
- CRI, CNI, CSI
- Ansible
- Variable, Loop, Condition
- Template
- Python Extension
- 缺乏RBAC
- Need Inventory File
Compare with......
Ansible Operator/ Helm 3
Same
- Template (jinja v.s. Go Template)
- Version Control
-
Containerlized
- 豐富的生態系
- 腳本為yaml 格式
- OCI Compatible
-
繼承所有 Kubernetes 特性
(CRI, CNI, CSI)
Diff
- Helm 3:
- 缺乏Lifecycle Management
-
Dry-run Usage
- Ansible Operator:
- Custom Image
- Agent 依賴性
- Python Extension
Live Demo
Ansible Operator 101
$ mkdir memcached-operator
$ cd memcached-operator
$ operator-sdk init --plugins=ansible --domain example.com
# or
$ git clone https://github.com/djzager/ansible-role-hello-world-k8s.git
- 準備資源
- Custom Image
- Ansible Operator
- HelloWorld Image
- Dockerfile / Containerfile
- Ansible Playbook / Role
- Kubernetes YAML file
- Deployment
- Role / RoleBinding / ServiceAccount
- CustomResource / CustomResourceDefinition
- Custom Image
FROM quay.io/operator-framework/ansible-operator:v0.8.1
COPY watches.yaml ${HOME}/watches.yaml
COPY roles/ ${HOME}/roles/
COPY playbook.yml ${HOME}/playbook.yml
- Dockerfile
- name: 'Set hello-world objects state={{ state }}'
k8s:
state: '{{ state }}'
definition: "{{ lookup('template', item.name) | from_yaml }}"
loop:
- name: deployment.yml.j2
- name: service.yml.j2
- Ansible Playbook
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: helloworlds.examples.djzager.io
spec:
group: examples.djzager.io
names:
kind: HelloWorld
listKind: HelloWorldList
plural: helloworlds
singular: helloworld
scope: Namespaced
version: v1alpha1
subresources:
status: {}
- Custom Resource Definition
# Prepare hello world operator image
$ docker build . -t hello-world-operator:latest
$ docker tag hello-world-operator:latest quay.io/rockwyc992/hello-world-operator:latest
$ docker push quay.io/rockwyc992/hello-world-operator:latest
# Create Kubernetes Resource from yaml file
$ kubectl create -f deploy/namespace.yaml \
-f deploy/service_account.yaml \
-f deploy/role.yaml \
-f deploy/role_binding.yaml \
-f deploy/deployment.yaml \
-f deploy/crds/helloworld_crd.yaml
$ kubectl -n mynamespace get pod -w
$ kubectl create -f deploy/crds/helloworld_cr.yaml
- Apply Kubernetes YAML file
$ docker build . -t hello-world-operator:latest
$ kubectl create -f deploy/deployment.yaml \
-f deploy/crds/helloworld_crd.yaml
$ kubectl create -f deploy/crds/helloworld_cr.yaml
- Apply Kubernetes YAML file
apiVersion: examples.djzager.io/v1alpha1
kind: HelloWorld
metadata:
name: example-helloworld
spec:
size: 3
image: nginx:alpine
- Custom Resource
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ meta.name }}
namespace: {{ meta.namespace }}
spec:
replicas: {{ size }}
template:
spec:
containers:
- image: {{ image }}
name: {{ meta.namespace }}
ports:
- containerPort: 80
protocol: TCP
- Deployment.yaml.j2
- From Ansible 2.9
- in-tree code => Community Module
$ ansible-galaxy collection install community.kubernetes
$ cat tasks/main.yaml
- name: Create a Deployment by reading the definition from a local file
community.kubernetes.k8s:
state: present
src: test-deployment.yml
$ cat tasks/main-legacy.yaml
- name: Create a Deployment by reading the definition from a local file
k8s:
state: present
src: test-deployment.yml
- From Ansible 2.9
- in-tree code => Community Module
- name: Create a Deployment by reading the definition from a local file
k8s:
state: present
src: test-deployment.yml
- Create From Local File
- name: Create a Service object from an inline definition
k8s:
state: present
definition:
apiVersion: v1
kind: Service
metadata:
name: web
spec:
selector:
app: galaxy
ports:
- protocol: TCP
targetPort: 8000
port: 8000
- Create From Inline Definition
- name: Read definition file from parsing template file
k8s:
state: present
definition: "{{ lookup('file', '/testing/deployment.yaml.j2') | from_yaml }}"
- name: Read definition template file
k8s:
state: present
template: '/testing/service.yaml.j2'
- Create From Jinja 2 Template
- name: Remove an existing Service object
k8s:
state: absent
api_version: v1
kind: Service
namespace: testing
name: web
- Remove an Object
Reference
Operator Framework:
- https://blog.pichuang.com.tw/20190407-how-look-like-operator-framework-in-production/
- https://coreos.com/blog/introducing-operator-framework
- https://cloudblogs.microsoft.com/sqlserver/2018/12/10/availability-groups-on-kubernetes-in-sql-server-2019-preview/
Shell Operator:
- https://medium.com/flant-com/shell-operator-for-kubernetes-update-2f1f9f9ebfb1
- https://medium.com/flant-com/meet-the-shell-operator-kubecon-36c14ba2f8fe
- http://dockone.io/m/article/8984
- https://github.com/kubesphere/ks-installer
Ansible Operator:
- https://sdk.operatorframework.io/docs/advanced-topics/operator-capabilities/operator-capabilities/
- https://github.com/djzager/ansible-role-hello-world-k8s
- https://www.openshift.com/blog/reaching-for-the-stars-with-ansible-operator
- https://medium.com/velotio-perspectives/getting-started-with-kubernetes-operators-ansible-based-part-2-472eb0d453b7
- https://docs.ansible.com/ansible/latest/collections/community/kubernetes/k8s_module.html
- https://galaxy.ansible.com/community/kubernetes
Q & A
Thanks for your attention !
Ansible Operator
By rockwyc992
Ansible Operator
- 1,159