Ansible Operator

超新手 Operator 入門 101

印章

2021/01/26

Agenda

1. Operator Framework Introduction
2. Shell Operator
3. Ansible Operator

4. Compare with Helm3

5. Demo (Ansible Operator 101)

這次分享不會講......

  • Kubernetes 基本操作
  • Ansible Playbook 撰寫
  • 使用 Operator SDK (Golang)
  • Operator Hub

Who Am I ?

  • 印章 (seal.tw ),不是海豹

  • 本名 吳易璋

  • Gitlab Taiwan社群成員

  • 「前」某醫學中心打雜

  • 兼任On-Permise Cloud 架構師

  • 兼任 Infra 維運

  • 兼任 SRE


     

  • 現職 無業遊民

賭神從來不拍照

GitLab 13.0 之後

Login 頁面的標語

Why Operator ?

Kubernetes is good, but ......

  • 維護困難 (yaml 缺乏變數、迴圈機制)
    • Deployment / DaemonSet
       
    • Service / Ingress / Network Policy
       
    • PV / PVC /ConfigMap / Secret
       
    • RBAC / ServiceAccount
       
    • CRD ?

Kubernetes is good, but ......

  • 缺乏Lifecycle管理、程式相依性管理
     
  • Stateful 機制不足,導致 CrashLoop
     
  • 專業知識要求高,例如:
    • MSSQL Always On
    • etcd cluster scale up
    • Virtual Machine (kubevirt)
    • Sidecar Injection (Istio)

Operator Framework

  • 此概念由CoreOS 提出,
    被大量用在 RedHat OpenShift 中
     
  • 為了管理 CRD 而產生,結合 Lifecycle 的概念
     
  • 復用 kube-controller 的機制
    (可視為,管理 Deployment 的一種 Operator)
     
  • Golang為主要的開發語言,提供變數、迴圈的機制

Operator Framework

  • Etcd Operator
  • Prometheus Operator
  • Rook.io (Ceph Operator)

Operator Framework

Operator not Popular?

Pros

  • 不用學習 Golang
     
  • Event Hook 機制
     
  • 既有 Shell Script 重複利用
     
  • 不用自行實作「監控CRD」
    (不重複造輪子)
     
  • 自帶 metrics 、 logging

Cons

  • 變數、迴圈不友善
     
  • 沒有 Template 機制 (envsubst)
     
  • Lifecycle 綁定既定框架
    (Auto-scale with custom Metrics)
     
  • 非聲明式 => IaC 難實現
    (Infrastructure as Code)

Shell Operator

FROM kubespheredev/shell-operator:v1.0.0-beta.5-alpine3.12

ENV  ANSIBLE_ROLES_PATH /kubesphere/installer/roles
WORKDIR /kubesphere
ADD controller/* /hooks/kubesphere/

ADD roles /kubesphere/installer/roles
ADD env /kubesphere/results/env
ADD playbooks /kubesphere/playbooks
USER kubesphere

Ansible / Kubernetes

 

其實是可以一起使用的......

Compare with......

Ansible / Kubernetes

Same

  • IaC 實現
  • 豐富的生態系
  • 商業支持 (RedHat)
     
  • 腳本為yaml 格式
  • 皆沒有 Lifecycle Management

Diff

  • Kubernetes:
    • Containerilzed
    • CRI, CNI, CSI
  • Ansible
    • Variable, Loop, Condition
    • Template
    • Python Extension
    • 缺乏RBAC
    • Need Inventory File

Compare with......

Ansible Operator/ Helm 3

Same

  • Template (jinja v.s. Go Template)
  • Version Control
  • Containerlized
     
  • 豐富的生態系
  • 腳本為yaml 格式
  • OCI Compatible
  • 繼承所有 Kubernetes 特性
    (CRI, CNI, CSI)
     

Diff

  • Helm 3:
    • 缺乏Lifecycle Management
    • Dry-run Usage
       
  • Ansible Operator:
    • Custom Image
    • Agent 依賴性
    • Python Extension

Live Demo

 

Ansible Operator 101

$ mkdir memcached-operator 
$ cd memcached-operator
$ operator-sdk init --plugins=ansible --domain example.com

# or

$ git clone https://github.com/djzager/ansible-role-hello-world-k8s.git
  • 準備資源
    • Custom Image
      • Ansible Operator
      • HelloWorld Image
      • Dockerfile / Containerfile
    • Ansible Playbook / Role
    • Kubernetes YAML file
      • Deployment
      • Role / RoleBinding / ServiceAccount
      • CustomResource / CustomResourceDefinition
FROM quay.io/operator-framework/ansible-operator:v0.8.1

COPY watches.yaml ${HOME}/watches.yaml

COPY roles/ ${HOME}/roles/
COPY playbook.yml ${HOME}/playbook.yml
  • Dockerfile
- name: 'Set hello-world objects state={{ state }}'
  k8s:
    state: '{{ state }}'
    definition: "{{ lookup('template', item.name) | from_yaml }}"
  loop:
    - name: deployment.yml.j2
    - name: service.yml.j2
  • Ansible Playbook
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: helloworlds.examples.djzager.io
spec:
  group: examples.djzager.io
  names:
    kind: HelloWorld
    listKind: HelloWorldList
    plural: helloworlds
    singular: helloworld
  scope: Namespaced
  version: v1alpha1
  subresources:
    status: {}
  • Custom Resource Definition
# Prepare hello world operator image
$ docker build . -t hello-world-operator:latest
$ docker tag hello-world-operator:latest quay.io/rockwyc992/hello-world-operator:latest
$ docker push quay.io/rockwyc992/hello-world-operator:latest

# Create Kubernetes Resource from yaml file
$ kubectl create -f deploy/namespace.yaml \
                 -f deploy/service_account.yaml \
                 -f deploy/role.yaml \
                 -f deploy/role_binding.yaml \
                 -f deploy/deployment.yaml \
                 -f deploy/crds/helloworld_crd.yaml
                 
$ kubectl -n mynamespace get pod -w
$ kubectl create -f deploy/crds/helloworld_cr.yaml
  • Apply Kubernetes YAML file
$ docker build . -t hello-world-operator:latest

$ kubectl create -f deploy/deployment.yaml \
                 -f deploy/crds/helloworld_crd.yaml
                 
$ kubectl create -f deploy/crds/helloworld_cr.yaml
  • Apply Kubernetes YAML file
apiVersion: examples.djzager.io/v1alpha1
kind: HelloWorld
metadata:
  name: example-helloworld
spec:
  size: 3
  image: nginx:alpine
  • Custom Resource
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ meta.name }}
  namespace: {{ meta.namespace }}
spec:
  replicas: {{ size }}
  template:
    spec:
      containers:
      - image: {{ image }}
        name: {{ meta.namespace }}
        ports:
        - containerPort: 80
          protocol: TCP
  • Deployment.yaml.j2
$ ansible-galaxy collection install community.kubernetes

$ cat tasks/main.yaml
- name: Create a Deployment by reading the definition from a local file
  community.kubernetes.k8s:
    state: present
    src: test-deployment.yml

$ cat tasks/main-legacy.yaml
- name: Create a Deployment by reading the definition from a local file
  k8s:
    state: present
    src: test-deployment.yml
- name: Create a Deployment by reading the definition from a local file
  k8s:
    state: present
    src: test-deployment.yml
  • Create From Local File
- name: Create a Service object from an inline definition
  k8s:
    state: present
    definition:
      apiVersion: v1
      kind: Service
      metadata:
        name: web
      spec:
        selector:
          app: galaxy
        ports:
        - protocol: TCP
          targetPort: 8000
          port: 8000
  • Create From Inline Definition
- name: Read definition file from parsing template file
  k8s:
    state: present
    definition: "{{ lookup('file', '/testing/deployment.yaml.j2') | from_yaml }}"

- name: Read definition template file
  k8s:
    state: present
    template: '/testing/service.yaml.j2'
  • Create From Jinja 2 Template
- name: Remove an existing Service object
  k8s:
    state: absent
    api_version: v1
    kind: Service
    namespace: testing
    name: web
  • Remove an Object

Reference

Q & A

Thanks for your attention !

Ansible Operator

By rockwyc992

Ansible Operator

  • 1,032