Kubernetes Concepts

high level overview

by Felix Faust

Agenda

What are we gonna talk about?

  • Origin Story
    What is Kubernetes and why is it so popular?
  • Architecture Overview
    and Master and Worker
  • Components
    Deployments, Services, Pods, etc.
  • Expandability
    Controllers and Customization
  • Plugins: CRI, CNI and CSI
    Bridging the gap of the underlying infrastructure
  • Everything else
    Storage, Security, Debugging

Origin Story

A wise man once said...

 

This is virgin sysadmin Larry. He manually deploys services via SFTP, then logs in via SSH to stop and restart a service. Don't be Larry.

Origin Story

A wise man once said...

 

This is Chad DevOps Brad. He thaught his cat to use kubernetes, which then introduced automated deployments of fresh food and created a cleanup cronjob for his littering box. Be like Chad.

Origin Story

What is Kubernetes and why is it so popular?

 

  • (Container) Orchestrator
    developed by Google, formerly known as Borg
  • Robust, Scalable, Portable, Self-Healing
  • All-Inclusive Solution
    No need to take care of servers, crazy networking, etc.
  • Vendor Neutral
    Kubernetes stays the same, no matter where you run it
  • Open Source Community with active community

Architecture Overview

Like microservices which take pieces of higher-level-configuration and "make it happen"

Declarative Configuration Framework

Declarative Configuration Framework

in simple terms

Architecture Overview

(YAML) Configuration Backend

AWS, GCP, Azure, ...

 

Virtual Machine or

Hardware

 

 

 

Virtual Machine or

Hardware

 

 

Master and Worker

  • The "brain"
  • Hosts control plane components
    • api server
    • etcd
    • managers and operators
  • Serves as server counter part to the kubectl CLI
  • Odd scaling (1, 3, 5) for High Availability
  • "brainless" Worker
  • Hosts any kinds of workloads
    • deployments
    • cronjobs
    • persistent storage
  • Serves customer traffic
  • Any scaling

Master

Worker

Components and Resources

What is that anyway?

 

  • (Custom) Resource Definition which is a resource itself
  • Organizational Resources
    Namespaces, ResourceQuotas
  • Workload Resources
    Pods, Deployments, ReplicaSets, StatefulSets, DaemonSets, Jobs, CronJobs
  • Storage Resources
    PersistentVolumes, PersistentVolumeClaims
  • Network Resources
    Services, Ingresses, NetworkPolicies
  • Other Resources
    ConfigMaps, Secrets, HorizontalPodAutoscalers

Components and Resources

What is that anyway?

 

  • Not every cluster is the same
    • Different core api versions
    • Different custom resources
  • Old and new stuff
    • Some things are deprecated and will be removed
  • Resources consist of
    • apiVersion
    • kind
    • metadata
    • spec
    • (sometimes) status

apps, core, storage.k8s.io, networking.k8s.io, ...

Deployment, Service, ConfigMap, ...

Components and Resources

What is that anyway?

 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
    spec:
      containers:
      - name: example-container
        image: example-image
        ports:
        - containerPort: 8080

Components and Resources

What is that anyway?

 

$ kubectl explain deployment.spec
KIND:     Deployment
VERSION:  apps/v1

RESOURCE: spec <Object>

DESCRIPTION:
     Specification of the desired behavior of the Deployment.

     DeploymentSpec is the specification of the desired behavior of the
     Deployment.

FIELDS:
   minReadySeconds      <integer>
     Minimum number of seconds for which a newly created pod should be ready
     without any of its container crashing, for it to be considered available.
     Defaults to 0 (pod will be considered available as soon as it is ready)

   paused       <boolean>
     Indicates that the deployment is paused.

   progressDeadlineSeconds      <integer>
     The maximum time in seconds for a deployment to make progress before it is
     considered to be failed. The deployment controller will continue to process
     failed deployments and a condition with a ProgressDeadlineExceeded reason
     will be surfaced in the deployment status. Note that progress will not be
     estimated during the time a deployment is paused. Defaults to 600s.

   replicas     <integer>
     Number of desired pods. This is a pointer to distinguish between explicit
     zero and not specified. Defaults to 1.

   revisionHistoryLimit <integer>
     The number of old ReplicaSets to retain to allow rollback. This is a
     pointer to distinguish between explicit zero and not specified. Defaults to
     10.

   selector     <Object> -required-
     Label selector for pods. Existing ReplicaSets whose pods are selected by
     this will be the ones affected by this deployment. It must match the pod
     templates labels.

   strategy     <Object>
     The deployment strategy to use to replace existing pods with new ones.

   template     <Object> -required-
     Template describes the pods that will be created. The only allowed
     template.spec.restartPolicy value is "Always".

Expandability

Controllers and Customization

 

Controllers
realize
things

Expandability

Controllers and Customization

 

Expandability

Practical Example: Prometheus Operator

 

  • Monitor the Kubernetes API server for changes to specific objects
  • Ensures that the current Prometheus deployments match these objects

The Prometheus operator automatically detects changes in the Kubernetes API server to any of the defined objects, and ensures that matching deployments and configurations are kept in sync.

To prevent invalid Prometheus alerting and recording rules from causing failures in a deployed Prometheus instance, an admission webhook is provided to validate PrometheusRule resources upon initial creation or update.

Expandability

Practical Example: Prometheus Operator

 

  • Prometheus, which defines a desired Prometheus deployment.
  • PrometheusAgent, which defines a desired Prometheus deployment, but running in Agent mode.
  • Alertmanager, which defines a desired Alertmanager deployment.
  • ThanosRuler, which defines a desired Thanos Ruler deployment.
  • ServiceMonitor, which declaratively specifies how groups of Kubernetes services should be monitored. The Operator automatically generates Prometheus scrape configuration based on the current state of the objects in the API server.
  • PodMonitor, which declaratively specifies how group of pods should be monitored. The Operator automatically generates Prometheus scrape configuration based on the current state of the objects in the API server.
  • Probe, which declaratively specifies how groups of ingresses or static targets should be monitored. The Operator automatically generates Prometheus scrape configuration based on the definition.
  • ScrapeConfig, which declaratively specifies scrape configurations to be added to Prometheus. This CustomResourceDefinition helps with scraping resources outside the Kubernetes cluster.
  • PrometheusRule, which defines a desired set of Prometheus alerting and/or recording rules. The Operator generates a rule file, which can be used by Prometheus instances.
  • AlertmanagerConfig, which declaratively specifies subsections of the Alertmanager configuration, allowing routing of alerts to custom receivers, and setting inhibit rules.

Plugins: CRI, CNI and CSI

Bridging the gap of the underlying infrastructure

 

Container

Interface

Runtime

Network

Storage

  • Container Interfaces implement provider specific elements
  • These are similar to Controllers
  • Kubernetes API is only an abstraction layer

Plugins: CRI, CNI and CSI

Container Runtime Interface

  • dockershim: because docker itself is not a CRI
  • containerd is the successor of docker
  • CRI-O: specifically designed for kubernetes
  • basically all the same, just different backing
  • On a side note
    • gVisor: application kernel written in go, additional system layer between system calls
    • runC (cli tool): uses linux containers
    • Kata Containers: lightweight virtual machines

Plugins: CRI, CNI and CSI

Container Network Interface

  • Multiple plugins can be installed side by side!
  • Managed Kubernetes Hyperscalers often ships their own CNI to support their platform
  • Many aspects for considering a CNI:
    • Feature Set (Network Policies, Service Discovery, ...)
    • Performance: Overhead, Throughput and Latency
    • Security
    • Compatibility with your environment
    • many more...

Plugins: CRI, CNI and CSI

Container Network Interface

  • Popular CNI plugins are:
    • Flannel
      lightweight, simple
    • Calico
      pure L3, strong feature set, high-performance
    • Weave
      lightweight, simple, service discovery, encryption
    • Cilium
      security focused, L7 capabilities
  • Reading on: https://www.cni.dev/docs/

Everything else

Storage

  • There are two kinds of storage
    • Persistent
    • Ephemeral
  • Persistent Storage can be claimed (requested) with a PersistentVolumeClaim
    • Claims typically refer to
      • storageClass (usually hdd, ssd, fast, slow, etc...)
      • size
      • accessMode (RWO, ROX, RWX, RWOP)
  • ​Ephemeral Storage typically refers to a in-memory tmpfs
    • Mostly used for mounting secrets, configmaps, ...

Everything else

Storage

  • You can take Snapshots of PersistentVolumes
    • if the CSI supports it
  • Common CSI implementations
    • Local Path Provisioner
    • Cinder (OpenStack)
    • Platform specific (hyperscaler), e.g. GCS, EBS, Azure Disk, ...
  • Reading on: https://kubernetes-csi.github.io/docs/drivers.html

Everything else

Security

  • Kubernetes API Server is secured with TLS
  • Authentication
    • Service Account
      • Tokens, which can have expiry date
      • Client Key and Secret
    • Supports OAuth
  • Authorization
    • Role Based Access Control (RBAC)
      • Allow or disallow access to certain components
      • Allow or disallow access to certain verbs (get, list, delete, ...)
  • Audit Logging

Everything else

Debugging

  • Kubernetes Components often have a status field
  • Events can be monitored
  • kubectl -w to watch components
  • kubectl debug to spawn and interact with containers
    Reading on
  • kubectl cp to copy files from/to containers
    Reading on

Most of the time, kubernetes is self-healing, but if not:

Questions?

If you haven't asked already

Ask all you want

Thank You

Felix Faust

DevOps Engineer

Schäfer Shop

turtledev.net

linkedin.com

contact@turtledev.net

Kubernetes Concepts

By amazingturtle

Made with Slides.com

Kubernetes Concepts

  • 23

More from amazingturtle