CaaS infrastructure

  • Raw K8s + Helm Charts
  • CaaS
  • Demo - kQueen
  • Demo - CD

Petr Michalec

Geek. Developer. Ex-IBMer. Vi(m) lover. DevOps/Cloud infrastructure and application architect with the passion for the edge thing. Daily at Mirantis.

 

Reach me on Twitter or Keybase as @epcim. 

 

icons8.com

Audience

  • Speakers
  • Attendees  

  • Running Kubernetes today? Hosted?

  • Application | Infrastructure focus?

  • Ops | Dev's | Decision makers?

Audience?

What is Kubernetes?

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

 

Stand alone Kubernetes, on-site or distributed as public platforms


UI, Tools to manage, operate, visualize, verify and lifecycle of "Deployments".

 

Physical infrastructure, Storage, Network. Integration with external services.

Quiz time

Basic Kubernetes objects:

Higher-level abstractions Controllers:

Quiz time

https://kubernetes.io/docs/concepts

ConfigMap, Labels/Selectors, Annotations, Storage classes, ...

Quiz time

Raw K8s experience

Logs & Metrics

Monitoring Solution

Requirements / specs.

  • Fully containerized
  • On-Premises, no ext. resources
  • Minimize # of components used
  • Serving 400+ node infrastructure
  • Network appliances > in_syslog
  • 3.5M~5M docs per hour / 200GB Elastic
  • Logs:
    • Fluent-bit
    • Fluentd
  • Metrics:
    • Telegraf
    • Fluent-bit
  • Data storage:
    • Elasticsearch
    • Influxdb
    • Graphite *
  • Presentation:
    • Kibana, Grafana

Helm Charts

Log collection

Metric collection

Master chart approach

Helm deployment

InfluxDB

InfluxDB - wheelhouse

Results, Challenges

  • Lack of CI/CD for Helm deployments at early stages
  • Limited K8s native integration (storage, network)
  • Helm charts
    • Different implementations (upstream, curated)
    • Update Charts to be production grade  
  • Optimize Fluentd and Elastic for high loads
    • Buffers, queues, frequency, stability at scale matter
  • K8s env. in Lab went down on its knees many many times!

Grafana

Grafana

Elasticsearch Helm Chart

Elasticsearch

  • Infrastructure:
    • Kubernetes
  • Orchestration
    • Helm
  • Applications
    • Helm Charts

Environment

  • Networking
    • node ports :(
  • Storage
    • NFS :(
  • CI / CD
    • Jenkins
    • Helm

"opinionated" K8s ​ecosystem?

  • Automated rollouts for apps
  • Painless platform upgrades
  • Management UI, API, Service discovery
  • SDN, SDS
  • Metric and Log collection capabilities
  • Security, Policies and governance
  • Service mash and discovery
  • Multi-clusters

Gartner CaaS stack

  • Storage, Networking, LB
  • Artefact repository
  • AAA + IDP Integrations
  • Continuous Integrations
  • Deployment pipelines, CD
  • Orchestration
  • Log collection
  • Monitoring
  • Sec. Audit
  • Multi-cluster
  • Monitoring
  • Data management
    • backups
    • volumes
  • Overlay networking
  • Vendor features lock-in

Infrastructure

Scheduling

  • Placement
  • Replication
  • Scaling
  • Resurrection
  • Rescheduling
  • Rolling deployments
  • Upgrades
  • Downgrades
  • Collocation

Resources

  • Memory
  • CPU, GPU
  • Volumes
  • IPs, Ports
  • Image/Artefacts

Service mgmt.

  • Labels
  • Groups
  • Namespaces
  • Dependencies
  • Load Balancing

Application

Voyage to CaaS

  • Developers that disagree with the monolithic IaaS/PaaS
  • Workloads more flexible, resilient, robust, scalable
  • CI/CD centric, continuous delivery of service
  • Mesh services, well-defined interfaces
  • Minimize overhead, resources
  • Visibility / Analytics

 

Google's Container Engine, EKS by Amazon, AKS by Microsoft, Tectonic by CoreOS, and OpenShift by RedHat, MCP Mirantis, ...

Demo - kQueen

Single Management plane for K8s clusters

github.com/mirantis/kqueen

  • Multi-tenant API + web UI + CLI
  • Enable to use multiple provisioners (Kubespray, Heat, GKE)
  • Kubernetes itinerary with workload visibility
    • Visualisation, management, operation, audit
    • Integration into 3rd-party systems
  • Pluggable Architecture
  • Extension by helm service catalogue (soon)

 

Visualizations

Demo - Continuous Delivery

Shoot simple app across K8s cluster

 

github.com/Mirantis/k8s-apps/tree/master/charts/rollout

Spinnaker

TweeViz

Look Forward

Look Forward

Multi-cluster managers 

Service management

Packaging

CD pipelines

Backup slides

 

Kubernetes as an container manager

 

 

Multi cloud application deployment & ops

 

Additional vectors

What you want

  • Own full-scale CaaS stack
  • Run apps only

 

 

 

 

 

What

  • Traditional apps
  • Cloud native apps

Where

  • On-site
  • Public
  • Hybrid

Role of ops teams evolves from managing the datacenter to helping developers deploy workloads to any cloud

One CI/CD  for Dev/Ops/Support

Services in a "big picture"

  • Standardization around of service management and cataloguing. (consumption, lifecycle, resource ctl.)
  • How do we create consumable services from a bunch of containers?
  • Where should rules affecting the behaviors of services reside?

K8s complexity at scale

Elasticsearch Helm Chart

Break

What makes K8s a modern infrastracture

By Petr Michalec

Create a presentation like this

What makes K8s a modern infrastracture

What makes K8s a modern infrastructure? Caas. Experience. kQueen demo. -- Kubernetes je často vnímáno jako “jen další” typ kontejnerové orchestrace, avšak Kubernetes API má svůj design zakořeněný ve více než 10-ti letech zkušeností s Borgem (Google předchůdce Kubernetes). Není tedy náhodou, že za poslední rok uvedlo na trh téměř 20 výrobců produkty okolo CaaS a PaaS založených právě na této technologii. Zároveň 3 největší public cloudy AWS, Azure a Google nabízejí managed Kubernetes.

  • 454
Loading comments...

More from Petr Michalec