Cloud Native Flink

Jowanza Joseph

@jowanza

Agenda

  • Flink
  • Stream Processing
  • Deployment
  • Cloud Native
  • Cloud Native Flink
  • Questions

Who Am I?

My Own Project

What Is Flink?

A Thought

Deployment

Why

What does Cloud Native Mean?

  1. Continuous Delivery

  2. Container Native
  3. Operator Friendly

Why Containers?

Container Flink

  • Stateless
  • Distributed
  • JVM

Container Orchestration

Container Orchestration

Metrics

rate(http_requests_total[5m])


sum(rate(http_requests_total[5m])) by (job)

sum(
  instance_memory_limit_bytes - instance_memory_usage_bytes
) by (app, proc) / 1024 / 1024

Tracing

Logging

Architecture

Pros

  • Monitoring
  • Scale up/down
  • Tracing
  • Isolation
  • Self-healing
  • Managed
  • Cheap

Cons

  • Maintaining Configs
  • Containers
  • Learning new query language
  • Learning new stuff

The Future

  • Canary
  • Alerting
  • Flink Clint Client

Questions?

Cloud Native Flink

By Jowanza Joseph

Cloud Native Flink

  • 900