Discreetly Studying the Effects of Individual Traffic Control Functions
KubeCon EU 2020
Lee Calcote
Prateek Sahu
Leader
Agent
Control Plane
Data Plane
namespace
intentions
Foo Pod
Proxy Sidecar
Service Foo
discovery, config,
tls certs
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Control flow
application traffic
Application traffic
application namespace
Consul Architecture
Follower
Consul Client
Consul Servers
Follower
policy
check
cloud native and its management
Enabler of Engineers. Enabler of Speed. Enabler of Business.
Research Partners
Technology Partners
layer5.io/landscape
It's meshy out there.
Service mesh abstractions to the rescue
Meshery is interoperable with each abstraction.
Service Mesh Interface
(SMI)
Multi-Vendor Service Mesh Interoperation (Hamlet)
Service Mesh Performance Specification (SMPS)
A standard interface for service meshes on Kubernetes.
A set of API standards for enabling service mesh federation.
A format for describing and capturing service mesh performance.
Service Mesh Performance
Service Mesh Performance Specification (SMPS)
A project and vendor-neutral specification for capturing details of:
-
Environment / Infrastructure
-
Number and size of nodes, orchestrator
-
-
Service mesh and its configuration
-
Service / application details
Bundled with test results.
layer5.io/performance
layer5.io/performance
Reviews v1
Reviews Pod
Reviews v2
Reviews v3
Product Pod
Details Container
Details Pod
Ratings Container
Ratings Pod
Product Container
Reviews Service
Ratings Service
Details Service
Product Service
BookInfo Sample App
BookInfo Sample App on Service Mesh
Reviews v1
Reviews Pod
Reviews v2
Reviews v3
Product Pod
Details Container
Details Pod
Ratings Container
Ratings Pod
Product Container
Envoy sidecar
Envoy sidecar
Envoy sidecar
Envoy sidecar
Envoy sidecar
Reviews Service
Enovy sidecar
Envoy ingress
Product Service
Ratings Service
Details Service
Service Meshes vs Other Functions
gRPC Load Balancing
multiplexing
Enforcing consistency is challenging.
Foo Container
Flow Control
Foo Pod
Go Library
A v1
Network Stack
Service Discovery
Circuit Breaking
Application / Business Logic
Bar Container
Flow Control
Bar Pod
Go Library
A v2
Network Stack
Service Discovery
Circuit Breaking
Application / Business Logic
Baz Container
Flow Control
Baz Pod
Java Library
B v1
Network Stack
Service Discovery
Circuit Breaking
Application / Business Logic
Retry Budgets
Rate Limiting
API Gateway functions
Microservices API Gateways
-
Ambassador uses Envoy
-
Kong uses Nginx
-
OpenResty uses Nginx
north-south vs. east-west
Service Mesh
Network Function Architectures
Pilot
Citadel
Mixer
Control Plane
Data Plane
istio-system namespace
policy check
Foo Pod
Proxy Sidecar
Service Foo
tls certs
discovery & config
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
reports
Control flow
application traffic
Application traffic
application namespace
telemetry reports
Istio Architecture
Galley
Ingress Gateway
Egress Gateway
Control Plane
Data Plane
octa-system namespace
policy check
Foo Pod
Proxy
Sidecar
Service Foo
discovery & config
Foo Container
Bar Pod
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
reports
Control flow
application traffic
Application traffic
application namespace
telemetry reports
Octarine Architecture
Policy
Engine
Security Engine
Visibility
Engine
+
Proxy
Sidecar
+
Control Plane
Data Plane
linkerd-system namespace
Foo Pod
Proxy Sidecar
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
scarping
Control flow during request processing
application traffic
Application traffic
application namespace
telemetry scraping
Architecture
destination
Prometheus
Grafana
tap
web
CLI
proxy-api
public-api
Linkerd
proxy-injector
Control Plane
Data Plane
Foo Pod
NSM Dataplane
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
scarping
Control flow during request processing
application traffic
Application traffic
application namespace
telemetry scraping
Architecture
destination
Prometheus
Registry
NSMe
domain
client
proxy-api
public-api
Network
Service
Mesh
proxy-injector
Configuration
Security
Telemetry
Control Plane
Data
Plane
service mesh ns
Foo Pod
Proxy Sidecar
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
Control flow
application traffic
http / gRPC
Application traffic
application namespace
Meshery Architecture
Ingress Gateway
Egress Gateway
Management
Plane
meshery
adapter
gRPC
kube-api
kube-system
layer5.io/meshery
Pod Memory Usage
Application resource consumption
layer5.io/meshery
Pod CPU Usage
Application resource consumption
layer5.io/meshery
Control Plane Memory
Istio
Linkerd
Consul
layer5.io/meshery
Control Plane CPU
Istio
Linkerd
Consul
hello@layer5.io
Service Mesh Deployment Models
Client
Edge Cache
Istio Gateway
(envoy)
Cache Generator
Collection of VMs running APIs
service mesh
Istio VirtualService
Istio VirtualService
Istio ServiceEntry
Situation:
- existing services running on VMs (that have little to no service-to-service traffic).
- nearly all traffic flows from client to the service and back to client.
Benefits:
- gain granular traffic control (e.g path rewrites).
- detailed service monitoring without immediately deploying a thousand sidecars.
Ingress
Out-of-band telemetry propagation
Application traffic
Control flow
Proxy per Node
Service A
Service A
Service A
linkerd
Node (server)
Service A
Service A
Service B
linkerd
Node (server)
Service A
Service A
Service C
linkerd
Node (server)
Advantages:
-
Less (memory) overhead.
-
Simpler distribution of configuration information.
-
primarily physical or virtual server based; good for large monolithic applications.
Disadvantages:
-
Coarse support for encryption of service-to-service communication, instead host-to-host encryption and authentication policies.
-
Blast radius of a proxy failure includes all applications on the node, which is essentially equivalent to losing the node itself.
-
Not a transparent entity, services must be aware of its existence.
Development Process
Application Architecture
Deployment and Packaging
Application Infrastructure
Agile
Waterfall
DevOps
N-Tier
Monolithic
Microservices
Cloud
Containers
Physical Servers
Virtual Servers
Data Center
Hosted
Evolution to Cloud Native
λ
Functions
Serverless
Events
SRE
(Unikernels)
Third step in your cloud native journey
Meshery is interoperable with each abstraction.
Container
Orchestrator
Mesh
5.5 years
(June 2014)
4.5 years
(July 2015)
1.5 years
(July 2018)
5.5 years ago
(June 2014)
7 years ago
(March 2013)
2.5 years ago
(May 2017)
v1.0
Announced
Istio
2 years ago
(Dec 2017)
1.5 years ago
(Sept 2018)
Linkerd v2
3 years
(Apr 2017)
4 years ago
(Feb 2016)
Linkerd v1