Lee Calcote
Clouds, containers, functions, applications and their management.
Service Meshes
why you need one
and
which one you need
November 2019
cloud native and its management
DEV
OPS
Decoupling at Layer 5
where Dev and Ops meet
Problem: too much infrastructure code in services
What do we need?
• Observability
• Logging
• Metrics
• Tracing
• Traffic Control
• Resiliency
• Efficiency
• Security
• Policy
...a Service Mesh
What is a Service Mesh?
a dedicated layer for managing service-to-service communication
So, a microservices platform?
obviously.
partially.
a services-first network
BookInfo Sample App
Reviews v1
Reviews Pod
Reviews v2
Reviews v3
Product Pod
Details Container
Details Pod
Ratings Container
Ratings Pod
Product Container
Reviews Service
Ratings Service
Details Service
Product Service
BookInfo Sample App on Service Mesh
Reviews v1
Reviews Pod
Reviews v2
Reviews v3
Product Pod
Details Container
Details Pod
Ratings Container
Ratings Pod
Product Container
Envoy sidecar
Envoy sidecar
Envoy sidecar
Envoy sidecar
Envoy sidecar
Reviews Service
Enovy sidecar
Envoy ingress
Product Service
Ratings Service
Details Service
Observability
what gets people hooked on service metrics
Goals
-
Metrics without instrumenting apps
-
Consistent metrics across fleet
-
Trace flow of requests across services
-
Portable across metric back-end providers
You get a metric! You get a metric! Everyone gets a metric!
Traffic Control
control over chaos
- Traffic splitting
- L7 tag based routing?
- Traffic steering
- Look at the contents of a request and route it to a specific set of instances.
- Ingress and egress routing
Resilency
- Systematic fault injection
-
Timeouts and Retries with timeout budget
-
Control connection pool size and request load
-
Circuit breakers and Health checks
content-based traffic steering
Missing: application lifecycle management, but not by much
Missing: distributed debugging; provide nascent visibility (topology)
Relating to
Service Meshes
Which is why...
I have client-side libraries.
Enforcing consistency is challenging.
Foo Container
Flow Control
Foo Pod
Go Library
A v1
Network Stack
Service Discovery
Circuit Breaking
Application / Business Logic
Bar Container
Flow Control
Bar Pod
Go Library
A v2
Network Stack
Service Discovery
Circuit Breaking
Application / Business Logic
Baz Container
Flow Control
Baz Pod
Java Library
B v1
Network Stack
Service Discovery
Circuit Breaking
Application / Business Logic
Retry Budgets
Rate Limiting
Which is why...
I have a container orchestrator.
Core
Capabilities
-
Cluster Management
-
Host Discovery
-
Host Health Monitoring
-
-
Scheduling
-
Orchestrator Updates and Host Maintenance
-
Service Discovery
-
Networking and Load Balancing
-
Stateful Services
-
Multi-Tenant, Multi-Region
Additional
Key Capabilities
-
Application Health and Performance Monitoring
-
Application Deployments
-
Application Secrets
minimal capabilities required to qualify as a container orchestrator
Service meshes generally rely on these underlying layers.
Which is why...
I have an API gateway.
Microservices API Gateways
-
Ambassador uses Envoy
-
Kong uses Nginx
-
OpenResty uses Nginx
north-south vs. east-west
Why use a Service Mesh?
to avoid...
-
Bloated service code
-
Duplicating work to make services production-ready
-
Load balancing, auto scaling, rate limiting, traffic routing...
-
-
Inconsistency across services
-
Retry, tls, failover, deadlines, cancellation, etc., for each language, framework
-
Siloed implementations lead to fragmented, non-uniform policy application and difficult debugging
-
-
Diffusing responsibility of service management
Help with Modernization
-
Can modernize your IT inventory without:
-
Rewriting your applications
-
Adopting microservices, regular services are fine
-
Adopting new frameworks
-
Moving to the cloud
-
address the long-tail of IT services
Get there for free
Service Mesh Architectures
Service Mesh Architecture
Data Plane
- Touches every packet/request in the system.
- Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
No control plane? Not a service mesh.
Control Plane
- Provides policy, configuration, and platform integration.
- Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
- Does not touch any packets/requests in the data path.
Data Plane
- Touches every packet/request in the system.
- Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
Control Plane
Data Plane
- Touches every packet/request in the system.
- Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
- Provides policy, configuration, and platform integration.
- Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
- Does not touch any packets/requests in the data path.
You need a management plane.
Ingress Gateway
Management
Plane
- Provides monitoring, backend system integration, expanded policy and application configuration.
Egress Gateway
Pilot
Citadel
Mixer
Control Plane
Data Plane
istio-system namespace
policy check
Foo Pod
Proxy Sidecar
Service Foo
tls certs
discovery & config
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
reports
Control flow
application traffic
Application traffic
application namespace
telemetry reports
Istio Architecture
Galley
Ingress Gateway
Egress Gateway
Control Plane
Data Plane
octa-system namespace
policy check
Foo Pod
Proxy
Sidecar
Service Foo
discovery & config
Foo Container
Bar Pod
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
reports
Control flow
application traffic
Application traffic
application namespace
telemetry reports
Octarine Architecture
Policy
Engine
Security Engine
Visibility
Engine
+
Proxy
Sidecar
+
Control Plane
Data Plane
linkerd-system namespace
Foo Pod
Proxy Sidecar
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
scarping
Control flow during request processing
application traffic
Application traffic
application namespace
telemetry scraping
Architecture
destination
Prometheus
Grafana
tap
web
CLI
proxy-api
public-api
Linkerd
proxy-injector
Control Plane
Data Plane
Foo Pod
Proxy Sidecar
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
scarping
Control flow during request processing
application traffic
Application traffic
application namespace
telemetry scraping
Architecture
destination
Prometheus
Grafana
NSMe
domain
client
proxy-api
public-api
Network
Service
Mesh
proxy-injector
let's wait until after NSMCon
Service Mesh Deployment Models
Client
Edge Cache
Istio Gateway
(envoy)
Cache Generator
Collection of VMs running APIs
service mesh
Istio VirtualService
Istio VirtualService
Istio ServiceEntry
Situation:
- existing services running on VMs (that have little to no service-to-service traffic).
- nearly all traffic flows from client to the service and back to client.
Benefits:
- gain granular traffic control (e.g path rewrites).
- detailed service monitoring without immediately deploying a thousand sidecars.
Ingress
Out-of-band telemetry propagation
Application traffic
Control flow
Proxy per Node
Service A
Service A
Service A
linkerd
Node (server)
Service A
Service A
Service B
linkerd
Node (server)
Service A
Service A
Service C
linkerd
Node (server)
Advantages:
-
Less (memory) overhead.
-
Simpler distribution of configuration information.
-
primarily physical or virtual server based; good for large monolithic applications.
Disadvantages:
-
Coarse support for encryption of service-to-service communication, instead host-to-host encryption and authentication policies.
-
Blast radius of a proxy failure includes all applications on the node, which is essentially equivalent to losing the node itself.
-
Not a transparent entity, services must be aware of its existence.
layer5.io/books
Advantages:
-
Good starting point for building a brand-new microservices architecture or for migrating from a monolith.
Disadvantages:
-
When the number of services increase, it becomes difficult to manage.
Router "Mesh"
Fabric Model
Advantages:
-
Granular encryption of service-to-service communication.
-
Can be gradually added to an existing cluster without central coordination.
Disadvantages:
-
Lack of central coordination. Difficult to scale operationally.
Ingress or Edge Proxy
Advantages:
-
Works with existing services that can be broken down over time.
Disadvantages:
-
Is missing the benefits of service-to-service visibility and control.
layer5.io/landscape
Service Mesh Abstractions
Service Mesh Interface (SMI)
Multi-Vendor Service Mesh Interoperation
Hamlet
Adopter’s Dilemma
Which service mesh to use?
What's the catch? Nothing's for free.
Playground
WHICH SERVICE MESH SHOULD I USE AND HOW DO I GET STARTED?
Learn about the functionality of different service meshes and visually manipulate mesh configuration.
Performance Benchmark
WHAT OVERHEAD DOES BEING ON THE SERVICE MESH INCUR?
Benchmark the performance of your application across different service meshes and compare their overhead.
layer5.io/meshery
@lcalcote
Configuration
Security
Telemetry
Control Plane
Data
Plane
service mesh ns
Foo Pod
Proxy Sidecar
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
Control flow
application traffic
http / gRPC
Application traffic
application namespace
Meshery Architecture
Ingress Gateway
Egress Gateway
Management
Plane
meshery
adapter
gRPC
kube-api
kube-system
Service Mesh Abstractions
+
Service Mesh Interface (SMI)
+
Multi-Vendor Service Mesh Interoperation
Hamlet
@lcalcote
layer5.io/meshery
Pod Memory Usage
Application resource consumption
@lcalcote
layer5.io/meshery
Pod CPU Usage
Application resource consumption
@lcalcote
layer5.io/meshery
Control Plane Memory
Istio
Linkerd
Consul
@lcalcote
layer5.io/meshery
Control Plane CPU
Istio
Linkerd
Consul
Relative
Showdown!
Slowdown
@lcalcote
layer5.io/meshery
| Cores | Threads | Istio (2) | Linkerd |
|---|---|---|---|
| 8 | 8 | 1 | 1 |
| 8 | 16 | 1.7 | 1.8 |
| 8 | 32 | 3.2 | 3.4 |
| 8 | 100 | 9.3 | 9.6 |
(2) mTLS on, tracing off
Relative
Showdown!Slowdown
@lcalcote
layer5.io/meshery
| Cores | Threads | Istio (1) | Istio (2) | Linkerd |
|---|---|---|---|---|
| 8 | 8 | 1 | 1 | 1 |
| 8 | 16 | 1.4 | 1.7 | 1.8 |
| 8 | 32 | 18.4 | 3.2 | 3.4 |
| 8 | 100 | 52.2 | 9.3 | 9.6 |
(1) mTLS on, tracing on
(2) mTLS on, tracing off
Service Mesh Benchmark Specification
A project and vendor-neutral specification for capturing details of:
-
Environment / Infrastructure
-
Number and size of nodes, orchestrator
-
-
Service mesh and its configuration
-
Service / application details
Bundled with test results.
github.com/layer5io/service-mesh-benchmark-spec
@lcalcote
layer5.io/meshery
@lcalcote
layer5.io/meshery
Lee Calcote
cloud native and its management
layer5.io/subscribe
Cloud Native Austin 2019: Service Meshes - why you need one and which one you need
By Lee Calcote
Cloud Native Austin 2019: Service Meshes - why you need one and which one you need
Presented at Cloud Native Austin in November 2019.
