Lee Calcote
Clouds, containers, functions, applications and their management.
It's a Multi-Mesh World
Lee Calcote
Founder, Layer5
@lcalcote
cloud native and its management
Service Mesh Patterns
slack.layer5.io
What is a Service Mesh?
a services-first network
layer5.io/books/the-enterprise-path-to-service-mesh-architectures
Traffic Control
control over chaos
Resilency
content-based traffic steering
Observability
what gets people hooked on service metrics
Security
identity and policy
Service Mesh Functionality
Expect more from your infrastructure
Business Logic
in-network application logic
DEVELOPER
OPERATOR
Decoupling at Layer 5
where Dev, Ops, Product meet
Empowered and independent teams can iterate faster
PRODUCT OWNER
DEVELOPER
OPERATOR
Decoupling at Layer 5
where Dev, Ops, Product meet
Empowered and independent teams can iterate faster
PRODUCT OWNER
layer5.io/landscape
It's meshy out there.
Strengths of Service Mesh Implementations
Different tools for different use cases
a sample
Service Mesh Architectures
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
No control plane? Not a service mesh.
Control Plane
Provides policy, configuration, and platform integration.
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
Does not touch any packets/requests in the data path.
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
Control Plane
Data Plane
-
Touches every packet/request in the system.
-
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
-
Provides policy, configuration, and platform integration.
-
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
-
Does not touch any packets/requests in the data path.
You need a management plane.
Ingress Gateway
Management
Plane
-
Provides backend system integration, expanded policy and governance, continuous delivery integration, workflow, chaos engineering,  configuration and performance management and multi-mesh federation.
Egress Gateway
Service Mesh Architecture
Pilot
Citadel
Mixer
Control Plane
Data Plane
istio-system namespace
policy check
Foo Pod
Proxy Sidecar
Service Foo
tls certs
discovery & config
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
Â
reports
Control flow
application traffic
Application traffic
application namespace
telemetry reports
Istio Architecture
Galley
Ingress Gateway
Egress Gateway
Leader
Agent
Control Plane
Data Plane
intentions
Foo Pod
Proxy Sidecar
Service Foo
discovery, config,
Â
tls certs
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Control flow
application traffic
Application traffic
application namespace
Follower
Consul Client
Consul Servers
Follower
policy
Â
check
Consul Architecture
layer5.io/service-mesh-architectures
WASM Filter
node
Control Plane
Data Plane
linkerd-system namespace
Foo Pod
Proxy Sidecar
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
Â
scarping
Control flow during request processing
application traffic
Application traffic
application namespace
telemetry scraping
Architecture
destination
Prometheus
Grafana
tap
web
CLI
proxy-api
public-api
Linkerd
proxy-injector
Control Plane
Data Plane
octa-system namespace
policy check
Foo Pod
Proxy
Sidecar
Service Foo
discovery & config
Foo Container
Bar Pod
Service Bar
Bar Container
Out-of-band telemetry propagation
telemetry
Â
reports
Control flow
application traffic
Application traffic
application namespace
telemetry reports
Octarine Architecture
Policy
Engine
Security Engine
Visibility
Engine
+
Proxy
Sidecar
+
Service Mesh Deployment Models
Client
Edge Cache
Istio Gateway
(envoy)
Cache Generator
Collection of VMs running APIs
service mesh
Istio VirtualService
Istio VirtualService
Istio ServiceEntry
Situation:
existing services running on VMs (that have little to no service-to-service traffic).
nearly all traffic flows from client to the service and back to client.
Â
Benefits:
gain granular traffic control (e.g path rewrites).
detailed service monitoring without immediately deploying a thousand sidecars.
Ingress
Out-of-band telemetry propagation
Application traffic
Control flow
Proxy per Node
Service A
Service A
Service A
maesh
Node (server)
Service A
Service A
Service B
maesh
Node (server)
Service A
Service A
Service C
maesh
Node (server)
Advantages:
-
Less (memory) overhead.
-
Simpler distribution of configuration information.
-
primarily physical or virtual server based; good for large monolithic applications.
Â
Disadvantages:
-
Coarse support for encryption of service-to-service communication, instead host-to-host encryption and authentication policies.
-
Blast radius of a proxy failure includes all applications on the node, which is essentially equivalent to losing the node itself.
-
Not a transparent entity, services must be aware of its existence.
layer5.io/books
Advantages:
-
Good starting point for building a brand-new microservices architecture or for migrating from a monolith.
Disadvantages:
-
When the number of services increase, it becomes difficult to manage.
Router "Mesh"
Fabric Model
Advantages:
-
Granular encryption of service-to-service communication.
-
Can be gradually added to an existing cluster without central coordination.
Disadvantages:
-
Lack of central coordination. Difficult to scale operationally.
Ingress or Edge Proxy
Advantages:
-
Works with existing services that can be broken down over time.
Disadvantages:
-
Is missing the benefits of service-to-service visibility and control.
Service mesh abstractionsÂ
Meshery is interoperable with these abstractions.
Service Mesh Interface
(SMI)
Multi-Vendor Service Mesh Interoperation (Hamlet)
Service Mesh Performance Specification (SMPS)
A standard interface for service meshes on Kubernetes.
A set of API standards for enabling service mesh federation.
A format for describing and capturing service mesh performance.
to the rescue
-
benchmarking of service mesh performance
 -
exchange of performance information from system-to-system / mesh-to-mesh
 -
apples-to-apples performance comparisons of service mesh deployments.
 -
MeshMark - a universal performance index to gauge a service mesh’s efficiency against deployments in other organizations’ environments
Â
Service Mesh Performance
https://smp-spec.io
Directly provides:
Indirectly facilitates:
- a vendor neutral specification for capturing details of infrastructure capacity, service mesh configuration, and workload metadata.
An optimization game
Latency, throughput, and the proxies’ CPU and memory consumption affected by these factors
Data Plane
Proxy sidecar
App Container
Pod
Â
- Number of client connections
- Target request rate
- Request size and Response size
- Number of proxy worker threads
- Protocol
- CPU cores
- Number and types of proxy filters
Â
with many variables
Data plane performance depends on many factors, for example:
Context-based routing
Understanding the trade-off between power and speed
Data Plane
Proxy sidecar
App Container
Pod
Speed
Data Plane
Proxy sidecar
App Container
Pod
Round robin load balancing
Data Plane
Proxy sidecar
App Container
Pod
Path-based routing
Comparing types of functions
Power
Speed
Comparing types of Data Plane filtering
Data Plane
Pod
Proxy sidecar
App Container
Comparing approaches to data plane filtering
Data Plane
App Container
Pod
Client Library Â
Proxy sidecar
Rate limiting with Go client library
-
100 RPS
- p50: 3.19ms
-
500 RPS
- p50: 2.44ms
-
Unlimited RPS - 4417
- p50: 0.66ms
Rate limiting with WASM module (Rust filter)
-
100 RPS
- p50: 2.1ms
-
500 RPS
- p50: 2.22ms
-
Unlimited RPS - 5781
- p50: 0.62ms
Power
Speed
Data Plane
Proxy sidecar
App Container
Pod
What is WebAssembly?
for the web, malware and beyond
- A small, fast binary format that promises near-native performance for web applications.
- Most modern browsers support it.
- Safe and sandboxed execution environment.
- Over 40 languages that support WASM as a compilation target.
- Originally used to speed up large web-applications.
webassembly.org
Image Access Container
Image Access Pod
Image Access Service
Image Hub on Consul
Envoy sidecar
github.com/layer5io/image-hub
WASM Filter
with a Rust-based WASM filter
apiVersion: apps/v1
kind: Deployment
spec:
template:
metadata:
labels:
app: api-v1
annotations:
"consul.hashicorp.com/connect-inject": "true"
"consul.hashicorp.com/service-meta-version": "1"
"consul.hashicorp.com/service-tags": "v1"
"consul.hashicorp.com/connect-service-protocol": "http"
"consul.hashicorp.com/connect-wasm-filter-add_header": "/filters/optimized.wasm"
spec:
containers:
- name: api
image: layer5/image-hub-api:latestLeader
Follower
Consul Servers
Follower
agent
node
Service Mesh Interface (SMI)
Service Mesh Performance (SMP)
Performance Management
Understand value vs overhead
Configuration Best Practices
Meshery analyzes your service mesh and workload configuration
operate with confidence
Assess your service mesh configuration against deployment and operational best practices with Meshery's configuration validator.
Service Mesh Interface (SMI) Conformance
Operate and upgrade with confirmation of SMI compatibility
✔︎ Learn Layer5 sample application used for validating test assertions.
Â
 ✔︎ Defines compliant behavior.
 ✔︎ Produces compatibility matrix.
 ✔︎ Ensures provenance of results.
 ✔︎ Runs a set of conformance tests.
 ✔︎ Built into participating service mesh’s release pipeline.
Configuration
Security
Telemetry
Control Plane
Data
Plane
service mesh ns
Foo Pod
Proxy Sidecar
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
Control flow
application
traffic
Application traffic
application namespace
Meshery Architecture
Ingress Gateway
Egress Gateway
Management
Plane
meshery
adapter
gRPC
kube-api
kube-system
generated load
http / gRPC traffic
fortio
wrk2
nighthawk
UI
server
workloads
Meshery WASM Filter
CLI
perf analysis
Lee Calcote
Join the discussion
Optimizing your average response time
identifying your optimal configuration for most requests
In the presence of Bucket 1...
...take your largest segment by count and divide by your number of cores
Bucket 2
Bucket 1
Bucket 3
Bucket 4
Performance Testing Best Practices
meshery.io
Use Meshery's powerful performance management features
- easily reproduce tests
- persist test results
- use different load generators
- baseline and compare over time
- test your workloads on and off the mesh
- tweak configurations and try again
- compare 6 different service meshes and counting...
Image Hub
github.com/layer5io/image-hub
| Functionality | In the app | In the filter |
|---|---|---|
| User / Token | ||
| Subscription Plans | ||
| Plan Enforcement |
a sample app
TwoÂ
application containers
Hub UI Pod
Image Storage Container
Image Storage Pod
Hub UI
Container
Image Storage Service
Hub UI Service
github.com/layer5io/image-hub
Image Hub on Docker Desktop
Hub UI Pod
Image Access Container
Image Access Pod
Hub UI
Container
Image Access Service
Hub UI Service
Image Hub on a Service Mesh
Envoy sidecar
Envoy sidecar
github.com/layer5io/image-hub
with Consul
Leader
Follower
Consul Servers
Follower
agent
Consul Client
node
What is a Service Mesh?
a services-first network
layer5.io/books/the-enterprise-path-to-service-mesh-architectures
Third step in Cloud Native journey
Container
Orchestrator
Mesh
5.5 years
(Jun 2014)
4.5 years
(Jul 2015)
3 years
(Apr 2017)
5.5 years ago
(Jun 2014)
7 years ago
(Mar 2013)
4 years ago
(Feb 2016)
v1.0
Announced
Open Infrastructure Summit 2020: It's a Multi-Mesh World
By Lee Calcote
Open Infrastructure Summit 2020: It's a Multi-Mesh World
Presented at Open Infrastructure Summit 2020 in October 2020.
