Lee Calcote
Clouds, containers, functions, applications and their management.
layer5.io
cloud native and its management
Service Mesh Patterns
calcotestudios.com/talks
Kush Trivedi
Layer5, Maintainer
@kush_1814
Join the community
slack.layer5.io
Third step in Cloud Native journey
Container
Orchestrator
Mesh
5.5 years
(Jun 2014)
4.5 years
(Jul 2015)
3 years
(Apr 2017)
5.5 years ago
(Jun 2014)
7 years ago
(Mar 2013)
4 years ago
(Feb 2016)
v1.0
Announced
layer5.io/landscape
It's meshy out there.
Strengths of Service Mesh Implementations
Different tools for different use cases
a sample
Service mesh abstractions
Meshery is interoperable with these abstractions.
Service Mesh Interface
(SMI)
Multi-Vendor Service Mesh Interoperation (Hamlet)
Service Mesh Performance (SMP)
A standard interface for service meshes on Kubernetes.
A set of API standards for enabling service mesh federation.
A format for describing and capturing service mesh performance.
to the rescue
DEV
OPS
Decoupling at Layer 5
where Dev and Ops meet
Empowered and independent teams can iterate faster
DEV
OPS
Decoupling at Layer 5
where Dev and Ops meet
Empowered and independent teams can iterate faster
Service Mesh Interface (SMI)
Service Mesh Performance (SMP)
Configuration
Security
Telemetry
Control Plane
Data
Plane
service mesh ns
Foo Pod
Proxy Sidecar
Service Foo
App Container
Bar Pod
Proxy Sidecar
Service Bar
App Container
Out-of-band telemetry propagation
Control flow
application
traffic
Application traffic
application namespace
Meshery Architecture
Ingress Gateway
Egress Gateway
Management
Plane
meshery
adapters
gRPC
kube-api
kube-system
generated load
http / gRPC traffic
fortio
wrk2
nighthawk
UI
API
workloads
Meshery WASM Filter
CLI
perf analysis
patterns
@mesheryio
operator
Service Mesh Interface
Four API specifications
Service Mesh Interface aims to provide:
• Standard interface for service mesh on Kubernetes
• Basic feature set for most common mesh use cases
• Extensible to support new features
• Space for the ecosystem to innovate
Traffic Split
Traffic Specs
Traffic Metrics
Traffic Access Control
APIs:
Service Mesh Interface (SMI) Conformance
Operate and upgrade with confirmation of SMI compatibility
✔︎ Learn Layer5 sample application used for validating test assertions.
✔︎ Defines compliant behavior.
✔︎ Produces compatibility matrix.
✔︎ Ensures provenance of results.
✔︎ Runs a set of conformance tests.
✔︎ Built into participating service mesh’s release pipeline.
Demo
SMI Conformance application - github.com/layer5io/learn-layer5
SMI Conformance
-
benchmarking of service mesh performance
-
exchange of performance information from system-to-system / mesh-to-mesh
-
apples-to-apples performance comparisons of service mesh deployments.
-
MeshMark - a universal performance index to gauge a service mesh’s efficiency against deployments in other organizations’ environments
Service Mesh Performance
https://smp-spec.io
Directly provides:
Indirectly facilitates:
- a vendor neutral specification for capturing details of infrastructure capacity, service mesh configuration, and workload metadata.
An optimization game
Latency, throughput, and the proxies’ CPU and memory consumption affected by these factors
Data Plane
Proxy sidecar
App Container
Pod
- Number of client connections
- Target request rate
- Request size and Response size
- Number of proxy worker threads
- Protocol
- CPU cores
- Number and types of proxy filters
with many variables
Data plane performance depends on many factors, for example:
Comparing types of Data Plane filtering
Data Plane
Pod
Proxy sidecar
App Container
Comparing approaches to data plane filtering
Data Plane
App Container
Pod
Client Library
Proxy sidecar
Power
Speed
Data Plane
Proxy sidecar
App Container
Pod
Context-based routing
Understanding the trade-off between power and speed
Data Plane
Proxy sidecar
App Container
Pod
Speed
Data Plane
Proxy sidecar
App Container
Pod
Round robin load balancing
Data Plane
Proxy sidecar
App Container
Pod
Path-based routing
Comparing types of functions
Power
Speed
Demo
https://smp-spec.io
Service Mesh Performance
Performance Testing Best Practices
Use Meshery's powerful performance management features
- easily reproduce tests
- persist test results
- use different load generators
- baseline and compare over time
- test your workloads on and off the mesh
- tweak configurations and try again
- manage 9 different service meshes and counting...
https://meshery.io
Hamlet
Hamlet specifies a set of API standards for enabling service mesh federation.
Specifications:
service catalog federation
https://github.com/vmware/hamlet
API to authenticate and securely distribute resources between federated service meshes.
API to discover, reach, authenticate, and securely communicate with federated services.
-
Federated Resource Discovery API
- Federated Service Discovery API
Service Mesh Patterns by the Book
Service Mesh Patterns
SIG Network: Service Mesh Working Group
Join the community
slack.layer5.io
Lee Calcote
Kush Trivedi
@kush_1814
Traffic Control
control over chaos
Resilency
content-based traffic steering
Observability
what gets people hooked on service metrics
Security
identity and policy
Service Mesh Functionality
Expect more from your infrastructure
Why use a Service Mesh?
-
Bloated service (application) code
-
Duplicating work to make services production-ready
-
Load balancing, auto scaling, rate limiting, traffic routing...
-
-
Inconsistency across services
-
Retry, tls, failover, deadlines, cancellation, etc., for each language, framework
-
Siloed implementations lead to fragmented, non-uniform policy application and difficult debugging
-
-
Diffusing responsibility of service management
to avoid...
Service Mesh Architectures
Control Plane
Data Plane
-
Touches every packet/request in the system.
-
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
-
Provides policy, configuration, and platform integration.
-
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
-
Does not touch any packets/requests in the data path.
You need a management plane.
Ingress Gateway
Management
Plane
-
Provides multi-mesh federation, backend system integration, expanded policy and governance, continuous delivery integration, workflow, chaos engineering, configuration and performance management.
Egress Gateway
Service Mesh Architecture
Image Hub
github.com/layer5io/image-hub
| Functionality | In the app | In the filter |
|---|---|---|
| User / Token | ||
| Subscription Plans | ||
| Plan Enforcement |
a sample app
Two
application containers
Hub UI Pod
Image Storage Container
Image Storage Pod
Hub UI
Container
Image Storage Service
Hub UI Service
github.com/layer5io/image-hub
Image Hub on Docker Desktop
Hub UI Pod
Image Access Container
Image Access Pod
Hub UI
Container
Image Access Service
Hub UI Service
Image Hub on a Service Mesh
Envoy sidecar
Envoy sidecar
github.com/layer5io/image-hub
with Consul
Leader
Follower
Consul Servers
Follower
agent
Consul Client
node
Leader
Agent
Control Plane
Data Plane
intentions
Foo Pod
Proxy Sidecar
Service Foo
discovery, config,
tls certs
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Control flow
application traffic
Application traffic
application namespace
Follower
Consul Client
Consul Servers
Follower
policy
check
Consul Architecture
layer5.io/service-mesh-architectures
WASM Filter
node
Image Access Container
Image Access Pod
Image Access Service
Image Hub on Consul
Envoy sidecar
github.com/layer5io/image-hub
WASM Filter
with a Rust-based WASM filter
apiVersion: apps/v1
kind: Deployment
spec:
template:
metadata:
labels:
app: api-v1
annotations:
"consul.hashicorp.com/connect-inject": "true"
"consul.hashicorp.com/service-meta-version": "1"
"consul.hashicorp.com/service-tags": "v1"
"consul.hashicorp.com/connect-service-protocol": "http"
"consul.hashicorp.com/connect-wasm-filter-add_header": "/filters/optimized.wasm"
spec:
containers:
- name: api
image: layer5/image-hub-api:latestLeader
Follower
Consul Servers
Follower
agent
node
PERFORMANCE
What is WebAssembly?
for the web, malware and beyond
- A small, fast binary format that promises near-native performance for web applications.
- Most modern browsers support it.
- Safe and sandboxed execution environment.
- Over 40 languages that support WASM as a compilation target.
- Originally used to speed up large web-applications.
webassembly.org
Optimizing your average response time
identifying your optimal configuration for most requests
In the presence of Bucket 1...
...take your largest segment by count and divide by your number of cores
Bucket 2
Bucket 1
Bucket 3
Bucket 4
Performance Management
Understand value vs overhead
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
No control plane? Not a service mesh.
Control Plane
Provides policy, configuration, and platform integration.
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
Does not touch any packets/requests in the data path.
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
Comparing types of Data Plane filtering
Data Plane
Pod
Proxy sidecar
App Container
+
Comparing approaches to data plane filtering
Data Plane
Proxy sidecar
App Container
Data Plane
App Container
Pod
Pod
Client Library
Proxy sidecar
Proxy sidecar
Power
Speed
KubeCon NA 2020: Service Mesh Specifications and Why They Matter in Your Deployment
By Lee Calcote
KubeCon NA 2020: Service Mesh Specifications and Why They Matter in Your Deployment
Presented at KubeCon NA 2020 in November 2020.
