Lee Calcote
Clouds, containers, functions, applications and their management.
layer5.io
cloud native and its management
Service Mesh Patterns
calcotestudios.com/talks
Join the community
slack.layer5.io
Third step in Cloud Native journey
Container
Orchestrator
Mesh
5.5 years
(Jun 2014)
4.5 years
(Jul 2015)
3 years
(Apr 2017)
5.5 years ago
(Jun 2014)
7 years ago
(Mar 2013)
4 years ago
(Feb 2016)
v1.0
Announced
layer5.io/landscape
It's meshy out there.
Strengths of Service Mesh Implementations
Different tools for different use cases
a sample
Service mesh abstractions
Meshery is interoperable with these abstractions.
Service Mesh Interface
(SMI)
Multi-Vendor Service Mesh Interoperation (Hamlet)
Service Mesh Performance Specification (SMPS)
A standard interface for service meshes on Kubernetes.
A set of API standards for enabling service mesh federation.
A format for describing and capturing service mesh performance.
to the rescue
Service Mesh Deployment Models
Client
Edge Cache
Istio Gateway
(envoy)
Cache Generator
Collection of VMs running APIs
service mesh
Istio VirtualService
Istio VirtualService
Istio ServiceEntry
Situation:
existing services running on VMs (that have little to no service-to-service traffic).
nearly all traffic flows from client to the service and back to client.
Benefits:
gain granular traffic control (e.g path rewrites).
detailed service monitoring without immediately deploying a thousand sidecars.
Ingress
Out-of-band telemetry propagation
Application traffic
Control flow
Proxy per Node
Service A
Service A
Service A
maesh
Node (server)
Service A
Service A
Service B
maesh
Node (server)
Service A
Service A
Service C
maesh
Node (server)
Advantages:
-
Less (memory) overhead.
-
Simpler distribution of configuration information.
-
primarily physical or virtual server based; good for large monolithic applications.
Disadvantages:
-
Coarse support for encryption of service-to-service communication, instead host-to-host encryption and authentication policies.
-
Blast radius of a proxy failure includes all applications on the node, which is essentially equivalent to losing the node itself.
-
Not a transparent entity, services must be aware of its existence.
Advantages:
-
Good starting point for building a brand-new microservices architecture or for migrating from a monolith.
Disadvantages:
-
When the number of services increase, it becomes difficult to manage.
Router "Mesh"
Fabric Model
Advantages:
-
Granular encryption of service-to-service communication.
-
Can be gradually added to an existing cluster without central coordination.
Disadvantages:
-
Lack of central coordination. Difficult to scale operationally.
Ingress or Edge Proxy
Advantages:
-
Works with existing services that can be broken down over time.
Disadvantages:
-
Is missing the benefits of service-to-service visibility and control.
Service Mesh Architectures
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
No control plane? Not a service mesh.
Control Plane
Provides policy, configuration, and platform integration.
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
Does not touch any packets/requests in the data path.
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
Control Plane
Data Plane
-
Touches every packet/request in the system.
-
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
-
Provides policy, configuration, and platform integration.
-
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
-
Does not touch any packets/requests in the data path.
You need a management plane.
Ingress Gateway
Management
Plane
-
Provides multi-mesh federation, backend system integration, expanded policy and governance, continuous delivery integration, workflow, chaos engineering, configuration and performance management.
Egress Gateway
Service Mesh Architecture
Service Mesh Interface (SMI)
Service Mesh Performance (SMP)
SIG Network:
Service Mesh WG
Configuration
Security
Telemetry
Control Plane
Data
Plane
service mesh ns
Foo Pod
Proxy Sidecar
Service Foo
App Container
Bar Pod
Proxy Sidecar
Service Bar
App Container
Out-of-band telemetry propagation
Control flow
application
traffic
Application traffic
application namespace
Meshery Architecture
Ingress Gateway
Egress Gateway
Management
Plane
meshery
adapters
gRPC
kube-api
kube-system
generated load
http / gRPC traffic
fortio
wrk2
nighthawk
UI
API
workloads
Meshery WASM Filter
CLI
perf analysis
patterns
@mesheryio
operator
Configuration Best Practices
Meshery analyzes your service mesh and workload configuration
operate with confidence
Assess your service mesh configuration against deployment and operational best practices with Meshery's configuration validator.
Service Mesh Interface
Four API specifications
Service Mesh Interface aims to provide:
• Standard interface for service mesh on Kubernetes
• Basic feature set for most common mesh use cases
• Extensible to support new features
• Space for the ecosystem to innovate
Traffic Split
Traffic Specs
Traffic Metrics
Traffic Access Control
APIs:
Service Mesh Interface (SMI) Conformance
Operate and upgrade with confirmation of SMI compatibility
✔︎ Learn Layer5 sample application used for validating test assertions.
✔︎ Defines compliant behavior.
✔︎ Produces compatibility matrix.
✔︎ Ensures provenance of results.
✔︎ Runs a set of conformance tests.
✔︎ Built into participating service mesh’s release pipeline.
Demo
SMI Conformance application - github.com/layer5io/learn-layer5
SMI Conformance
-
benchmarking of service mesh performance
-
exchange of performance information from system-to-system / mesh-to-mesh
-
apples-to-apples performance comparisons of service mesh deployments.
-
MeshMark - a universal performance index to gauge a service mesh’s efficiency against deployments in other organizations’ environments
Service Mesh Performance
https://smp-spec.io
Directly provides:
Indirectly facilitates:
- a vendor neutral specification for capturing details of infrastructure capacity, service mesh configuration, and workload metadata.
Performance Management
Understand value vs overhead
Demo
https://smp-spec.io
Service Mesh Performance
What is WebAssembly?
for the web, malware and beyond
- A small, fast binary format that promises near-native performance for web applications.
- Most modern browsers support it.
- Safe and sandboxed execution environment.
- Over 40 languages that support WASM as a compilation target.
- Originally used to speed up large web-applications.
webassembly.org
Comparing types of Data Plane filtering
Data Plane
Pod
Proxy sidecar
App Container
+
Comparing approaches to data plane filtering
Data Plane
Proxy sidecar
App Container
Data Plane
App Container
Pod
Pod
Client Library
Proxy sidecar
Proxy sidecar
Power
Speed
An optimization game
Latency, throughput, and the proxies’ CPU and memory consumption affected by these factors
Data Plane
Proxy sidecar
App Container
Pod
- Number of client connections
- Target request rate
- Request size and Response size
- Number of proxy worker threads
- Protocol
- CPU cores
- Number and types of proxy filters
with many variables
Data plane performance depends on many factors, for example:
Service Mesh Patterns
Service Mesh Patterns
SIG Network: Service Mesh Working Group
by the book
Join the community
slack.layer5.io
Lee Calcote
DEV
OPS
Decoupling at Layer 5
where Dev and Ops meet
Empowered and independent teams can iterate faster
DEV
OPS
Decoupling at Layer 5
where Dev and Ops meet
Empowered and independent teams can iterate faster
Context-based routing
Understanding the trade-off between power and speed
Data Plane
Proxy sidecar
App Container
Pod
Speed
Data Plane
Proxy sidecar
App Container
Pod
Round robin load balancing
Data Plane
Proxy sidecar
App Container
Pod
Path-based routing
Comparing types of functions
Power
Speed
Performance Testing Best Practices
Use Meshery's powerful performance management features
- easily reproduce tests
- persist test results
- use different load generators
- baseline and compare over time
- test your workloads on and off the mesh
- tweak configurations and try again
- manage 9 different service meshes and counting...
https://meshery.io
Hamlet
Hamlet specifies a set of API standards for enabling service mesh federation.
Specifications:
service catalog federation
https://github.com/vmware/hamlet
API to authenticate and securely distribute resources between federated service meshes.
API to discover, reach, authenticate, and securely communicate with federated services.
-
Federated Resource Discovery API
- Federated Service Discovery API
Traffic Control
control over chaos
Resilency
content-based traffic steering
Observability
what gets people hooked on service metrics
Security
identity and policy
Service Mesh Functionality
Expect more from your infrastructure
What is a Service Mesh?
a services-first network
layer5.io/books/the-enterprise-path-to-service-mesh-architectures
Image Hub
github.com/layer5io/image-hub
| Functionality | In the app | In the filter |
|---|---|---|
| User / Token | ||
| Subscription Plans | ||
| Plan Enforcement |
a sample app
Two
application containers
Hub UI Pod
Image Storage Container
Image Storage Pod
Hub UI
Container
Image Storage Service
Hub UI Service
github.com/layer5io/image-hub
Image Hub on Docker Desktop
Hub UI Pod
Image Access Container
Image Access Pod
Hub UI
Container
Image Access Service
Hub UI Service
Image Hub on a Service Mesh
Envoy sidecar
Envoy sidecar
github.com/layer5io/image-hub
with Consul
Leader
Follower
Consul Servers
Follower
agent
Consul Client
node
Leader
Agent
Control Plane
Data Plane
intentions
Foo Pod
Proxy Sidecar
Service Foo
discovery, config,
tls certs
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Control flow
application traffic
Application traffic
application namespace
Follower
Consul Client
Consul Servers
Follower
policy
check
Consul Architecture
layer5.io/service-mesh-architectures
WASM Filter
node
Image Access Container
Image Access Pod
Image Access Service
Image Hub on Consul
Envoy sidecar
github.com/layer5io/image-hub
WASM Filter
with a Rust-based WASM filter
apiVersion: apps/v1
kind: Deployment
spec:
template:
metadata:
labels:
app: api-v1
annotations:
"consul.hashicorp.com/connect-inject": "true"
"consul.hashicorp.com/service-meta-version": "1"
"consul.hashicorp.com/service-tags": "v1"
"consul.hashicorp.com/connect-service-protocol": "http"
"consul.hashicorp.com/connect-wasm-filter-add_header": "/filters/optimized.wasm"
spec:
containers:
- name: api
image: layer5/image-hub-api:latestLeader
Follower
Consul Servers
Follower
agent
node
PERFORMANCE
Optimizing your average response time
identifying your optimal configuration for most requests
In the presence of Bucket 1...
...take your largest segment by count and divide by your number of cores
Bucket 2
Bucket 1
Bucket 3
Bucket 4
Why use a Service Mesh?
-
Bloated service (application) code
-
Duplicating work to make services production-ready
-
Load balancing, auto scaling, rate limiting, traffic routing...
-
-
Inconsistency across services
-
Retry, tls, failover, deadlines, cancellation, etc., for each language, framework
-
Siloed implementations lead to fragmented, non-uniform policy application and difficult debugging
-
-
Diffusing responsibility of service management
to avoid...
All Day DevOps 2020: Solving the Service Mesh Adopter's Dilemma
By Lee Calcote
All Day DevOps 2020: Solving the Service Mesh Adopter's Dilemma
Presented at All Day DevOps 2020 in November 2020.
