Lee Calcote
Clouds, containers, functions, applications and their management.
Cloud Native Austin
September 2020
6:00pm-6:15pm: Welcome, Socializing, and Announcements
6:15pm-7:10pm:
-
Talk 1: Service Meshes, But at What Cost?
Lee Calcote, Layer5
Agenda
Us
Local Organizers
Vendor Neutral
Fosters of Community
Open Source Friendly
You
Who’s Hiring?
Who’s Looking?
Other?
Announcements
Presentations
#cloudnativeATX
Service Meshes, but at what cost?
layer5.io
cloud native and its management
Service Mesh Patterns
calcotestudios.com/talks
Prateek Sahu
slack.layer5.io
@PrateekSahu22
prateeksahu.github.io
Third step in Cloud Native journey
Container
Orchestrator
Mesh
5.5 years
(Jun 2014)
4.5 years
(Jul 2015)
3 years
(Apr 2017)
5.5 years ago
(Jun 2014)
7 years ago
(Mar 2013)
4 years ago
(Feb 2016)
v1.0
Announced
What is a Service Mesh?
a services-first network
layer5.io/books/the-enterprise-path-to-service-mesh-architectures
Traffic Control
control over chaos
Resilency
content-based traffic steering
Observability
what gets people hooked on service metrics
Security
identity and policy
Service Mesh Functionality
Expect more from your infrastructure
Help with Modernization
-
Can modernize your IT inventory without:
-
Rewriting your applications
-
Adopting microservices, regular services are fine
-
Adopting new frameworks
-
Moving to the cloud
-
address the long-tail of IT services
Get there for free
Why use a Service Mesh?
-
Bloated service (application) code
-
Duplicating work to make services production-ready
-
Load balancing, auto scaling, rate limiting, traffic routing...
-
-
Inconsistency across services
-
Retry, tls, failover, deadlines, cancellation, etc., for each language, framework
-
Siloed implementations lead to fragmented, non-uniform policy application and difficult debugging
-
-
Diffusing responsibility of service management
to avoid...
DEV
OPS
Decoupling at Layer 5
where Dev and Ops meet
Empowered and independent teams can iterate faster
DEV
OPS
Decoupling at Layer 5
where Dev and Ops meet
Empowered and independent teams can iterate faster
Service Mesh Architectures
Control Plane
Data Plane
-
Touches every packet/request in the system.
-
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
-
Provides policy, configuration, and platform integration.
-
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
-
Does not touch any packets/requests in the data path.
You need a management plane.
Ingress Gateway
Management
Plane
-
Provides multi-mesh federation, backend system integration, expanded policy and governance, continuous delivery integration, workflow, chaos engineering, configuration and performance management.
Egress Gateway
Service Mesh Architecture
layer5.io/landscape
It's meshy out there.
Service mesh abstractions
Meshery is interoperable with these abstractions.
Service Mesh Interface
(SMI)
Multi-Vendor Service Mesh Interoperation (Hamlet)
Service Mesh Performance Specification (SMPS)
A standard interface for service meshes on Kubernetes.
A set of API standards for enabling service mesh federation.
A format for describing and capturing service mesh performance.
to the rescue
- A vendor neutral specification for capturing details of infrastructure capacity, service mesh configuration, and workload metadata.
-
a universal performance index to gauge a service mesh’s efficiency against deployments in other organizations’ environments.
-
apples-to-apples performance comparisons of service mesh deployments.
Service Mesh Performance
vendor neutral service mesh performance measurement standard
Directly provides:
Facilitates:
SIG Network: Service Mesh WG
Service Mesh Interface (SMI)
Service Mesh Performance (SMP)
Performance Management
Understand value vs overhead
Configuration Best Practices
Meshery analyzes your service mesh and workload configuration
operate with confidence
Assess your service mesh configuration against deployment and operational best practices with Meshery's configuration validator.
Configuration
Security
Telemetry
Control Plane
Data
Plane
service mesh ns
Foo Pod
Proxy Sidecar
Service Foo
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Out-of-band telemetry propagation
Control flow
application
traffic
Application traffic
application namespace
Meshery Architecture
Ingress Gateway
Egress Gateway
Management
Plane
meshery
adapter
gRPC
kube-api
kube-system
generated load
http / gRPC traffic
load generator
fortio
wrk2
nighthawk
UI
server
workloads
Meshery WASM Filter
CLI
Image Hub
github.com/layer5io/image-hub
| Functionality | In the app | In the filter |
|---|---|---|
| User / Token | ||
| Subscription Plans | ||
| Plan Enforcement |
a sample app
Two
application containers
Hub UI Pod
Image Storage Container
Image Storage Pod
Hub UI
Container
Image Storage Service
Hub UI Service
github.com/layer5io/image-hub
Image Hub on Docker Desktop
Hub UI Pod
Image Access Container
Image Access Pod
Hub UI
Container
Image Access Service
Hub UI Service
Image Hub on a Service Mesh
Envoy sidecar
Envoy sidecar
github.com/layer5io/image-hub
with Consul
Leader
Follower
Consul Servers
Follower
agent
Consul Client
node
Leader
Agent
Control Plane
Data Plane
intentions
Foo Pod
Proxy Sidecar
Service Foo
discovery, config,
tls certs
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Control flow
application traffic
Application traffic
application namespace
Follower
Consul Client
Consul Servers
Follower
policy
check
Consul Architecture
layer5.io/service-mesh-architectures
WASM Filter
node
Image Access Container
Image Access Pod
Image Access Service
Image Hub on Consul
Envoy sidecar
github.com/layer5io/image-hub
WASM Filter
with a Rust-based WASM filter
apiVersion: apps/v1
kind: Deployment
spec:
template:
metadata:
labels:
app: api-v1
annotations:
"consul.hashicorp.com/connect-inject": "true"
"consul.hashicorp.com/service-meta-version": "1"
"consul.hashicorp.com/service-tags": "v1"
"consul.hashicorp.com/connect-service-protocol": "http"
"consul.hashicorp.com/connect-wasm-filter-add_header": "/filters/optimized.wasm"
spec:
containers:
- name: api
image: layer5/image-hub-api:latestLeader
Follower
Consul Servers
Follower
agent
node
PERFORMANCE
An optimization game
Latency, throughput, and the proxies’ CPU and memory consumption affected by these factors
Data Plane
Proxy sidecar
App Container
Pod
- Number of client connections
- Target request rate
- Request size and Response size
- Number of proxy worker threads
- Protocol
- CPU cores
- Number and types of proxy filters
with many variables
Data plane performance depends on many factors, for example:
What is WebAssembly?
for the web, malware and beyond
- A small, fast binary format that promises near-native performance for web applications.
- Most modern browsers support it.
- Safe and sandboxed execution environment.
- Over 40 languages that support WASM as a compilation target.
- Originally used to speed up large web-applications.
webassembly.org
Comparing types of Data Plane filtering
Data Plane
Pod
Proxy sidecar
App Container
Comparing approaches to data plane filtering
Data Plane
App Container
Pod
Client Library
Proxy sidecar
Rate limiting with Go client library
-
100 RPS
- p50: 3.19ms
-
500 RPS
- p50: 2.44ms
-
Unlimited RPS - 4417
- p50: 0.66ms
Rate limiting with WASM module (Rust filter)
-
100 RPS
- p50: 2.1ms
-
500 RPS
- p50: 2.22ms
-
Unlimited RPS - 5781
- p50: 0.62ms
Power
Speed
Data Plane
Proxy sidecar
App Container
Pod
Context-based routing
Understanding the trade-off between power and speed
Data Plane
Proxy sidecar
App Container
Pod
Speed
Data Plane
Proxy sidecar
App Container
Pod
Round robin load balancing
Data Plane
Proxy sidecar
App Container
Pod
Path-based routing
Comparing types of functions
Power
Speed
Optimizing your average response time
identifying your optimal configuration for most requests
In the presence of Bucket 1...
...take your largest segment by count and divide by your number of cores
Bucket 2
Bucket 1
Bucket 3
Bucket 4
Performance Testing Best Practices
meshery.io
Use Meshery's powerful performance management features
- easily reproduce tests
- persist test results
- use different load generators
- baseline and compare over time
- test your workloads on and off the mesh
- tweak configurations and try again
- compare 6 different service meshes and counting...
Lee Calcote
Join the discussion
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
No control plane? Not a service mesh.
Control Plane
Provides policy, configuration, and platform integration.
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
Does not touch any packets/requests in the data path.
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Service Mesh Architecture
Cloud Native Austin 2020: Service Meshes, But at What Cost?
By Lee Calcote
Cloud Native Austin 2020: Service Meshes, But at What Cost?
Presented at Cloud Native Austin 2020 in September 2020.
