Einstieg in die Architektur reaktiver Softwaresysteme
Speaker
Alexander Troppmann
Software Craftsman & Architect
- software architecture
- full-stack development
- DevOps / k8s & cloud
- training for dev teams
- agile transformation
a slightly opinionated time travel ;)
Monolithic Software Architecture
Monolithic Software Architecture
Benefits
- simple to model
- simple to develop
- simple to build
- simple to test
- simple to deploy
- simple to scale
- simple to fail ;-)
Monolithic Software Architecture
Drawbacks
- large growing code base
- entropy increases over time (valid not only for the universe but also for software projects :)
- implementation speed slows down
- time-to-market increases
- continuous deployment is difficult
- long startup times of the application
- eats memory & cpu time
Monolithic Software Architecture
More Drawbacks
- overloaded APIs
- onboarding new developers is fun :)
- scales in one dimension
- scaling of dev teams may be difficult
- often lack of reusability of application components
- long-term technology stack
- mostly synchronized function calls
Monolithic Software Architecture
Misunderstanding
Monolithic Software Architecture
meanwhile in a secret room ...
Microservices Architecture
Benefits
-
Continuous Delivery
- better testability
- improved maintainability
- independent deployment of services
-
Bounded Context
- smaller code base
- faster implementation of new features
- faster application startup
- enables multiple, autonomous development teams
Microservices Architecture
More Benefits
-
fault isolation
- fail-safe
- resiliency
- reliability
- (high) availability
- improved scalability
- no long-term technology stack
- enables polyglott services
- replaceable
- enabler for migration strategies
Microservices Architecture
Drawbacks
- complexity of a Distributed System
- inter-service communication
- need of dealing with partial failures
- distributed transactions
- integration testing
- coordination between teams
- experienced developers needed
- complex infrastructure setups needed
Microservices Architecture
Microservices Architecture
Principles I
- Boundary Context
- testability
- polyglott development landscape
- containerization
Reactive Microservices Architecture
Principles II
- redundancy
- resilience
- elasticity
Reactive Microservices Architecture
Principles III
- loose-coupling and strong bonding
- often provides asynchronous API
- communication enforced via Event Bus
- shared event schema
- implementation-agnostic protocol
- communication enforced via Event Bus
- API-driven design
- internal RESTful API
- callback strategies
Reactive Microservices Architecture
Principles IV
- Microservices don't have to be micro ;)
- Miniservices / CNCF
- Self Contained Systems
- use the well-known patterns, e.g.
- 12-Factor
- Init-container
- Side-car
- Ambassador
- Adapter / Proxy / Facade
- Service Discovery / Mesh
Reactive Microservices Architecture
Message Bus
Subject-Based Messaging
- naming conventions for subjects
- use descriptive, natural language names
- do not remove the verbs
- use past tense sentences for events
- build subject hierarchies
Message Bus
Message Bus
Publish-Subscribe
- Single-Writer Principle
- multiple consumers
Message Bus
Request-Reply
- receiver client (subscriber) listens to a subject
- requestor client (publisher) makes a request
- receiver consumes the request and sends reply
Message Bus
Queue Groups
- implements load-balancing
- enables scaling of consumer replicas
Message Bus
Streams I
- Message Store
- duration, size, and interest are limited
- retention policy
- successfully processed messages get ACKnowledged
- re-delivery of failed messages
- support for durable consumers
- rate limiting for consumers
Message Bus
Streams II
- on a Message Streaming Bus you can
- start consuming from the very beginning
- jump in at any point of time or sequence
- continue after exit and restart
- consume just new messages
- possible scenarios
- re-connection after failure
- connection of a very new client
- re-playing of messages having been consumed before
Message Bus
Delivery Modes /QoS
- No-Guarantee
- zero, one time, or multiple deliveries
- At-Most-Once
- zero or more message deliveries
- At-Least-Once
- one or more message deliveries
- duplicates are possible
- Exactly-Once
- one and only one message delivery
- one and only one message delivery
Message Bus
Acknowledgement
- Auto-ACK
- Manual-ACK
- timeout / waiting for ACK
- Max-In-Flight
- acts as Rate Limiter for message delivery
- "1" ensures every message is processed in order
- attention: never ACK message before it has successfully been processed
Message Format
Principles
- specification of event schema via IDL
- protoc compiles IDL spec files to target language
- ProtoBuf is a binary format, but JSON is supported
- support for evolutionary schema design
- allows structuring of messages
Message Format
Structure of a Message
- new types are defined via "messages"
- inner (nested) messages are supported
- import & re-use of existing message definitions
- "oneof" syntax
- "repeated" syntax
- "reserved" and "deprecated" fields
- byte order of message fields
Message Format
Best Practices
- create a centralized Git repo to store event schema
- run a protoc build per target language
- auto-commit generated assets by CI/CD pipeline
- separate event schema into
- meta data
- payload
- think of
- using envelope strategy for e.g. Dead-letters
- merging of messages on binary level
Message Format
Design Patterns
Incremental Retry
- Scenario
- handler fails during consuming a message
- Problem Statement
- event has not been processed successfully
- message should be processed by another instance
- maybe, this issues will reoccur repeatedly
- Proposed Solution
- don't ACK message, it will get redelivered
- increment time delay between redeliveries
Design Patterns
Dead Letter
- Scenario
- handler fails during consuming a message
- processing already failed previously
-
Problem Statement
- messages will be kept on the bus
- retry-count will increase rapidly
-
Proposed Solution
- re-publish failed events as dead letter message
- use a wrapper for dead letter event
- ACK the origin failed message on the bus
Design Patterns
Full Message Payload
- Scenario
- events have lots of data to share
-
Problem Statement
- Microservices should be operated independently
- message bus is able to act as data store
- inter-service communications are too high
-
Proposed Solution
- publisher sends full data payload along with message
- meta data allows the receiver to filter relevant data
Design Patterns
Lightweight Msg Payload
- Scenario
- events have big binary data to share, e.g. media assets
-
Problem Statement
- Microservices should be operated independently
- message bus should not act as data store
- inter-service communications can be high traffic
-
Proposed Solution
- publisher sends RESTful resource link only
- receiver uses resource link to fetch relevant data
Design Patterns
Asynchronous Data Sources
- Scenario
- data is provided by two or more external sources
-
Problem Statement
- all sources are needed to complete the data import
-
Proposed Solution
- implement a light-weight Merger Svc
- wait for all data expected from all related sources
- merge singular events and emit as new message
Design Patterns
Handle Traffic
- Scenario
- our system is composed by a large number of services
-
Problem Statement
- too many events on the bus
- some messages are to big for some consumers
-
Proposed Solution
- implement a light-weight Splitter Svc that slices big message payloads into smaller messages
- use a light-weight Router Svc that consumes messages and pushes them to more specific topics on the bus
Design Patterns
Updated Service Impl
- Scenario
- some Microservice has to be replaced by a new implementation or new version
-
Problem Statement
- we need backwards compatibility
- previous msg schema version needs to be supported
-
Proposed Solution
- implement new container that consumes only messages starting from version X
- keep deprecated container in cluster for version < X
Design Patterns
Chaos Engineering
- Scenario
- we would like to see how Microservices react to events
- testing service resiliency with defect messages
-
Problem Statement
- our system publishes correct messages only ;)
-
Proposed Solution
- implement your own Randomized Event Generator to inject defect messages on the bus
- use the same generator to produce correct messages for testing the behavior of a new service during dev
Design Patterns
Summary: How to build reactive Microservices
Reactive Microservices Architecture
Messaging
- define a message schema via IDL
- platform independency
- evolutionary schema
- meta data and data payload are separated
- apply naming conventions
- consider use of full message payloads
- Single-Writer Principle
- rethink: software components are driven by events
Reactive Microservices Architecture
Application Lifecycle
- what happens during startup?
- chores at shutdown?
Reactive Microservices Architecture
Health Checks
- startup probes: cold start or warm-up needed?
- readiness: ready to serve traffic?
- liveness: alive or dead?
Instrumentation
- Structured Logging
- export of application metrics
- request / event consumer counter
- operational success / failure counter
- latency / duration
- Distributed Tracing
- e.g. add correlation-id to message meta data
Reactive Microservices Architecture
Resiliency & Reliability
- services are allowed and probable to fail
- ability to re-join the running system
- other instances will take over workloads
- idempotency
- services get back into correct and defined state
- incremental retry and failure strategies such as publishing dead letter messages
- redundant data storage
Reactive Microservices Architecture
Microservices Container
- acts as a black box
- is configurable via orchestration best-practices
- k8s config maps
- k8s secrets
- volume mounts
- ENV variables
- behaves like a plugin to our system
- follows at least the https://12factor.net guidelines
- supports asynchronous operations
- supports elasticity by running multiple instances
Reactive Microservices Architecture
THANK YOU!
Alexander Troppmann
Software Craftsman & Architect
- XING: bit.ly/2P4UZoo
- https://www.linkedin.com/in/atroppmann/
- me@talex.dev
Einstieg in die Architektur reaktiver Softwaresysteme
By Alexander Troppmann
Einstieg in die Architektur reaktiver Softwaresysteme
Slides für CloudLand Talk am Donnerstag, 30. Juni 2022 um 10 Uhr, in der Green Area.
- 108