Einstieg in die Architektur reaktiver Softwaresysteme

a slightly opinionated time travel ;)

Monolithic Software Architecture

https://microservices.io/patterns/monolithic.html

Monolithic Software Architecture

Benefits

• simple to model
• simple to develop
• simple to build
• simple to test
• simple to deploy
• simple to scale
• simple to fail ;-)

Monolithic Software Architecture

Drawbacks

• large growing code base
• entropy increases over time (valid not only for the universe but also for software projects :)
• implementation speed slows down
• time-to-market increases
• continuous deployment is difficult
• long startup times of the application
• eats memory & cpu time

Monolithic Software Architecture

More Drawbacks

• overloaded APIs
• onboarding new developers is fun :)
• scales in one dimension
• scaling of dev teams may be difficult
• often lack of reusability of application components
• long-term technology stack
• mostly synchronized function calls

Monolithic Software Architecture

Misunderstanding

Monolithic Software Architecture

meanwhile in a secret room ...

Microservices Architecture

https://medium.com/hengky-sanjaya-blog/monolith-vs-microservices-b3953650dfd

Benefits

• Continuous Delivery
  • better testability
  • ​improved maintainability
  • independent deployment of services
• ​Bounded Context
  • ​​smaller code base
  • faster implementation of new features
  • faster application startup​
• enables multiple, autonomous development teams

Microservices Architecture

More Benefits

• fault isolation
  • ​fail-safe
  • resiliency
  • reliability
  • (high) availability
• improved scalability
• ​no long-term technology stack
• enables polyglott services
• replaceable
• enabler for migration strategies

Microservices Architecture

Drawbacks

• complexity of a Distributed System
  • inter-service communication
  • need of dealing with partial failures
  • distributed transactions
  • integration testing
  • coordination between teams
• experienced developers needed
• complex infrastructure setups needed

Microservices Architecture

Microservices Architecture

https://microservices.io/patterns/monolithic.html

https://www.reactivemanifesto.org/​

Principles I

• Boundary Context
• testability
• polyglott development landscape
• containerization

Reactive Microservices Architecture

Principles II

• redundancy
• resilience
• elasticity

Reactive Microservices Architecture

Principles III

• loose-coupling and strong bonding
• often provides asynchronous API
  • communication enforced via Event Bus
    • shared event schema
    • implementation-agnostic protocol
• API-driven design
  • internal RESTful API
  • callback strategies

Reactive Microservices Architecture

Principles IV

• Microservices don't have to be micro ;)
  • Miniservices / CNCF
  • Self Contained Systems
• use the well-known patterns, e.g.
  • 12-Factor
  • Init-container
  • Side-car
  • Ambassador
  • Adapter / Proxy / Facade
  • Service Discovery / Mesh

Reactive Microservices Architecture

Message Bus

https://nats.io

Subject-Based Messaging

• naming conventions for subjects
  • use descriptive, natural language names
  • do not remove the verbs
  • use past tense sentences for events
• build subject hierarchies

Message Bus

Message Bus

Publish-Subscribe

• Single-Writer Principle
• multiple consumers

Message Bus

Request-Reply

• receiver client (subscriber) listens to a subject
• requestor client (publisher) makes a request
• receiver consumes the request and sends reply

Message Bus

Queue Groups

• implements load-balancing
• enables scaling of consumer replicas

Message Bus

Streams I

• Message Store
  • duration, size, and interest are limited
  • retention policy
• successfully processed messages get ACKnowledged
• re-delivery of failed messages
• support for durable consumers
• rate limiting for consumers

Message Bus

Streams II

• on a Message Streaming Bus you can
  • start consuming from the very beginning
  • jump in at any point of time or sequence
  • continue after exit and restart
  • consume just new messages
• possible scenarios
  • re-connection after failure
  • connection of a very new client
  • re-playing of messages having been consumed before

Message Bus

Delivery Modes /QoS

• No-Guarantee
  • zero, one time, or multiple deliveries
• At-Most-Once
  • zero or more message deliveries
• At-Least-Once
  • one or more message deliveries
  • duplicates are possible
• Exactly-Once
  • one and only one message delivery
     

Message Bus

Acknowledgement

• Auto-ACK
• Manual-ACK
• timeout / waiting for ACK
• Max-In-Flight
  • acts as Rate Limiter for message delivery
  • "1" ensures every message is processed in order
• attention: never ACK message before it has successfully been processed

Message Format

https://developers.google.com/protocol-buffers

Principles

• specification of event schema via IDL
• protoc compiles IDL spec files to target language
• ProtoBuf is a binary format, but JSON is supported
• support for evolutionary schema design
• allows structuring of messages

Message Format

Structure of a Message

• new types are defined via "messages"
• inner (nested) messages are supported
• import & re-use of existing message definitions
• "oneof" syntax
• "repeated" syntax
• "reserved" and "deprecated" fields
• byte order of message fields

Message Format

Best Practices

• create a centralized Git repo to store event schema
  • run a protoc build per target language
  • auto-commit generated assets by CI/CD pipeline
• separate event schema into
  • meta data
  • payload
• think of
  • using envelope strategy for e.g. Dead-letters
  • merging of messages on binary level

Message Format

Design Patterns

Incremental Retry

• Scenario
  • handler fails during consuming a message
• Problem Statement
  • event has not been processed successfully
  • message should be processed by another instance
  • maybe, this issues will reoccur repeatedly
• Proposed Solution
  • don't ACK message, it will get redelivered
  • increment time delay between redeliveries

Design Patterns

Dead Letter

• Scenario
  • handler fails during consuming a message
  • processing already failed previously
• Problem Statement
  • messages will be kept on the bus
  • retry-count will increase rapidly
• Proposed Solution
  • re-publish failed events as dead letter message
  • use a wrapper for dead letter event
  • ACK the origin failed message on the bus

Design Patterns

Full Message Payload

• Scenario
  • events have lots of data to share
• Problem Statement
  • Microservices should be operated independently
  • message bus is able to act as data store
  • inter-service communications are too high
• Proposed Solution
  • publisher sends full data payload along with message
  • meta data allows the receiver to filter relevant data

Design Patterns

Lightweight Msg Payload

• Scenario
  • events have big binary data to share, e.g. media assets
• Problem Statement
  • Microservices should be operated independently
  • message bus should not act as data store
  • inter-service communications can be high traffic
• Proposed Solution
  • publisher sends RESTful resource link only
  • receiver uses resource link to fetch relevant data

Design Patterns

Asynchronous Data Sources

• Scenario
  • data is provided by two or more external sources
• Problem Statement
  • all sources are needed to complete the data import
• Proposed Solution
  • implement a light-weight Merger Svc
  • wait for all data expected from all related sources
  • merge singular events and emit as new message

Design Patterns

Handle Traffic

• Scenario
  • our system is composed by a large number of services
• Problem Statement
  • too many events on the bus
  • some messages are to big for some consumers
• Proposed Solution
  • implement a light-weight Splitter Svc that slices big message payloads into smaller messages
  • use a light-weight Router Svc that consumes messages and pushes them to more specific topics on the bus

Design Patterns

Updated Service Impl

• Scenario
  • some Microservice has to be replaced by a new implementation or new version
• Problem Statement
  • we need backwards compatibility
  • previous msg schema version needs to be supported
• Proposed Solution
  • implement new container that consumes only messages starting from version X
  • keep deprecated container in cluster for version < X

Design Patterns

Chaos Engineering

• Scenario
  • we would like to see how Microservices react to events
  • testing service resiliency with defect messages
• Problem Statement
  • our system publishes correct messages only ;)
• Proposed Solution
  • implement your own Randomized Event Generator to inject defect messages on the bus
  • use the same generator to produce correct messages for testing the behavior of a new service during dev

Design Patterns

Summary: How to build reactive Microservices

Reactive Microservices Architecture

Messaging

• define a message schema via IDL
  • platform independency
  • evolutionary schema
  • meta data and data payload are separated
  • apply naming conventions
• consider use of full message payloads
• Single-Writer Principle
• rethink: software components are driven by events

Reactive Microservices Architecture

Application Lifecycle

• what happens during startup?
• chores at shutdown?

Reactive Microservices Architecture

Health Checks

• startup probes: cold start or warm-up needed?
• readiness: ready to serve traffic?
• liveness: alive or dead?

Instrumentation

• Structured Logging
• export of application metrics
  • request / event consumer counter
  • operational success / failure counter
  • latency / duration
• Distributed Tracing
  • e.g. add correlation-id to message meta data

Reactive Microservices Architecture

Resiliency & Reliability

• services are allowed and probable to fail
• ability to re-join the running system
  • other instances will take over workloads
  • idempotency
  • services get back into correct and defined state
• incremental retry and failure strategies such as publishing dead letter messages
• redundant data storage

Reactive Microservices Architecture

Microservices Container

• acts as a black box
• is configurable via orchestration best-practices
  • k8s config maps
  • k8s secrets
  • volume mounts
  • ENV variables
• behaves like a plugin to our system
• follows at least the https://12factor.net guidelines
• supports asynchronous operations
• supports elasticity by running multiple instances

Reactive Microservices Architecture