Einstieg in die Architektur reaktiver Softwaresysteme

Speaker

Alexander Troppmann

Software Craftsman & Architect

 

  • software architecture
  • full-stack development
  • DevOps / k8s & cloud
  • training for dev teams
  • agile transformation

a slightly opinionated time travel ;)

Monolithic Software Architecture

Monolithic Software Architecture

Benefits

  • simple to model
  • simple to develop
  • simple to build
  • simple to test
  • simple to deploy
  • simple to scale
  • simple to fail ;-)

Monolithic Software Architecture

Drawbacks

  • large growing code base
  • entropy increases over time (valid not only for the universe but also for software projects :)
  • implementation speed slows down
  • time-to-market increases
  • continuous deployment is difficult
  • long startup times of the application
  • eats memory & cpu time

Monolithic Software Architecture

More Drawbacks

  • overloaded APIs
  • onboarding new developers is fun :)
  • scales in one dimension
  • scaling of dev teams may be difficult
  • often lack of reusability of application components
  • long-term technology stack
  • mostly synchronized function calls

Monolithic Software Architecture

Misunderstanding

Monolithic Software Architecture

meanwhile in a secret room ...

Microservices Architecture

Benefits

  • Continuous Delivery
    • better testability
    • ​improved maintainability
    • independent deployment of services
  • ​Bounded Context
    • ​smaller code base
    • faster implementation of new features
    • faster application startup
  • enables multiple, autonomous development teams

Microservices Architecture

More Benefits

  • fault isolation
    • ​fail-safe
    • resiliency
    • reliability
    • (high) availability
  • improved scalability
  • ​no long-term technology stack
  • enables polyglott services
  • replaceable
  • enabler for migration strategies

Microservices Architecture

Drawbacks

  • complexity of a Distributed System
    • inter-service communication
    • need of dealing with partial failures
    • distributed transactions
    • integration testing
    • coordination between teams
  • experienced developers needed
  • complex infrastructure setups needed

Microservices Architecture

Microservices Architecture

Principles I

  • Boundary Context
  • testability
  • polyglott development landscape
  • containerization

Reactive Microservices Architecture

Principles II

  • redundancy
  • resilience
  • elasticity

Reactive Microservices Architecture

Principles III

  • loose-coupling and strong bonding
  • often provides asynchronous API
    • communication enforced via Event Bus
      • shared event schema
      • implementation-agnostic protocol
  • API-driven design
    • internal RESTful API
    • callback strategies

Reactive Microservices Architecture

Principles IV

  • Microservices don't have to be micro ;)
    • Miniservices / CNCF
    • Self Contained Systems
  • use the well-known patterns, e.g.
    • 12-Factor
    • Init-container
    • Side-car
    • Ambassador
    • Adapter / Proxy / Facade
    • Service Discovery / Mesh

Reactive Microservices Architecture

Message Bus

Subject-Based Messaging

  • naming conventions for subjects
    • use descriptive, natural language names
    • do not remove the verbs
    • use past tense sentences for events
  • build subject hierarchies

Message Bus

Message Bus

Publish-Subscribe

  • Single-Writer Principle
  • multiple consumers

Message Bus

Request-Reply

  • receiver client (subscriber) listens to a subject
  • requestor client (publisher) makes a request
  • receiver consumes the request and sends reply

Message Bus

Queue Groups

  • implements load-balancing
  • enables scaling of consumer replicas

Message Bus

Streams I

  • Message Store
    • duration, size, and interest are limited
    • retention policy
  • successfully processed messages get ACKnowledged
  • re-delivery of failed messages
  • support for durable consumers
  • rate limiting for consumers

Message Bus

Streams II

  • on a Message Streaming Bus you can
    • start consuming from the very beginning
    • jump in at any point of time or sequence
    • continue after exit and restart
    • consume just new messages
  • possible scenarios
    • re-connection after failure
    • connection of a very new client
    • re-playing of messages having been consumed before

Message Bus

Delivery Modes /QoS

  • No-Guarantee
    • zero, one time, or multiple deliveries
  • At-Most-Once
    • zero or more message deliveries
  • At-Least-Once
    • one or more message deliveries
    • duplicates are possible
  • Exactly-Once
    • one and only one message delivery
       

Message Bus

Acknowledgement

  • Auto-ACK
  • Manual-ACK
  • timeout / waiting for ACK
  • Max-In-Flight
    • acts as Rate Limiter for message delivery
    • "1" ensures every message is processed in order
  • attention: never ACK message before it has successfully been processed

Message Format

Principles

  • specification of event schema via IDL
  • protoc compiles IDL spec files to target language
  • ProtoBuf is a binary format, but JSON is supported
  • support for evolutionary schema design
  • allows structuring of messages

Message Format

Structure of a Message

  • new types are defined via "messages"
  • inner (nested) messages are supported
  • import & re-use of existing message definitions
  • "oneof" syntax
  • "repeated" syntax
  • "reserved" and "deprecated" fields
  • byte order of message fields

Message Format

Best Practices

  • create a centralized Git repo to store event schema
    • run a protoc build per target language
    • auto-commit generated assets by CI/CD pipeline
  • separate event schema into
    • meta data
    • payload
  • think of
    • using envelope strategy for e.g. Dead-letters
    • merging of messages on binary level

Message Format

Design Patterns

Incremental Retry

  • Scenario
    • handler fails during consuming a message
  • Problem Statement
    • event has not been processed successfully
    • message should be processed by another instance
    • maybe, this issues will reoccur repeatedly
  • Proposed Solution
    • don't ACK message, it will get redelivered
    • increment time delay between redeliveries

Design Patterns

Dead Letter

  • Scenario
    • handler fails during consuming a message
    • processing already failed previously
  • Problem Statement
    • messages will be kept on the bus
    • retry-count will increase rapidly
  • Proposed Solution
    • re-publish failed events as dead letter message
    • use a wrapper for dead letter event
    • ACK the origin failed message on the bus

 

 

Design Patterns

Full Message Payload

  • Scenario
    • events have lots of data to share
  • Problem Statement
    • Microservices should be operated independently
    • message bus is able to act as data store
    • inter-service communications are too high
  • Proposed Solution
    • publisher sends full data payload along with message
    • meta data allows the receiver to filter relevant data

 

 

Design Patterns

Lightweight Msg Payload

  • Scenario
    • events have big binary data to share, e.g. media assets
  • Problem Statement
    • Microservices should be operated independently
    • message bus should not act as data store
    • inter-service communications can be high traffic
  • Proposed Solution
    • publisher sends RESTful resource link only
    • receiver uses resource link to fetch relevant data

 

 

Design Patterns

Asynchronous Data Sources

  • Scenario
    • data is provided by two or more external sources
  • Problem Statement
    • all sources are needed to complete the data import
  • Proposed Solution
    • implement a light-weight Merger Svc
    • wait for all data expected from all related sources
    • merge singular events and emit as new message

 

 

Design Patterns

Handle Traffic

  • Scenario
    • our system is composed by a large number of services
  • Problem Statement
    • too many events on the bus
    • some messages are to big for some consumers
  • Proposed Solution
    • implement a light-weight Splitter Svc that slices big message payloads into smaller messages
    • use a light-weight Router Svc that consumes messages and pushes them to more specific topics on the bus

 

 

Design Patterns

Updated Service Impl

  • Scenario
    • some Microservice has to be replaced by a new implementation or new version
  • Problem Statement
    • we need backwards compatibility
    • previous msg schema version needs to be supported
  • Proposed Solution
    • implement new container that consumes only messages starting from version X
    • keep deprecated container in cluster for version < X

 

 

Design Patterns

Chaos Engineering

  • Scenario
    • we would like to see how Microservices react to events
    • testing service resiliency with defect messages
  • Problem Statement
    • our system publishes correct messages only ;)
  • Proposed Solution
    • implement your own Randomized Event Generator to inject defect messages on the bus
    • use the same generator to produce correct messages for testing the behavior of a new service during dev

 

 

Design Patterns

Summary: How to build reactive Microservices

Reactive Microservices Architecture

Messaging

  • define a message schema via IDL
    • platform independency
    • evolutionary schema
    • meta data and data payload are separated
    • apply naming conventions
  • consider use of full message payloads
  • Single-Writer Principle
  • rethink: software components are driven by events

Reactive Microservices Architecture

Application Lifecycle

  • what happens during startup?
  • chores at shutdown?

Reactive Microservices Architecture

Health Checks

  • startup probes: cold start or warm-up needed?
  • readiness: ready to serve traffic?
  • liveness: alive or dead?

Instrumentation

  • Structured Logging
  • export of application metrics
    • request / event consumer counter
    • operational success / failure counter
    • latency / duration
  • Distributed Tracing
    • e.g. add correlation-id to message meta data

Reactive Microservices Architecture

Resiliency & Reliability

  • services are allowed and probable to fail
  • ability to re-join the running system
    • other instances will take over workloads
    • idempotency
    • services get back into correct and defined state
  • incremental retry and failure strategies such as publishing dead letter messages
  • redundant data storage

Reactive Microservices Architecture

Microservices Container

  • acts as a black box
  • is configurable via orchestration best-practices
    • k8s config maps
    • k8s secrets
    • volume mounts
    • ENV variables
  • behaves like a plugin to our system
  • follows at least the https://12factor.net guidelines
  • supports asynchronous operations
  • supports elasticity by running multiple instances

Reactive Microservices Architecture

THANK YOU!

Alexander Troppmann

Software Craftsman & Architect

 

  • XING: bit.ly/2P4UZoo
  • https://www.linkedin.com/in/atroppmann/
  • me@talex.dev

Einstieg in die Architektur reaktiver Softwaresysteme

By Alexander Troppmann

Einstieg in die Architektur reaktiver Softwaresysteme

Slides für CloudLand Talk am Donnerstag, 30. Juni 2022 um 10 Uhr, in der Green Area.

  • 117