Cross-Domain Events with AsyncAPI and AWS

@level_out

Luke Hedger

The LEGO Group

AsyncAPI Conf - Nov 2022

@level_out

Domain A

Domain B

Domain C

Event-driven architecture: Phase 1

Inter-domain events

@level_out

Domain A

Domain B

Domain C

Event-driven architecture: Phase 2

Cross-domain events

@level_out

Produce rich payment events in realtime to consumers via transport convenient to them

@level_out

Domains

Shopper checkout

Order submission

Payment requests

Asynchronous authorisations

Order fulfilment

@level_out

Current: pull

Is payment authorised?

Order submitted

Payment authorised

1

2

3

@level_out

Future: push

Order submitted

Payment authorised

1

1

1

2

@level_out

Cross-domain events

• Consumable - simple onboarding, familiar delivery
• Discoverable - well-documented
• Standard - standards-compliant, schema-based, interoperable
• Versioned - predictable payloads, backwards-compatible
• Secure - encrypted, verifiable messages
• Scalable - handle high volume (100s events p/second)
• Resilient - fault tolerant, replayable, high-availability

Standards

AsyncAPI

• The AsyncAPI specification is the industry standard for defining asynchronous APIs
• Growing ecosystem of tools for building and operating event-driven APIs

@level_out

Amazon EventBridge

• A serverless event bus with powerful rule-based filtering
• Already used in domain - no new tech, subscribe to existing events
• Target AWS workloads (cross-account events) and non-AWS consumers (HTTP API Destinations)

@level_out

CloudEvents

• A specification for describing event data in a common way
• Predictable, portable event payloads
• Reduced handling logic

@level_out

Field mapping

@level_out

@level_out

Cross-domain events flow

Discovery

AsyncAPI docs generation

@level_out

$ ag asyncapi.yml @asyncapi/html-template -o asyncapi.out

Published to Backstage

@level_out

Event hierarchy

• Topical hierarchy of event types allows for granular filtering upon consumption
• {Entity}.{Event}.{Metadata}
• Metadata could be locale, platform etc

@level_out

Payment.Authorised.Ecom

Payment.Authorised.Pos

Order.Created.Uk

Basket.Paid.Eu

Event evolution

• Handle changes to events via additions whenever possible
• Introduce new event types if breaking changes are required e.g. Payment.Authorised, Payment.AuthorisedV2, Payment.Completed
• Produce both the old event and the new event for some time/forever to minimise disruption
• Increment AsyncAPI version accordingly - new events = major, updated events = minor, fixes = patch

@level_out

Enrichment

JSON Patch

• Standards-based, schema-driven event payload enrichment
• Reduces custom logic and risk of regression
• JSON Patch operations: add, remove, replace, copy, move

@level_out

Event IDs

• Use ULID (Universally Unique Lexicographically Sortable Identifier) github.com/ulid/spec
• Events can be chronologically sorted by ID

@level_out

01AN4Z07BY79KA1307SR9X4MV3

Idempotency

• Cannot guarantee exactly-once message production/delivery (3rd party and upstream services)
• Support idempotent consumption
• Use content hash (e.g. MD5) for deterministic idempotency key

@level_out

0800fc577294c34e0b28ad2839435945

Security

Nested JSON Web Tokens

• Signed - "is this event from trusted source?"
• Encrypted - sensitive data is encrypted in transit
• Encoded - compressed payloads

@level_out

shared HMAC secret

private key (verified with public key)

Testing

Linting

• Lint AsyncAPI definitions using Spectral
• Built-in support for AsyncAPI
• Custom rules can be used to enforce API styleguides

@level_out

Validation

• Events can be described as JSON Schema documents
• Validate event payloads against event schema in unit tests (e.g. with AJV)
• Producer (and consumers) could also validate at runtime

@level_out

Testing integration points

• Build-time integrations - test Infrastructure as Code (workflow is triggered by event, event bus target config)
• Runtime integrations - test deployed resources (enrichment, event production)
• Balance with observability

@level_out

Contract testing

• Event contract between producer and consumers
• Broker-less contract testing - replace traditional "broker" with a central registry of schema/contracts e.g. GitHub Packages
• Schemas need to be versioned and distributed to consumers
• Test workflow: Assert HTTP response expect.toMatchSchema

@level_out

Observability

Health

• Leverage AWS service metrics to monitor health of resources
• Configure CloudWatch Alarms on key metrics - undeliverable events, workflow failures
• Use custom metrics to collect data points about the application e.g. frequency of known errors

@level_out

Performance

• Trace timelines help identify bottlenecks and inefficient processes
• Dashboards provide an overall view of system performanc

@level_out

Recap

Cross-domain events

• Use AsyncAPI to define, document and distribute event-driven APIs
• Use CloudEvents to structure event payloads
• Use Amazon EventBridge to choreograph events at scale
• Use JSON Patch for event enrichment
• Secure events with signed and encrypted JWTs
• Devise comprehensive test strategy with variety of tests/linting
• Balance testing with observability of application health and performance in production

@level_out

• Still learning, still discussing
• More from me twitter.com/level_out
• These slides 🤳👇

@level_out

Thanks!