Event-Driven Python
Serverless Microservices
bene@theodo.co.uk
Ben Ellerby
@EllerbyBen
Ben Ellerby
@EllerbyBen
serverless-transformation
@EllerbyBen
Serverless
What is this Serverless thing?
Serverless is the future of the Cloud, a range of services that allow you to rapidly build and run applications without compromising on security or cost.
@EllerbyBen
What is this Serverless thing?
-
Architectural movement
- “allows you to build and run applications and services without thinking about servers” — AWS
- Developers send application code which is run by the cloud provider in isolated containers abstracted from the developer.
- Use 3rd party services used to manage backend logic and state (e.g. Firebase, Cognito)
- A framework with the same name
@EllerbyBen
Why Serverless?
💰 Cost reduction
👷♂️ #NoOps... well LessOps
💻 Developers focus on delivering business value
📈 More scalable
🌳 Greener
@EllerbyBen
Not just Lambda (FaaS)
Lambda
S3
Dynamo
API Gateway
Compute
Storage
Data
API Proxy
Cognito
Auth
SQS
Queue
Step Functions
Workflows
EventBridge
Bus
Runtimes
@EllerbyBen
@EllerbyBen
3.7, 3.6, 2.7
A Simple Function
@EllerbyBen
def handler_name(event, context):
...
return some_valueEvent: The event body
dict, list, str, int, float, or NoneType
Context: Meta Data
dict
Lambda Triggers
@EllerbyBen
Power and Flexibility to build...
@EllerbyBen
Big Ball of Mud
A software system that lacks a perceivable architecture. Although undesirable from a software engineering point of view, such systems are common in practice due to business pressures, developer turnover and code entropy. They are a type of design anti-pattern.
@EllerbyBen
Microservices
@EllerbyBen
The Lambda Pinball
@EllerbyBen
Lambda pinball architectures characteristically lose sight of important domain logic in the tangled web of lambdas, buckets and queues as requests bounce around increasingly complex graphs of cloud services.
Big Ball of Mud
A software system that lacks a perceivable architecture. Although undesirable from a software engineering point of view, such systems are common in practice due to business pressures, developer turnover and code entropy. They are a type of design anti-pattern.
@EllerbyBen
Microservices work well when:
@EllerbyBen
- Are split into clear Services
- Can be deployed independently
- Only communicate with each other asynchronously
- Master their own data
This is nothing new...
@EllerbyBen
- The Service-Oriented Architecture (SOA) movement, that preceded Microservices, had many of the same tenants
The ESB
@EllerbyBen
The ESB
@EllerbyBen
- Enterprise: Implies use in large Enterprise organisation as often used to tackle complexity in these domains along with the historically large infrastructure investment needed.
The ESB
@EllerbyBen
-
Service: As this is providing a way for the different Services (logical self-contained representations of business processes) to communicate.
The ESB
@EllerbyBen
- Bus: Referencing the hardware element of computers that allows the transfer of signals between different components.
The Event
@EllerbyBen
- Event: “A significant change in state” — K. Mani Chandy
Enter EventBridge
@EllerbyBen
Amazon EventBridge: "Serverless event bus that connects application data from your own apps, SaaS, and AWS services" -AWS
Enter EventBridge
@EllerbyBen
The biggest Serverless announcement since the release of AWS Lambda.
The key component to building state-of-the-art Serverless EDAs.
@EllerbyBen
The ESB is dead, long live the ESB
@EllerbyBen
- The difference is it's completely Serverless, with no management required.
- Simple integration with existing AWS Services
"Avoiding the Lambda Pinball"
@EllerbyBen
How do I start?
@EllerbyBen
🤷♂️
@EllerbyBen
Domain-Driven Design and Event Storming
- Event Storming, an extension to DDD, from Alberto Brandolini is a workshop based approach to discover your domain Events, Boundaries and Entities (or Aggregates)
@EllerbyBen
EventBridge Storming
EventBridge Storming: "A specific variant of EventStorming that reduces rework and tight-coupling for teams building state-of-the-art Serverless Event-Driven Architectures with EventBridge."
@EllerbyBen
EventBridge Storming
@EllerbyBen
1. Event Discovery
@EllerbyBen
2. Temporal Sequencing
@EllerbyBen
4. Categorize into Entities (& Aggregates)
@EllerbyBen
5. Categorization into Bounded Contexts
@EllerbyBen
Bounded Context
A set of Language Consistency.
-
Single team
-
Few stakeholders
Optimize to eliminate dependencies across bounded contexts
Same Aggregate can appear in more than one BC, this implies both use it in different ways and duplication will reduce coupling.
@EllerbyBen
6. Name Microservices
- One Bounded Context may have multiple Microservices corresponding to its underlying Aggregates & Entities
@EllerbyBen
6. Name Microservices
- ⚠️If an Entity or Aggregate appears in two Bounded Contexts, this implies there should be two independent Microservices to handle the divergent business processes in these two contexts.
@EllerbyBen
7. Creating a Single EventBridge Event Bus
@EllerbyBen
8. Building a Shared Schema
- Teams should be able to work on and deploy Services independently.
- Teams need to agree on the structure of Events, their Schema.
- Title, structure and types.
- Teams should share Schema, not data and code.
@EllerbyBen
EventBridge Schema Registry
@EllerbyBen
EventBridge Schema Registry
"Event": {
"type": "object",
"properties": {
"ordinal": {
"type": "number",
"format": "int64"
},
"name": {
"type": "string"
},
"price": {
"type": "number",
"format": "double"
},
"address": {
"type": "string"
},
"comments": {
"type": "array",
"items": {
"type": "string"
}
},
"created_at": {
"type": "string",
"format": "date-time"
}
}
}@EllerbyBen
Result
@EllerbyBen
EventBridge Storming
EventBridge Storming: "A specific variant of EventStorming that reduces rework and tight-coupling for teams building state-of-the-art Serverless Event-Driven Architectures with EventBridge."
@EllerbyBen
EventBridge Storming
@EllerbyBen
How to trigger an Event?
🤷♂️
Sidebar
@EllerbyBen
@EllerbyBen
@EllerbyBen
@EllerbyBen
@EllerbyBen
@SlsDevTools
@EllerbyBen
Building State-of-the-art Event-Driven Python Microservices with Serverless in AWS - Python Web Conf
By Ben Ellerby
Building State-of-the-art Event-Driven Python Microservices with Serverless in AWS - Python Web Conf
Serverless is often thought of in terms of FaaS (like AWS Lambda). There are though many other Serverless services, for instance in AWS there is DynamoDB, API Gateway, Cognito and so on. Serverless microservices can be composed of these underlying services to achieve huge flexibility and speed. However, there is the trap of building a distributed monolith if the right tooling and approach are not used. This talk will go through the steps to build a state-of-the-art Python Serverless Microservice architecture based on Amazon Eventbridge through the "EventBridge Storming" approach (an extension of Event Storming from Domain-Driven Design). From idea to code in 8 steps.
