A practical DDD approach to Nameko microservices
Alicante, 5th of October
Julio Trigo
About me
● Computer Science Engineer
● University of Alicante Polytechnic School
● Based in London
● Certified Scrum Master (CSM)
● Software developer
juliotrigo
juliotrigo
About me
● ~17 platform services
● ~5 user applications
ClearView Flex
● ~7 external client applications
● Dev team
– Cross-functional
– Works on the whole product
Agenda
➊ Nameko
➋ A (practical) DDD approach to microservices
➌ Splitting code into components
What is this about❓
✅ My experience working with Nameko, microservices, and DDD across several teams and products that has proven successful and beneficial when working on a product codebase
✅ A practical DDD approach
✅ End up with consistent components that are easy to understand, extend and replace
✅ Just a way of splitting code into components using microservices
✅ Nameko introduction and overview
✅ Product codebase that we plan to maintain for a long time
What is this not about❓
❌ The only way of doing things
❌ The "right" way of doing things: DDD, microservices, Nameko, splitting code, etc.
❌ What microservices are
❌ How to deal with microservices: deployment, logging, etc.
❌ Advantages/disadvantages of using microservices
❌ Not the "best way" for all use cases: POC, research, short-lived components and many other cases
❌ Code performance
❌ Nameko internals
Agenda
➊ Nameko
➋ A (practical) DDD approach to microservices
➌ Splitting code into components
Nameko
● Microservices
● What's Nameko?
● Services: how to write and run a service
● Communication protocols: HTTP, RPC, AMQP
● Extensions (entrypoints, dependencies)
● Workers
– What they are and how to use them
– Built-in and community
● Testing
Microservices
"The term "Microservice Architecture" has sprung up over the last few years to describe a particular way of designing software applications as suites of independently deployable services.
While there is no precise definition of this architectural style, there are certain common characteristics around organization around business capability, automated deployment, intelligence in the endpoints, and decentralized control of languages and data." †
† Microservices - James Lewis, Martin Fowler
https://martinfowler.com/articles/microservices.html
Microservices
● Microservices can be a "Replaceable Component Architecture" †
† GOTO 2016 - Software that Fits in Your Head - Dan North
https://www.youtube.com/watch?v=4Y0tOi7QWqM
– if you choose to optimise for replaceability and consistency
Microservices
● Having small services normally comes as a result
● Small services ≠ Microservices
● End goal ≠ to have a big number of small services
● "A framework for building microservices in Python"
● Japanese mushroom, which grows in clusters
● Encourages the dependency injection pattern
● Encourages testability
● Define your own transport mechanisms and service dependencies: mix & match
● Is not a web framework
● "Extensible"
– Compatible with almost any protocol, transport or database
– Built-in extensions, community extensions or "build your own"
● "Focus on business logic"
● "Distributed and scalable"
– Write regular Python methods and classes
– Nameko manages connections, transports and concurrency
– Spin up multiple service instances to easily scale out
– Concurrency by yielding workers when they wait for I/O
Service
● A Nameko service is just a Python class
● Declares dependencies as attributes
● Encapsulates the application logic in its methods
● Methods are exposed with entrypoint decorators
# service.py
from nameko.rpc import rpc
class PaymentService:
name = "payment"
@rpc
def healthcheck(self):
return "OK"
$ nameko run service
starting services: payment
Connected to amqp://guest:**@127.0.0.1:5672//
Running services
# config.yaml
AMQP_URI: "amqp://guest:guest@localhost:5672"
WEB_SERVER_ADDRESS: "0.0.0.0:8000"
max_workers: 10
$ nameko run service --config ./config.yaml
starting services: payment
Connected to amqp://guest:**@127.0.0.1:5672//
● Config
$ nameko shell --broker amqp://guest:guest@localhost:5672/
Nameko Python 3.6.8 (default, Mar 9 2019, 12:20:08)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.38)] shell on darwin
Broker: amqp://guest:guest@localhost:5672/
>>> n.rpc.payment.healthcheck()
'OK'
Interacting with running services
AMQP broker
$ nameko run service
starting services: payment
Connected to amqp://guest:**@127.0.0.1:5672//
Communication protocols (HTTP)
● Built on top of werkzeug
● Supports all the standard HTTP methods
– GET / POST / DELETE / PUT / OPTIONS / ...
Communication protocols (RPC)
● RPC (Remote procedure call) over AMQP
● Normal RPC calls block until the remote method completes
● Asynchronous calling mode to parallelize RPC calls: call_async
● AMQP messages are ack’d only after the request has been successfully processed
● Request and response payloads are serialized into JSON for transport over the wire
Communication protocols (Pub-Sub)
● Publish-Subscriber pattern
● How event messages are received in a cluster:
– SERVICE_POOL (default): event handlers are pooled by service name and one instance from each pool receives the event (similar to RPC)
– BROADCAST: every listening service instance receives the event
– SINGLETON: exactly one listening service instance receives the event
● Request and response payloads are serialized into JSON for transport over the wire
Entrypoints
● Gateways into the service methods they decorate
● They normally monitor an external entity
– Message queue, Redis event, etc.
● On a relevant event:
– The entrypoint “fires”
– The decorated method is executed by a service worker
Entrypoints (built-in: HTTP)
● @http entrypoint
# admin_facade.service.py
from nameko.web.handlers import http
class AdminFacade:
name = "admin-facade"
@http(
"POST",
"/projects/<int:project_id>/orders/<int:order_id>"
)
def confirm_order_payment(
self, request, project_id, order_id
):
# ...
return 200, "OK"
Entrypoints (built-in: HTTP)
$ curl -X POST -i http://localhost:8000/projects/1/orders/200
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 2
Date: Thu, 03 Oct 2019 21:47:07 GMT
# config.yaml
AMQP_URI: "amqp://guest:guest@localhost:5672"
WEB_SERVER_ADDRESS: "0.0.0.0:8000"
$ nameko run admin_facade.service --config ./config.yaml
starting services: admin-facade
Entrypoints (built-in: RCP)
● RpcProxy dependency for services to talk to other services
# admin_facade.service.py
from nameko.rpc import RpcProxy
from nameko.web.handlers import http
class AdminFacade:
name = "admin-facade"
payment_service = RpcProxy("payment")
@http(
"POST",
"/projects/<int:project_id>/orders/<int:order_id>"
)
def confirm_order_payment(
self, request, project_id, order_id
):
# ...
payment = self.payment_service.pay_order(order_id)
# ...
return 200, "OK"
Entrypoints (built-in: RCP)
# payment.service.py
from nameko.rpc import rpc
class PaymentService:
name = "payment"
@rpc
def pay_order(self, order_id):
"""Business logic to pay orders."""
● @rpc entrypoint
Entrypoints (built-in: RCP)
● ServiceRpcProxy: standalone proxy that non-Nameko clients can use to make RPC calls to a cluster
from nameko.standalone.rpc import ServiceRpcProxy
AMQP_URI = "amqp://guest:guest@localhost:5672"
class NonNamekoService:
def process_order(self, order_id):
# ...
with ServiceRpcProxy("payment", AMQP_URI) as rpc:
rpc.pay_order(order_id)
# ...
Entrypoints (built-in: Pub-Sub)
● EventDispatcher dependency for services to emit messages
● @event_handler entrypoint
# payment.service.py
from nameko.events import EventDispatcher
from nameko.rpc import rpc
class PaymentService:
name = "payment"
dispatch = EventDispatcher()
@rpc
def pay_order(self, order_id):
# ...
event_body = {
"order": {"id": order_id},
"payment_method": {"id": 100},
"project": {"id": 200},
"paid_by": {"id": 300},
}
self.dispatch("payment_taken", event_body)
# ...
Entrypoints (built-in: Pub-Sub)
# messaging.service.py
from nameko.events import event_handler
PAYMENT_SERVICE_NAME = "payment"
PAYMENT_TAKEN_EVENT = "payment_taken"
class MessagingService:
name = "messaging"
@event_handler(PAYMENT_SERVICE_NAME, PAYMENT_TAKEN_EVENT)
def send_order_receipt_email(self, payload):
# handler_type = "service_pool"
order = payload["order"]
project = payload["project"]
payment_method = payload["payment_method"]
paid_by = payload["paid_by"]
# ...
Entrypoints (community)
/nameko/nameko-amqp-retry
/nameko/nameko-salesforce
/iky/nameko-slack
/sohonetlabs/nameko-rediskn
– Nameko extension allowing AMQP entrypoints to retry later
– A Nameko extension for handling Salesforce Streaming API
– Nameko extension for interaction with Slack APIs
– Redis Keyspace Notifications extension for Nameko services
Entrypoints (build your own)
● Extensions should subclass nameko.extensions.Entrypoint
● Some methods can be overridden to add functionality
from nameko.extensions import Entrypoint # Inherits from `Extension`
class ExampleEntrypoint(Entrypoint):
# Extension` methods
def setup(self):
"""Called before the container starts.
Do any required initialisation here.
"""
def start(self):
"""Called when the container has successfully started.
Start acting upon external events.
"""
def stop(self):
"""Called when the service container begins to shut down.
Do any graceful shutdown here.
"""
def kill(self):
"""Called to stop this extension without grace.
Urgently shut down here.
"""
# Usage: `@entrypoint_name`
entrypoint_name = ExampleEntrypoint.decorator
Dependencies
● Nameko encourages to declare service dependencies
● Hide/encapsulate code that isn’t part of the core service logic
● Simple interface to the service
● Gateway between service code and everything else
– Other services, external APIs, DBs...
● The class attribute
● DependencyProvider
= a DependencyProvider
≠ the interface that workers can actually use
= a declaration
– Responsible for providing an object that is injected into service workers
– Implement get_dependency(): the result is injected into a newly created worker
– Live for the duration of the service
worker = Service()
worker.other_rpc = worker.other_rpc.get_dependency()
worker.method()
del worker
● The injected dependency: unique to each worker
Dependency injection
Dependencies (built-in)
● RpcProxy for services to talk to other services
● EventDispatcher for services to emit messages
● Config for read-only access to config values at run time
# service.py
from nameko.dependency_providers import Config
from nameko.rpc import rpc
class PaymentService:
name = "payment"
config = Config()
@rpc
def healthcheck(self):
amqp_uri = self.config.get("AMQP_URI")
return "OK"
Dependencies (community)
/nameko/nameko-sqlalchemy
/nameko/nameko-sentry
/etataurov/nameko-redis
/fraglab/nameko-redis-py
– SQLAlchemy dependency for nameko services
– Captures entrypoint exceptions and sends tracebacks to a Sentry server
– Redis dependency for nameko services
– Redis dependency and utils for Nameko
Dependencies (community)
/nameko/nameko-tracer
/nameko/nameko-salesforce
/sohonetlabs/nameko-statsd
/marcuspen/nameko-stripe
– Logs traces: comprehensive information about entrypoints fired
– Nameko dependency provider for easy communication with the Salesforce REST API
– Statsd Nameko dependency provider
– Stripe dependency for Nameko services
Dependencies (build your own)
● Subclass nameko.extensions.DependencyProvider
● Implement a get_dependency() method: returns the object to be injected into service workers
from nameko.extensions import DependencyProvider # Inherits from `Extension`
class ExampleDependencyProvider(DependencyProvider):
# `Extension` methods
# `DependencyProvider` methods
def get_dependency(self, worker_ctx):
""" Called before worker execution.
Return an object to be injected into the worker instance by the container.
"""
def worker_result(self, worker_ctx, result=None, exc_info=None):
""" Called with the result of a service worker execution.
Dependencies that need to process the result should do it here.
This method is called for all `Dependency` instances on completion
of any worker.
"""
def worker_setup(self, worker_ctx):
""" Called before a service worker executes a task.
Dependencies should do any pre-processing here, raising exceptions
in the event of failure.
"""
def worker_teardown(self, worker_ctx):
""" Called after a service worker has executed a task.
Dependencies should do any post-processing here, raising
exceptions in the event of failure.
"""
Workers
Event
Entrypoint fires
Worker is created
Decorated method is executed
Worker is destroyed
Dependencies injected into worker
Instance of the service class
Only lives for the method execution
Result of get_dependency()
Testing
● Nameko’s conventions: make testing as easy as possible
● Dependency injection
● pytest
● Utilities:
– Makes it simple to replace and isolate pieces of functionality
worker_factory()
entrypoint_hook()
replace_dependencies()
pytest fixtures: web_config, rabbit_config, container_factory
Agenda
➊ Nameko
➋ A (practical) DDD approach to microservices
➌ Splitting code into components
DDD & Microservices
● How the design of the solution is driven by the domain of the application
● Contracts between services
– Service boundaries
– Service APIs
● Types of services and responsibilities
– Application services (facades)
– Domain services
● Applications and services
– User applications (internal / external)
– Platform services
– External systems
– Client applications
Web app
Support CP app
Admin CP app
Data analysis app
Applications and services
● External systems
● User applications (internal / external)
● Client applications
● Platform services
Web Facade
Payment
Service
Network app
Web app
Support CP app
Admin CP app
Data analysis app
Single application
API + Business logic
Web app
Support CP app
Admin CP app
Data analysis app
API
Business logic
Web app
Support CP app
Admin CP app
Data analysis app
These architecture approaches
● Single application
● API + business
● Most of the components are deployed together
● Difficult to scale individual parts on demand
● Refactor: riskier to replace / rewrite components
● API / business
● No real separation between components
● Test / debug: more difficult to isolate single components
Web app
Support CP app
Admin CP app
Data analysis app
Web Facade
Business logic
Mobile Facade
Admin Facade
Support CP Facade
Data Facade
Application services (facades)
● Provide an interface to the external application that serves
● Contract between platform services and the user application
– Deploying a facade can only affect one user application
● It only serves one application
– Simplify smoke tests after deployments
– No risk of introducing bugs into other parts of the system
Support CP app
Support CP Facade
Data (request)
Data (response)
Protocol
(e.g. deploying Support CP Facade can only affect support users)
Web app
Support CP app
Admin CP app
Data analysis app
Web
Facade
Payment
Service
Mobile
Facade
Admin
Facade
Support CP Facade
Data
Facade
User
Service
Auth
Service
Project
Service
Messaging
Service
Domain services
● Changes within a business domain only affect one service
✅ Group logic and responsibilities around business domains
Payment
Service
User
Service
Auth
Service
Project
Service
Messaging
Service
● Contract (namespace) shared with the outside world
Contracts between services
● Messages that services produce and consume
● The service API defines the contract with the outside world
● Service boundaries (API):
– Define the endpoints to be exposed
– Define the "domain language" to be exposed
– Define the domain exceptions to be exposed
– Define the communication protocol to be exposed
● Encapsulation
A practical DDD approach
● The design of the solution is driven by the domain of the application
● A practical approach to DDD
✅ "Intelligence in the endpoints"
@rpc
def confirm_payment_intent(self, payment_intent_id):
# ...
@rpc
def get_viewer_scores(self, streaming_session_id, email):
# ...
@rpc
def get_multiple_transactions_by_id(self, transaction_ids):
# ...
A practical DDD approach
✅ Makes refactoring:
✅ Do not share DBs
– Separation between DBs: Redis, SQL, Elasticsearch...
– Could potentially share the same server/instance (reduce costs)
– Different services can't access the same piece of data
– Easier
– Less time consuming
– Less riskier
– e.g. "refactor internal endpoints logic"
– e.g. "replace external 3rd party systems (Stripe, auth provider...)"
A practical DDD approach
❌ Create services about technologies with CRUD operations
– e.g. Elasticsearch service, Redis service...
– e.g. DB service, API service...
❌ Create services about types of tech with CRUD operations
❌ Couple technologies to services (contract)
– Would need a new service per technology
– Instead: facades, domain services, and intelligent endpoints
– Would affect the whole architecture
– Instead: service dependencies and intelligent endpoints (vs. CRUD)
– Replace technologies without affecting the service contract
Agenda
➊ Nameko
➋ A (practical) DDD approach to microservices
➌ Splitting code into components
Code components
● Namespaces
● Complexity metric
● Components
– Purpose and boundaries
● Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
Code components
● “Simple is better than complex.”
● Single responsibility principle
● Simple, intuitive and elegant API
– "A class should have only one reason to change" - Robert C. Martin
PAYMENT_TAKEN_EVENT = "payment_taken"
class TaxCalculationError(Exception):
pass
def validate_country_format(country):
# ...
from nameko_sqlalchemy import DatabaseSession
class PaymentMethod(marshmallow.Schema):
# ...
class PaymentService():
name = "payment"
db = Database()
stripe = Stripe()
@rpc
def list_payment_methods(self, customer_id):
# ...
return schemas.dump(
schemas.PaymentMethod, payment_methods, many=True
)
Namespaces
● “Namespaces are one honking great idea -- let's do more of those!”
● Avoid name collisions
● "Name" / encapsulate / group components
self.db.transactions.get_by_order_id
self.stripe.list_payment_methods
self.db.NotFound
remote_err.NotFound
http_err.NotFound
schemas.dump
schemas.Payment
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
● “If the implementation is hard to explain, it's a bad idea.”
● “Readability counts.”
– More time reading than writing
Admin CP app
Payment Service
Messaging Service
Admin Facade
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
Admin CP app
● Front-end application
● Calls the platform services
– Contract with the Admin Facade
● No back-end logic
● Serverless? AWS Cloudfront, Lambda@Edge, S3
Payment Service
Messaging Service
/admin_facade
http_errors.py
http_handlers.py
remote_errors.py
schemas.py
service.py
/payment_service
dependencies
db.py
nameko_stripe.py
exceptions.py
states.py
/messaging_service
service.py
service.py
schemas.py
models.py
Admin Facade
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# admin_facade.schemas.py
# Responsible of the service messages serializations (in/out)
# Defines the (data) contract with the app that it serves
from marshmallow import fields, Schema
class Payment(Schema):
"""Contract with the Admin CP application."""
id = fields.Str()
status = fields.Str(required=True)
# ...
def dump(schema_cls, data, strict=True, **schema_kwargs):
schema = schema_cls(strict=strict, **schema_kwargs)
return schema.dump(data).data
Schemas
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# admin_facade.http_handlers.py
import json
from werkzeug.wrappers import Response
JSON_CONTENT_TYPE = "application/json"
def json_response(data, status=200, content_type=JSON_CONTENT_TYPE):
return Response(
response=json.dumps(data),
content_type=content_type,
status=status
)
Module
Functions
Constants
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# admin_facade.http_errors.py
class HttpError(Exception):
status_code = 500
def __init__(self, description):
self.payload = description
super().__init__(self.payload)
class BadRequest(HttpError):
status_code = 400
class NotFound(HttpError):
status_code = 404
Exceptions
# admin_facade.remote_errors.py
from nameko.exceptions import registry
def remote_error(exc_path):
def _wrapper(exc_type):
registry[exc_path] = exc_type
return exc_type
return _wrapper
@remote_error('payment.exceptions.BadRequest')
class BadRequest(Exception):
pass
@remote_error('payment.exceptions.NotFound')
class NotFound(Exception):
pass
@remote_error('payment.exceptions.TransactionNotFound')
class TransactionNotFound(Exception):
pass
Exceptions
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Classes
Methods
Functions
Exceptions
Constants
Domain services
# admin_facade.service.py
from nameko.rpc import RpcProxy
from nameko.web.handlers import http
from nameko_sentry import SentryReporter
from . import http_errors as http_err # Namespaces
from . import remote_errors as remote_err
from . import schemas
from .http_handlers import json_response # Widely used / not ambiguous
PAYMENT_SERVICE_NAME = "payment"
class AdminFacade:
name = "admin-facade"
# Dependencies
payment_service = RpcProxy(PAYMENT_SERVICE_NAME)
sentry = SentryReporter()
# ...
Service class
Constants
Endpoints
Admin Facade
# ...
@http(
"POST",
"/projects/<int:project_id>/orders/<int:order_id>"
)
def confirm_order_payment(self, request, project_id, order_id):
# ...
try:
payment = self.payment_service.pay_order(order_id)
# Translate remote "expected exceptions"
except remote_err.BadRequest as err:
raise http_err.BadRequest(err.args[0])
except remote_err.NotFound as err:
raise http_err.NotFound(err.args[0])
except remote_err.TransactionNotFound as err:
# Translate to a different HTTP error
raise http_err.BadRequest(err.args[0])
# Endpoint does not know about formatting/serializing the response
response = schemas.dump(schemas.Payment, payment)
return json_response(response)
Payment Service
Messaging Service
/admin_facade
http_errors.py
http_handlers.py
remote_errors.py
schemas.py
service.py
/payment_service
dependencies
db.py
nameko_stripe.py
exceptions.py
states.py
/messaging_service
service.py
service.py
schemas.py
models.py
Admin Facade
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# payment.dependencies.db.py
from nameko_sqlalchemy import DatabaseSession
from .. import models # The dependency uses the SQLAlchemy models
# Logic related to the dependency lives in the dependency module / package:
# - Constants, exceptions, functions...
# Service endpoints:
# - Do not know about SQL-related objects/actions
# - Interact with the DB objects through "collections"
# Dependency exceptions live in the dependency namespace
class NotFound(LookupError):
pass
Service dependencies
Exceptions
Constants
3rd party packages
# ...
class Transaction:
"""Collection class to provide an interface for the DB.
Provide an abstraction (contract) between the service and the dependency.
"""
name = "transactions"
model = models.Transaction
def __init__(self, session):
self.session = session
def get_by_order_id(self, order_id):
"""Logic to interact with the DB to get the transaction.
Raises `NotFound`
"Intelligent endpoints", like services, not just CRUD operations.
"""
# ...
class DatabaseWrapper:
# Services access the dependency expcetions through the wrapper object
NotFound = NotFound
def __init__(self, session, collections):
for collection in collections:
setattr(self, collection.name, collection(session))
class Database(DatabaseSession):
"""Database dependency provider.
Wrapper around the Nameko SQLAlchemy dependeny provider.
"""
def __init__(self):
super().__init__(models.DeclarativeBase)
def get_dependency(self, worker_ctx):
# A `DatabaseWrapper` instance will be injected to the service
# as a dependency.
return DatabaseWrapper(
session=super().get_dependency(worker_ctx),
collections=[Transaction]
)
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# payment.schemas.py
# Responsible of the service messages serializations (in/out)
# Defines the (data) contract with the rest of the platform services
from marshmallow import fields, Schema
class Payment(Schema):
"""Contract with the rest of the platform services."""
id = fields.Str()
status = fields.Str(required=True)
# ...
def dump(schema_cls, data, strict=True, **schema_kwargs):
schema = schema_cls(strict=strict, **schema_kwargs)
return schema.dump(data).data
Schemas
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# payment.exceptions.py
# Remote exceptions are as explicit and simple as possible:
# - Exception name: specific and explicit
# - Exception argument: error message (string)
class NotFound(Exception):
pass
class BadRequest(Exception):
pass
class TransactionNotFound(Exception):
pass
Exceptions
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# payment.states.py
from enum import auto, Enum
class AutoName(Enum):
def _generate_next_value_(name, start, count, last_values):
return name.lower()
class PaymentState(AutoName):
"""Used to define all the states a Payment can be in.
Encapsulate the states in the `PaymentState` class and
all the logic around them in the `states` module.
"""
AWAITING_CONFIRM = auto()
PAID = auto()
# ...
Module
Classes
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# payment.service.py
from nameko.events import EventDispatcher
from nameko.rpc import rpc
from nameko_sentry import SentryReporter
from .dependencies.db import Database
from .dependencies.nameko_stripe import Stripe
from . import exceptions as exc # Namespaces
from . import schemas
from . import states
PAYMENT_TAKEN_EVENT = "payment_taken"
class PaymentService:
name = "payment"
# Dependencies
db = Database()
dispatch = EventDispatcher()
sentry = SentryReporter()
stripe = Stripe()
# ...
Payment Service
Service class
Constants
Endpoints
# ...
@rpc
def pay_order(self, order_id):
try:
# Endpoint does not know about SQL-related objects/actions
# Interacts with the DB objects through "collections"
transaction = self.db.transactions.get_by_order_id(order_id)
# Translate dependency "expected exceptions"
# Services provide their own exception (contract)
except self.db.NotFound as err:
message = f"Unable to pay order ID {order_id}."
raise exc.TransactionNotFound(message)
try:
payment_methods = self.stripe.list_payment_methods(customer_id)
# Translate dependency "expected exceptions"
# Services provide their own exception (contract)
except self.stripe.NotFound as err:
raise exc.NotFound(err.args[0])
except self.stripe.PaymentMethodError as err:
raise exc.BadRequest(err.args[0])
# ...
# ...
# Compare `transaction` DB collection with payment state
if transaction.state is states.PaymentState.PAID:
raise exc.BadRequest(f"Order {order_id} is already paid.")
try:
payment = self.stripe.make_payment(
customer_id, order_id, payment_method_id
)
# Translate dependency "expected exceptions"
# Services provide their own exception (contract)
except self.stripe.NotFound as err:
raise exc.NotFound(err.args[0])
# Dispatch an event (async) with the payment info
event_body = {
"order": {"id": order_id},
"payment_method": {"id": payment_method_id},
"project": {"id": transaction.project_id},
"paid_by": {"id": user_id},
}
self.dispatch(PAYMENT_TAKEN_EVENT, event_body)
# Endpoint does not know about formatting/serializing
# the response
return schemas.dump(schemas.Payment, payment)
Admin Facade
Payment Service
Messaging Service
/admin_facade
http_errors.py
http_handlers.py
remote_errors.py
schemas.py
service.py
/payment_service
dependencies
db.py
nameko_stripe.py
exceptions.py
states.py
/messaging_service
service.py
service.py
schemas.py
models.py
Splitting code into components
User applications
Facades
Service class
Endpoints
Serialization schemas
Service dependencies
3rd party packages
Modules
Methods
Functions
Exceptions
Constants
Domain services
Classes
# messaging.service.py
from nameko.events import event_handler
# Replace with the Nameko "email_provider" dependency of your choice
from nameko_email_provider import EmailProvider
from nameko_sentry import SentryReporter
PAYMENT_SERVICE_NAME = "payment"
PAYMENT_TAKEN_EVENT = "payment_taken"
class MessagingService:
name = "messaging"
# Dependencies
email_provider = EmailProvider()
sentry = SentryReporter()
# ...
Service class
Constants
Endpoints
Messaging Service
Methods
# ...
@event_handler(PAYMENT_SERVICE_NAME, PAYMENT_TAKEN_EVENT)
def send_order_paid_confirmation_email(self, payload):
"""Create and send order paid confirmation email.
Not just a dumb and generic "send email" endpoint.
It contains the logic to generate and send order paid
confirmation emails.
"""
order = payload["order"]
project = payload["project"]
payment_method = payload["payment_method"]
paid_by = payload["paid_by"]
# ...
self._send_email(recipients, subject, content)
def _send_email(self, recipients, subject, content, **kwargs):
"""Use the email provider dependency to send out emails.
Helper method that acts as a wrapper around the email_provider
dependency. Another approach could be a new "dependency wrapper".
Reused by all the endpoints in the service that need to
send emails.
"""
# ...
self.email_provider.send(
recipients, subject, content, sender, bcc=bcc
)
Complexity metric
● "Code that fits in your head" †
† GOTO 2016 - Software that Fits in Your Head - Dan North. Pattern described by Dan North, referring to James Lewis for coming up with the expression
https://www.youtube.com/watch?v=4Y0tOi7QWqM
● Also refers to a physical head (screen) 🙂
● "You can only reason about something that fits in your head" †
● Concentrate in one component at a time (ignore other parts)
● Fully understand what a component does
● Consistency:
Code formatting, idioms, component design, contracts, technologies (MySQL, PostgreSQL, MariaDB...)
Resources
https://www.nameko.io
nameko
- Clean Code - Robert C. Martin
- Implementing Domain-Driven Design - Vaughn Vernon
- Domain-Driven Design - Eric Evans
>>> import this
The Zen of Python, by Tim Peters
- Microservices - James Lewis, Martin Fowler
https://martinfowler.com/articles/microservices.html
- GOTO 2016 - Software that Fits in Your Head - Dan North
https://www.youtube.com/watch?v=4Y0tOi7QWqM
Thank you!
juliotrigo
juliotrigo
https://slides.com/juliotrigo/pycones2019-a-practical-ddd-approach-to-nameko-microservices
PyConES2019 - A practical DDD approach to Nameko microservices
By juliotrigo
PyConES2019 - A practical DDD approach to Nameko microservices
- 1,856