Jalex Chang
2022.07.31
Clean Architecture in Go
The Crescendo Way
Jalex Chang
- Saff Software Engineer @ Crescendo Lab
- Gopher
- Love software engineering, database systems, and distributed systems
Agenda
-
Introduction
-
Architecture overview
-
Architectural designs
-
Component designs
-
Discussions
-
Summary
References
[1] Crescendo's go-clean-arch, https://github.com/chatbotgang/go-clean-arch
[2] Go clean architecture project - wild-workouts, https://github.com/ThreeDotsLabs/wild-workouts-go-ddd-example
[3] Uncle Bob's clean architecture, https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html
Introduction
In this tech talk, we want to introduce a simple but maintainable Go clean architecture, commonly used in Crescendo Lab.
Topics will be covered in the talk:
-
Engineering challenges in Crescendo Lab
-
Design goals of a new architecture
-
Designs and decisions we have made
-
Pros and cons of the proposed architecture
The proposed architecture and its designs are open to the public.
- Template on Github (chatbotgang/go-clean-arch).
Engineering challenges in Crescendo Lab
Crescendo Lab is a regional Martech start-up:
- Provide B2B enterprise solutions for marketing, sales, and customer service needs.
- Most clients are amateur players.
- High-growth in businesses
Challenges in eng teams:
- Complex business logic
- Good adaptability for rapid market changes
- Good observability for responding to client's questions
- More and more members are onboarding.
- Tech stack is changing, from Python to Golang.
Design goals
-
Low communication overhead
- No barrier between business and engineering people.
-
Testable
- Business logic is well protected and validated.
-
Flexible
- Features and logic could be added and modified easily to fit market changes.
-
Observable
- Errors are easy to handle and address.
- Internal behaviors of requests could be traceable.
-
Simple and straight
- Any new member could pick up the architecture within days.
- Everyone could have the same programming style without effort.
What concepts can be leveraged?
-
Low communication overhead
- No barrier between business and engineering people.
-
Testable
- Business logic is well protected and validated.
-
Flexible
- Features and logic could be added and modified easily to fit market changes.
-
Observable
- Errors are easy to handle and address.
- Internal behaviors of requests could be traceable.
-
Simple and straight
- Any new member could pick up the architecture within days.
- Everyone could have the same programming style without effort.
Domain-Driven Design
(Ubiquitous language)
Clean Architecture
Go Practices
Try and Error Loop......
Architecture Overview
Layered architecture
4 layers, making things simple but organized:
-
Router handles input requests.
- Framework-dependent
- E.g. routing HTTP requests, authentication, access control, and parameter validation.
-
Adapter handles output requests
- External-system-dependent
- E.g. accessing DB, publishing events, and interacting with other services.
-
Domain handles business logic.
- Introduce Ubiquitous Language.
-
Application handles orchestration.
- Focus on control flows
- Coordinate Domain and Adapter
Architecture overview
Dependency rules
Codebase structure
internal
│
├── adapter
│ ├── eventbroker
│ ├── repository
│ │ └── postgres
│ │ ├── good_repository.go
│ │ ├── good_repository_test.go
│ │ ├── postgres_repository.go
│ │ └── postgres_repository_test.go
│ └── server
│
├── app
│ └── service
│ ├── auth
│ └── barter
│ ├── exchange_service.go
│ ├── exchange_service_test.go
│ ├── good_service.go
│ ├── good_service_test.go
│ ├── interface.go
│ ├── service.go
│ └── service_test.go
internal
│
├── domain
│ ├── barter
│ │ ├── exchange.go
│ │ ├── exchange_test.go
│ │ ├── good.go
│ │ ├── good_test.go
│ │ ├── trader.go
│ │ └── trader_test.go
│ └── common
│ ├── error.go
│ └── error_test.go
│
└── router
├── handler.go
├── handler_auth_trader.go
├── handler_barter_exchange.go
├── handler_barter_good.go
├── middleware.go
├── middleware_auth.go
├── middleware_logger.go
└── response.go
Architectural Designs
Domain layer
Design principles
- Name things by business language
- No Getter/Setter pattern
- Testable via unit tests
There is nothing wrong with only having data structures in domain.
We aggregate business logic through refactorization.
Applicaiton layer
Design principles
- Coarse-grained Service
- Groups of use cases
- No dependency on each other
- Interact Adapter through Interface
- Testable via unit tests
Router layer
Design principles
- An API maps to a handler.
- A handler can call multiple use cases.
-
API testing only.
- Postman-driven development
Adapter layer
Design principles
- A dependent system maps to an adapter.
- No transaction leaks.
- Integration tests only
-
Form DB integration tests as unit tests
Testing landscape
Data modeling
API model ≠ Domain model ≠ DB model
type CustomerTag struct {
ID int
CustomerID int
Type TagType
Name string
CreatedAt time.Time
}
type repoCsutomerTag struct {
ID int `db:"id"`
CustomerID int `db:"customer_id"`
TagID int `db:"tag_id"`
CreatedAt time.Time `db:"created_at"`
}
type repoChannelTag struct {
ID int `db:"id"`
ChannelID int `db:"channel_id"`
Name string `db:"name"`
Type string `db:"type"`
}
func ListCustomerTags(app *app.Application) gin.HandlerFunc {
type Tag struct {
ID int `json:"id"`
Name string `json:"name"`
CreatedAt int `json:"created_at"`
...
}
type Response struct {
CustomerID int `json:"customer_id"`
MarketingTags []Tag `json:"marketing_tags"`
ServiceTags []Tag `json:"service_tags"`
SalseTags []Tag `json:"salse_tags"`
}
}
Data modeling
Adapter
Domain
Router
Component Designs
Good designs are invisible.
Giant application - Glue things together
// internal/app/application.go
func NewApplication(ctx context.Context, wg *sync.WaitGroup, params ApplicationParams) (*Application, error) {
// 1. Init adapters
pgRepo := postgres.NewPostgresRepository(ctx, db)
authServer := server.NewAuthServer(ctx, server.AuthServerParam{})
// 2. Create the applicaiton and init services
app := &Application{
AuthService: auth.NewAuthService(ctx, auth.AuthServiceParam{
AuthServer: authServer,
TraderRepo: pgRepo,
}),
BarterService: barter.NewBarterService(ctx, barter.BarterServiceParam{
GoodRepo: pgRepo,
}),
}
return app, nil
}
- Manage all services in a single application instance
- Help inject adapters to services
- Every Router handler leverages the application directly
Giant application - Leveraged by handlers directly
func registerAPIHandlers(router *gin.Engine, app *app.Application) {
barterGroup := v1.Group("/barter", BearerToken.Required())
{
barterGroup.POST("/goods", PostGood(app))
barterGroup.GET("/goods", ListMyGoods(app))
barterGroup.DELETE("/goods/:good_id", RemoveMyGood(app))
}
}
func PostGood(app *app.Application) gin.HandlerFunc {
return func(c *gin.Context) {
good, err := app.BarterService.PostGood(ctx, barter.PostGoodParam{...})
if err != nil {...}
}
}
Inject application to handlers:
Handlers call services methods through the application:
func (s *BarterService) PostGood(ctx context.Context, param PostGoodParam) (*barter.Good, common.Error) {
good, err := s.goodRepo.CreateGood(ctx, barter.NewGood(param.Trader, param.GoodName))
if err != nil {...}
}
Services call adapter methods through interface:
Error Handling
// internal/domain/common/error.go
type Error interface {
Error() string
ClientMsg() string
}
func NewError(code ErrorCode, err error, opts ...ErrorOption) Error {
e := DomainError{code: code, err: err}
// Handle error options
for _, o := range opts {
o(&e)
}
return e
}
type DomainError struct {
code ErrorCode
err error
clientMsg string
remoteStatus int
detail map[string]interface{}
}
type ErrorCode struct {
Name string
StatusCode int
}
// Authentication and Authorization error codes
var ErrorCodeAuthPermissionDenied = ErrorCode{
Name: "AUTH_PERMISSION_DENIED",
StatusCode: http.StatusForbidden,
}
// Business-related errors
var ErrorCodeParameterEmailNotExist = ErrorCode{
Name: "PARAMETER_EMAIL_NOT_EXIST",
StatusCode: http.StatusBadRequest,
}
var ErrorCodeParameterInvalidPassword = ErrorCode{
Name: "PARAMETER_INVALID_PASSWORD",
StatusCode: http.StatusBadRequest,
}
var ErrorCodePaymentExpiredPlan = ErrorCode{
Name: "PAYMENT_EXPIRED_PLAN",
StatusCode: http.StatusPaymentRequired,
}
Custom Error Domain
- Errors are self-described.
- Specific Error Code for business needs
- Options Pattern for extensions
Pre-defined ErrorCode:
Error Handling - Leverage Error Domain everywhere
func ParseTraderFromToken(signedToken string, signingKey []byte) (*Trader, common.Error) {
if !token.Valid {
msg := "invalid token"
return nil, common.NewError(common.ErrorCodeParameterInvalid, errors.New(msg), common.WithMsg(msg))
}
}
func (s *BarterService) ExchangeGoods(ctx context.Context, param ExchangeGoodsParam) common.Error {
if !requestGood.IsMyGood(param.Trader){
return common.NewError(common.ErrorCodeAuthPermissionDenied, nil)
}
}
func (r *PostgresRepository) GetTraderByEmail(ctx context.Context, email string) (*barter.Trader, common.Error) {
if err = r.db.GetContext(ctx, &row, query, args...); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, common.NewError(common.ErrorCodeResourceNotFound, err)
}
return nil, common.NewError(common.ErrorCodeRemoteProcess, err)
}
}
Domain
Application
Adapter
Error Handling - Describe itself before API responding
// internal/router/response.go
func respondWithError(c *gin.Context, err error) {
// 1. parse error mesages
errMessage := parseError(err)
// 2. log out error messages
ctx := c.Request.Context()
zerolog.Ctx(ctx).Error().Err(err).Str("component", "handler").Msg(errMessage.Message)
// 3. respond error messages
c.AbortWithStatusJSON(errMessage.Code, errMessage)
}
func parseError(err error) ErrorMessage {
var domainError common.DomainError
_ = errors.As(err, &domainError)
return ErrorMessage{
Name: domainError.Name(),
Code: domainError.HTTPStatus(),
Message: domainError.ClientMsg(),
RemoteCode: domainError.RemoteHTTPStatus(),
Detail: domainError.Detail(),
}
}
Traceable logging - Pass request ID through context
// internal/router/middleware_logger.go
func SetGeneralMiddlewares(ctx context.Context, ginRouter *gin.Engine) {
ginRouter.Use(requestid.New())
ginRouter.Use(LoggerMiddleware(ctx))
}
func LoggerMiddleware(rootCtx context.Context) gin.HandlerFunc {
return func(c *gin.Context) {
// 1. add RequestID into the logger of the request context
requestID := requestid.Get(c)
zlog := zerolog.Ctx(rootCtx).With().
Str("requestID", requestID).
Logger()
c.Request = c.Request.WithContext(zlog.WithContext(rootCtx))
// 2. call next handlers
c.Next()
}
}
Traceable logging - Wrap context when logging
// logger wraps the execution context with component info
func (s *ChatService) logger(ctx context.Context) *zerolog.Logger {
l := zerolog.Ctx(ctx).With().Str("component", "chat-service").Logger()
return &l
}
func (s *ChatService) SendMessageToCutomer(ctx context.Context, param Param) common.Error {
if err := s.lineServer.SendMessages(ctx, messages); err != nil {
s.logger(ctx).Error().Err(err).Msg("failed to send messages to Line")
return err
}
}
Traceable logging - Trace request behavior internally
Disccussions
Q1: What are the pros and cons of the proposed architecture?
Pros
- Simple and straight. Easy to pick up.
- Organized. Only one programming style everywhere.
- Flexible. Easy to modify orchestration/business logic when requirements changed
Cons
- The codebase is long-winded.
- Too comfortable. Coding feels not challenging sometimes.
Disccussion
Q2: What use cases are the proposed architecture?
- Implement and maintain complex business logic
- Teams and businesses are high-growth and changeable.
Q3: How to introduce new tech/architecture to teams?
- (X) Be a boss or manager
- (O) Make hands dirty
- Let the new things seem practical and easy to follow
Future works
Architecture-wise
- More code-gen on codebases to reduce boring coding tasks
- E.g. auto-gen API handlers and data repositories.
- More automation in software testing
- Continue to collect teams' comments and evolve the architecture, with the business growth.
Community-wise
- Continue to share more internal architectural/component designs in public
- E.g. access control, event-driven patterns, etc.
Takeaways
In this sharing, we have introduced a Go Clean Architecture that is internally used in Crescendo Lab.
-
To make the codebase simple and straight, the architecture has only four layers: domain, application, router, and adapter layers.
-
To make developers happy and easily handle their jobs, the details matter.
-
E.g. error domain and traceable logging.
-
-
The proposed architecture and its designs are open to the public (chatbotgang/go-clean-arch).
Good architecture comes from observations and comminucations.
Thanks for listening
We are hiring now! Feel free to chat with us in Room TR 312 (研揚大樓 312室).
Clean Architecture in Go: The Crescendo Way
By Jalex Chang
Clean Architecture in Go: The Crescendo Way
Share our experience in improving engineering teams’ productivity and happiness in Crescendo by introducing DDD (Domain-Driven Design) and Clean Architecture to Go projects.
- 2,073