Jalex Chang

2022.07.31

Clean Architecture in Go

The Crescendo Way

Jalex Chang

  • Saff Software Engineer @ Crescendo Lab
  • Gopher
  • Love software engineering, database systems, and distributed systems

 

Agenda

  • Introduction

  • Architecture overview

  • Architectural designs

  • Component designs

  • Discussions

  • Summary

References

[1] Crescendo's go-clean-arch, https://github.com/chatbotgang/go-clean-arch

[2] Go clean architecture project - wild-workouts, https://github.com/ThreeDotsLabs/wild-workouts-go-ddd-example
[3] Uncle Bob's clean architecture,
https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html

Introduction

In this tech talk, we want to introduce a simple but maintainable Go clean architecture, commonly used in Crescendo Lab.

 

Topics will be covered in the talk:

  • Engineering challenges in Crescendo Lab

  • Design goals of a new architecture

  • Designs and decisions we have made

  • Pros and cons of the proposed architecture

 

The proposed architecture and its designs are open to the public.

Engineering challenges in Crescendo Lab

Crescendo Lab is a regional Martech start-up:

  • Provide B2B enterprise solutions for marketing, sales, and customer service needs.
  • Most clients are amateur players.
  • High-growth in businesses

 

Challenges in eng teams:

  • Complex business logic
  • Good adaptability for rapid market changes
  • Good observability for responding to client's questions
  • More and more members are onboarding.
  • Tech stack is changing, from Python to Golang.

Design goals

  • Low communication overhead
    • ​No barrier between business and engineering people.
  • Testable
    • Business logic is well protected and validated.
  • Flexible
    • ​Features and logic could be added and modified easily to fit market changes.
  • Observable
    • ​Errors are easy to handle and address.
    • ​Internal behaviors of requests could be traceable.
  • Simple and straight​
    • ​Any new member could pick up the architecture within days.
    • Everyone could have the same programming style without effort.

What concepts can be leveraged?

  • Low communication overhead
    • ​No barrier between business and engineering people.
  • Testable
    • ​Business logic is well protected and validated.
  • Flexible
    • ​Features and logic could be added and modified easily to fit market changes.
  • Observable
    • ​Errors are easy to handle and address.
    • ​Internal behaviors of requests could be traceable.
  • Simple and straight​
    • ​Any new member could pick up the architecture within days.
    • Everyone could have the same programming style without effort.

Domain-Driven Design
(Ubiquitous language)

Clean Architecture

Go Practices

Try and Error Loop......

Architecture Overview

Layered architecture

4 layers, making things simple but organized:

  • Router handles input requests.
    • Framework-dependent
    • E.g. routing HTTP requests, authentication, access control, and parameter validation.
  • Adapter handles output requests
    • External-system-dependent
    • E.g. accessing DB, publishing events, and interacting with other services.
  • Domain handles business logic.
    • ​Introduce Ubiquitous Language.
  • Application handles orchestration.
    • Focus on control flows
    • Coordinate Domain and Adapter

Architecture overview

Dependency rules

Codebase structure

internal
│
├── adapter
│   ├── eventbroker
│   ├── repository
│   │   └── postgres
│   │       ├── good_repository.go
│   │       ├── good_repository_test.go
│   │       ├── postgres_repository.go
│   │       └── postgres_repository_test.go
│   └── server
│
├── app
│   └── service
│       ├── auth
│       └── barter
│           ├── exchange_service.go
│           ├── exchange_service_test.go
│           ├── good_service.go
│           ├── good_service_test.go
│           ├── interface.go
│           ├── service.go
│           └── service_test.go
internal
│
├── domain
│   ├── barter
│   │   ├── exchange.go
│   │   ├── exchange_test.go
│   │   ├── good.go
│   │   ├── good_test.go
│   │   ├── trader.go
│   │   └── trader_test.go
│   └── common
│       ├── error.go
│       └── error_test.go
│
└── router
    ├── handler.go
    ├── handler_auth_trader.go
    ├── handler_barter_exchange.go
    ├── handler_barter_good.go
    ├── middleware.go
    ├── middleware_auth.go
    ├── middleware_logger.go
    └── response.go

Architectural Designs

Domain layer

Design principles

  • Name things by business language
  • No Getter/Setter pattern
  • Testable via unit tests

 

 

 

 

There is nothing wrong with only having data structures in domain.

We aggregate business logic through refactorization.

Applicaiton layer

Design principles

Router layer

Design principles

  • An API maps to a handler.
  • A handler can call multiple use cases. 
  • API testing only.
    • ​Postman-driven development 

Adapter layer

Design principles

  • A dependent system maps to an adapter.
  • No transaction leaks.
  • Integration tests only
  • Form DB integration tests as unit tests 

 

Testing landscape

Data modeling

API model Domain model ≠ DB model

type CustomerTag struct {
    ID         int
    CustomerID int    
    Type       TagType
    Name       string
    CreatedAt  time.Time
}
type repoCsutomerTag struct {
    ID          int       `db:"id"`
    CustomerID  int       `db:"customer_id"`
    TagID       int       `db:"tag_id"`
    CreatedAt   time.Time `db:"created_at"`
}

type repoChannelTag struct {
    ID        int       `db:"id"`
    ChannelID int       `db:"channel_id"`
    Name      string    `db:"name"`
    Type      string    `db:"type"`
}
func ListCustomerTags(app *app.Application) gin.HandlerFunc {
    type Tag struct {
        ID          int      `json:"id"`
        Name        string   `json:"name"`
        CreatedAt   int      `json:"created_at"`
        ...
    }
    type Response struct {
        CustomerID     int   `json:"customer_id"` 
        MarketingTags  []Tag `json:"marketing_tags"`
        ServiceTags    []Tag `json:"service_tags"`
        SalseTags      []Tag `json:"salse_tags"`
    }
}

Data modeling

Adapter

Domain

Router

Component Designs

Good designs are invisible.

Giant application - Glue things together

// internal/app/application.go
func NewApplication(ctx context.Context, wg *sync.WaitGroup, params ApplicationParams) (*Application, error) {
    // 1. Init adapters
    pgRepo := postgres.NewPostgresRepository(ctx, db)
    authServer := server.NewAuthServer(ctx, server.AuthServerParam{})
    
    // 2. Create the applicaiton and init services
    app := &Application{
        AuthService: auth.NewAuthService(ctx, auth.AuthServiceParam{
            AuthServer:     authServer,
            TraderRepo:     pgRepo,
        }),
        
        BarterService: barter.NewBarterService(ctx, barter.BarterServiceParam{
            GoodRepo: pgRepo,
        }),
    }
    return app, nil
}
  • Manage all services in a single application instance
  • Help inject adapters to services
  • Every Router handler leverages the application directly

Giant application -  Leveraged by handlers directly

func registerAPIHandlers(router *gin.Engine, app *app.Application) {
    barterGroup := v1.Group("/barter", BearerToken.Required())
    {
        barterGroup.POST("/goods", PostGood(app))
        barterGroup.GET("/goods", ListMyGoods(app))
        barterGroup.DELETE("/goods/:good_id", RemoveMyGood(app))
    }
}
func PostGood(app *app.Application) gin.HandlerFunc {
    return func(c *gin.Context) {
        good, err := app.BarterService.PostGood(ctx, barter.PostGoodParam{...})
        if err != nil {...}
    }
}

Inject application to handlers:

Handlers call services methods through the application:

func (s *BarterService) PostGood(ctx context.Context, param PostGoodParam) (*barter.Good, common.Error) {
    good, err := s.goodRepo.CreateGood(ctx, barter.NewGood(param.Trader, param.GoodName))
    if err != nil {...}
}

Services call adapter methods through interface:

Error Handling

// internal/domain/common/error.go
type Error interface {
    Error() string
    ClientMsg() string
}

func NewError(code ErrorCode, err error, opts ...ErrorOption) Error {
    e := DomainError{code: code, err: err}
    // Handle error options
    for _, o := range opts {
        o(&e)
    }
    return e
}

type DomainError struct {
    code         ErrorCode              
    err          error               
    clientMsg    string                 
    remoteStatus int                    
    detail       map[string]interface{}
}
type ErrorCode struct {
    Name       string
    StatusCode int
}

// Authentication and Authorization error codes
var ErrorCodeAuthPermissionDenied = ErrorCode{
    Name:       "AUTH_PERMISSION_DENIED",
    StatusCode: http.StatusForbidden,
}

// Business-related errors
var ErrorCodeParameterEmailNotExist = ErrorCode{
    Name:       "PARAMETER_EMAIL_NOT_EXIST",
    StatusCode: http.StatusBadRequest,
}

var ErrorCodeParameterInvalidPassword = ErrorCode{
    Name:       "PARAMETER_INVALID_PASSWORD",
    StatusCode: http.StatusBadRequest,
}

var ErrorCodePaymentExpiredPlan = ErrorCode{
    Name:       "PAYMENT_EXPIRED_PLAN",
    StatusCode: http.StatusPaymentRequired,
}

Custom Error Domain

  • Errors are self-described.
  • Specific Error Code for business needs
  • Options Pattern for extensions

Pre-defined ErrorCode:

Error Handling - Leverage Error Domain everywhere

func ParseTraderFromToken(signedToken string, signingKey []byte) (*Trader, common.Error) {
    if !token.Valid {
        msg := "invalid token"
        return nil, common.NewError(common.ErrorCodeParameterInvalid, errors.New(msg), common.WithMsg(msg))
    }
}
func (s *BarterService) ExchangeGoods(ctx context.Context, param ExchangeGoodsParam) common.Error {
    if !requestGood.IsMyGood(param.Trader){
        return common.NewError(common.ErrorCodeAuthPermissionDenied, nil)
    }
}
func (r *PostgresRepository) GetTraderByEmail(ctx context.Context, email string) (*barter.Trader, common.Error) {
    if err = r.db.GetContext(ctx, &row, query, args...); err != nil {
        if errors.Is(err, sql.ErrNoRows) {
            return nil, common.NewError(common.ErrorCodeResourceNotFound, err)
        }
        return nil, common.NewError(common.ErrorCodeRemoteProcess, err)
    }
}

Domain

Application

Adapter

Error Handling - Describe itself before API responding

// internal/router/response.go
func respondWithError(c *gin.Context, err error) {
    // 1. parse error mesages
    errMessage := parseError(err)
    
    // 2. log out error messages
    ctx := c.Request.Context()
    zerolog.Ctx(ctx).Error().Err(err).Str("component", "handler").Msg(errMessage.Message)
    
    // 3. respond error messages
    c.AbortWithStatusJSON(errMessage.Code, errMessage)
}

func parseError(err error) ErrorMessage {
    var domainError common.DomainError
    _ = errors.As(err, &domainError)

    return ErrorMessage{
        Name:       domainError.Name(),
        Code:       domainError.HTTPStatus(),
        Message:    domainError.ClientMsg(),
        RemoteCode: domainError.RemoteHTTPStatus(),
        Detail:     domainError.Detail(),
    }
}

Traceable logging - Pass request ID through context

// internal/router/middleware_logger.go
func SetGeneralMiddlewares(ctx context.Context, ginRouter *gin.Engine) {
    ginRouter.Use(requestid.New())
    ginRouter.Use(LoggerMiddleware(ctx))
}

func LoggerMiddleware(rootCtx context.Context) gin.HandlerFunc {
    return func(c *gin.Context) {
        // 1. add RequestID into the logger of the request context
        requestID := requestid.Get(c)
        zlog := zerolog.Ctx(rootCtx).With().
            Str("requestID", requestID).
            Logger()
        c.Request = c.Request.WithContext(zlog.WithContext(rootCtx))
        
        // 2. call next handlers
        c.Next()
    }
}

Traceable logging - Wrap context when logging

// logger wraps the execution context with component info
func (s *ChatService) logger(ctx context.Context) *zerolog.Logger {
    l := zerolog.Ctx(ctx).With().Str("component", "chat-service").Logger()
    return &l
}

func (s *ChatService) SendMessageToCutomer(ctx context.Context, param Param) common.Error {
    if err := s.lineServer.SendMessages(ctx, messages); err != nil {
        s.logger(ctx).Error().Err(err).Msg("failed to send messages to Line")
        return err
    }
}

Traceable logging - Trace request behavior internally

Disccussions

Q1: What are the pros and cons of the proposed architecture?

Pros

  • Simple and straight. Easy to pick up.
  • Organized. Only one programming style everywhere.
  • Flexible.  Easy to modify orchestration/business logic when requirements changed

Cons

  • The codebase is long-winded.
  • Too comfortable. Coding feels not challenging sometimes.

Disccussion

Q2: What use cases are the proposed architecture?

  • Implement and maintain complex business logic
  • Teams and businesses are high-growth and changeable.

 

Q3: How to introduce new tech/architecture to teams?

  • (X) Be a boss or manager
  • (O) Make hands dirty
    • Let the new things seem practical and easy to follow

Future works

Architecture-wise

  • More code-gen on codebases to reduce boring coding tasks
    • E.g. auto-gen API handlers and data repositories.
  • More automation in software testing
  • Continue to collect teams' comments and evolve the architecture, with the business growth.

 

Community-wise

  • Continue to share more internal architectural/component designs in public
    • E.g. access control, event-driven patterns, etc.

Takeaways

In this sharing, we have introduced a Go Clean Architecture that is internally used in Crescendo Lab.

  • To make the codebase simple and straight, the architecture has only four layers: domain, application, router, and adapter layers.

  • To make developers happy and easily handle their jobs, the details matter.

    • E.g. error domain and traceable logging.​

  • The proposed architecture and its designs are open to the public (chatbotgang/go-clean-arch).

 

 

​​Good architecture comes from observations and comminucations.

Thanks for listening

We are hiring now! Feel free to chat with us in Room TR 312 (研揚大樓 312室).

Clean Architecture in Go: The Crescendo Way

By Jalex Chang

Clean Architecture in Go: The Crescendo Way

Share our experience in improving engineering teams’ productivity and happiness in Crescendo by introducing DDD (Domain-Driven Design) and Clean Architecture to Go projects.

  • 2,127