Let's talk about automation tests in Golang

David Chou @ Crescendo Lab

CC-BY-SA-3.0-TW

Engineering Mgr @ Crescendo Lab

Co-organizer @ Golang Taipei

Golang Taipei Telegram

Golang Taipei Facebook

Unit-test in Golang is simple

because of the build-in testing framework

go test

go test -v -run={TestName}
go test -race
go test -count=1
go test -c -o {TestExe}
go test -coverprofile=coverage.out
go tool cover -html=coverage.out

My make test

$ go test -race -cover -coverprofile cover.out

$ go tool cover -func=cover.out | tail -n 1

$ go test -race -coverprofile cover.out
ok      github.com/.../internal/pkg/apiutil       0.334s  coverage: 53.3% of statements
ok      github.com/.../internal/pkg/emailutil     17.611s coverage: 90.7% of statements
ok      github.com/.../internal/pkg/integrationutil       2.026s  coverage: 0.0% of statements
ok      github.com/.../internal/pkg/lambdautil    0.829s  coverage: 62.5% of statements
ok      github.com/.../internal/pkg/s3util        2.056s  coverage: 81.8% of statements
ok      github.com/.../internal/pkg/smtp/processor/attachment     0.627s  coverage: 76.4% of statements
ok      github.com/.../internal/pkg/smtp/processor/event  0.082s  coverage: 71.4% of statements
ok      github.com/.../internal/pkg/smtp/processor/integration    0.060s  coverage: 15.4% of statements
ok      github.com/.../internal/pkg/smtp/processor/lambda 0.415s  coverage: 82.4% of statements
ok      github.com/.../internal/pkg/smtp/processor/payload        0.053s  coverage: 68.8% of statements
ok      github.com/.../internal/pkg/videoutil     0.413s  coverage: 75.9% of statements

$ go tool cover -func=cover.out | tail -n 1
57.8%

Test helpers

save time by writing less test codes

func TestSomething(t *testing.T) {
    value, err := DoSomething()
    if err != nil {
        t.Fatalf("DoSomething() failed: %s", err)
    }
    if value != 100 {
        t.Fatalf("expected 100, got: %d", value)
    }
}

import "github.com/stretchr/testify/assert"

func TestSomething(t *testing.T) {
    value, err := DoSomething()
    assert.NoError(t, err)
    assert.EqualValues(t, 100, value)
}

testify

Nil(), NoNil()
Error(), NoError()
EqualValues(), Equal(), JSONEq()
Len(), True(), False(), Empty()

func TestEventuallyExample(t *testing.T) {
	status := "pending"

	// 模擬一個異步操作，1 秒後將 status 更新為 "done"
	go func() {
		time.Sleep(1 * time.Second)
		status = "done"
	}()

	// 使用 assert.Eventually 確保 status 在 2 秒內變為 "done"
	assert.Eventually(t, func() bool {
		return status == "done"
	}, 2*time.Second, 100*time.Millisecond, "status should eventually be 'done'")
}

testify

Test async function with eventually()

func (suite *MyDBTestSuite) TestCreateUser() {
   suite.NoError(createUser())
}

func (suite *MyDBTestSuite) TestDeleteUser() {
   suite.NoError(deleteUser())
}

testify

Use suite to organize test functions with
fine-grained setup/teardown

type MyDBTestSuite struct {
    suite.Suite
}

func (suite *MyDBTestSuite) SetupSuite() {
    // run db migrations, setup all tables
}

func (suite *MyDBTestSuite) TearDownSuite() {
    // do nothing
}

func (suite *MyDBTestSuite) SetupTest() {
    // insert mock data
}

func (suite *MyDBTestSuite) TearDownTest() {
    // remove all data
}

Test helper functions

func testTempFile(t *testing.T) (string, func()) {
	f, err := ioutil.TempFile("", "test")
	require.NoError(t, err)
	f.Close()
	return f.Name(), func() { os.Remove(f.Name()) }
}

func TestThing(t *testing.T) {
	f, remove := testTempFile(t)
    defer remove()
    
    // do something with the temp file
}

Advanced testing in Go

Test helper functions

func testTempFile(t *testing.T) (string, func()) {
	f, err := ioutil.TempFile("", "test")
	require.NoError(t, err)
	f.Close()
	return f.Name(), func() { os.Remove(f.Name()) }
}

func TestThing(t *testing.T) {
	f, remove := testTempFile(t)
    defer remove()
    
    // do something with the temp file
}

Don't return error, but pass in testing.T and fail
No error return; the usage could be quite clean

Test helper functions

func testTempFile(t *testing.T) (string, func()) {
	f, err := ioutil.TempFile("", "test")
	require.NoError(t, err)
	f.Close()
	return f.Name(), func() { os.Remove(f.Name()) }
}

func TestThing(t *testing.T) {
	f, remove := testTempFile(t)
    defer remove()
    
    // do something with the temp file
}

Return a func() to clean up resource or recover state
The func() can access testing.T to also fail

Table-driven test

golang officially recommends

func IsStrExist(slice []string, find string) bool {
	for _, v := range slice {
		if v == find {
			return true
		}
	}
	return false
}

func TestIsStrExistSimple(t *testing.T) {
	ok := IsStrExist([]string{"a", "b"}, "a")
	require.True(t, ok)

	ok = IsStrExist([]string{"a", "b"}, "c")
	require.False(t, ok)

	ok = IsStrExist([]string{}, "a")
	require.False(t, ok)

	ok = IsStrExist(nil, "a")
	require.False(t, ok)
}

Table-driven test

func TestIsStrExist(t *testing.T) {
	tests := map[string]struct {
		slice []string
		find  string
		want  bool
	}{
		"find":    {[]string{"a", "b"}, "a", true},
		"notfind": {[]string{"a", "b"}, "c", false},
		"empty":   {[]string{}, "a", false},
		"nil":     {nil, "a", false},
	}
	for name, tt := range tests {
		t.Run(name, func(t *testing.T) {
        	got := IsStrExist(tt.slice, tt.find)
			require.EqualValues(t, tt.want, got)
		})
	}
}

Table-driven test

func TestIsStrExistP(t *testing.T) {
	tests := map[string]struct {
		slice []string
		find  string
		want  bool
	}{
		"find":    {[]string{"a", "b"}, "a", true},
		"notfind": {[]string{"a", "b"}, "c", false},
		"empty":   {[]string{}, "a", false},
		"nil":     {nil, "a", false},
	}
	for name, tt := range tests {
		tt := tt // fix loop iterator variable issue; no need if >= 1.22
		t.Run(name, func(t *testing.T) {
			t.Parallel()
			require.EqualValues(t, tt.want, IsStrExist(tt.slice, tt.find))
		})
	}
}

Table-driven test in parallel

Table-driven test

Easier to add new test cases
Easier to exhaustive corner cases
Make reproducing issues simple
Try to do this pattern if possible

Writing testable code

testable code != TDD

that code is testable when we don’t have to change the code itself when we’re adding a unit test to it

SOLID principle

dependency inversion

Accept interface, return concrete type

concrete type is usually pointer or struct

Do not define interfaces on the implementor side

Go interfaces generally belong in the package that uses values of the interface type, not the package that implements those values.

package s3

type S3 struct {...}

func New(...) *S3 {...}

func (s *S3) CopyObject(*s3.CopyObjectInput) (*s3.CopyObjectOutput, error)
func (s *S3) CopyObjectWithContext(aws.Context, *s3.CopyObjectInput, ...request.Option) (*s3.CopyObjectOutput, error)
func (s *S3) CopyObjectRequest(*s3.CopyObjectInput) (*request.Request, *s3.CopyObjectOutput)
	
func (s *S3) GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error)
func (s *S3) GetObjectWithContext(aws.Context, *s3.GetObjectInput, ...request.Option) (*s3.GetObjectOutput, error)
func (s *S3) GetObjectRequest(*s3.GetObjectInput) (*request.Request, *s3.GetObjectOutput)

----
type S3API interface {
	CopyObject(*s3.CopyObjectInput) (*s3.CopyObjectOutput, error)
	CopyObjectWithContext(aws.Context, *s3.CopyObjectInput, ...request.Option) (*s3.CopyObjectOutput, error)
	CopyObjectRequest(*s3.CopyObjectInput) (*request.Request, *s3.CopyObjectOutput)
	
	GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error)
	GetObjectWithContext(aws.Context, *s3.GetObjectInput, ...request.Option) (*s3.GetObjectOutput, error)
	GetObjectRequest(*s3.GetObjectInput) (*request.Request, *s3.GetObjectOutput)
    
	// ... over 300 functions
}

package s3client

type S3Service interface {
	GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error)
}

func Download(s3Srv S3Service, bucket, key string) error {...}

func Main() {
	s3Srv := s3.New()
	err := Download(s3Srv, "bucket", "key")
	...
}

package s3client_test

type S3ServiceMock struct {}

func (m S3ServiceMock) GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error) {...}

func TestDownload(t *testing.T) {
	var m S3ServiceMock
	err := Download(m, "mybucket", "mykey")
	...
}

Mocking

mocking is the super power of your tests

type OrderModel interface {
	GetOrderByID(orderID string) (Order, error)
}

func IsOrderPaid(orderModel OrderModel, orderID string) (ok bool, err error) {
	var order Order
	if order, err = orderModel.GetOrderByID(orderID); err != nil {
		return false, err
	}

	return order.status == "completed", nil
}

IsOrderPaid()

type OrderModelMock struct{}

func (m OrderModelMock) GetOrderByID(orderID string) (Order, error) { ... }

func TestIsOrderPaidSimple(t *testing.T) {
	var orderMock OrderModelMock

	ok, err := IsOrderPaid(orderMock, "orderId")
 	...
}

Too boilerplate to write all these mockers
Hard to add different behaviors into mockers
Need manually update mockers if interface changes

gomock

The official mocking framework
Build-in mocking generator
Type-safe mocking objects
Powerful stub API

gomock

//go:generate mockgen -destination automocks/ordermodel.go -package=automocks . OrderModel
type OrderModel interface {
	GetOrderByID(orderID string) (Order, error)
}

$ go generate -x ./...
mockgen -destination automocks/ordermodel.go -package=automocks . OrderModel

run go generate to generate mocking objects

gomock

m := automocks.NewMockOrderModel(ctrl)
m.EXPECT().
	GetOrderByID(gomock.Any()).
	Return(Order{}, nil).
	Times(1)
   
    
m.EXPECT().
	GetOrderByID("correctOrderID").
	DoAndReturn(func(orderID string) (Order, error) {
		...
	}).MinTimes(3)

create mocker with expectation API

gomock + table-driven test?

func TestIsOrderPaid(t *testing.T) {
	tests := []struct {
		name    string
		args    args
		wantOk  bool
		wantErr error
	}{
		{
			name:    "paid",
			args:    args{...},
			wantOk:  true,
			wantErr: nil,
		},
		{
			name:    "notpaid",
			args:    args{...},
			wantOk:  false,
			wantErr: nil,
		},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			ctrl := gomock.NewController(t)
			defer ctrl.Finish()

			orderModel := getOrderModel(t, ctrl, tt.args)
            
			gotOk, err := IsOrderPaid(orderModel, tt.args.orderID)
			...
		})
	}
}

type args struct {
	orderID  string
	order    Order
	orderErr error
}

func getOrderModel(c *gomock.Controller, args args) OrderModel {
	orderModel := automocks.NewMockOrderModel(c)
	orderModel.EXPECT().
		GetOrderByID(args.orderID).
		Return(args.order, args.orderErr).
		Times(1)
	return orderModel
}

Fake data

Fake it until you make it

It's annoying to create test data for each test cases

testAnswer := "test123"
testResult := 1024

faker

Generate fake data with no effort
https://github.com/go-faker/faker

type AStruct struct {
	Number int64
	Height int64
	B      BStruct
}

type BStruct struct {
	Image string
}

func Example_withoutTag() {
	a := AStruct{}
	faker.FakeData(&a)
	fmt.Printf("%+v", a)
	/*
		Result:
		{
			Number: 1
			Height: 26
			B: {
				Image: RmmaaHkAkrWHldVbBNuQSKlRb
			}
		}
	*/
}

faker.Email()           // => mJBJtbv@OSAaT.com
faker.URL()             // => https://www.oEuqqAY.org/QgqfOhd
faker.FirstName()       // => Whitney
faker.E164PhoneNumber() // => +724891571063
faker.UUIDHyphenated()  // => 8f8e4463-9560-4a38-9b0c-ef24481e4e27
faker.IPv4()            // => 99.23.42.63
faker.IPv6()            // => 975c:fb2c:2133:fbdd:beda:282e:1e0a:ec7d
faker.Word())           // => nesciunt
faker.Sentence()        // => Consequatur perferendis voluptatem accusantium.
faker.TimeString()      // => 03:10:25
faker.MonthName()       // => February
faker.YearString()      // => 1994
faker.DayOfWeek()       // => Sunday
faker.DayOfMonth()      // => 20
faker.Timestamp()       // => 1973-06-21 14:50:46
...

// For deterministic result
faker.SetRandomSource(faker.NewSafeSource(rand.NewSource(seed)))

testfixtures

Insert predefined data into database
https://github.com/go-testfixtures/testfixtures

# comments.yml
- id: 1
  post_id: 1
  content: A comment...
  author_name: John Doe
  author_email: john@doe.com
  created_at: 2020-12-31 23:59:59
  updated_at: 2020-12-31 23:59:59
  
 - id: 2
  post_id: 2
  content: Another comment...
  author_name: John Doe
  author_email: john@doe.com
  created_at: 2020-12-31 23:59:59
  updated_at: 2020-12-31 23:59:59

loader, err := testfixtures.New(
	testfixtures.Database(db.DB),
    testfixtures.Dialect("postgres"),
    testfixtures.Location(time.UTC),
    // Load predefined data
    testfixtures.Files(
  	    "fixtures/users.yml",
        "fixtures/comments.yml",
    ),
)

loader.Load()

Integration test

the next step beyond unit test

a real environment?

expensive and hard to maintain

docker-compose up?

good, but we could do better

ory/dockertest

A swiss knife to boot up ephemeral container easily

func LaunchDB(t *testing.T) (*sql.DB, func()) {
    pool, _ := dockertest.NewPool("")

	// pulls an image, creates a container based on it and runs it
	resource, _ := pool.RunWithOptions(&dockertest.RunOptions{
		Repository: "mysql",
		Tag:        "8.0",
		Env:        []string{"MYSQL_ROOT_PASSWORD=secret"},
	})

	// exponential backoff-retry
	var db *sql.DB
	pool.Retry(func() error {
		var err error
		db, err = sql.Open("mysql", DSN)
		if err != nil {
			return err
		}
		return db.Ping()
	})

	return db, func() { pool.Purge(resource) }
}

var db *sql.DB

func TestMain(m *testing.M) {
	var close func()
	db, close = testutil.LaunchDB()

	// Run all integration tests
	code := m.Run()

	// Close db container
	close()

	os.Exit(code)
}

func TestSomething(t *testing.T) {
	// db.Query()
}

ory/dockertest

Launch container just in your go test code
Remove containers just after test finishes
Smoothly integrate with go test framework and your IDE
Could use build-tag to run integration test explicitly

// +build integration

import testing

$ go test --tags integration ./...

Continuous smoke test

keep testing in production

Don't let the customer tell you the system is down

Continuous smoke test

Steps:

Identify critical user journey
Build smoke tests for those critical cases
Continuously run those tests on production
Send alerts if these critical smoke tests fail

Continuous smoke test

Benefits

Proactively find issues from the user's perspective
More possible to find the problems before the user calls us
Can be used to calculate SLI

go test
Test helpers
Table-driven test
Testable code
Mocking
Fake data
Integration test
Continuous smoke test

Happy Go Test

We are a regional SaaS startup with two major products
Provide B2B solutions for marketing, sales, and customer service needs

We send > 4 billion messages per year

even more than 中華電信's SMS

Let's talk about automation tests in Golang

Engineering Mgr @ Crescendo Lab

Co-organizer @ Golang Taipei

Golang Taipei Telegram

Golang Taipei Facebook

Unit-test in Golang is simple

because of the build-in testing framework

go test

go test -v -run={TestName}

go test -race

go test -count=1

go test -c -o {TestExe}

go test -coverprofile=coverage.out go tool cover -html=coverage.out

My make test

$ go test -race -cover -coverprofile cover.out

$ go tool cover -func=cover.out | tail -n 1

Test helpers

save time ​by writing ​less test codes

testify

testify

Nil(), NoNil()

Error(), NoError()

EqualValues(), Equal(), JSONEq()

Len(), True(), False(), Empty()

testify

Test async function with eventually()

testify

Use suite to organize test functions with fine-grained setup/teardown

Test helper functions

Test helper functions

Don't return error, but pass in testing.T and fail

No error return; the usage could be quite clean

Test helper functions

Return a func() to clean up resource or recover state

The func() can access testing.T to also fail

Table-driven test

golang officially recommends

Table-driven test

Table-driven test

Table-driven test in parallel

Table-driven test

Easier to add new test cases

Easier to exhaustive corner cases

Make reproducing issues simple

Try to do this pattern if possible

Writing testable code

testable code != TDD

that code is testable when we don’t have to change the code itself when we’re adding a unit test to it

SOLID principle

dependency inversion

Accept interface, return concrete type

Do not define interfaces on the implementor side

Mocking

mocking is the super power of your tests

IsOrderPaid()

Too boilerplate to write all these mockers

Hard to add different behaviors into mockers

Need manually update mockers if interface changes

gomock

The official mocking framework

Build-in mocking generator

Type-safe mocking objects

Powerful stub API

gomock

run go generate to generate mocking objects

gomock

create mocker with expectation API

gomock + table-driven test?

Fake data

Fake it until you make it

It's annoying to create test data for each test cases

faker

testfixtures

Integration test

the next step beyond unit test

a real environment?

expensive and hard to maintain

docker-compose up?

good, but we could do better

ory/dockertest

go test -coverprofile=coverage.out
go tool cover -html=coverage.out

save time by writing less test codes

Use suite to organize test functions with
fine-grained setup/teardown