Let's talk about automation tests in Golang

David Chou @ Crescendo Lab

CC-BY-SA-3.0-TW

Engineering Mgr @ Crescendo Lab

Co-organizer @ Golang Taipei

 

Golang Taipei Telegram

Golang Taipei Facebook

Unit-test in Golang is simple

because of the build-in testing framework

go test

  • go test -v -run={TestName}

  • go test -race

  • go test -count=1

  • go test -c -o {TestExe}

  • go test -coverprofile=coverage.out
    go tool cover -html=coverage.out

My make test

$ go test -race -cover -coverprofile cover.out 

$ go tool cover -func=cover.out | tail -n 1

$ go test -race -coverprofile cover.out
ok      github.com/.../internal/pkg/apiutil       0.334s  coverage: 53.3% of statements
ok      github.com/.../internal/pkg/emailutil     17.611s coverage: 90.7% of statements
ok      github.com/.../internal/pkg/integrationutil       2.026s  coverage: 0.0% of statements
ok      github.com/.../internal/pkg/lambdautil    0.829s  coverage: 62.5% of statements
ok      github.com/.../internal/pkg/s3util        2.056s  coverage: 81.8% of statements
ok      github.com/.../internal/pkg/smtp/processor/attachment     0.627s  coverage: 76.4% of statements
ok      github.com/.../internal/pkg/smtp/processor/event  0.082s  coverage: 71.4% of statements
ok      github.com/.../internal/pkg/smtp/processor/integration    0.060s  coverage: 15.4% of statements
ok      github.com/.../internal/pkg/smtp/processor/lambda 0.415s  coverage: 82.4% of statements
ok      github.com/.../internal/pkg/smtp/processor/payload        0.053s  coverage: 68.8% of statements
ok      github.com/.../internal/pkg/videoutil     0.413s  coverage: 75.9% of statements

$ go tool cover -func=cover.out | tail -n 1
57.8%

Test helpers

save time ​by writing ​less test codes

func TestSomething(t *testing.T) {
    value, err := DoSomething()
    if err != nil {
        t.Fatalf("DoSomething() failed: %s", err)
    }
    if value != 100 {
        t.Fatalf("expected 100, got: %d", value)
    }
}
import "github.com/stretchr/testify/assert"

func TestSomething(t *testing.T) {
    value, err := DoSomething()
    assert.NoError(t, err)
    assert.EqualValues(t, 100, value)
}

testify

testify

  • Nil(), NoNil()

  • Error(), NoError()

  • EqualValues(), Equal(), JSONEq()

  • Len(), True(), False(), Empty()

func TestEventuallyExample(t *testing.T) {
	status := "pending"

	// 模擬一個異步操作，1 秒後將 status 更新為 "done"
	go func() {
		time.Sleep(1 * time.Second)
		status = "done"
	}()

	// 使用 assert.Eventually 確保 status 在 2 秒內變為 "done"
	assert.Eventually(t, func() bool {
		return status == "done"
	}, 2*time.Second, 100*time.Millisecond, "status should eventually be 'done'")
}

testify

Test async function with eventually() 

func (suite *MyDBTestSuite) TestCreateUser() {
   suite.NoError(createUser())
}

func (suite *MyDBTestSuite) TestDeleteUser() {
   suite.NoError(deleteUser())
}

testify

Use suite to organize test functions with
fine-grained setup/teardown

type MyDBTestSuite struct {
    suite.Suite
}

func (suite *MyDBTestSuite) SetupSuite() {
    // run db migrations, setup all tables
}

func (suite *MyDBTestSuite) TearDownSuite() {
    // do nothing
}

func (suite *MyDBTestSuite) SetupTest() {
    // insert mock data
}

func (suite *MyDBTestSuite) TearDownTest() {
    // remove all data
}

Test helper functions

func testTempFile(t *testing.T) (string, func()) {
	f, err := ioutil.TempFile("", "test")
	require.NoError(t, err)
	f.Close()
	return f.Name(), func() { os.Remove(f.Name()) }
}

func TestThing(t *testing.T) {
	f, remove := testTempFile(t)
    defer remove()
    
    // do something with the temp file
}

Advanced testing in Go

Test helper functions

func testTempFile(t *testing.T) (string, func()) {
	f, err := ioutil.TempFile("", "test")
	require.NoError(t, err)
	f.Close()
	return f.Name(), func() { os.Remove(f.Name()) }
}

func TestThing(t *testing.T) {
	f, remove := testTempFile(t)
    defer remove()
    
    // do something with the temp file
}

  • Don't return error, but pass in testing.T and fail

  • No error return; the usage could be quite clean

Test helper functions

func testTempFile(t *testing.T) (string, func()) {
	f, err := ioutil.TempFile("", "test")
	require.NoError(t, err)
	f.Close()
	return f.Name(), func() { os.Remove(f.Name()) }
}

func TestThing(t *testing.T) {
	f, remove := testTempFile(t)
    defer remove()
    
    // do something with the temp file
}

  • Return a func() to clean up resource or recover state

  • The func() can access testing.T to also fail

Table-driven test

golang officially recommends

func IsStrExist(slice []string, find string) bool {
	for _, v := range slice {
		if v == find {
			return true
		}
	}
	return false
}
func TestIsStrExistSimple(t *testing.T) {
	ok := IsStrExist([]string{"a", "b"}, "a")
	require.True(t, ok)

	ok = IsStrExist([]string{"a", "b"}, "c")
	require.False(t, ok)

	ok = IsStrExist([]string{}, "a")
	require.False(t, ok)

	ok = IsStrExist(nil, "a")
	require.False(t, ok)
}

Table-driven test

func TestIsStrExist(t *testing.T) {
	tests := map[string]struct {
		slice []string
		find  string
		want  bool
	}{
		"find":    {[]string{"a", "b"}, "a", true},
		"notfind": {[]string{"a", "b"}, "c", false},
		"empty":   {[]string{}, "a", false},
		"nil":     {nil, "a", false},
	}
	for name, tt := range tests {
		t.Run(name, func(t *testing.T) {
        	got := IsStrExist(tt.slice, tt.find)
			require.EqualValues(t, tt.want, got)
		})
	}
}

Table-driven test

func TestIsStrExistP(t *testing.T) {
	tests := map[string]struct {
		slice []string
		find  string
		want  bool
	}{
		"find":    {[]string{"a", "b"}, "a", true},
		"notfind": {[]string{"a", "b"}, "c", false},
		"empty":   {[]string{}, "a", false},
		"nil":     {nil, "a", false},
	}
	for name, tt := range tests {
		tt := tt // fix loop iterator variable issue; no need if >= 1.22
		t.Run(name, func(t *testing.T) {
			t.Parallel()
			require.EqualValues(t, tt.want, IsStrExist(tt.slice, tt.find))
		})
	}
}

Table-driven test in parallel

Table-driven test

  • Easier to add new test cases

  • Easier to exhaustive corner cases

  • Make reproducing issues simple

  • Try to do this pattern if possible

Writing testable code

testable code != TDD

that code is testable when we don’t have to change the code itself when we’re adding a unit test to it

SOLID principle

dependency inversion

Accept interface, return concrete type

concrete type is usually pointer or struct

Do not define interfaces on the implementor side

Go interfaces generally belong in the package that uses values of the interface type, not the package that implements those values.

package s3

type S3 struct {...}

func New(...) *S3 {...}

func (s *S3) CopyObject(*s3.CopyObjectInput) (*s3.CopyObjectOutput, error)
func (s *S3) CopyObjectWithContext(aws.Context, *s3.CopyObjectInput, ...request.Option) (*s3.CopyObjectOutput, error)
func (s *S3) CopyObjectRequest(*s3.CopyObjectInput) (*request.Request, *s3.CopyObjectOutput)
	
func (s *S3) GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error)
func (s *S3) GetObjectWithContext(aws.Context, *s3.GetObjectInput, ...request.Option) (*s3.GetObjectOutput, error)
func (s *S3) GetObjectRequest(*s3.GetObjectInput) (*request.Request, *s3.GetObjectOutput)

----
type S3API interface {
	CopyObject(*s3.CopyObjectInput) (*s3.CopyObjectOutput, error)
	CopyObjectWithContext(aws.Context, *s3.CopyObjectInput, ...request.Option) (*s3.CopyObjectOutput, error)
	CopyObjectRequest(*s3.CopyObjectInput) (*request.Request, *s3.CopyObjectOutput)
	
	GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error)
	GetObjectWithContext(aws.Context, *s3.GetObjectInput, ...request.Option) (*s3.GetObjectOutput, error)
	GetObjectRequest(*s3.GetObjectInput) (*request.Request, *s3.GetObjectOutput)
    
	// ... over 300 functions
}
package s3client

type S3Service interface {
	GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error)
}

func Download(s3Srv S3Service, bucket, key string) error {...}

func Main() {
	s3Srv := s3.New()
	err := Download(s3Srv, "bucket", "key")
	...
}
package s3client_test

type S3ServiceMock struct {}

func (m S3ServiceMock) GetObject(*s3.GetObjectInput) (*s3.GetObjectOutput, error) {...}

func TestDownload(t *testing.T) {
	var m S3ServiceMock
	err := Download(m, "mybucket", "mykey")
	...
}

Mocking

mocking is the super power of your tests

type OrderModel interface {
	GetOrderByID(orderID string) (Order, error)
}

func IsOrderPaid(orderModel OrderModel, orderID string) (ok bool, err error) {
	var order Order
	if order, err = orderModel.GetOrderByID(orderID); err != nil {
		return false, err
	}

	return order.status == "completed", nil
}

IsOrderPaid()

type OrderModelMock struct{}

func (m OrderModelMock) GetOrderByID(orderID string) (Order, error) { ... }

func TestIsOrderPaidSimple(t *testing.T) {
	var orderMock OrderModelMock

	ok, err := IsOrderPaid(orderMock, "orderId")
 	...
}

  • Too boilerplate to write all these mockers

  • Hard to add different behaviors into mockers

  • Need manually update mockers if interface changes

gomock

  • The official mocking framework

  • Build-in mocking generator

  • Type-safe mocking objects

  • Powerful stub API

gomock

//go:generate mockgen -destination automocks/ordermodel.go -package=automocks . OrderModel
type OrderModel interface {
	GetOrderByID(orderID string) (Order, error)
}
$ go generate -x ./...
mockgen -destination automocks/ordermodel.go -package=automocks . OrderModel

run go generate to generate mocking objects

gomock

m := automocks.NewMockOrderModel(ctrl)
m.EXPECT().
	GetOrderByID(gomock.Any()).
	Return(Order{}, nil).
	Times(1)
   
    
m.EXPECT().
	GetOrderByID("correctOrderID").
	DoAndReturn(func(orderID string) (Order, error) {
		...
	}).MinTimes(3)

create mocker with expectation API

gomock + table-driven test?

func TestIsOrderPaid(t *testing.T) {
	tests := []struct {
		name    string
		args    args
		wantOk  bool
		wantErr error
	}{
		{
			name:    "paid",
			args:    args{...},
			wantOk:  true,
			wantErr: nil,
		},
		{
			name:    "notpaid",
			args:    args{...},
			wantOk:  false,
			wantErr: nil,
		},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			ctrl := gomock.NewController(t)
			defer ctrl.Finish()

			orderModel := getOrderModel(t, ctrl, tt.args)
            
			gotOk, err := IsOrderPaid(orderModel, tt.args.orderID)
			...
		})
	}
}
type args struct {
	orderID  string
	order    Order
	orderErr error
}
func getOrderModel(c *gomock.Controller, args args) OrderModel {
	orderModel := automocks.NewMockOrderModel(c)
	orderModel.EXPECT().
		GetOrderByID(args.orderID).
		Return(args.order, args.orderErr).
		Times(1)
	return orderModel
}

Fake data

Fake it until you make it

It's annoying to create test data for each test cases

testAnswer := "test123"
testResult := 1024

faker

Generate fake data with no effort
https://github.com/go-faker/faker

type AStruct struct {
	Number int64
	Height int64
	B      BStruct
}

type BStruct struct {
	Image string
}

func Example_withoutTag() {
	a := AStruct{}
	faker.FakeData(&a)
	fmt.Printf("%+v", a)
	/*
		Result:
		{
			Number: 1
			Height: 26
			B: {
				Image: RmmaaHkAkrWHldVbBNuQSKlRb
			}
		}
	*/
}
faker.Email()           // => mJBJtbv@OSAaT.com
faker.URL()             // => https://www.oEuqqAY.org/QgqfOhd
faker.FirstName()       // => Whitney
faker.E164PhoneNumber() // => +724891571063
faker.UUIDHyphenated()  // => 8f8e4463-9560-4a38-9b0c-ef24481e4e27
faker.IPv4()            // => 99.23.42.63
faker.IPv6()            // => 975c:fb2c:2133:fbdd:beda:282e:1e0a:ec7d
faker.Word())           // => nesciunt
faker.Sentence()        // => Consequatur perferendis voluptatem accusantium.
faker.TimeString()      // => 03:10:25
faker.MonthName()       // => February
faker.YearString()      // => 1994
faker.DayOfWeek()       // => Sunday
faker.DayOfMonth()      // => 20
faker.Timestamp()       // => 1973-06-21 14:50:46
...
// For deterministic result
faker.SetRandomSource(faker.NewSafeSource(rand.NewSource(seed)))

testfixtures

Insert predefined data into database
https://github.com/go-testfixtures/testfixtures

# comments.yml
- id: 1
  post_id: 1
  content: A comment...
  author_name: John Doe
  author_email: john@doe.com
  created_at: 2020-12-31 23:59:59
  updated_at: 2020-12-31 23:59:59
  
 - id: 2
  post_id: 2
  content: Another comment...
  author_name: John Doe
  author_email: john@doe.com
  created_at: 2020-12-31 23:59:59
  updated_at: 2020-12-31 23:59:59
loader, err := testfixtures.New(
	testfixtures.Database(db.DB),
    testfixtures.Dialect("postgres"),
    testfixtures.Location(time.UTC),
    // Load predefined data
    testfixtures.Files(
  	    "fixtures/users.yml",
        "fixtures/comments.yml",
    ),
)

loader.Load()

Integration test

the next step beyond unit test

a real environment?

expensive and hard to maintain

docker-compose up?

good, but we could do better

ory/dockertest

A swiss knife to boot up ephemeral container easily

func LaunchDB(t *testing.T) (*sql.DB, func()) {
    pool, _ := dockertest.NewPool("")

	// pulls an image, creates a container based on it and runs it
	resource, _ := pool.RunWithOptions(&dockertest.RunOptions{
		Repository: "mysql",
		Tag:        "8.0",
		Env:        []string{"MYSQL_ROOT_PASSWORD=secret"},
	})

	// exponential backoff-retry
	var db *sql.DB
	pool.Retry(func() error {
		var err error
		db, err = sql.Open("mysql", DSN)
		if err != nil {
			return err
		}
		return db.Ping()
	})

	return db, func() { pool.Purge(resource) }
}
var db *sql.DB

func TestMain(m *testing.M) {
	var close func()
	db, close = testutil.LaunchDB()

	// Run all integration tests
	code := m.Run()

	// Close db container
	close()

	os.Exit(code)
}

func TestSomething(t *testing.T) {
	// db.Query()
}

ory/dockertest

  • Launch container just in your go test code

  • Remove containers just after test finishes

  • Smoothly integrate with go test framework and your IDE

  • Could use build-tag to run integration test explicitly

// +build integration

import testing
$ go test --tags integration ./...

Continuous smoke test

keep testing in production

Don't let the customer tell you the system is down

Continuous smoke test

      Steps:

  • Identify critical user journey

  • Build smoke tests for those critical cases

  • Continuously run those tests on production

  • Send alerts if these critical smoke tests fail

Continuous smoke test

Benefits

  • Proactively find issues from the user's perspective

  • More possible to find the problems before the user calls us

  • Can be used to calculate SLI

  • go test

  • Test helpers

  • Table-driven test

  • Testable code

  • Mocking

  • Fake data

  • Integration test

  • Continuous smoke test

Happy Go Test

  • We are a regional SaaS startup with two major products

  • Provide B2B solutions for marketing, sales, and customer service needs

We send > 4 billion messages per year

even more than 中華電信's SMS

We are on journey from 1 to 100 with unique challenges and opportunities

Our Tech Blog

We are hiring!!!

Let's talk about automation tests in Golang

By Ting-Li Chou

Made with Slides.com

Let's talk about automation tests in Golang

  • 80

More from Ting-Li Chou