Look at unit testing in Golang through the eyes of a front-end engineer

← Back to list

Posted on 23.11.2022

Last updated on 04.12.2024

Image by Glenn Carstens-Peters on Unsplash

Refill!

In this article I'm gonna share my learning regarding unit testing in Golang, and how it is different from JavaScript.

The code I made in my previous article works, but, unfortunately, it is not testable. In JavaScript one can just mock any kind of internal dependency just by mocking an entire module declaration with jest.mock('modulename'). This is possible due to the fact that JavaScript is an interpreted language. By its nature it allows code monkey-patching at runtime. However, this is not the case with Golang.

So, in Golang everything about unit testing rests on three main pillars: code split, dependency injection + inversion of control patterns and mocks.

# Code split and DI

In order to properly make things eligible for testing, we need to split one big chunk of code into logically isolated parts. This will lead us to implementing the DI pattern:

🎓 Dependency injection assumes that every unit may have other units as dependencies that are passed as arguments.

It is plain to see, that we have at least two separate logical units here:

an service or client that is in charge of downloading files - that would be an HTTP Client,
an service or client responsible for uploading the files to a bucket - a GCP Storage Client.

Let's move the corresponding functions and compose them under the packages described below.

# HTTP Client

Let's go ahead and define the logic of the HTTP client:

👉 📃 internal/clients/http/client.go

package http

import (
    "copyfiletestable/internal"
    "errors"
    "fmt"
    "net/http"
    "net/url"
    "strings"
)

type Client struct{}

func NewClient() *Client {
    return &Client{}
}

func (c *Client) ExtractFileName(fileURL string) (string, error) {
    parsedURL, err := url.Parse(fileURL)
    if err != nil {
        return "", err
    }

    splitPath := strings.Split(parsedURL.Path, "/")

    if len(splitPath) == 0 {
        return "", nil
    }

    return splitPath[len(splitPath)-1], nil
}

func (c *Client) DownloadFile(url string) (handle *internal.DocumentReadHandle, err error) {
    req, err := http.NewRequest("GET", url, nil)
    if err != nil {
        return nil, err
    }

    client := &http.Client{
        Transport: &http.Transport{},
    }

    resp, err := client.Do(req)
    if err != nil {
        return nil, err
    }

    if resp.StatusCode != http.StatusOK {
        resp.Body.Close()
        return nil, errors.New("could not load the file " + fmt.Sprintf("%d", resp.StatusCode))
    }

    return &internal.DocumentReadHandle{
        Reader: resp.Body,
    }, nil
}

The code is licensed under the MIT license

# GCP Storage Client

Then we define the logic of the GCP Storage client:

👉 📃 internal/clients/gcp/storage.go

package gcp

import (
    "cloud.google.com/go/storage"
    "context"
    "copyfiletestable/internal"
    "github.com/pborman/uuid"
    "io"
    "time"
)

type StorageClient struct{}

func NewStorageClient() *StorageClient {
    return &StorageClient{}
}

func (c *StorageClient) UploadFile(handle *internal.DocumentReadHandle, bucketName string, objectPath string) error {
    uploaderCtx := context.Background()

    uploaderCtx, cancel := context.WithTimeout(uploaderCtx, time.Second*50)
    defer cancel()

    client, err := storage.NewClient(uploaderCtx)
    if err != nil {
        return err
    }

    object := client.Bucket(bucketName).Object(objectPath)
    objectWriter := object.NewWriter(uploaderCtx)

    if _, err := io.Copy(objectWriter, handle.Reader); err != nil {
        return err
    }
    if err := objectWriter.Close(); err != nil {
        return err
    }

    return nil
}

func (c *StorageClient) PrepareObjectPath(objectPathPrefix, fileName string) string {
    return objectPathPrefix + "/" + uuid.New() + "-" + fileName
}

The code is licensed under the MIT license

Before we begin with the FileCopier service, let me remake the main() function like the following:

👉 📃 cmd/main.go

package main

import (
    "copyfiletestable/internal/clients/gcp"
    "copyfiletestable/internal/clients/http"
    "copyfiletestable/internal/services/filecopier"
    "log"
    "os"
)

func main() {
    fileURL := os.Getenv("FILE_URL")
    bucketName := os.Getenv("BUCKET_NAME")
    objectPath := os.Getenv("OBJECT_PATH")

    storageClient := gcp.NewStorageClient()
    httpClient := http.NewClient()

    fileCopier := filecopier.New(&filecopier.ServiceDependencies{
        StorageClient: storageClient,
        HttpClient:    httpClient,
    })

    err := fileCopier.CopyFile(fileURL, bucketName, objectPath)
    if err != nil {
        panic(err)
    }

    log.Println("Done!")
}

The code is licensed under the MIT license

Now, if we just directly import these dependencies into the filecopier.go file, it won't do us any good, because then it won't allow injecting just any dependency, but only of one certain type. Clearly this is not how the IoC pattern is implemented.

# Testability and IoC

🎓 Inversion of control demands that a unit only provides a main flow, a framework, but it does not know anything on how its dependencies are implemented or created.

This is where Go's implicit interfaces come into play.

Implicit interfaces in Golang is a way of achieving loose coupling by enabling "Duck Typing". An interface only states what kind of methods a structure should have, and then during the build time the compiler (and your IDE by doing static analysis) will make sure all loose ends are properly attached.

This works kind of similar to what happens in TypeScript, but from the other perspective - not quite, really, no.

👉 📃 internal/services/filecopier/filecopier.go

package filecopier

import "copyfiletestable/internal"

type gcpStorageClient interface {
    UploadFile(handle *internal.DocumentReadHandle, bucketName string, objectPath string) error
    PrepareObjectPath(objectPathPrefix, fileName string) string
}

type httpClient interface {
    ExtractFileName(fileURL string) (string, error)
    DownloadFile(url string) (handle *internal.DocumentReadHandle, err error)
}

type ServiceDependencies struct {
    StorageClient gcpStorageClient
    HttpClient    httpClient
}

type Service struct {
    storageClient gcpStorageClient
    httpClient    httpClient
}

func New(deps *ServiceDependencies) *Service {
    return &Service{
        storageClient: deps.StorageClient,
        httpClient:    deps.HttpClient,
    }
}

func (s *Service) CopyFile(fileURL, bucketName, objectPath string) (err error) {
    fileName, err := s.httpClient.ExtractFileName(fileURL)
    if err != nil {
        return err
    }

    documentHandle, err := s.httpClient.DownloadFile(fileURL)
    if err != nil {
        return err
    }
    defer func() {
        err := documentHandle.Reader.Close()
        if err != nil {
        }
    }()

    targetObjectPath := s.storageClient.PrepareObjectPath(objectPath, fileName)
    err = s.storageClient.UploadFile(documentHandle, bucketName, targetObjectPath)
    if err != nil {
        return err
    }

    return
}

The code is licensed under the MIT license

See? We define local interfaces and use them as types. We directly import neither HTTP Client nor GCP Storage Client, but only state there what kind of methods the provided dependencies must offer.

I've decided to move the DocumentReadHandle type out into a separate file, to prevent it from being "owned" by one or another entity, so it remains neutral and re-usable.

👉 📃 internal/type.go

package internal

import "io"

type DocumentReadHandle struct {
    Reader io.ReadCloser
}

The code is licensed under the MIT license

io.ReadCloser is already an interface, so any object that has both Read() and Close() methods of a certain signatue will fit there.

# Mocks

So, assume now we wish to test the CopyFile() method. As the Service now has two dependencies, they must be properly mocked.

Generally speaking, when having interfaces in place, any kind of object can become a mock. But, just like in JS we have jest, in order to drastically improve experience with mocking, there is a library for Golang, called Testify.

So let's go ahead and create mocks for both HTTP Client and GCP Storage Client.

# GCP Storage Client

Let's create a new file called storage_mock.go and put it next to the client itself:

👉 📃 internal/clients/gcp/storage_mock.go

package gcp

import (
    "copyfiletestable/internal"
    "github.com/stretchr/testify/mock"
)

type StorageClientMock struct {
    mock.Mock
}

func NewStorageClientMock() *StorageClientMock {
    return &StorageClientMock{}
}

func (m *StorageClientMock) UploadFile(handle *internal.DocumentReadHandle, bucketName string, objectPath string) error {
    args := m.Called(handle, bucketName, objectPath)
    return args.Error(0)
}

func (m *StorageClientMock) PrepareObjectPath(objectPathPrefix, fileName string) string {
    args := m.Called(objectPathPrefix, fileName)
    return args.Get(0).(string)
}

The code is licensed under the MIT license

What is going on here? Few things:

We define a mock StorageClientMock and mix it with the mock.Mock trait.
We mock every method of the original entity:
- m.Called informs Testify that the method was called with such and such arguments,
- args.Get(), args.Error() return the pre-defined result back to the calling code.

# HTTP Client

Same goes for the HTTP Client:

👉 📃 internal/clients/http/client_mock.go

package http

import (
    "copyfiletestable/internal"
    "github.com/stretchr/testify/mock"
)

type ClientMock struct {
    mock.Mock
}

func NewClientMock() *ClientMock {
    return &ClientMock{}
}

func (m *ClientMock) ExtractFileName(fileURL string) (string, error) {
    args := m.Called(fileURL)
    return args.Get(0).(string), args.Error(1)
}

func (m *ClientMock) DownloadFile(url string) (handle *internal.DocumentReadHandle, err error) {
    args := m.Called(url)

    var arg1 *internal.DocumentReadHandle
    if args.Get(0) != nil {
        arg1 = args.Get(0).(*internal.DocumentReadHandle)
    }

    return arg1, args.Error(1)
}

The code is licensed under the MIT license

For the DownloadFile method the code is a bit more tricky, because we need to make sure the type cast won't fail on meeting nil.

# Reader

We also need to implement the ReaderMock to match the io.ReadCloser interface and emulate file reads.

👉 📃 internal/io/io_mock.go

package io

import "github.com/stretchr/testify/mock"

type ReaderMock struct {
    mock.Mock
}

func NewReaderMock() *ReaderMock {
    return &ReaderMock{}
}

func (m *ReaderMock) Read(p []byte) (n int, err error) {
    args := m.Called(p)
    return args.Get(0).(int), args.Error(0)
}

func (m *ReaderMock) Close() (err error) {
    args := m.Called()
    return args.Error(0)
}

The code is licensed under the MIT license

# Writing tests

The time has come to write our first test. The structure is somewhat similar to what we have in jest:

We describe the test by prefixing the method we test with the Test prefix: TestCopyFile.
Define test cases and run them one by one. This is important if we rely on some external resource like a real database.

We have mocks now, but we need to define the behaviour (so-called "expectations"). To do that, the mock.On() method exists:

httpClientMock.On("ExtractFileName", mock.AnythingOfType("string")).Return("", nil)

The code is licensed under the MIT license

This declaration says "Whenever the ExtractFileName() method is called on the mock, with a string argument, return an empty string and a nil error".

If we compare Golang and TypeScript, mock.AnythingOfType("string") matches string type of TS. mock.Anything is the same as any of TS.

The paradigm behind the expectations is to match a set of static parameters into a static result. However, there is also a way to define the result as a "function of arguments" (or, spoken on the Jest language, "create a temporary mocked implementation of a method").

To enable this behaviour, tweak the mocked method in the following way:

func (m *ClientMock) ExtractFileName(fileURL string) (string, error) {
    args := m.Called(fileURL)

    var arg0 string
    var arg1 error

    if mi, ok := args.Get(0).(func(string) (string, error)); ok {
        arg0, arg1 = mi(fileURL)
    } else {
        if args.Get(0) != nil {
            arg0 = args.Get(0).(string)
        }

        arg1 = args.Error(1)
    }

    return arg0, arg1
}

The code is licensed under the MIT license

This code upgrade unlocks the following syntax:

httpClientMock.On("ExtractFileName", mock.Anything).Return(func(fileURL string) (string, error) {
    return fileURL+"abracadabra", nil
})

The code is licensed under the MIT license

As for assertions, those are pretty straightforward to use since they resemble the corresponding ones from Jest: assert.* methods perform assertions, and to check if a mock was called, there is a mock.AssertCalled() function.

All right. As we have all our mocks in place, the behaviour of FileCopier.CopyFile() can be tested right away:

👉 📃 internal/services/filecopier/filecopier_test.go

package filecopier_test

import (
    "io"
    "testing"

    "copyfiletestable/internal"
    "copyfiletestable/internal/clients/gcp"
    "copyfiletestable/internal/clients/http"
    ioMock "copyfiletestable/internal/io"
    "copyfiletestable/internal/services/filecopier"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/mock"
)

type TestFlow func(t *testing.T)

func TestCopyFile(t *testing.T) {
    testCases := map[string]TestFlow {
        "Should download a file": func(t *testing.T) {
            // Create mocks and define expectations. This code can be moved out to a helper to avoid code duplication for every test.
            storageClientMock := gcp.NewStorageClientMock()
            storageClientMock.On("UploadFile", mock.Anything, mock.AnythingOfType("string"), mock.AnythingOfType("string")).Return(nil)
            storageClientMock.On("PrepareObjectPath", mock.AnythingOfType("string"), mock.AnythingOfType("string")).Return("")

            readerMock := ioMock.NewReaderMock()
            readerMock.On("Read", mock.Anything).Return(0, io.EOF)
            readerMock.On("Close").Return(nil)

            httpClientMock := http.NewClientMock()
            httpClientMock.On("ExtractFileName", mock.AnythingOfType("string")).Return("", nil)
            httpClientMock.On("DownloadFile", mock.AnythingOfType("string")).Return(&internal.DocumentReadHandle{
                Reader: readerMock,
            }, nil)

            // Create the client and provide the dependencies.
            fileCopier := filecopier.New(&filecopier.ServiceDependencies{
                HttpClient:    httpClientMock,
                StorageClient: storageClientMock,
            })

            // Execute the method we test here.
            err := fileCopier.CopyFile("http://path.to/file.pdf", "my-bucket", "my-object")

            // Check the outcome.
            assert.NoError(t, err)
            httpClientMock.AssertCalled(t, "DownloadFile", mock.Anything)
            storageClientMock.AssertCalled(t, "UploadFile", mock.Anything, mock.Anything, mock.Anything)
            readerMock.AssertCalled(t, "Close")
        },
    }

    // Run the test cases, one by one.
    for name, tc := range testCases {
        t.Run(name, testFlow)
    }
}

The code is licensed under the MIT license

# Setup-verify pattern

If I look at the previously created unit test, I won't help to notice that the code is a bit muddy: it is unclear where the preparations end, and the checking start. To structure the code a bit better, there is a pattern called setup-verify, involving two functions to be put in place. Let's revrite the existing code in that fashion.

👉 📃 internal/services/filecopier/filecopier_test.go

package filecopier_test

import (
    "io"
    "testing"

    "copyfiletestable/internal"
    "copyfiletestable/internal/clients/gcp"
    "copyfiletestable/internal/clients/http"
    ioMock "copyfiletestable/internal/io"
    "copyfiletestable/internal/services/filecopier"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/mock"
)

func TestCopyFile(t *testing.T) {
    createMocks := func() (*gcp.StorageClientMock, *ioMock.ReaderMock, *http.ClientMock) {
        storageMock := gcp.NewStorageClientMock()
        storageMock.On("UploadFile", mock.Anything, mock.AnythingOfType("string"), mock.AnythingOfType("string")).Return(nil)
        storageMock.On("PrepareObjectPath", mock.AnythingOfType("string"), mock.AnythingOfType("string")).Return("")

        readerMock := ioMock.NewReaderMock()
        readerMock.On("Read", mock.Anything).Return(0, io.EOF)
        readerMock.On("Close").Return(nil)

        httpClient := http.NewClientMock()
        httpClient.On("ExtractFileName", mock.AnythingOfType("string")).Return("", nil)
        httpClient.On("DownloadFile", mock.AnythingOfType("string")).Return(&internal.DocumentReadHandle{
            Reader: readerMock,
        }, nil)

        return storageMock, readerMock, httpClient
    }

    type setup struct {
        FileURL     string
        BucketName  string
        ObjectName  string
        StorageMock *gcp.StorageClientMock
        ReaderMock  *ioMock.ReaderMock
        ClientMock  *http.ClientMock
    }

    type setupFunc func(t *testing.T) *setup
    type verifyFunc func(t *testing.T, setup *setup, err error)

    testCases := map[string]struct {
        setupFunc  setupFunc
        verifyFunc verifyFunc
    }{
        "Should download a file": {
            setupFunc: func(t *testing.T) *setup {
                storageMock, readerMock, httpClient := createMocks()
                return &setup{
                    FileURL:     "http://path.to/file.pdf",
                    BucketName:  "my-bucket",
                    ObjectName:  "my-object",
                    StorageMock: storageMock,
                    ReaderMock:  readerMock,
                    ClientMock:  httpClient,
                }
            },
            verifyFunc: func(t *testing.T, setup *setup, err error) {
                assert.NoError(t, err)
                setup.ClientMock.AssertCalled(t, "DownloadFile", mock.Anything)
                setup.StorageMock.AssertCalled(t, "UploadFile", mock.Anything, mock.Anything, mock.Anything)
                setup.ReaderMock.AssertCalled(t, "Close")
            },
        },
    }

    for name, tc := range testCases {
        t.Run(name, func(t *testing.T) {
            setup := tc.setupFunc(t)

            fileCopier := filecopier.New(&filecopier.ServiceDependencies{
                HttpClient:    setup.ClientMock,
                StorageClient: setup.StorageMock,
            })

            err := fileCopier.CopyFile(setup.FileURL, setup.BucketName, setup.ObjectName)

            tc.verifyFunc(t, setup, err)
        })
    }
}

The code is licensed under the MIT license

The method induce more boilerplate, however, it is clearly seen what the input data is, and what to expect as a result.

# Executing tests

Golang has a built-in test runner, that can be invoked via the go test CLI command. While it's useful as part of a CI/CD pipeline, for local execution it's recommended to use richgo, because it can provide colored output.

go install github.com/kyoh86/richgo@latest

The code is licensed under the MIT license

Extend the Makefile by adding a new action:

test:
    PROJECT_DIR=${PWD} godotenv -f ./env.local richgo test ./... -v -p=1 -count=1 | tee test.output

The code is licensed under the MIT license

So, by running make test we hopefully can see the following picture:

START| CopyFile
START|   CopyFile/Should_download_a_file
PASS | CopyFile (0.00s)
PASS |   CopyFile/Should_download_a_file (0.00s)
PASS | copyfiletestable/internal/services/filecopier

The code is licensed under the MIT license

Right. Now we know how testing works in Golang. Hope this article was helpful! The code can be found here. Enjoy!

Sergei Gannochenko

Business-oriented fullstack engineer, in ❤️ with Tech.
Golang, React, TypeScript, Docker, AWS, Jamstack.
19+ years in dev.

Look at unit testing in Golang through the eyes of a front-end engineer

Table of contents

# Code split and DI

# HTTP Client

# GCP Storage Client

# Testability and IoC

# Mocks

# GCP Storage Client

# HTTP Client

# Reader

# Writing tests

# Setup-verify pattern

# Executing tests

Sergei Gannochenko