The hooks pattern in Go: a Golang kata

← Back to list

Posted on 09.11.2023

Last updated on 04.12.2024

Image by AI on Midjourney

Refill!

The hooks pattern

Implementation

A note on an alternative approach

Not so long ago I had to solve a performance issue. When reading an enormous amount of records I had to filter it by another massive left join. I started to play around with the query, trying to optimise it, but ultimately the query time was still way above the tolerable.

Use of a materialized view was also not an option, because re-building the view on every change was costly, and I couldn't even defer it by delegating to some background task, because eventual consistency was not an option.

Then I've decided giving de-normalisation a try. Instead of joining tables and filtering by some condition, I'll have a boolean flag in the main database and then filter by its value. This de-normalisation came with a price, like any others: I had to keep the field up-to-date on every change of a joined table.

For the sake of demo I've named the entities books and authors.

So the plan was to do two things:

create that boolean field,
back-fill it with a query,
keep it up-to-date.

First two items of the plan were easy to do, but what about the last one? I had around 10 places in the codebase, where the data undergone changes, and making 10 unique functions was off the table. And then, what about imports? I shouldn't have updated the flag in a cycle, ideally I should be making changes in the end, when the import is over.

That's when I've come up with the hooks pattern.

# The hooks pattern

So the idea was to introduce another service called updater, that will take a list of book IDs and update the corresponding authors. In order to make the list, I had to accumulate it somehow. For that, another service called hooks was introduced. So, after every call of the update() method of the books repository, I send an event to the hooks service. On every event I dump the book IDs into a common heap. When the request is done, just before the end, I take that heap, check if it's not empty, and then update the authors.

Here is a few notable statements to mention:

The updater and the books services don't talk to each other directly, they use the hooks service as a mediator.
An event can be triggered in any place, at any depth, and this doesn't matter. The event will be recorded anyway, and later processed in the request middleware.
The code that triggers an event should be placed inside a service right after (the name "OnAfter" implies that) the repository function call and the corresponding error check.
The code should not be called from the repository, because the repository should act merely as a wrapper around the database, and should not ideally have other dependencies. In fact, a repository cannot depend on a service by definition.
The context should only hold the values, not the instance of the updater itself. Also, when the request is over, the context is garbage collected, so no need to clean anything manually. If you re-use the context for some reason, inside the Process() there is a deferred call of a function that cleans the mess up.
It is important to run the update outside any transaction. It's okay, because in the worst case scenario it's a noop. Should the update be executed in a transaction, it would immediately cause issues with concurrent requests.
On every update the database will only lock a small (I hope) fraction of the authors table, which is tolerable.
The update of the authors table should happen on the same hit, because the user should be able to immediately see the results. If I wanted to defer it, I would have gone with a materialized view instead.
The whole approach works well if there is no really intense writing happening on the books table. If there is, then there might be jamming due to database row locking.

# Implementation

The application turned to be quite massive, so I will only show the code of the updater service, the hooks service and the fragment of the books service.

So the updater service subscribes to the events using the Init() method, and then latter processes the result using Process().

👉 📃 internal/service/updater/updater.go

package updater

import (
    "context"
    "fmt"
    "net/http"

    "hookspattern/internal/constants"
    "hookspattern/internal/interfaces"
    "hookspattern/pkg/slice"
)

type CtxKeyType string

const (
    CtxKey CtxKeyType = "updater-values"
)

type Values struct {
    affectedBooks []string
}

type Service struct {
    hooksService     interfaces.HooksService
    bookRepository   interfaces.BookRepository
    authorRepository interfaces.AuthorRepository
}

func New(hooksService interfaces.HooksService, bookRepository interfaces.BookRepository, authorRepository interfaces.AuthorRepository) *Service {
    return &Service{
        hooksService:     hooksService,
        authorRepository: authorRepository,
        bookRepository:   bookRepository,
    }
}

func (s *Service) WithValue(ctx context.Context) context.Context {
    return context.WithValue(ctx, CtxKey, &Values{})
}

func (s *Service) GetValues(ctx context.Context) (*Values, error) {
    if value, ok := ctx.Value(CtxKey).(*Values); ok {
        return value, nil
    }

    return nil, fmt.Errorf("values missing from the context")
}

func (s *Service) Init() {
    s.hooksService.On(constants.EventOnAfterBookDelete, func(ctx context.Context, args interface{}) {
        values, err := s.GetValues(ctx)
        if err != nil {
            fmt.Printf("error: %s", err.Error())
            return
        }

        if ids, ok := args.([]string); ok {
            values.affectedBooks = slice.Merge(values.affectedBooks, ids)
        } else {
            fmt.Println("the argument is not of correct type, this is a noop")
            return
        }

        fmt.Println("Deletion event processed!")
    })
}

func (s *Service) Process(ctx context.Context) {
    select {
    case <-ctx.Done():
        fmt.Println("Context is done, this is a noop")
        return
    default:
    }

    fmt.Println("Processing!")
    values, err := s.GetValues(ctx)
    if err != nil {
        fmt.Printf("error: %s", err.Error())
        return
    }

    defer func() {
        values.affectedBooks = []string{}
    }()

    if len(values.affectedBooks) == 0 {
        return
    }

    err = s.authorRepository.RefreshHasBooksFlag(ctx, s.bookRepository.GetAuthorIDsByBookIDsSQL(values.affectedBooks))
    if err != nil {
        fmt.Printf("error: %s", err.Error())
    }
}

// GetHTTPMiddleware returns a middleware that can later be injected into the HTTP server
func (s *Service) GetHTTPMiddleware() func(next http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            ctx := s.WithValue(r.Context())
            // fmt.Printf("Request received: %s %s\n", r.Method, r.URL.Path)

            next.ServeHTTP(w, r.WithContext(ctx))

            s.Process(ctx)
        })
    }
}

The code is licensed under the MIT license

The hooks service implements the publish-subscribe pattern:

👉 📃 internal/service/hooks/hooks.go

package hooks

import "context"

type Handler = func(ctx context.Context, args interface{})

type Service struct {
    handlers map[string][]Handler
}

func (s *Service) On(eventName string, handler Handler) {
    if s.handlers == nil {
        s.handlers = make(map[string][]Handler)
    }

    if _, ok := s.handlers[eventName]; !ok {
        s.handlers[eventName] = make([]Handler, 0)
    }

    s.handlers[eventName] = append(s.handlers[eventName], handler)
}

func (s *Service) Trigger(ctx context.Context, eventName string, args interface{}) {
    if s.handlers == nil {
        return
    }

    if handlers, ok := s.handlers[eventName]; ok {
        for _, handler := range handlers {
            handler(ctx, args)
        }
    }
}

The code is licensed under the MIT license

Only a fragment of the books service illustrating how the event is triggered.

👉 📃 internal/service/book/book.go

package book

import (
    "context"

    "hookspattern/internal/constants"
    "hookspattern/internal/domain/book"
    "hookspattern/internal/interfaces"
)

type Service struct {
    BookRepository interfaces.BookRepository
    HooksService   interfaces.HooksService
}

...

func (s *Service) DeleteBook(ctx context.Context, bookID string) error {
    err := s.BookRepository.DeleteBook(bookID)
    if err != nil {
        return err
    }

    s.HooksService.Trigger(ctx, constants.EventOnAfterBookDelete, []string{bookID})

    return nil
}

...

The code is licensed under the MIT license

The whole thing comes together in the main.go file: the middleware, the services and the repositories.

👉 📃 cmd/main.go

...

    booksRepo := &bookRepository.Repository{
        Session: session,
    }
    authorRepo := &authorRepository.Repository{
        Session: session,
    }

    hooksSvc := &hooks.Service{}
    bookSvc := &bookService.Service{
        BookRepository: booksRepo,
        HooksService:   hooksSvc,
    }
    updaterSvc := updater.New(hooksSvc, booksRepo, authorRepo)

    bookController := book.Controller{
        BookService: bookSvc,
    }

    updaterSvc.Init() // not forgetting to init the updater service

    updaterMiddleware := updaterSvc.GetHTTPMiddleware()

    mux := http.NewServeMux()
    mux.Handle("/books", updaterMiddleware(http.HandlerFunc(bookController.GetBooks)))
    mux.Handle("/books/delete", updaterMiddleware(http.HandlerFunc(bookController.DeleteBook)))

...

The code is licensed under the MIT license

# A note on an alternative approach

There could be an alternative solution to the problem. Instead of keeping the IDs in the context and addressing the change at the end of the operation, it is possible to create a go routine that will listen to a channel. Then, in the other part of the application a change is sent to that channel and immediately addressed inside the go routine.

This method looks simpler, but it disallows cancellation of event processing in case the operation the event is originated from eventually fails and returns an error.

And that's it. Hooks are a good pattern should there a necessity to de-couple two services. The fact that interfaces are used brings danger of having cycle dependency to a minimum.

The complete code is, as usual, here. Enjoy.

Sergei Gannochenko

Business-oriented fullstack engineer, in ❤️ with Tech.
Golang, React, TypeScript, Docker, AWS, Jamstack.
19+ years in dev.

The hooks pattern in Go: a Golang kata

Table of contents

# The hooks pattern

# Implementation

# A note on an alternative approach

Sergei Gannochenko