Google PubSub and Go: a Golang kata

Posted on 07.03.2023

Image by Sharon Waldron on Unsplash

Refill!

Time for a new Go kata, a really brief one! This time, I've decided to refresh my knowledge on the Pub/Sub service of GCP.

The Pub/Sub service is extremely popular to enable indirect communication between microservices hosted in the GCP cloud. It works like a chat, but for applications. If a service wants to communicate, it sends a message to a specific topic. Other services, should they be interested in receiving messages, subscribe to that topic. A subscription can have a filter based on message attributes.

Anyway.

For this kata I haven't gone too much into abstractions, everything remains in just a single main() function. Also, I have the resources created by hand, not with Terraform, for I wanted this kata to be fully dedicated to Go.

There are two ways working with Google Pub/Sub locally: with an emulator, or using the actual cloud. While with the emulator it is more or less clear, when using the cloud, a service account (basically unmanned account) must be created. There are millions of articles on how to do it, here is just the very first one I stumbled upon.

After the account is created, download the JSON credentials file and place it somewhere on your local machine. You are gonna need it.

The code

The service is really simple, here it is:

👉 📃 cmd/main.go

package main

import (
    "context"
    "log"
    "os"

    googlePubSub "cloud.google.com/go/pubsub"
    "google.golang.org/api/option"
)

func main() {
    projectID := os.Getenv("PROJECT_ID")
    topicID := os.Getenv("TOPIC_ID")
    subscriptionID := os.Getenv("SUBSCRIPTION_ID")
    credentialsFile := os.Getenv("CREDENTIALS_FILE")

    credentials, err := os.ReadFile(credentialsFile)

    ctx := context.Background()
    ctx, cancelCtx := context.WithCancel(ctx)

    pubSubClient, err := googlePubSub.NewClient(ctx, projectID, option.WithCredentialsJSON(credentials))
    if err != nil {
        panic(err)
    }

    topic := pubSubClient.Topic(topicID)

    attributes := map[string]string{
        "Hello": "There",
    }

    message := &googlePubSub.Message{
        Data:       []byte("Hello!"),
        Attributes: attributes,
    }

    publishResult := topic.Publish(ctx, message)
    messageID, err := publishResult.Get(ctx)
    if err != nil {
        panic(err)
    }

    log.Printf("Message was published with id %s", messageID)

    subscription := pubSubClient.Subscription(subscriptionID)

    err = subscription.Receive(ctx, func(receiveCtx context.Context, message *googlePubSub.Message) {
        log.Printf("Message received %s", string(message.Data))
        message.Ack()

        cancelCtx()
    })

    log.Printf("Done")

    if err != nil {
        panic(err)
    }
}

The code is licensed under the MIT license

What it does is the following:

First, it reads the content of the JSON credentials file.
Then, it makes an instance of the Pub/Sub client, and after that makes a reference to the topic. The topic itself must be created beforehand, either with Terraform or via the GCP web interface! It is a resource.
Then it makes a message and publishes it to the topic.
Shortly after, it creates a reference to a subscription. Just like with the topic, the subscription must be explicitly provisioned, as it's a resource too. I used the one that was automatically created when I made the topic.
Then we wait for a message to arrive. As soon as we have it, we acknowledge it. Acknowledgment tells GCP the message was successfully consumed and now should not be delivered anywhere else. If the message is not acknowledged (message.Nask()), it will be re-delivered immediately to another subscriber, with little delay, until the TTL expires. You should always either ask or nask a message.
Inside of the callback we call cancelCtx as we don't want to receive more messages and wish to interrupt the execution.

Simple as it is.

You can read more about the Pub\Sub to understand it better. Here is a set of laconic yet informative articles: one, two, three.

Here goes my .env.local file, use it as a template:

👉 📃 .env.local

PROJECT_ID=project-id
TOPIC_ID=test-topic
SUBSCRIPTION_ID=test-topic-sub
CREDENTIALS_FILE=/Users/my-user/credentials.json

The code is licensed under the MIT license

And the Makefile:

👉 📃 Makefile

run:
    @godotenv -f ./.env.local go run ./cmd/main.go

The code is licensed under the MIT license

That's all, folks!

Here is the reference on the Go pub/sub client.

As usual, the code of the kata is available here. Have fun!

← Back to list

Sergei Gannochenko

Business-oriented fullstack engineer, in ❤️ with Tech.
Golang, React, TypeScript, Docker, AWS, Jamstack.
15+ years in dev.