Cover image from github.com/MariaLetta/free-gophers-pack

👉🏻 Read part 2 here

How to achieve real-time

Apps that need real-time capabilities such as chat, gaming, live dashboard and etc. These can achieve with the following ways:

Managed services

Managed services such as Firebase, Pusher and etc can setup quickly. But it may have limited controls, and potentially incur high cost if your apps has a lot of traffics. It is good to use for small app with occasional traffics and proof of concepts apps, but not recommended for medium to large traffics apps in my opinion.

HTTP Polling

There are two types of polling, short and long.

• Short polling will make request on every interval. The interval is fixed however.
• Long polling will make new request to server only if the previous request received response, or timeout.

There were a lot of shortcomings in both pollings  such as header overhead, latency, timeouts, caching, and etc.

This however easy to scale, because it is stateless, horizontal scale can be done easily.

Server-sent event

SSE pushes data to the client on a unidirectional way. Client cannot push data to server. Client and server maintain a TCP connection

Websocket

Websocket is can send data in bi-directional way from client to server, or sever to client. Client and server maintain a TCP socket connection.

Problem with stateful connection

Stateful connections such as websocket and SSE can send data in real-time, but not easy to scale. Websocket has a hard limit of 65,536 on a single server. It can horizontal scale by adding more servers, but you will need to ensure all servers will receive the data, as different clients may connect to different servers.

Code example

Checkout the source code scaling-ws on problem branch

The code example has the following structure

Clone the project, checkout problem branch and open terminal

# on terminal
# start web ui
cd web
npm i && npm run serve

# open another terminal
# start ws server
go run main.go --addr :4000 --skipWs

# open another terminal
# start api server
go run main.go

Open two browser tabs on localhost:8080 Both tabs select ok on the dialog, to use ws server 2

On the left is client A, right is client B. Both receives messages when then Call API button is clicked on either side.

Now we want to scale the websocket server, by adding additional server The structure now becomes this

Now refresh both tabs, first tab select cancel on dialog to use ws server 1, second tab select Ok on dialog to use ws server 2,

Now client on the left can't receive any data, but client on the right can received all data when either of the buttons is clicked. Because the API server only send signal to WS server 2, but client on the left is connecting to WS server 1

Code snippet of API server send message to Websocket

 r.POST("/ping", func(c *gin.Context) {
 // send to ws
  conn.WriteJSON(map[string]string{"sendTime": time.Now().Format(time.ANSIC)})
  c.JSON(200, gin.H{"message": "pong"})
})

and WS server receives, and broadcast to connected client

Solution

Checkout the source code scaling-ws on solution-nats-ws branch

We can sync up both WS server by adding NATS in between as communication bus.

NATS.io is a simple, secure and high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.

The structure now updated with NATS

Download and install NATS, and refactor with pub/sub pattern

Checkout to solution-nats-ws branch, close all terminal that run Go code and start new terminal

# open another terminal
# start nats-server
# make sure it is installed
nats-server

# open another terminal
# start api server
make start-api

# open another terminal
# start ws server
make start-ws

Refresh both tabs, first tab select cancel on dialog to use ws server 1, second tab select Ok on dialog to use ws server 2,

Now both clients will received data regardless which WS server they connected to.

This can achieve with the code below

Publish on API server

const subject = "com.scaling-ws.updates"

r.POST("/ping", func(c *gin.Context) {
    // publish to nats
    message := map[string]string{"sendTime": time.Now().Format(time.ANSIC)}
        if err := ec.Publish(subject, message); err != nil {
            log.Fatal(err)
    }

    c.JSON(200, gin.H{"message": "pong"})
})

Subscribe on ws server

// subscribe nats
sub, err := nc.Subscribe(subject, func(m *nats.Msg) {
     // do something after received data 
    // such as sending to clients
})

We have successfully horizontal scaled websockets servers with NATS. If you interested on running the example repo, can be found on Github or Gitee, instruction is written on README.

That's all for part 1. For the coming part 2, we will refactor to NATS websocket client when GA (currently still in preview), which potentially can remove our own websocket server.

👉🏻 Read part 2 here

Feel free to comment below if you have any suggestions. Thanks for reading 🍻