Highload websocket project architecture?

Question

go

Good evening everyone, we are designing a highly loaded service in the go language. You need to keep a lot of simultaneous connections.
So far, the logic is as follows:
1. The client connects to the sockets with the transfer of a unique channel number through which messages will be received
2. The service looks for the channel object in the map, if it does not find it, it looks in the database (while we are looking towards redis) for an entry with the number this channel and then puts it in the map, if it doesn't find anything, it disconnects the client.
3. Further in the goroutine, it launches the pub/sub listener and subscribes to the channel ID. When messages arrive, it sends them over sockets to the user.
A channel is something like a room for multiple connections.
The question is architecture. Is the option of storing channels in an array suitable, or are there any other ideas?
An example of such a service is PUSHER
UPD ::
There are no ideas yet about storing channels, but it will be expensive to create a listener in each routine, therefore we create a global listener for all channels, and parse the message into a channel / data.
That is, in pub we write {channel: kwb9162, message: hello} and parse on the server side.

Answer 1

Due to the nature of the garbage collector in Go, you are likely to have problems if the keys in the map contain pointers (including structures containing them).
Here is a way to store sockets in a slightly different form without pointers in the keys:

const n = 4 // длина идентификатора комнаты в байтах
type (
  noop struct{}
  WebSocket struct{} // сокет

  Sockets []WebSocket
  RoomKey [n]byte
  
  Index uint32

  Room map[Index] struct{}
  Rooms map[RoomKey] Room
)

var sockets = make(Sockets, 0, 2000000)
var rooms = make(Rooms, 1000)
var lastSocketRoomKey RoomKey

func insert(ws WebSocket, rk RoomKey){
  room, _ := rooms[rk]
  // тут создание комнаты, если ее нет
  // ...
  
  sockets = append(sockets, ws)
  
  last := len(sockets) - 1
  room[Index(last)] = noop{}
  
        // сохраним ключ комнаты последнего сокета
        // чтобы можно было исправить индекс при удалении
        copy(lastSocketRoomKey[:], rk[:])
}

func remove(i Index, rk RoomKey){
  // удаляем сокет,
  // переместив последний в массиве на место удаляемого

  last := len(sockets) - 1
  sockets[i] = sockets[last]
  sockets = sockets[:last]

  // удаляем сокет из комнаты
  room, _ := rooms[rk]
  delete(room, Index(i))

  // чиним индекс перемещенного сокета
  roomOfLast := rooms[lastSocketRoomKey]
  delete(roomOfLast, Index(last))
  roomOfLast[Index(i)] = noop{}
}

When you need to broadcast messages, in my example the socket access would be something like this:

for _, index := range rooms[roomkey]{
     // sockets[index] -- сокет в комнате с roomkey
     
    // т.к. из-за особенности структуры искать сокет в комнате получается невыгодно долго, 
    // придется удалять закрытые сокеты "ленивым" способом во время рассылки, 
    // либо заполнять массив индексов и после уже удалять эти сокеты из общего хранилища
    // функцией remove(index, roomkey)
}

Obviously, in this example, access to sockets and rooms should not come from different goroutines.

Answer 2

Searching for a channel in a normal array is O(n). At least a dichotomy, but it's better to use map

Answer 3

I'll leave it here, in case you need the concept https://github.com/alfssobsd/demowebsocket-server

Answer 4

Each user has his own channel, key=value storage will do here:

{
   user: 1,
   channel: "asdsad12e123"
}