Node.js, socket.io, cluster. How to organize the architecture of a multiplayer game?

Question

Highload

Good afternoon!
There is a game server on node.js + socket.io. Game rooms are spinning on the server. Each room is for 2-4 players. The room is essentially an instance of the game engine with the current state of the game + the players that are connected to this game room (from the point of view of the code, the room is a js object, and the players differ by socket id).
Clients interact with the server using socket.io and only enter game moves, get the updated game state from the server.
On the server, game events are routed - from which player the move came, to which game room this move should be given.
Now the task is to improve the application for a higher load. There are two goals for optimization - 1) a greater number of simultaneous connections and 2) parallel calculation of game moves. The option of using the cluster module is being considered. It would be desirable to take out costly operations in worker'y. But there is the following difficulty. An array of game rooms with their current state should be stored centrally. And this data (rather complex js-objects) cannot be placed in the database. It seems like the storage of this data on the master suggests itself. And from workers it is required to provide access to them. Moreover, some other locking mechanism is required so that two workers do not try to work simultaneously with one piece of memory.
Has anyone encountered a similar problem? How did you implement the interaction between workers and master?
Any other options are also being considered - maybe it’s worth doing something differently at all? Share your experience, you have probably already faced similar tasks.

Answer 1

In general, you yourself answered your own question: we use a cluster, a master and several workers (usually by the number of cores). The data that everyone needs is stored either on the master or in the noskl, as indicated above. Workers ask permission to update the data, and if the current record is now blocked by someone, then they wait. If not locked, then lock and update, then release. At least he did just that.

Answer 2

И эти данные (достаточно сложные js-объекты) нельзя поместить в БД

noSQL. MongoDB, for example, stores everything in the form of JS objects.
You can also try an in-memory solution like Redis.
The database in your case can then, if necessary, also be sharded to improve performance.

Answer 3

1. We scattered the server through the cluster http server (we did not use our own long-polling, socket.io) into several threads according to the number of cores in the system. There was also a need to store data in the main thread and process it there if necessary. Cluster provides the ability to communicate between child threads and the head one, this is done as follows:
In the head thread (master) we set the message event - which waits for a message from child threads
, we divide all the logic into 2 files server.js (Master) and worker.js

var config = {
    numWorkers: require('os').cpus().length,
    refreshTime: 1000, // Milliseconds between data refreshes.
    waitTime: 90, //Время в секундах когда пользователь считается оффлайн
    worker: {
        port: XXXX,
        setNoDelay: true,
        mongoUrl: 'mongodb://XXX:[email protected]/tmp/mongodb-270XX.sock'
    }
};

here we store the settings, so it’s more convenient so that you don’t rummage through the code later, if you need to fix it

cluster.setupMaster({
    exec: "worker.js"      
});

we tell the cluster in which files the executable code of the workers

for (var i = 0; i < config.numWorkers; i++)
    (function(worker) {
        worker.on('message', function(data) {
      switch(data.routeType){   // мы  в сообщениях преедавали  массив , routeType  -  переменная масисива которую мы ввели и в нашем случае она  обозначала какоето действие
                case 'act1':
                    //  Что то делаем, выполняем какойто код   массив  data  может  содержать еще какието элементы котоыре вы будете обрабатывать
                    break;
                case 'act1':
            // Второе действие
                    break;
        default:
          break;
      }
        });

        worker.on('exit', function(code, signal) {
            console.log('Worker died: ', worker.process.pid, code, signal);  //  Сообщаем в консоль что у нас подох один из потоков
        });
    })(cluster.fork({WORKER_CONFIG: JSON.stringify(config.worker)}));  //  Форкаем  потоки

worker.js -- code in streams
var config = JSON.parse(process.env.WORKER_CONFIG || "{}"); -- this way you can get some settings from the main thread in the child thread

process.on('message', function(data){   // получаем сообщение от  головного потока 
    switch(data.routeType){
      case 'myMess1': 
        // Выполняем свой код
            break;
            case 'myMess2':
            // Второе действие
            break;
      default:
      break;
      }
}

send message to head

var procMess = {
  routeType: 'myMess',
//..........  любые ваши данные которые нужно передать
};

process.send(procMess);	    // Отправка данных  в головной поток

I advise you to collect this example habrahabr.ru/post/123154 in order to visually see how messages are transmitted from the head thread to the workers and back.
We assembled the first version of our engine according to a similar principle and tested it under high loads - there were no problems.

Answer 4

2.After the new year, we decided to try out socket.io - we liked the fact that it immediately implements both web sockets and long-polling in itself, we assembled the solution without any problems in one thread.
An interesting thing began when they tried to parallelize
until a solution was found, they ran into people with the same problems, they used Reddis - until they really want to install it.
adamnengland.wordpress.com/2013/01/30/node-js-clus...

Answer 5

Александр, доброго времени суток. Полагаю, на свой вопрос ответ уже нашли, теперь хотел бы обратиться к вам за консультацией. Сам проектирую браузерку — клиент на Unity, сервер на node.js, бд на redis (orm для неё на jugglingdb, очень похожа на mongoose), связь на tcp. Хочу сразу попробовать кластерную архитектуру, дабы потом не пришлось в спешке переписывать логику под неё в случае неожиданного роста онлайна.
Как и у вас, игрушка будет пошаговой (боёвка — в стиле jrpg). Единственное, что в бою завязано на таймер — выдача тому или иному игроку права хода на определённый лимит времени (таймаут хода) и централизованное управление просмотра анимаций ударов и заклинаний («анимация удара файрболлом 5 секунд, после передача хода следующему игроку»).
Вне боёв персонажам нужен временной луп для постепенного восстановления параметров.
Какого рода задачи логично распределить в воркеры? Допустимо ли использовать redis pub-sub для обмена очень большим количеством сообщений между потоками? Буду очень признателен вашим советам.