How to deal with socket.io load?

Question

Socket.io

Good afternoon everyone
The problem is that when there is a sharp influx to the site, socket.io raises the CPU to 100% and problems begin (Socket.io + redis sub pub) What should I do? What are your tips?

All on one server
Technologies: php-fpm,nginx,redis,node,socket.io

UPD: Processor 16 X QEMU Virtual CPU version 2.5+ 2593.748 MHz
RAM 16 GB
UBUNTU 16.04

Problems start when there are 1000 connections
Thank you

Answer 1

Launched in one process most likely a socket server?
Do sharding so that requests are processed in several threads. Even on the same server.
Because most likely you have only one core loaded.
There are no other options other than this.
If, after all, the entire server is loaded (which I doubt), then shard to different servers already.
Well, for prevention, of course, it’s worth checking the code for bottlenecks, although 1000 connections for one core are already normal, especially if there is any kind of logic there. So this ceiling is just for one core/process most likely on your server.

Answer 2

1 The socket.io logging mode is debug.
2 Application logging mode (php-fpm, nginx, redis, node) - debug.
3 Analysis of socket.io logs.
4 Analysis of application logs (php-fpm, nginx, redis, node).
5 Analysis of other logs (maillog, messages, secure, auditd, what database do you have there).
6 System load analysis (htop, iotop, ...).
7 Replacing the OS with a more recent one with a stable kernel, stable applications, stable updates.
8 OS profiling for the appropriate type of load.
9 Profiling applications for the appropriate type of load.
10 Load testing of each application and the entire server as a whole.
11 Planning to move to a dedicated server. with normal iron. You can have a greedy neighbor who saps resources mercilessly. Maybe you are that greedy neighbor, so the admins screwed up your capabilities. Maybe the greedy owner of VDS and you are fighting among yourself for the missing resources.
Looking towards docker or LXC is useless. Especially in the case of a virtual machine.