How to configure HAProxy and OS to process 30k requests/second?

Question

HAproxy

Good afternoon!
There are two sites:
1. jMeter (50 nodes + control);
2. HAProxy -> 6 nodes with web servers;
The sites are located in different regions of the country, both have a channel - 1 Gbps.
jMeter is used to simulate the same type of action - 10 steps to fill out a form (getting statics, directories, sending a form as a json request). Exposed primitive Assertions on respone. From each node, 300 "users" are launched, filling out the form twice. In total, we get 50 x 300 x 10 x 2 = 300k requests.
HAProxy, it seems to me, is not experiencing any particular difficulties, because at the beginning of the test the CPU is loaded, but almost immediately the load drops.
Using Ubuntu 18.04 Server LTS.
Tests show (in both HTTP and TCP mode) that the first requests are processed quite quickly, about 12k requests / sec, with an average response time of 0.5 sec. And then the trouble begins, as I suppose, at the network level - I sin on the lack of free ports, because in OS TIME_WAIT lasts 60 seconds. And if we compare the "sticking" time of the jMeter results and a few Connection Refused, it turns out that during the test the connection pool limit is exhausted, it goes into the TIME_WAIT state and the clients wait until "there is free space".
I tried using Round Robin, but there is only one channel on the site and HAProxy sits behind NAT.
I do not apply technical details, because I have not "fixed" the configuration yet.
I would appreciate any recommendation.

Answer 1

Socket sharing, multiple instances listening on the same socket.