Why does reading from a TCP socket in Java lag?

Question

linux

Hello!
There is a problem that has already taken out my whole brain ..
The bottom line is this: when my java 8 application starts, it immediately creates 800 connections to a remote server (there is no access to it), in new threads - threads are created normally, connections are also all connected (the connect method does not throw any exceptions).
netstat command | grep 4444 | grep ESTABLISHED | wc -l outputs 800 (i.e. all connections are successfully established).
In a new thread, a blocking read is called in a loop, which waits for a packet of this type:
*length of data* *data private*
.

Now the problem:
Not all connections receive responses from the server.
Scheme of interaction between the client and the server: the client sends a packet of 42 bytes to the server, the server responds with 1502 bytes.
In practice, all packets left the client (probably because write did not throw IOException), but the responses came in at best 500 connections, the rest apparently blocked on read..

How can this be debugged? Kick in the right direction pliz ..
Or perhaps tell me what options to put on the socket so that there is an answer.

P.s. I don't have access to the remote host. I can only debug from the client...
Thanks in advance for your replies!!

Answer 1

Hmm, what if, say, only 100 threads run? Also, not all will work?
You can try to see what packets are actually coming from the server using tcpdump or something similar.

Answer 2

Threads are created in java, 8 GB of RAM is set on jvm, everything is ok.
Connections are also established - everything, it can be seen through netstat (ESTABLISHED everywhere) ...
but there is no answer ..
========================== ============++
!!!!!!!!!!!!!!!!!! It is important when a bug occurs:
Launched 1 time - all threads started and connections connected, completed the application - all connections went to WAIT_TIME.
Again I start - half works.
I wait until all connections leave with WAIT TIME, I start it - it works)
In short, the error occurs due to tcp previous segment not captured and out-of-order , who knows how to fix it:?

Answer 3

Have you tried to create your own test server for tests, which, for example, would simply print all requests to the console?
Connections close or reuse? Have you tried to change this moment of logic? Does the server side support it?