Why isn't a TCP/IP packet sent with the RST flag?

Question

Computer networks

Maybe I'm just dumb, but it always seemed to me that if machine A sends a TCP packet to machine B, which refers to a (no longer) existing TCP connection, then machine B should respond to it with an RST packet (if, of course, there are such left packets are not DROPed by the firewall, etc.). But at the moment I see a slightly different picture, and I can’t understand whether I don’t know something about TCP, or whether these are some bugs in the Linux kernel (to be honest, I traditionally sinned on conntrack, but disabling the firewall did nothing changed).

So, there are two linux servers (let's call them A and B). They are connected to one (unmanaged) switch. The firewall on both servers was initially enabled (and used conntrack - "-m state --state NEW"), but then I reset it (deleted all rules in all tables and set the default to ACCEPT in all chains). Thus, nothing should block/modify packets between these servers.

Now, we start a TCP service on server A, which accepts incoming connections and starts sending data continuously to connected clients. It doesn't try to read anything from clients, and expects to get an EPIPE on write() to a client that has failed.

Next, we launch a TCP client on server B (normal nc - netcat), it connects to server A, starts receiving data from it, and then we disable nc (by Ctrl-C).

After that, the TCP service on server A blocks when calling write () on this already disconnected client, without receiving (even after a couple of minutes) either EPIPE or another error.

I ran tcpdump on both servers and this is what it shows:

1. after nc is interrupted on server B, B sends FIN to A, to which A correctly answers with an ACK - thus, at this stage we get a completely correct semi-open (one-way) TCP connection;
   
2. further, A tries to continue sending data to B (in normal packets with ACK and PSH,ACK flags) - which is also expected and correct;
   
3. BUT, B does not respond to these packets from A in any way (although B knows that the nc process has already exited, so this TCP connection is already closed / does not exist, and, as I expect, it should respond to A with an RST packet) ;
   
4. A, having received no response, stops accepting new data from the TCP service (which at this moment is blocked in the write() call) and starts resending B packets with previous data (for which it also does not receive any response from B).
   

Interestingly, if you run both the client and the service on the same physical server, then everything works as I expect - the RST packet is sent, the service receives EPIPE when writing () immediately after stopping nc by Ctrl-C.

Hence the question: because of what the RST packet may not be sent when the client and the service are running on different servers (and do I understand correctly that it must be sent at all)?

Both servers have updated Hardened Gentoo Linux, kernel 2.6.39-hardened-r8, no specific network settings via sysctl.

I don't know if this is important, but there is quite a lot of network activity on both servers - ` netstat -alnp` produces about 5000 entries at any given moment, and I think about 1000 tcp connections are established and disconnected every second. This activity has nothing to do with the TCP service described above - it refers to an HTTP proxy (3proxy), and in the kernel logs something like this is constantly issued on the 3proxy port:


TCP: Possible SYN flooding on port XXXXX. Sending cookies.
net_ratelimit: 19 callbacks suppressed

Here is an example dump of the above TCP session: i54.tinypic.com/1zz10mx.jpg

Answer 1

2.6.39-hardened-r8
Try the latest vanilla kernel from kernel.org
Maybe they screwed something up with hardened.

Answer 2

And in my opinion B should not swear, he is in the FIN_WAIT_2 state and expects only FIN from A

Answer 3

Can you post the session recording in wireshark format?

Answer 4

The behavior depends on whether the SO_LINGER option is set on the socket . If the client sets SO_LINGER, then there will be no resets while the packets go within the TCP connection.

Answer 5

Is the server code very secret? Can you publish? Very interesting problem, I want to get to the bottom of it

Answer 6

I also remembered that ru.wikipedia.org/wiki/TCP_hijacking
may be somehow connected

Answer 7

Check out Richard Stevens. TCP/IP protocols.
It can be found or bought online. This issue is discussed in detail there.
The chapter “Detecting half-open connections” clearly states that such an action (write to a half-closed socket) should indeed receive RST.
Look also at the behavior of ARP at this point - does the server that should send RST lose information about where to send it?