How to fix End of Stream encountered before parsing was completed error?

Question

.NET

There are two .NET applications - client and server. The client sends a hash of the settings to the server via TCP every five seconds (using TcpClient). The server, in turn, waits for a TCP request from the client (TcpListener) and, upon receipt, checks the hash of the client's settings with the hash of the current settings. If the hashes differ, the server generates a response to the client and sends the new settings to the agent via TCP in binary format (via BinaryFormatter). The client, in turn, waits in a separate thread for new settings, and as soon as the settings arrive, it deserializes them with a BinaryFormatter and applies them. If there are few settings on the server, then everything works out successfully. But as soon as the number of settings increases, the client crashes when trying to deserialize the object (Error:
So, a short scheme of work.
1. When the client starts, one TCPListener is opened on port 8765 and the condition is checked in the while(true) infinite loop: if (listener.Pending()), then we read the response from the server, deserialize it and apply the settings received from the server.
2. In the second thread of the client, every five seconds a TCP request is sent to the server, this request stores the hash of the current client settings.
3. When the server starts, one TcpListener is opened on port 8766 and the condition is checked in the while(true) infinite loop: if (listener.Pending()), then we compare the hash of the client settings with the current hash of the settings on the server. If the hashes differ, then we send new settings to the client on port 8765.
== Client ==

TcpListener creation code

try
            {
                listener = TcpListener.Create(ListenPortNumber);
                listener.Start();
            }
            catch (Exception ex)
            {
                _listenToServerThrLive = false;
            }

            while (_listenToServerThrLive)
            {
                try
                {
                    if (listener.Pending())
                    {
                        ThreadPool.QueueUserWorkItem(HandleServerRespose, listener);
                    }
                }
                catch (Exception ex)
                {
                    Thread.Sleep(100);
                    continue; // --> while(_listenToServerThrLive)
                }

                Thread.Sleep(100);
            }

HandleServerRespose

private void HandleServerRespose(object objListener)
        {
            TcpListener listener = objListener as TcpListener;

            try
            {
                ResponseData hbRespData = null;
            byte[] data = new byte[256];

            using (MemoryStream memStream = new MemoryStream())
            {
                using (TcpClient client = listener.AcceptTcpClient())
                {
                    using (NetworkStream stream = client.GetStream())
                    {
                        do
                        {
                            int bytes = stream.Read(data, 0, data.Length);
                            memStream.Write(data, 0, bytes);
                        } while (stream.DataAvailable);
                    }
                }

                var frmt = new BinaryFormatter();
                memStream.Flush();
                memStream.Position = 0;
                hbRespData = (ResponseData)frmt.Deserialize(memStream); // здесь вылетает ошибка "End of Stream encountered before parsing was completed"
            }

                if (resp.NeedSettingsUpdate)
                {
                    _settingsProvider.UpdateSettings();
                }
            }
            catch (Exception exc)
            {

            }
        }

== Server ==

Creating a TcpListener

listener = TcpListener.Create(_settings.AgentHeartbeatListenPort);
 listener.Start();
 while (_heartbeatsListenerThrLive)
 {
try
                {
                    if (listener.Pending())
                    {
                        TcpClient client = listener.AcceptTcpClient();
                        ThreadPool.QueueUserWorkItem(RequestsHandler, client);
                    }
                }
                catch (Exception ex)
                {
                    continue; // --> while (_heartbeatsListenerThrLive)
                }

                Thread.Sleep(100);
            }
}

Processing a request from a client

private void RequestsHandler(object objClient)
        {
            IPAddress clientIpAddress = IPAddress.None;
            HeartbeatData hbdata = null;

            using (TcpClient client = objClient as TcpClient)
            {
                clientIpAddress = ((IPEndPoint)client.Client.RemoteEndPoint).Address.MapToIPv4();

                try
                {
                    hbdata = ReadHeartbeatData(client);
                }
                catch (Exception exc)
                {
                    _log.Error("Unable to read client heartbeat data.", exc);
                }
                if (!hbdata.SettingsHash.Equals(commonHash, StringComparison.InvariantCultureIgnoreCase))
                {
                     try
                    {
                        SendHeartbeatResponse(clientIpAddress, ClientHeartbeatResponsePort, response);
                    }
                    catch (Exception exc) { }

SendHeartbeatResponse

private void SendHeartbeatResponse(IPAddress ipAddress, int portNumber, ResponseData response)
        {
            lock (lockMarker)
            {
                using (TcpClient tcpClient = new TcpClient())
                {
                    tcpClient.Connect(ipAddress, portNumber);
                    using (NetworkStream stream = tcpClient.GetStream())
                    {
                        using (var binStream = new MemoryStream())
                        {
                            var frmt = new BinaryFormatter();
                            frmt.Serialize(binStream, response);
                            binStream.Flush();
                            binStream.Position = 0;
                            binStream.CopyTo(stream);
                        }

                        tcpClient.Close();
                    }
                }
            }
        }

Answer 1

When transmitting data over the network, there are sometimes pauses.
Your code waits for the first such pause:
while (stream.DataAvailable)
...and ends reading without waiting for the rest of the data.
It is necessary to read not yet DataAvailable, but until the required number of bytes arrives. To do this, the length of the message must be communicated to the client before the message itself is sent.