How can one send megabyte data over a socket?

Question

Python

It is not possible to implement normal buffering. I accept data on 4096 and as long as my data fits in this gap, everything works. But if there is more data, they seem to "get stuck", what solution can be to get them in a loop and then check that the buffer is empty, for example. Found something like:

def listen(conn):
    conn.send(command.upper().encode())
    buffer = conn.recv(4096).decode()
    buffering = True
    while buffering:
        if "\n" in buffer:
            (line, buffer) = buffer.split("\n", 1)
            yield line + "\n"
        else:
            more = conn.recv(4096).decode()
            buffer += more
            if not more:
                buffering = False
    if buffer:
        yield buffer

But for me it doesn't work. How can one send megabyte data over a socket?

Answer 1

recv() will return an empty byte array only in one case - if the other side has closed the data connection (well, or in both directions).
It's not very clear from your description whether you're expecting a connection close, or some kind of end-of-message marker, or even a message of known length.
Also, what happens if the transmitted data is exactly 4096 bytes long?
If you receive data to the end, then you can simplify the code for receiving data to something like

msg_parts = ""
while True:
  part = conn.recv(4096)
  if part:
    msg_parts += part.decode() #по умолчанию это ascii. 
    #С многобайтными кодировками типа utf-8 могут быть проблемы, 
    #если многобайтный символ будет разбит границей пакета
    while "\n" in msg_parts:
      line, _, msg_parts = msg_parts.partition("\n")
      yield line + "\n" 
  else:
    break
yield msg_parts

Well, or in the new python it can be even shorter

msg_parts = ""
while part := conn.recv(4096):
  msg_parts += part.decode() #
  while "\n" in msg_parts:
    line, _, msg_parts = msg_parts.partition("\n")
    yield line + "\n" 
yield msg_parts

Answer 2

For myself, I found a way to put a tag at the beginning and end of the message, for example, </data>and read the data until the end tag is encountered.