How to work with libavformat without falling into big delays?

Question

linux

My task is simple: to write a simple audio player with http and rtmp support. The main thing is the minimum delays and work with low bitrates. For such things, the all-in-one set, namely libav*, is a good fit, where working with media data is quite simple. However, I have problems:
1. Lack of support for the ICY protocol, so just ffurl_open () does not call. Not a problem, there is libcurl and then you can parse it with your hands.
2. The parsing of the FLV stream (namely, it is inside RTMP) occurs after loading a decent piece of data (max_analyze_duration will not help here). For low-bitrate streams, 64kb is too much.
At the moment I'm trying to receive data manually (libcurl/librtmp) and feed data via AVIOContext:

AVIOContext *io=avio_alloc_context(buffer, BUFFER_SIZE, 0, NULL, my_byte_reader_callback, NULL, NULL);
context=avformat_alloc_context();
context->pb=io;
...
while(1){
av_read_frame...
}

And here I have a big problem: the read callback is called with too large buffer sizes. Often there are requests for 4MB packages (in the case of low bitrates, this can be an entire hour of audio stream). I decided to play with the returned values ​​(after all, it is not necessary to fill the entire buffer at once), I got something like this:
I gave a buffer of 1280 bytes, no more than got request for 2816 bytes, sent 1280
got packet size 105
got packet size 108
got packet size 93
got packet size 106
got packet size 101
got packet size 106
got packet size 106
got packet size 107
got packet size 100
got packet size 101
got packet size 100
got request for 1536 bytes, sent 1280
got packet size 101
got packet size 101
got packet size 105
got packet size 99
got packet size 109
got packet size 105
got packet size 100
got packet size 107
got packet size 103
got packet size 99
got request for 256 bytes, sent 256
got packet size 100
got packet size 103
got packet size 99
got request for 4096 bytes, sent 1280
got packet size 105
got request for 7339101 bytes, sent 1280
got request for 7337821 bytes, sent 1280
got request for 7336541 bytes, sent 1280
got request for 7335261 bytes, sent 1280
got request for 7333981 bytes, sent 1280
At first everything works fine, then it breaks down and asks for big chunks. The flow itself is valid, without jumps. Gave a 128 -
byte buffer _ _ request for 6711330 bytes, sent 128 got request for 6711202 bytes, sent 128 got request for 6711074 bytes, sent 128
In general, he didn’t want to give packets (except for the first one), the size of which is on average 100 bytes, instead he wants a huge amount of data. The flow, as you might guess, is the same.
I've been stuck on a simple task for the third week now, any ideas?

Answer 1

Apparently, it's worth a trite look into the source code and see what and why such a buffer size is formed from. Open source.

Answer 2

why return all 4 megabytes from the callback? Return as many as there are, and then the callback will be called the required number of times.

Answer 3

Why don't you use avformat_open_input, etc?
After all, you can feed it any link that ffmpeg understands.
Or does everything rest on ICY?