How to lower the latency of h.264 video in FFMPEG without resorting to zerolatency?

Question

Video processing

The ffmpeg wiki says :

You may be able to decrease initial "startup" latency by specifing that I-frames come "more frequently" (or basically always, in the case of x264 's zerolatency setting), though this can increase frame size and decrease quality, see here for some more background. Basically for typical x264 streams, it inserts an I-frame every 250 frames. This means that new clients that connect to the stream may have to wait up to 250 frames before they can start receiving the stream (or start with old data). So increasing I-frame frequency (makes the stream larger, but might decrease latency).

How to turn on zero latency is quite clear - "-tune zerolatency", but I still do not understand how to specify a higher (but still not maximum) frequency of I-frames, which this paragraph advises ("by specifing that I-frames come more frequently "). What does the description "Basically for typical x264 streams, it inserts an I-frame every 250 frames" refer to (what is meant by this "it" sentence)?

Answer 1

In general, the delay in zerolatency means the difference between when the frame came to the input of the encoder and when it gave it away. The delay may be caused by the decoder buffering a number of frames. As far as I remember, the default is 40.
Without buffering, it will not be possible to encode B-frames. Also, by analyzing frames from the future, the encoder more accurately distributes the bitrate between frames. The number of buffered frames is controlled by the rc_lookahead parameter.
The number of B-frames used by default is 3, so it is worth reducing their number or turning it off if necessary in a very small delay.
I also advise you to take a closer look at other parameters that are set via --tune zerolatency:

- zerolatency:
    --bframes 0 --force-cfr --no-mbtree
    --sync-lookahead 0 --sliced-threads
    --rc-lookahead 0

If you need to reduce the interval between I-frames, as far as I understand, this is done in ffmpeg with the -g option. But this does not affect the delay. Perhaps even slightly increase it, since I-frames are compressed worse.