What tools to use to create dynamic video programmatically?

Question

Video

There is a task of generating a programmatic generation of a video file according to the data.
The input contains some data, such as a list of words/images tied to time. It is necessary to create a scene, place objects there according to the data, change them over time, animate, add audio and output the result to a file.
I considered drawing on the canvas api of the browser in headless mode (there are many libs for drawing) and capturing screenshots and generating video through puppeteer. But there are problems with audio, with time binding inside the web application to video frames due to possible freezes. In general, this approach with web view seems to be a crutch.
I wanted to know the tools that are best practice in this area and how, in the ideal case, this is solved.

Answer 1

Blender and python scripts

Answer 2

Alternatively, smoke towards FFMpeg. It has a bunch of wrappers for all sorts of languages.
What is a language anyway?

Answer 3

Just working with a similar project, to create a video, WPF/.Net.
The first library is Accord.Video.FFMPEG;
Example:

writer = new VideoFileWriter();
writer.Open("_temp.mkv", 1280, 720, 5, VideoCodec.MPEG4, 1900000);
// Затем закидываем кадры:
RenderTargetBitmap rtb = new RenderTargetBitmap(
            (int)canvas.ActualWidth, (int)canvas.ActualHeight, 96, 96,
                 PixelFormats.Default);
rtb.Render(canvas);
var bi = rtb.GetBitmap(1280, 720);
writer.WriteVideoFrame(bi);

Why grab frames, it doesn’t matter, I have a Canvas, you can also use a WebView.
The sound is a little more complicated, either - NAudio.Wave

var mixer = new MixingSampleProvider();
// Закидываем звук по времени.
var off = new OffsetSampleProvider(sampleSource);
off.DelayBy = TimeSpan.FromSeconds(sec);
mixer.AddMixerInput(off);
// Энкодим в файл
MediaFoundationEncoder.EncodeToMp3(mixer.ToWaveProvider16(), audioTempFileName, AUDIO_BITRATE);
// Затем эту пару скармливаем FFMpeg
string args = @"-err_detect ignore_err -i " + audioTempFileName + " -i " +
               "_temp.mkv" + " -c copy -c:v libx264 -c:a libmp3lame -shortest " + outFileName; //libvorbis libfaac libmp3lame
RunProcess(args);

A more radical solution is the Unity game engine (C #), there is a lot of goodies, although you will have to fully learn it: 3D, GPU, cameras, real-time and frame-by-frame recording, animation and special effects editors.
There is a component for recording a camera to a video file, prepare a scene according to the data, start recording. No fuss with timings and intermediate formats.