D
D
Dmitry2018-09-12 15:44:04
fmpeg
Dmitry, 2018-09-12 15:44:04

What is the fastest way to generate video with subtitles on node.js + ffmpeg?

Introductory: Pieces
of text fly from the user from the site
Pieces arrive at the node.js server
At the output, you need to get a video glued from all pieces of text, voiced by a machine voice, with added subtitles and an audio background. So that it can be rummaged in the social. networks (vk does not accept the .mkv format)
Options that we tried:
1. Get all the text at once, generate all the speech, generate a file with subtitles, burn subtitles into the video (vk does not support the .mkv container) and give it away. It took 12 seconds of operations for a 45-second video on a local computer.
2. For each piece of text generate audio and video files (with added subtitles). It took 1 second for one piece for one client .On the final request, only glue everything and give back the glued. The last request took 2-3 seconds, which is already tolerable.
Option 2 looks acceptable in terms of speed, but if you run 50 clients simultaneously, then the computer (tested on a MacBook PRO 2013, 2.4 GHz i7, 8gb 1600 Mhz DDR3, SSD 256gb) processed only 1 piece from 1 client in 60 seconds (60 times slower), then the computer hung tightly.
Audio generation done via Amazon Polly
Subtitle generation via https://github.com/gsantiago/subtitle.js#readme
Call ffmpeg from node.js via https://github.com/fluent-ffmpeg/node-fluent-ffmpeg
video - 1280x720
Used commands:

ffmpeg -i import/back.mov -i export_0/tmp.srt -scodec mov_text -t 6 export_0/output.mov
- burning video with subtitles and trimming to conditional 6 seconds (we send unix timestamp in the code)
ffmpeg -i audio1.mp3  .... -i audio15.mp3  merged.mp3
- merging all audio
ffmpeg -i merged.mp3  -i back.mp3 -filter_complex amerge -ac 2-c:a libmp3lame -q:a 4 -shortest audio.mp3
- overlay audio background on the text
ffmpeg -i video.txt -f concat -c copy video.mp4- gluing all videos
ffmpeg -i audio.mp3 -i video.mp4 -i test.mp4 -i export/output.mp3 -c:v copy -c:a aac -map 0:v:0 -map 1:a:0 -shortest output.mp4
- overlaying audio on video
Questions that torment:
1. Is it possible faster?
2. Can you use other codecs or gluing methods without recoding?
3. Try to call ffmpeg commands directly without a wrapper? (actually gives 50-100ms of speed)
4. Try not to save to disk, but write data to Stream and glue them together in the end?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
Sergey Sokolov, 2018-09-12
@sergiks

  1. you can: use codecs in intermediate files that allow contact
    They can then be concatenated quickly-i "concat:video1.ts|video2.ts"
  2. it's probably faster to run the shell script once with all the ffmpeg commands
  3. disk is usually fast enough

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question