R
R
ruboss2015-10-02 15:49:22
Java
ruboss, 2015-10-02 15:49:22

How to split a part of a video into frames without making unnecessary gestures with tambourines (Java, streams)?

Hi everybody!
There is a task in which you need to split the video into frames with a certain frequency.
Moreover, it is not always necessary to split the entire video.
For example: I need to get frames from 00:11:10 to 00:12:20 from the video "Movie1" at a rate of 2 frames per second.
70 s = 140 frames. Moreover, "Movie1" can be either a file on the server with the program, or a remote file that will have to be uploaded via the Internet.
Those. If I need 70 seconds of video, then it will not be optimal to download the entire two-hour recording for this.
Problem essence:
1) How it is possible to make seek for a remote file? What if I know its height and width but don't know fps ?
2) What are the working libraries for working with video files in java? Support for different formats, separation of sound from video, etc.?
I tried to make it on FFmpeg from javacv, I can't figure out how to skip part of the frames. It seems that apart from grab() and grabFrame() there is no way to skip it. Therefore, processing a video of 30 seconds takes 28 seconds, which is not suitable at all.

public void loadVideo() {
    av_log_set_level(AV_LOG_QUIET);										//off logs
    FrameGrabber videoGrabber = new FFmpegFrameGrabber(fileName);		//init
      CanvasFrame canvas = new CanvasFrame("test1");						//canvas
    try {
      //videoGrabber.setFrameRate(48.0);								//not working
      videoGrabber.setFormat(format);									// mp4 for example
      videoGrabber.start();											//start
    } catch (com.googlecode.javacv.FrameGrabber.Exception e) {
      e.printStackTrace();
    }

    Frame vFrame = null;
    double fps = videoGrabber.getFrameRate();							//get fps
    
    System.out.println(fps);
    
    int cnt = 0;
    int each = (int) (Math.round(fps / framePerSec));					//count which frame w need to get
    if (each > 1)		
      each--;															// for example each 12`th if we have fps = 25
                                      // that mean we have 2 image per second
    do {
      try {
        vFrame = videoGrabber.grabFrame();							//grab
        if (vFrame != null) {
          IplImage img = vFrame.image;
          if (img != null) {
             canvas.showImage(img);								//show image
            cnt++;
            if (cnt % each == 0) {
              int seconds = (int) (cnt / fps);
              list.add(cvCloneImage(img));					//add to some list of images for processing
              timeList.add(seconds);
              cvSaveImage("frames/f_"+seconds+".png",img);	//save image
            }
          }
        }
      } catch (com.googlecode.javacv.FrameGrabber.Exception e) {
        e.printStackTrace();
      }
    } while (vFrame != null);
  }

I also tried to do it through Xuggler - seekKeyFrame does not work:
public void videoSeparator() throws IOException{
    String filename = fileName;										
    
    IContainer container = IContainer.make();								//создаем контейнер

    if (container.open(filename, IContainer.Type.READ, null) < 0)			//пытаемся открыть
      throw new IllegalArgumentException("could not open file: "
          + filename);
    
    int numStreams = container.getNumStreams();								//количество потоков
    int videoStreamId = -1;
    IStreamCoder videoCoder = null;
    double fps = 24.0;
                                        // нужно найти видео поток
    for (int i = 0; i < numStreams; i++) {
      IStream stream = container.getStream(i);
      fps = stream.getFrameRate().getDouble();							//fps
      IStreamCoder coder = stream.getStreamCoder();
      if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO) {
        videoStreamId = i;
        videoCoder = coder;
        break;
      }
    }
    if (videoStreamId == -1)
                                        //если не нашли
      throw new RuntimeException(
          "could not find video stream in container: " + filename);
                                        //пытаемся открыть кодек
    if (videoCoder.open() < 0)
      throw new RuntimeException(
          "could not open video decoder for container: " + filename);

    IPacket packet = IPacket.make();
    
    long start = 0;
    long end = container.getDuration();										//длина видео в микросекундах
    
    System.out.println("s = "+start+" e = "+end);
                                        // микросекунды - пол секунды
    long step = 1000*500;		
    
    int cnt = 0;
    CanvasFrame canvas = new CanvasFrame("test1");							//холст для отображения
    END: while (container.readNextPacket(packet) >= 0) {					//читаем пакеты с контейнера
      if (packet.getStreamIndex() == videoStreamId) {						//если это видео
        
        cnt++;
        
        IVideoPicture picture = IVideoPicture.make(								
            videoCoder.getPixelType(), videoCoder.getWidth(),
            videoCoder.getHeight());
        int offset = 0;
        while (offset < packet.getSize()) {								//считываем в изображение	
          int bytesDecoded = videoCoder.decodeVideo(picture, packet,
              offset);
          // Если что-то пошло не так
          if (bytesDecoded < 0)
            throw new RuntimeException(
                "got error decoding video in: " + filename);
          offset += bytesDecoded;
          if (picture.isComplete()) {									// когда все считали
            IVideoPicture newPic = picture;
            // в микросекундах
            long timestamp = picture.getTimeStamp();
            BufferedImage javaImage = Utils							
                  .videoPictureToImage(newPic);
            canvas.showImage(javaImage);							//выводим изображение
          
            if (timestamp > end) {									//выходим если установленое время вышло
              break END;
            }
          }
        }
        //container.seekKeyFrame(videoStreamId, step, IContainer.SEEK_FLAG_BYTE);	//пытаемся перскочить пол секунды - не работает
      }
    }
    System.out.println("total count = "+cnt);
    if (videoCoder != null) {
      videoCoder.close();
      videoCoder = null;
    }
    if (container != null) {
      container.close();
      container = null;
    }
    
    System.out.println("Count = " + count);
  }

Answer the question

In order to leave comments, you need to log in

1 answer(s)
R
ruboss, 2015-10-04
@ruboss

videoGrabber.setTimestamp(curts);

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question