javacv 合并图像和 mp3 创建的 mp4 比原始 mp3 长

Question

我正在使用下面的代码来合并一个 mp3 和一个图像

IplImage ipl = cvLoadImage(path2ImageFile);
int height = ipl.height();
int width = ipl.width();
if(height%2!=0) height = height+1;
if(width%2!=0) width = width+1;

OpenCVFrameConverter.ToIplImage grabberConverter = new OpenCVFrameConverter.ToIplImage();  
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(path2OutputFile,width,height); 
FrameGrabber audioFileGrabber = new FFmpegFrameGrabber(path2AudioFile);
try 
{  
    audioFileGrabber.start();

    recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264 );//AV_CODEC_ID_VORBIS
    recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);//AV_CODEC_ID_MP3 //AV_CODEC_ID_AAC

    recorder.setFormat("mp4");  
    recorder.setAudioChannels(2);
    recorder.start();  

    Frame frame = null;
    //audioFileGrabber.getFrameNumber()
    while ((frame = audioFileGrabber.grabFrame())!=null) 
    {   recorder.record(grabberConverter.convert(ipl));  
        recorder.record(frame);
    }

    recorder.stop();  
    audioFileGrabber.stop();
 }  

这会产生一个 mp4，但要长约 1 分半钟。
为什么会这样？
有什么办法可以解决这个问题吗？

Answer 1

这是一个很好的解决问题。 IMO，问题似乎有两个原因

• 这种情况下video的帧率需要和Audio的帧率一样，所以没有时间增加
• 写入时的音频帧应写入相同的时间戳

以下是适合我的最终代码

import org.bytedeco.ffmpeg.global.*;
import org.bytedeco.javacv.*;


public class AudioExample {
    public static void main(String[] args) {
        String path2ImageFile = "WhatsApp Image 2018-12-29 at 21.54.58.jpeg";
        String path2OutputFile = "test.mp4";
        String path2AudioFile = "GMV_M_Brice_English_Greeting-Menus.mp3";

        FFmpegFrameGrabber imageFileGrabber = new FFmpegFrameGrabber(path2ImageFile);
        FFmpegFrameGrabber audioFileGrabber = new FFmpegFrameGrabber(path2AudioFile);
        try {
            audioFileGrabber.start();
            imageFileGrabber.start();

            FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(path2OutputFile,
                    imageFileGrabber.getImageWidth(), imageFileGrabber.getImageHeight());

            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);//AV_CODEC_ID_VORBIS
            recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);//AV_CODEC_ID_MP3 //AV_CODEC_ID_AAC
            recorder.setSampleRate(audioFileGrabber.getSampleRate());
            recorder.setFrameRate(audioFileGrabber.getAudioFrameRate());
            recorder.setVideoQuality(1);
            recorder.setFormat("mp4");
            recorder.setAudioChannels(2);
            recorder.start();

            Frame frame = null;

            Frame imageFrame = imageFileGrabber.grab();
            while ((frame = audioFileGrabber.grab()) != null) {
                recorder.setTimestamp(frame.timestamp);
                recorder.record(imageFrame);
                recorder.record(frame);
            }
            recorder.stop();
            audioFileGrabber.stop();
            imageFileGrabber.stop();
        } catch (FrameRecorder.Exception | FrameGrabber.Exception e) {
            e.printStackTrace();
        }
    }
}

我删除了图像的 opencv，因为我们也可以使用另一个帧抓取器来获取图像作为帧。您还可以看到生成的音频与 mp3 大小相同

尝试 #2

由于第一次尝试对视频和音频使用相同的帧率，这导致了一些问题，我想修复帧率，然后在顶部交错音频

import org.bytedeco.ffmpeg.global.*;
import org.bytedeco.javacv.*;


public class AudioExample {
    public static void main(String[] args) {
        String path2ImageFile = "/Users/tarunlalwani/Downloads/WhatsApp Image 2018-12-29 at 21.54.58.jpeg";
        String path2OutputFile = "/Users/tarunlalwani/Downloads/test.mp4";
        String path2AudioFile = "/Users/tarunlalwani/Downloads/GMV_M_Brice_English_Greeting-Menus.mp3";

        FFmpegFrameGrabber imageFileGrabber = new FFmpegFrameGrabber(path2ImageFile);
        FFmpegFrameGrabber audioFileGrabber = new FFmpegFrameGrabber(path2AudioFile);
        try {
            audioFileGrabber.start();
            imageFileGrabber.start();

            FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(path2OutputFile,
                    imageFileGrabber.getImageWidth(), imageFileGrabber.getImageHeight());

            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);//AV_CODEC_ID_VORBIS
            recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);//AV_CODEC_ID_MP3 //AV_CODEC_ID_AAC
            recorder.setSampleRate(audioFileGrabber.getSampleRate());
            int frameRate = 30;
            recorder.setFrameRate(frameRate);
            recorder.setFormat("mp4");
            recorder.setAudioChannels(2);
            recorder.start();

            Frame frame = null;

            Frame imageFrame = imageFileGrabber.grab();
            int frames = (int)((audioFileGrabber.getLengthInTime() / 1000000f) * frameRate) + 1;
            int i = 0;

            //create video using fixed images and framerate
            while (i ++ <frames) {
                recorder.record(imageFrame);
            }

            // Add audio to given timestamps
            while ((frame = audioFileGrabber.grabFrame()) != null) {
                recorder.setTimestamp(frame.timestamp);
                recorder.record(frame);
            }

            recorder.stop();
            audioFileGrabber.stop();
            imageFileGrabber.stop();
        } catch (FrameRecorder.Exception | FrameGrabber.Exception e) {
            e.printStackTrace();
        }
    }
}