从视频的音频流生成波形图像

Question

我需要生成视频音频流的波形， 目前我正在使用 xuggler 和 java 做一些小事情，但似乎我无法从 IAudioSamples 获取我的视频的输入流音频的字节数组。

现在我正在寻找一种更简单的方法来做到这一点，因为 xuggler 真的变得难以理解，我在网上搜索并找到了这个： http://codeidol.com/java/swing/Audio/Build-an-Audio-Waveform-Display/

应该在 .wav 文件上工作，但是当我在视频或 .mp3 上尝试代码时，AudioInputStream 会返回“找不到音频输入流”

有人能告诉我一种方法来获取一个视频的音频流的字节 [] 数组，以便我可以按照教程创建波形吗？ 另外，如果您有可以帮助我的建议或其他图书馆，我会很高兴

Answer 1

由于 mp3 是一种编码格式，因此您需要先对其进行解码才能从中获取光线数据（字节）。

class Mp3FileXuggler {

private boolean DEBUG = true;

private String _sInputFileName;
private IContainer _inputContainer;
private int _iBitRate;
private IPacket _packet;
private int _iAudioStreamId;
private IStreamCoder _audioCoder;

private int _iSampleBufferSize;
private int _iInputSampleRate;

private static SourceDataLine mLine;

private int DECODED_AUDIO_SECOND_SIZE = 176375; /** bytes */
private int _bytesPerPacket;

private byte[] _residualBuffer;

/**
 * Constructor, prepares stream to be readed
 * @param input input File
 * @throws UnsuportedSampleRateException 
 */
public Mp3FileXuggler(String sFileName) throws UnsuportedSampleRateException{
    this._sInputFileName = sFileName;
    this._inputContainer = IContainer.make();
    this._iSampleBufferSize = 18432;
    this._residualBuffer = null;

    /** Open container **/
    if (this._inputContainer.open(this._sInputFileName, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Could not read the file: " + this._sInputFileName);

    /** How many streams does the file actually have */
    int iNumStreams = this._inputContainer.getNumStreams();
    this._iBitRate = this._inputContainer.getBitRate();

    if (DEBUG) System.out.println("Bitrate: " + this._iBitRate);

    /** Iterate the streams to find the first audio stream */
    this._iAudioStreamId = -1;
    this._audioCoder = null;
    boolean bFound = false;
    int i = 0;
    while (i < iNumStreams && bFound == false){

        /** Find the stream object */
        IStream stream = this._inputContainer.getStream(i);
        IStreamCoder coder = stream.getStreamCoder();

        /** If the stream is audio, stop looking */
        if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_AUDIO){
            this._iAudioStreamId = i;
            this._audioCoder = coder;
            this._iInputSampleRate = coder.getSampleRate();
            bFound = true;
        }
        ++i;
    }

    /** If none was found */
    if (this._iAudioStreamId == -1)
        throw new RuntimeException("Could not find audio stream in container: " + this._sInputFileName);

    /** Otherwise, open audiocoder */

    if (this._audioCoder.open(null,null) < 0)
        throw new RuntimeException("could not open audio decoder for container: " + this._sInputFileName);

    this._packet = IPacket.make();

    //openJavaSound(this._audioCoder);

    /** Dummy read one packet to avoid problems in some audio files */
    this._inputContainer.readNextPacket(this._packet);


    /** Supported sample rates */
    switch(this._iInputSampleRate){
        case 22050:
                this._bytesPerPacket = 2304;
            break;

        case 44100:
                this._bytesPerPacket = 4608;
            break;


    }

}

public byte[] getSamples(){
    byte[] rawBytes = null;


        /** Go to the correct packet */
        while (this._inputContainer.readNextPacket(this._packet) >= 0){
            //System.out.println(this._packet.getDuration());
            /** Once we have a packet, let's see if it belongs to the audio stream */

            if (this._packet.getStreamIndex() == this._iAudioStreamId){
                IAudioSamples samples = IAudioSamples.make(this._iSampleBufferSize, this._audioCoder.getChannels());


                //  System.out.println(">> " + samples.toString());
                /** Because a packet can contain multiple set of samples (frames of samples). We may need to call
                 * decode audio multiple times at different offsets in the packet's data */

                int iCurrentOffset = 0;
                while(iCurrentOffset < this._packet.getSize()){

                    int iBytesDecoded = this._audioCoder.decodeAudio(samples, this._packet, iCurrentOffset);
                    iCurrentOffset += iBytesDecoded;

                    if (samples.isComplete()){
                        rawBytes = samples.getData().getByteArray(0, samples.getSize());

                        //playJavaSound(samples);
                    }
                }
                return rawBytes;
            }
            else{
                /** Otherwise drop it */
                do{}while(false);
            }
        }

    return rawBytes; /** This will return null at this point */
}

}

使用此类从 mp3 文件中获取原始数据，并将它们提供给您的频谱抽屉。