使用 jfreechart 绘制音频信号（幅度与时间）

Question

我继承了一个代码 sn-p，它绘制给定文件的音频波形。但是这个波形是一个使用 JAVA 矢量图形构建的简单图像，没有任何标签、轴信息等。我想将它移植到 jfreechart 以增加它的信息价值。我的问题是代码至少可以说是神秘的。

public class Plotter {
AudioInputStream audioInputStream;
Vector<Line2D.Double> lines = new Vector<Line2D.Double>();
String errStr;
Capture capture = new Capture();
double duration, seconds;
//File file;
String fileName = "out.png";
SamplingGraph samplingGraph;
String waveformFilename;
Color imageBackgroundColor = new Color(20,20,20);

public Plotter(URL url, String waveformFilename) throws Exception {
    if (url != null) {
        try {
            errStr = null;
            this.fileName = waveformFilename;
            audioInputStream = AudioSystem.getAudioInputStream(url);
            long milliseconds = (long)((audioInputStream.getFrameLength() * 1000) / audioInputStream.getFormat().getFrameRate());
            duration = milliseconds / 1000.0;
            samplingGraph = new SamplingGraph();
            samplingGraph.createWaveForm(null);     

        } catch (Exception ex) { 
            reportStatus(ex.toString());
            throw ex;
        }
    } else {
        reportStatus("Audio file required.");
    }
}
/**
 * Render a WaveForm.
 */
class SamplingGraph implements Runnable {

    private Thread thread;
    private Font font10 = new Font("serif", Font.PLAIN, 10);
    private Font font12 = new Font("serif", Font.PLAIN, 12);
    Color jfcBlue = new Color(000, 000, 255);
    Color pink = new Color(255, 175, 175);


    public SamplingGraph() {
    }


    public void createWaveForm(byte[] audioBytes) {

        lines.removeAllElements();  // clear the old vector

        AudioFormat format = audioInputStream.getFormat();
        if (audioBytes == null) {
            try {
                audioBytes = new byte[
                    (int) (audioInputStream.getFrameLength() 
                    * format.getFrameSize())];
                audioInputStream.read(audioBytes);
            } catch (Exception ex) { 
                reportStatus(ex.getMessage());
                return; 
            }
        }
        int w = 500;
        int h = 200;
        int[] audioData = null;
        if (format.getSampleSizeInBits() == 16) {
             int nlengthInSamples = audioBytes.length / 2;
             audioData = new int[nlengthInSamples];
             if (format.isBigEndian()) {
                for (int i = 0; i < nlengthInSamples; i++) {
                     /* First byte is MSB (high order) */
                     int MSB = (int) audioBytes[2*i];
                     /* Second byte is LSB (low order) */
                     int LSB = (int) audioBytes[2*i+1];
                     audioData[i] = MSB << 8 | (255 & LSB);
                 }
             } else {
                 for (int i = 0; i < nlengthInSamples; i++) {
                     /* First byte is LSB (low order) */
                     int LSB = (int) audioBytes[2*i];
                     /* Second byte is MSB (high order) */
                     int MSB = (int) audioBytes[2*i+1];
                     audioData[i] = MSB << 8 | (255 & LSB);
                 }
             }
         } else if (format.getSampleSizeInBits() == 8) {
             int nlengthInSamples = audioBytes.length;
             audioData = new int[nlengthInSamples];
             if (format.getEncoding().toString().startsWith("PCM_SIGN")) {
                 for (int i = 0; i < audioBytes.length; i++) {
                     audioData[i] = audioBytes[i];
                 }
             } else {
                 for (int i = 0; i < audioBytes.length; i++) {
                     audioData[i] = audioBytes[i] - 128;
                 }
             }
        }

        int frames_per_pixel = audioBytes.length / format.getFrameSize()/w;
        byte my_byte = 0;
        double y_last = 0;
        int numChannels = format.getChannels();
        for (double x = 0; x < w && audioData != null; x++) {
            int idx = (int) (frames_per_pixel * numChannels * x);
            if (format.getSampleSizeInBits() == 8) {
                 my_byte = (byte) audioData[idx];
            } else {
                 my_byte = (byte) (128 * audioData[idx] / 32768 );
            }
            double y_new = (double) (h * (128 - my_byte) / 256);
            lines.add(new Line2D.Double(x, y_last, x, y_new));
            y_last = y_new;
        }
        saveToFile();
    }


    public void saveToFile() {            
        int w = 500;
        int h = 200;
        int INFOPAD = 15;

        BufferedImage bufferedImage = new BufferedImage(w, h, BufferedImage.TYPE_INT_RGB);
        Graphics2D g2 = bufferedImage.createGraphics();

        createSampleOnGraphicsContext(w, h, INFOPAD, g2);            
        g2.dispose();
        // Write generated image to a file
        try {
            // Save as PNG
            File file = new File(fileName);
            System.out.println(file.getAbsolutePath());
            ImageIO.write(bufferedImage, "png", file);
            JOptionPane.showMessageDialog(null, 
                    new JLabel(new ImageIcon(fileName)));
        } catch (IOException e) {
        }
    }


    private void createSampleOnGraphicsContext(int w, int h, int INFOPAD, Graphics2D g2) {            
        g2.setBackground(imageBackgroundColor);
        g2.clearRect(0, 0, w, h);
        g2.setColor(Color.white);
        g2.fillRect(0, h-INFOPAD, w, INFOPAD);

        if (errStr != null) {
            g2.setColor(jfcBlue);
            g2.setFont(new Font("serif", Font.BOLD, 18));
            g2.drawString("ERROR", 5, 20);
            AttributedString as = new AttributedString(errStr);
            as.addAttribute(TextAttribute.FONT, font12, 0, errStr.length());
            AttributedCharacterIterator aci = as.getIterator();
            FontRenderContext frc = g2.getFontRenderContext();
            LineBreakMeasurer lbm = new LineBreakMeasurer(aci, frc);
            float x = 5, y = 25;
            lbm.setPosition(0);
            while (lbm.getPosition() < errStr.length()) {
                TextLayout tl = lbm.nextLayout(w-x-5);
                if (!tl.isLeftToRight()) {
                    x = w - tl.getAdvance();
                }
                tl.draw(g2, x, y += tl.getAscent());
                y += tl.getDescent() + tl.getLeading();
            }
        } else if (capture.thread != null) {
            g2.setColor(Color.black);
            g2.setFont(font12);
            //g2.drawString("Length: " + String.valueOf(seconds), 3, h-4);
        } else {
            g2.setColor(Color.black);
            g2.setFont(font12);
            //g2.drawString("File: " + fileName + "  Length: " + String.valueOf(duration) + "  Position: " + String.valueOf(seconds), 3, h-4);

            if (audioInputStream != null) {
                // .. render sampling graph ..
                g2.setColor(jfcBlue);
                for (int i = 1; i < lines.size(); i++) {
                    g2.draw((Line2D) lines.get(i));
                }

                // .. draw current position ..
                if (seconds != 0) {
                    double loc = seconds/duration*w;
                    g2.setColor(pink);
                    g2.setStroke(new BasicStroke(3));
                    g2.draw(new Line2D.Double(loc, 0, loc, h-INFOPAD-2));
                }
            }
        }
    }

    public void start() {
        thread = new Thread(this);
        thread.setName("SamplingGraph");
        thread.start();
        seconds = 0;
    }

    public void stop() {
        if (thread != null) {
            thread.interrupt();
        }
        thread = null;
    }

    public void run() {
        seconds = 0;
        while (thread != null) {
            if ( (capture.line != null) && (capture.line.isActive()) ) {
                long milliseconds = (long)(capture.line.getMicrosecondPosition() / 1000);
                seconds =  milliseconds / 1000.0;
            }
            try { thread.sleep(100); } catch (Exception e) { break; }                              
            while ((capture.line != null && !capture.line.isActive())) 
            {
                try { thread.sleep(10); } catch (Exception e) { break; }
            }
        }
        seconds = 0;
    }
} // End class SamplingGraph

/** 
 * Reads data from the input channel and writes to the output stream
 */
class Capture implements Runnable {

    TargetDataLine line;
    Thread thread;

    public void start() {
        errStr = null;
        thread = new Thread(this);
        thread.setName("Capture");
        thread.start();
    }

    public void stop() {
        thread = null;
    }

    private void shutDown(String message) {
        if ((errStr = message) != null && thread != null) {
            thread = null;
            samplingGraph.stop();                
            System.err.println(errStr);
        }
    }

    public void run() {

        duration = 0;
        audioInputStream = null;

        // define the required attributes for our line, 
        // and make sure a compatible line is supported.

        AudioFormat format = audioInputStream.getFormat();
        DataLine.Info info = new DataLine.Info(TargetDataLine.class, 
            format);

        if (!AudioSystem.isLineSupported(info)) {
            shutDown("Line matching " + info + " not supported.");
            return;
        }

        // get and open the target data line for capture.

        try {
            line = (TargetDataLine) AudioSystem.getLine(info);
            line.open(format, line.getBufferSize());
        } catch (LineUnavailableException ex) { 
            shutDown("Unable to open the line: " + ex);
            return;
        } catch (SecurityException ex) { 
            shutDown(ex.toString());
            //JavaSound.showInfoDialog();
            return;
        } catch (Exception ex) { 
            shutDown(ex.toString());
            return;
        }

        // play back the captured audio data
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        int frameSizeInBytes = format.getFrameSize();
        int bufferLengthInFrames = line.getBufferSize() / 8;
        int bufferLengthInBytes = bufferLengthInFrames * frameSizeInBytes;
        byte[] data = new byte[bufferLengthInBytes];
        int numBytesRead;

        line.start();

        while (thread != null) {
            if((numBytesRead = line.read(data, 0, bufferLengthInBytes)) == -1) {
                break;
            }
            out.write(data, 0, numBytesRead);
        }

        // we reached the end of the stream.  stop and close the line.
        line.stop();
        line.close();
        line = null;

        // stop and close the output stream
        try {
            out.flush();
            out.close();
        } catch (IOException ex) {
            ex.printStackTrace();
        }

        // load bytes into the audio input stream for playback

        byte audioBytes[] = out.toByteArray();
        ByteArrayInputStream bais = new ByteArrayInputStream(audioBytes);
        audioInputStream = new AudioInputStream(bais, format, audioBytes.length / frameSizeInBytes);

        long milliseconds = (long)((audioInputStream.getFrameLength() * 1000) / format.getFrameRate());
        duration = milliseconds / 1000.0;

        try {
            audioInputStream.reset();
        } catch (Exception ex) { 
            ex.printStackTrace(); 
            return;
        }

        samplingGraph.createWaveForm(audioBytes);
    }
} // End class Capture    

}

我已经经历了几次，知道下面的部分是计算音频值的地方，但我的问题是我不知道如何检索当时的时间信息，即该值属于什么时间间隔。

 int frames_per_pixel = audioBytes.length / format.getFrameSize()/w;
            byte my_byte = 0;
            double y_last = 0;
            int numChannels = format.getChannels();
            for (double x = 0; x < w && audioData != null; x++) {
                int idx = (int) (frames_per_pixel * numChannels * x);
                if (format.getSampleSizeInBits() == 8) {
                     my_byte = (byte) audioData[idx];
                } else {
                     my_byte = (byte) (128 * audioData[idx] / 32768 );
                }
                double y_new = (double) (h * (128 - my_byte) / 256);
                lines.add(new Line2D.Double(x, y_last, x, y_new));
                y_last = y_new;
            }

我想使用 jfreechart 的 XYSeriesPLot 绘制它，但在计算所需的 x(time) 和 y 值时遇到问题（这是幅度，但在这段代码中是 y_new）？

我知道这是一件非常容易的事情，但我对整个音频内容还是陌生的，我了解音频文件背后的理论，但这似乎是一个简单的问题，但解决方案却很艰难

enter link description here

Answer 1

要实现的关键是，在提供的代码中，预计绘图的分辨率将比实际音频数据低得多。例如，考虑以下波形：

然后，绘图代码将数据表示为图表中的蓝色框：

当框为 1 像素宽时，这对应于端点为 (x,y_last) 和 (x,y_new) 的线。如您所见，当波形足够平滑时，从y_last 到y_new 的幅度范围是框内样本的公平近似值。

现在，当尝试以逐像素方式（光栅显示）渲染波形时，这种表示可以很方便。但是，对于 XYPlot 图形（可以在 jfreechart 中找到），您只需要指定 (x,y) 点的序列，并且 XYPlot 负责在这些点之间绘制线段。这对应于下图中的绿线：

理论上，您可以将每个样本按原样提供给 XYPlot。但是，除非您的样本很少，否则这往往很难绘制。因此，通常人们会首先对数据进行下采样。如果波形足够平滑，则下采样过程会减少到抽取（即每 N 个样本取 1 个）。然后抽取因子 N 控制渲染性能和波形近似精度之间的权衡。请注意，如果在提供的代码中使用抽取因子frames_per_pixel 来生成良好的光栅显示（即，您希望看到的波形特征不会被块状像素外观隐藏，并且不会显示混叠伪影） )，对于 XYPlot 来说，相同的因子应该仍然足够（实际上您可能可以再下采样一点）。

至于将样本映射到时间/幅度轴，我不会使用 x 和 y 参数，因为它们在提供的绘图代码中定义：它们只是适用于栅格类型的像素索引显示（如上面的蓝色框表示）。

我宁愿将样本索引（提供的代码中的idx）通过除以采样率（您可以从format.getFrameRate() 获得）直接映射到时间轴。 同样，我将通过将audioData[idx] 样本除以 128（每样本 8 位数据）和 32768（每样本 16 位数据）将全尺寸样本值映射到 [-1,+1] 范围。

w 和 h 参数的主要目的仍然是配置绘图区域大小，但不再需要直接计算 XYPlot 输入（XYPlot 本身负责将时间/幅度值映射到像素坐标）。另一方面，w 参数还用于确定要绘制的点数。现在您可能希望根据波形可以维持多少抽取而不会显示太多失真来控制点数，或者您可以保持原样以最大可用绘图分辨率显示波形（有一些性能成本）。 但请注意，如果您希望显示少于 w 样本的波形，则可能必须将 frames_per_pixel 转换为浮点值。