在处理中从相机获取两个连续的帧

Question

我一直在尝试使用 Processing 来实现这一目标的几种方法，但每次都没有得到一些完全连续的方法。有人知道这样做的“正确方法”吗？

提前致谢！

Answer 1

理论上，听captureEvent() 获取新帧并跟踪是否先前记录了第一帧，如果是之后记录第二帧，则应该是这样。

这是一个基本的注释草图来说明这一点（按任意键抓取另一对帧）：

import processing.video.*;

Capture camera;

PImage firstFrame;
PImage secondFrame;

void setup(){
  size(1920,480);

  camera = new Capture(this,640,480);
  camera.start();
}
void draw(){
  image(camera,0,0);
  if(firstFrame != null){
    image(firstFrame,640,0);
  }
  if(secondFrame != null){
    image(secondFrame,1280,0);
  }
}
//this is the callback from the video library when a new camera frame is available
void captureEvent(Capture c){
  //read a new frame
  c.read();
  //if the first frame wasn't recorded yet, record(copy) it's pixels
  if(firstFrame == null){
    firstFrame = c.get();
  }
  //same for the second frame, but check if the first frame has been recorded first
  if(firstFrame != null && secondFrame == null){
    secondFrame = c.get();
  }
}

void keyPressed(){
  //reset consecutive frames on keypress
  firstFrame = secondFrame = null;
}

理论上（正如您在Processing Video Library's source code 中看到的那样），仅当新的相机样本准备好时才会触发 captureEvent。 在实践中，您会发现两个连续的帧可能看起来相同（即使它们在时间上可能相差一秒），甚至是您在 cmets 中指出的噪声。

感觉你所追求的是一个连续的帧，但与前一个足够不同。如果是这种情况，您可以使用FrameDifferencing example（Processing > Examples > Libraries > Video > Capture > FrameDifferencing）

这是上述草图的修改版本，使用 Golan Levin 的 FrameDifferencing 代码仅在略有不同时抓取第二帧：

import processing.video.*;

Capture camera;

PImage firstFrame;
PImage secondFrame;
PImage diff;

void setup(){
  size(1920,960);

  camera = new Capture(this,640,480);
  camera.start();

  diff = createImage(640,480,RGB);
}
void draw(){
  image(camera,0,0);
  if(firstFrame != null){
    image(firstFrame,640,0);
  }
  if(secondFrame != null){
    image(secondFrame,1280,0);
  }
  image(diff,0,480);
}
//this is the callback from the video library when a new camera frame is available
void captureEvent(Capture c){
  //read a new frame
  c.read();
  //if the first frame wasn't recorded yet, record(copy) it's pixels
  if(firstFrame == null){
    firstFrame = c.get();
    println("recorded first frame at",new java.util.Date());
  }
  //same for the second frame, but check if the first frame has been recorded first
  if(firstFrame != null && secondFrame == null){
    //if the difference between the first frame cand the current frame is even ever so slightly off, record the second frame
    if(difference(firstFrame,camera) > 100){
      secondFrame = c.get();
    }
  }

}

int difference(PImage first,PImage second){
  final int numPixels = 640*480;
  camera.loadPixels();
  int movementSum = 0; // Amount of movement in the frame
  for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
    color currColor = first.pixels[i];
    color prevColor = second.pixels[i];
    // Extract the red, green, and blue components from current pixel
    int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
    int currG = (currColor >> 8) & 0xFF;
    int currB = currColor & 0xFF;
    // Extract red, green, and blue components from previous pixel
    int prevR = (prevColor >> 16) & 0xFF;
    int prevG = (prevColor >> 8) & 0xFF;
    int prevB = prevColor & 0xFF;
    // Compute the difference of the red, green, and blue values
    int diffR = abs(currR - prevR);
    int diffG = abs(currG - prevG);
    int diffB = abs(currB - prevB);
    // Render the difference image to the screen
    diff.pixels[i] = color(diffR, diffG, diffB);
    // Add these differences to the running tally
    movementSum += diffR + diffG + diffB;
  }
  diff.updatePixels();
  return movementSum;
}

void keyPressed(){
  //reset consecutive frames on keypress
  firstFrame = secondFrame = null;
}

在上面的例子中，100 是一个任意值。 最大值为 255*3*640*480（每个通道 0-255 * 通道数 * 宽度 * 高度）

Answer 2

我看了科泰相机sources和issue reports，尤其是this one。不幸的是，此代码不是为提供实时相机帧而构建的。

你可以尝试以AndroidCapture项目为starter，修改原生Androidclass来达到你的目的。