在 Android 中与 ORB 匹配时出错

Question

我的代码运行良好，但是当它提取关键点时，这两个图像的匹配度很差。 在这里你可以找到我的代码，但是我不知道如何在JAVA Android中绘制好的匹配

 descriptors = new Mat();
        keypoints = new MatOfKeyPoint();
        detector = FeatureDetector.create(FeatureDetector.ORB);
        detector.detect(img1, keypoints);
        descriptor = DescriptorExtractor.create(DescriptorExtractor.ORB);
        descriptor.compute(img1, keypoints, descriptors);
        matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
     ColorDetection.cvt_YUVtoRGBtoHSV(mYuv,mGraySubmat);
          MatOfKeyPoint mKeyPoints = new MatOfKeyPoint();
        MatOfDMatch  matches = new MatOfDMatch();

          detector.detect(mGraySubmat, mKeyPoints);
          descriptor.compute(mGraySubmat, mKeyPoints, mIntermediateMat);

        matcher.match(mIntermediateMat,descriptors,matches);
        mIntermediateMat2.create(resultSize, CvType.CV_8UC1);
        Features2d.drawMatches(img1, keypoints, mGraySubmat, mKeyPoints, matches, 
                mIntermediateMat2,GREEN, RED,  MATCH_MASK, Features2d.NOT_DRAW_SINGLE_POINTS);

          Imgproc.resize(mIntermediateMat2, mIntermediateMat2, mRgba.size());
          Imgproc.cvtColor(mIntermediateMat2, mRgba, Imgproc.COLOR_RGBA2BGRA, 4);
     Utils.matToBitmap(mRgba, bmp);

      DMatch dm[] = matches.toArray();
          List<Point> lp1 = new ArrayList<Point>(dm.length);
          List<Point> lp2 = new ArrayList<Point>(dm.length);
          KeyPoint tkp[] = keypoints.toArray();
          KeyPoint qkp[] = mKeyPoints.toArray();
          for (int i = 0; i < dm.length; i++) {
              DMatch dma = dm[i];
              lp1.add(tkp[dma.trainIdx].pt);
              lp2.add(qkp[dma.queryIdx].pt);
          }
          MatOfPoint2f pointsPrev = new MatOfPoint2f(lp1.toArray(new Point[0]));
          MatOfPoint2f pointsAct  = new MatOfPoint2f(lp2.toArray(new Point[0]));
        Log.i("pointsPrev", pointsPrev.size().toString());
        Log.i("pointsAct", pointsAct.size().toString());
          fundamental_matrix.create(resultSize, CvType.CV_8UC1);
        fundamental_matrix = Calib3d.findFundamentalMat(
                  pointsAct, pointsPrev, Calib3d.FM_RANSAC, 3, 0.99);

有什么建议吗？

编辑：

我无法将匹配项转换为列表！因为Feature2d.drawMatches() 需要MatOfDmatch 而不是List<Dmatch>

MatOfDMatch matches, matches12, matches21;
matcher.match( descriptors1, descriptors2, matches12 );
matcher.match( descriptors2, descriptors1, matches21 );

iterate matches12
    DMatch forward = matches12[i];  
    DMatch backward = matches21[forward.trainIdx]; 
    if( backward.trainIdx == forward.queryIdx ) 
 //add forward to matches 
Features2d.drawMatches(img1, keypoints, mGraySubmat, mKeyPoints, matches,mIntermediateMat2);

Answer 1

你的代码应该是这样的：

 FeatureDetector detector = FeatureDetector.create(FeatureDetector.ORB);
 DescriptorExtractor descriptor = DescriptorExtractor.create(DescriptorExtractor.ORB);;
 DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);

 //first image
 Mat img1 = Highgui.imread("<image1 path>");
 Mat descriptors1 = new Mat();
 MatOfKeyPoint keypoints1 = new MatOfKeyPoint();

 detector.detect(img1, keypoints1);
 descriptor.compute(img1, keypoints1, descriptors1);

 //second image
 Mat img2 = Highgui.imread("<image2 path>");
 Mat descriptors2 = new Mat();
 MatOfKeyPoint keypoints2 = new MatOfKeyPoint();

 detector.detect(img2, keypoints2);
 descriptor.compute(img2, keypoints2, descriptors2);

 //matcher should include 2 different image's descriptors
 MatOfDMatch  matches = new MatOfDMatch();             
 matcher.match(descriptors1,descriptors2,matches);
 //feature and connection colors
 Scalar RED = new Scalar(255,0,0);
 Scalar GREEN = new Scalar(0,255,0);
 //output image
 Mat outputImg = new Mat();
 MatOfByte drawnMatches = new MatOfByte();
 //this will draw all matches, works fine
 Features2d.drawMatches(img1, keypoints1, img2, keypoints2, matches, 
 outputImg, GREEN, RED,  drawnMatches, Features2d.NOT_DRAW_SINGLE_POINTS);

此外，如果您只想显示功能，您可以添加以下代码：

 Mat featuredImg = new Mat();
 Scalar kpColor = new Scalar(255,159,10);//this will be color of keypoints
 //featuredImg will be the output of first image
 Features2d.drawKeypoints(img1, keypoints1, featuredImg , kpColor, 0);
 //featuredImg will be the output of first image
 Features2d.drawKeypoints(img1, keypoints1, featuredImg , kpColor, 0);

然后你可以像这样显示匹配点：

  Bitmap imageMatched = Bitmap.createBitmap(outputImg.cols(), outputImg.rows(), Bitmap.Config.RGB_565);//need to save bitmap
  Utils.matToBitmap(outputImg, imageMatched);
  ImageView.setImageBitmap(imageMatched);

最终您可以实现良好的匹配。我希望this thread 会有所帮助。

Answer 2

良好匹配方法基于从您的 MatOfDMatch matches = new MatOfDMatch(); 列表中删除具有不同描述符或空间位置的匹配点。我建议做的是循环匹配列表并将满足以下条件的匹配放入新列表中：

int DIST_LIMIT = 80;
List<DMatch> matchesList = matches.toList();
List<DMatch> matches_final= new ArrayList<DMatch>();
for(int i=0; i<matchesList.size(); i++)
   if(matchesList .get(i).distance <= DIST_LIMIT){
       matches_final.add(matches.toList().get(i));
   }
}

MatOfDMatch matches_final_mat = new MatOfDMatch();
matches_final_mat.fromList(matches_final);

你可以对匹配的点坐标做同样的事情。 Here 是有用的链接。