为什么精确的图像没有精确的描述符？

Question

我正在开发一个图像识别应用程序，但有些东西困扰着我。

我正在处理双色调图像（文档）。如果我复制同一个图像文件并比较两者（实际上是比较两个 [同一个子区域] 的子区域），我不会使用 SURF 获得 100% 的匹配。这是我的部分代码：

public double FindPercentMatch(Bitmap obj, Bitmap scene)
{

    // get scene key points
    VectorOfKeyPoint scenePoints = GetKeyPoints((Bitmap)scene.Clone());

    // if no scene points found, no need to go any further
    if (scenePoints == null || scenePoints.Size == 0) { return 0; }

    // get object key points
    VectorOfKeyPoint objectPoints = GetKeyPoints((Bitmap)obj.Clone());            

    // if not enough object key points found vs scene key points, then match can't be close enough (since
    // you can't have more matches than scene points anyway, so don't even try to match
    if (objectPoints == null || objectPoints.Size == 0 || (scenePoints.Size / objectPoints.Size < .15)) { return 0; }

    // we have enough key points, so compute descriptors for scene and object
    Matrix<float> objectDescriptors = GetDescriptors(objectPoints,(Bitmap)obj.Clone());
    Matrix<float> sceneDescriptors = GetDescriptors(scenePoints, (Bitmap)scene.Clone());
    int objectDescriptorCount = objectDescriptors == null ? 0 : objectDescriptors.Size.Height;
    int sceneDescriptorCount = sceneDescriptors == null ? 0 : sceneDescriptors.Size.Height;

    // find matches
    int sim = FindMatches(objectDescriptors, sceneDescriptors);

    // for testing so we know how many were found so we can monitor it
    log.Debug("descriptors1 = " + objectDescriptorCount + ", 2 = " + sceneDescriptorCount + ", matches=" + sim);

    double percent = 0;
    if (sim != 0)
    {
        percent = objectDescriptorCount != 0 ? (double)sim / (double)objectDescriptorCount : 0;
        log.Debug(percent * 100 + "%");
    }
    return percent;

}

public int FindMatches(Matrix<float> dbDescriptors, Matrix<float> queryDescriptors)
{

    double uniquenessThreshold = 0.6;

    if (dbDescriptors == null || queryDescriptors == null)
    {
        return 0;
    }
    var indices = new Matrix<int>(queryDescriptors.Rows, 2); // matrix that will contain indices of the 2-nearest neighbors found
    var dists = new Matrix<float>(queryDescriptors.Rows, 2); // matrix that will contain distances to the 2-nearest neighbors found

    // create FLANN index with 4 kd-trees and perform KNN search over it look for 2 nearest neighbours
    var flannIndex = new Index(dbDescriptors, 4);

    flannIndex.KnnSearch(queryDescriptors, indices, dists, 2, 24);

    // for eatch match over a certain threshold, add +1 to the number of 'good' matches
    int sim = 0;
    for (int i = 0; i < indices.Rows; i++)
    {
        // filter out all inadequate pairs based on distance between pairs
        if (dists.Data[i, 0] < (uniquenessThreshold * dists.Data[i, 1]))
        {
            sim++;
        }
    }
    return sim;
}

在图像的同一子区域上使用 FindPercentMatch 时的预期结果是 100%，但根据图像的不同，它可能是 70%、99%，我什至见过 101%。

Answer 1

我想我已经通过使用蛮力匹配器而不是 flann 索引解决了我的问题。此外，我没有使用 2nn 并比较它们，而是只找到了第一个最近的邻居，并将任何低于某个阈值的邻居都算作“良好匹配”。然后我将好的匹配除以 OBJECT（不是场景）的关键点总数，所以我现在最终得到 0-100% 的匹配。

        int numberOfNearestNeighbors = 1;

        double maxDistance = .30;

        BruteForceMatcher<float> matcher = new BruteForceMatcher<float>(DistanceType.L2);

        matcher.Add(dbDescriptors);

        var indices = new Matrix<int>(queryDescriptors.Rows, numberOfNearestNeighbors);
        using (Matrix<float> dist = new Matrix<float>(queryDescriptors.Rows, numberOfNearestNeighbors))
        {
            // null = no mask, use whole image
            matcher.KnnMatch(queryDescriptors, indices, dist, numberOfNearestNeighbors, null);

            int matchesUnderMaxDistance = 0;
            float distance = 0;

            // filter matches that are too different
            for (int i = 0; i < indices.Rows; i++)
            {
                distance = dist.Data[i, 0];

                if (distance < maxDistance)
                {
                    matchesUnderMaxDistance++;
                }
            }

            return matchesUnderMaxDistance;
        }

Answer 2

据我所知，您的过滤不会产生一对一的对象描述符 - 场景描述符（匹配超过某个阈值），这就是您可以获得 100+ % 的原因

另一件事是“objectDescriptorCount 等于sceneDescriptorCount”这一事实并不总是意味着“objectDescriptors 等于sceneDescriptors”。手动检查描述符，如果它们相同，则意味着您的处理逻辑有问题（计算“sim”）

希望对你有帮助