序列化 OpenCV Mat_<Vec3f>

Question

我正在从事一个机器人研究项目，我需要序列化 ​​3D 点的 2D 矩阵：基本上每个像素都是一个 3 向量的浮点数。这些像素保存在 OpenCV 矩阵中，需要通过进程间通信发送并保存到文件中，以便在多台计算机上处​​理。我想以 endian/architecture-independent、space-efficient 的方式尽快将它们序列化。 cv::imencode here 将是完美的，除了它只适用于 8 位和 16 位元素，我们不想失去任何精度。这些文件不需要是人类可读的（尽管我们现在这样做是为了确保数据的可移植性，而且速度非常慢）。有没有这方面的最佳实践，或者优雅的方式来做到这一点？

谢谢！

Answer 1

较早的答案很好，但它们不适用于当您想要序列化感兴趣区域（除其他外）时出现的非连续矩阵。另外，没有必要序列化elemSize()，因为这是从type 值派生的。

这里有一些代码无论连续性如何都可以工作（带有包含/命名空间）

#pragma once

#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/utility.hpp>
#include <opencv2/opencv.hpp>

namespace boost {
namespace serialization {

template<class Archive>
void serialize(Archive &ar, cv::Mat& mat, const unsigned int)
{
    int cols, rows, type;
    bool continuous;

    if (Archive::is_saving::value) {
        cols = mat.cols; rows = mat.rows; type = mat.type();
        continuous = mat.isContinuous();
    }

    ar & cols & rows & type & continuous;

    if (Archive::is_loading::value)
        mat.create(rows, cols, type);

    if (continuous) {
        const unsigned int data_size = rows * cols * mat.elemSize();
        ar & boost::serialization::make_array(mat.ptr(), data_size);
    } else {
        const unsigned int row_size = cols*mat.elemSize();
        for (int i = 0; i < rows; i++) {
            ar & boost::serialization::make_array(mat.ptr(i), row_size);
        }
    }

}

} // namespace serialization
} // namespace boost

Answer 2

编辑：Christoph Heindl 对这篇文章发表了评论，并附上了指向他的博客的链接，他在该博客中改进了这个序列化代码。强烈推荐！

http://cheind.wordpress.com/2011/12/06/serialization-of-cvmat-objects-using-boost/

--

对于可能受益的人：一些使用 boost::serialization 序列化 Mat& 的代码
我还没有测试过多通道数据，但一切都应该可以正常工作。

#include <iostream>
#include <fstream>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/serialization/split_free.hpp>
#include <boost/serialization/vector.hpp>

BOOST_SERIALIZATION_SPLIT_FREE(Mat)
namespace boost {
namespace serialization {

    /*** Mat ***/
    template<class Archive>
    void save(Archive & ar, const Mat& m, const unsigned int version)
    {
      size_t elemSize = m.elemSize(), elemType = m.type();

      ar & m.cols;
      ar & m.rows;
      ar & elemSize;
      ar & elemType; // element type.
      size_t dataSize = m.cols * m.rows * m.elemSize();

      //cout << "Writing matrix data rows, cols, elemSize, type, datasize: (" << m.rows << "," << m.cols << "," << m.elemSize() << "," << m.type() << "," << dataSize << ")" << endl;

      for (size_t dc = 0; dc < dataSize; ++dc) {
          ar & m.data[dc];
      }
    }

    template<class Archive>
    void load(Archive & ar, Mat& m, const unsigned int version)
    {
        int cols, rows;
        size_t elemSize, elemType;

        ar & cols;
        ar & rows;
        ar & elemSize;
        ar & elemType;

        m.create(rows, cols, elemType);
        size_t dataSize = m.cols * m.rows * elemSize;

        //cout << "reading matrix data rows, cols, elemSize, type, datasize: (" << m.rows << "," << m.cols << "," << m.elemSize() << "," << m.type() << "," << dataSize << ")" << endl;

        for (size_t dc = 0; dc < dataSize; ++dc) {
                  ar & m.data[dc];
        }
    }

}
}

现在，mat 可以被序列化和反序列化如下：

    void saveMat(Mat& m, string filename) {
            ofstream ofs(filename.c_str());
            boost::archive::binary_oarchive oa(ofs);
            //boost::archive::text_oarchive oa(ofs);
            oa << m;
    }

    void loadMat(Mat& m, string filename) {
            std::ifstream ifs(filename.c_str());
            boost::archive::binary_iarchive ia(ifs);
            //boost::archive::text_iarchive ia(ifs);
            ia >> m;
    }

我在这里使用了 binary_oarchive 和 binary_iarchive 来降低内存使用率。二进制格式不提供平台之间的可移植性，但如果需要，可以使用 text_oarchive/iarchive。

Answer 3

您可以为此使用boost::serialization。它经过高度优化，非常容易集成。

您的情况可能的加速包括将每个对象序列化为原始二进制块（请参阅boost::serialization::make_binary）和禁用版本跟踪（BOOST_SERIALIZATION_DISABLE_TRACKING）。

此外，您可以尝试在序列化例程中添加压缩以节省空间（如果数据易于压缩，还可以节省时间）。例如，这可以使用boost::iostreams 来实现。

Answer 4

我最近问自己一个类似的问题，尽管我特别想序列化 opencv 的 Mat 和 MatND 对象。使用boost::serialize 很好，但需要一些技巧。由于您不想修改 OpenCV 本身的内部结构来序列化这些对象，因此您不得不使用所谓的“免费”功能。由于序列化 OpenCV 对象很复杂，我发现我不得不将序列化操作拆分为保存和加载，每个操作的实现都略有不同。您需要使用 boost/serialization/split_free.hpp 来完成此任务。 Boost 在这里为此提供了很好的文档：http://www.boost.org/doc/libs/1_45_0/libs/serialization/doc/index.html。

祝你好运！

Answer 5

将您的 Mat 转换为向量并使用 fwrite 怎么样？

转换为矢量的过程可能会影响性能，但它是安全的。我怀疑上面的所有答案，无论是在接受的答案中循环图像数据，还是在 Christoph 帖子中使用 make_array，都假设您的 Mat 数据是连续的，但情况不一定如此。当您的 Mat 数据不连续时，这些答案的输出将不正确。

Answer 6

我写了这段代码：

/*
Will save in the file:
cols\n
rows\n
elemSize\n
type\n
DATA
*/
void serializeMatbin(Mat& mat, std::string filename){
    if (!mat.isContinuous()) {
        cout << "Not implemented yet" << endl;
        exit(1);
    }
    int elemSizeInBytes = (int)mat.elemSize();
    int elemType        = (int)mat.type();
    int dataSize        = (int)(mat.cols * mat.rows * mat.elemSize());

    FILE* FP = fopen(filename.c_str(), "wb");
    int sizeImg[4] = {mat.cols, mat.rows, elemSizeInBytes, elemType };
    fwrite(/*buffer*/ sizeImg, /*howmanyelements*/ 4, /* size of each element */ sizeof(int), /*file*/ FP);
    fwrite(mat.data, mat.cols * mat.rows, elemSizeInBytes, FP);
    fclose(FP);
}

Mat deserializeMatbin(std::string filename){
    FILE* fp = fopen(filename.c_str(), "r");
    int header[4];
    fread(header, sizeof(int), 4, fp);
    int cols = header[0]; 
    int rows = header[1];
    int elemSizeInBytes = header[2];
    int elemType = header[3];

    Mat outputMat = Mat(rows, cols, elemType);

    fread(outputMat.data, elemSizeInBytes, cols * rows, fp);
    fclose(fp);
    return outputMat;
}

void testSerializeMatbin(){
    Mat a = Mat::ones(/*cols*/ 10, /* rows */ 5, CV_8U) * 2;
    std::string filename = "test.matbin";
    serializeMatbin(a, filename);
    Mat b = deserializeMatbin(filename);
    cout << "Rows: " << b.rows << " Cols: " << b.cols << " type: " << b.type()<< endl;
}

Answer 7

你也可以使用 msgpack 创建适配器https://github.com/msgpack/msgpack-c/wiki/v2_0_cpp_adaptor 这是示例代码。它可能有用：

namespace clmdep_msgpack {
MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS) {
    namespace adaptor {

        //I am sending as bin (int)(int)(int)(char*)
        //Mat values:         rows,cols,type,data
        template<>
        struct convert<cv::Mat> {
            clmdep_msgpack::object const &operator()(clmdep_msgpack::object const &o, cv::Mat &v) const
            {
                if(o.type != clmdep_msgpack::type::BIN) throw clmdep_msgpack::type_error();
                char *buffer = (char *) o.via.bin.ptr;
                int buffer_size = o.via.bin.size;
                int rows, cols, type;
                rows = *reinterpret_cast<int *>(buffer);
                cols = *reinterpret_cast<int *>(buffer + 1 * sizeof(int));
                type = *reinterpret_cast<int *>(buffer + 2 * sizeof(int));
                cv::Mat(rows, cols, type, (void *) (buffer + 3 * sizeof(int))).copyTo(v);

                return o;
            }
        };

        template<>
        struct pack<cv::Mat> {
            template<typename Stream>
            clmdep_msgpack::packer<Stream> &operator()(clmdep_msgpack::packer<Stream> &o, cv::Mat const &v) const
            {
                // packing member variables as bin.
                size_t mat_size;
                if(v.isContinuous()) {
                    mat_size = v.total() * v.elemSize();
                } else {
                    mat_size = v.step[v.dims - 1];
                    for(int t = 0; t < v.dims; t++) {
                        // calculate total size of multi dimensional matrix by multiplying dimensions
                        mat_size *= v.size[t];
                    }
                }
                int extra_ints = 3;
                int buffer_size = extra_ints * sizeof(int) + mat_size;
                // Allocate destination image buffer
                char *imagebuffer = new char[buffer_size];
                int type = v.type();
                std::memcpy(imagebuffer, &(v.rows), sizeof(int));
                std::memcpy(imagebuffer + 1 * sizeof(int), &(v.cols), sizeof(int));
                std::memcpy(imagebuffer + 2 * sizeof(int), &type, sizeof(int));

                if(v.isContinuous()) {
                    std::memcpy((imagebuffer + 3 * sizeof(int)), (char *) v.data, mat_size);
                } else {
                    const size_t rowsize = v.step[v.dims - 1] * v.size[v.dims - 1];
                    size_t coordinates[v.dims - 1] = {0};
                    size_t srcptr = 0, dptr = extra_ints * sizeof(int);
                    while(dptr < buffer_size) {
                        // we copy entire rows at once, so lowest iterator is always [dims-2]
                        // this is legal since OpenCV does not use 1 dimensional matrices internally (a 1D matrix is a 2d matrix with only 1 row)
                        std::memcpy(&imagebuffer[dptr], &(((char *) v.data)[srcptr]), rowsize);
                        // destination matrix has no gaps so rows follow each other directly
                        dptr += rowsize;
                        // src matrix can have gaps so we need to calculate the address of the start of the next row the hard way
                        // see *brief* text in opencv2/core/mat.hpp for address calculation
                        coordinates[v.dims - 2]++;
                        srcptr = 0;
                        for(int t = v.dims - 2; t >= 0; t--) {
                            if(coordinates[t] >= v.size[t]) {
                                if(t == 0) break;
                                coordinates[t] = 0;
                                coordinates[t - 1]++;
                            }
                            srcptr += v.step[t] * coordinates[t];
                        }
                    }
                }
                o.pack_bin(buffer_size);
                o.pack_bin_body(imagebuffer, buffer_size);
                return o;
            }
        };

        template<>
        struct object_with_zone<cv::Mat> {
            void operator()(clmdep_msgpack::object::with_zone &o, cv::Mat const &v) const
            {
                size_t mat_size;
                if(v.isContinuous()) {
                    mat_size = v.total() * v.elemSize();
                } else {
                    mat_size = v.step[v.dims - 1];
                    for(int t = 0; t < v.dims; t++) {
                        // calculate total size of multi dimensional matrix by multiplying dimensions
                        mat_size *= v.size[t];
                    }
                }
                int extra_ints = 3;
                int buffer_size = extra_ints * sizeof(int) + mat_size;
                // Allocate destination image buffer
                char *imagebuffer = new char[buffer_size];
                int type = v.type();
                std::memcpy(imagebuffer, &(v.rows), sizeof(int));
                std::memcpy(imagebuffer + 1 * sizeof(int), &(v.cols), sizeof(int));
                std::memcpy(imagebuffer + 2 * sizeof(int), &type, sizeof(int));

                if(v.isContinuous()) {
                    std::memcpy((imagebuffer + 3 * sizeof(int)), (char *) v.data, mat_size);
                } else {
                    const size_t rowsize = v.step[v.dims - 1] * v.size[v.dims - 1];
                    size_t coordinates[v.dims - 1] = {0};
                    size_t srcptr = 0, dptr = extra_ints * sizeof(int);
                    while(dptr < buffer_size) {
                        // we copy entire rows at once, so lowest iterator is always [dims-2]
                        // this is legal since OpenCV does not use 1 dimensional matrices internally (a 1D matrix is a 2d matrix with only 1 row)
                        std::memcpy(&imagebuffer[dptr], &(((char *) v.data)[srcptr]), rowsize);
                        // destination matrix has no gaps so rows follow each other directly
                        dptr += rowsize;
                        // src matrix can have gaps so we need to calculate the address of the start of the next row the hard way
                        // see *brief* text in opencv2/core/mat.hpp for address calculation
                        coordinates[v.dims - 2]++;
                        srcptr = 0;
                        for(int t = v.dims - 2; t >= 0; t--) {
                            if(coordinates[t] >= v.size[t]) {
                                if(t == 0) break;
                                coordinates[t] = 0;
                                coordinates[t - 1]++;
                            }
                            srcptr += v.step[t] * coordinates[t];
                        }
                    }
                }
                o.type = type::BIN;
                o.via.bin.size = buffer_size;
                o.via.bin.ptr = imagebuffer;
            }
        };


    } // namespace adaptor
} // MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS)
} // names