如何在 c++ opencv 代码中将 PyObject 变量转换为 Mat

Question

我在 opencv 中有一个 c++ 人脸识别代码和一个 python 代码。在 python 代码中，我从机器人读取帧，我想将此帧发送到我的 c++ 代码。 我使用 this link 在 c++ 函数中调用 python 函数。

我的 c++ 函数是 embed.cpp:

#include <Python.h>
#include <stdlib.h>
#include "opencv2/core/core.hpp"
#include "opencv2/contrib/contrib.hpp"
#include "opencv2/highgui/highgui.hpp"

#include <iostream>
#include <fstream>
#include <sstream>
#include </home/nao/Desktop/python/boost_1_61_0/boost/python/extract.hpp>
#include </home/nao/Desktop/python/boost_1_61_0/boost/python/converter/registry.hpp>

namespace py = boost::python;
int main()
{

// Set PYTHONPATH TO working directory
   setenv("PYTHONPATH",".",1);
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *presult;
// Initialize the Python Interpreter
Py_Initialize();
// Build the name object
pName = PyString_FromString((char*)"alvideo2");
// Load the module object
 pModule = PyImport_Import(pName);
// pDict is a borrowed reference 
pDict = PyModule_GetDict(pModule);
// pFunc is also a borrowed reference 
pFunc = PyDict_GetItemString(pDict, (char*)"showNaoImage");

if (PyCallable_Check(pFunc))
{
 pValue=Py_BuildValue("(z)",(char*)"something");
PyErr_Print();
printf("Let's give this a shot!\n");
presult=PyObject_CallObject(pFunc,pValue);
PyErr_Print();
} else{
PyErr_Print();
}

cv::imshow("Original_image",presult);

cvWaitKey(0);

Py_DECREF(pValue);
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
// Finish the Python Interpreter
 Py_Finalize();
 return 0;
}

我的python代码是alvideo2.py：

# -*- encoding: UTF-8 -*-
# Get an image from NAO. Display it and save it using PIL.

import cv2
import os #new
import sys
import time 
import numpy as np
from PIL import Image
from naoqi import ALProxy
from PIL import ImageFont
from PIL import ImageDraw 

#---------------------------------------------------------------------------
def showNaoImage(text):
  """
  First get an image from Nao, then show it on the screen with PIL.
  """
  IP = "192.168.1.18"  # Replace here with your NaoQi's IP address.
  PORT = 9559

  # Read IP address from first argument if any.
  #if len(sys.argv) > 1:
    #IP = sys.argv[1]

  camProxy = ALProxy("ALVideoDevice", IP, PORT)
  resolution = 2    # VGA
  colorSpace = 11   # RGB

  videoClient = camProxy.subscribe("python_client", resolution, colorSpace, 5)

  #t0 = time.time()

  # Get a camera image.
  # image[6] contains the image data passed as an array of ASCII chars.
  naoImage = camProxy.getImageRemote(videoClient)

  # Time the image transfer.
  #print "acquisition delay ", t1 - t0

  camProxy.unsubscribe(videoClient)


  # Now we work with the image returned and save it as a PNG  using ImageDraw
  # package.

  # Get the image size and pixel array.
  imageWidth = naoImage[0]
  imageHeight = naoImage[1]
  array = naoImage[6]

  # Create a PIL Image from our pixel array.
  im = Image.frombytes("RGB", (imageWidth, imageHeight), array)
  frame=np.array(im)
  #cv2.imshow("Faces found", frame)
  #cv2.waitKey(0) 
  return frame

所以，我的问题是如何在 cv::imshow("Original_image",presult); 中使用返回的帧？ .否则我如何将 pyobject 转换为 Mat？

非常感谢。

Answer 1

对于 Python 3.x C-API：

在 Python 端以bytearray 格式返回frame：

return bytearray(frame)

在 cpp 端通过PyByteArray_AsString 函数获取它：

pData = PyObject_CallObject(pFunc, pArgs);
uchar *data = (uchar *)PyByteArray_AsString(pData );
cv::Mat img(rows, cols, CV_8UC3, data)

Answer 2

由于showNaoImage 返回一个numpy 数组，您可以使用numpy 的C API 从帧中提取值，或获取指向保存这些值的内存的指针。见the documentation，具体是处理array data access的部分。

要将数据转换为cv::imshow 接受的Mat 数组，请使用带有数据及其维度的Mat 构造函数。例如：

// numpy array created from a PIL image is 3-dimensional:
// height x width x num_channels (num_channels being 3 for RGB)
assert (PyArray_NDIM(presult) == 3 && PyArray_SHAPE(presult)[2] == 3);

// Extract the metainformation and the data.
int rows = PyArray_SHAPE(presult)[0];
int cols = PyArray_SHAPE(presult)[1];
void *frame_data = PyArray_DATA(presult);

// Construct the Mat object and use it.
Mat cv_frame(rows, cols, CV_8UC3, frame_data);
cv::imshow("Original_image", cv_frame);

Answer 3

如果我没记错的话，opencv for python 中应该有一个特定的函数，叫做 pyopencv_to(pyobject, cv::mat, ... )处理灰色和彩色图像。