boost python：使用return_internal_reference将参数的生命周期与返回值联系起来

Question

我开始学习使用 boost python 并且有一个菜鸟问题。

我想编写一个函数，可以将其参数的生命周期与其结果联系起来，这样当我调用r = func(a) 时，如果我仍然有r 的引用，则参数a 将永远不会被销毁.文档建议对此类请求使用return_internal_reference 呼叫策略。但这是否要求r 是a 的内部引用，正如其名称所暗示的那样？

在下面的（过度简化的）示例中，假设我想将输入数组 a 的生命周期与生成的 lambda 函数联系起来，该函数不是输入 a 的内部引用。

#include <functional>
#include <boost/python.hpp>
#include <boost/python/return_internal_reference.hpp>

using namespace std;
using namespace boost::python;

function<float(int)> func(const float* a) {
  return [=](int n) { return a[n]; };
}

BOOST_PYTHON_MODULE(test) {
  def("func", func, return_internal_reference<1>());
}

我希望能够在python中做到以下几点：

f = func(a)   # 'a' can be a temporary variable, say returned by another function
f(5)          # but 'a' should not be destroyed at this step, 
              # because its lifetime is tied to 'f'

当我尝试编译上面的代码时，我遇到了下面列出的错误墙，但如果我删除return_internal_reference<1>() 调用策略，代码编译成功。

我很确定我错误地使用了这个呼叫策略，但不知道如何使它正确。任何指针将不胜感激。非常感谢！

$ g++ -std=c++11 -shared Test.cc -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -L/opt/local/lib -lboost_python-mt -lpython2.7 -o test.so
In file included from /opt/local/include/boost/preprocessor/iteration/detail/iter/forward1.hpp:52:0,
                 from /opt/local/include/boost/python/detail/invoke.hpp:63,
                 from /opt/local/include/boost/python/detail/caller.hpp:16,
                 from /opt/local/include/boost/python/object/function_handle.hpp:8,
                 from /opt/local/include/boost/python/converter/arg_to_python.hpp:19,
                 from /opt/local/include/boost/python/call.hpp:15,
                 from /opt/local/include/boost/python/object_core.hpp:14,
                 from /opt/local/include/boost/python/args.hpp:25,
                 from /opt/local/include/boost/python.hpp:11,
                 from Test.cc:2:
/opt/local/include/boost/python/detail/invoke.hpp: In instantiation of 'PyObject* boost::python::detail::invoke(boost::python::detail::invoke_tag_<false, false>, const RC&, F&, AC0&) [with RC = boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<std::function<float(int)> >; F = std::function<float(int)> (*)(const float*); AC0 = boost::python::arg_from_python<const float*>; PyObject = _object]':
/opt/local/include/boost/python/detail/caller.hpp:223:13:   required from 'PyObject* boost::python::detail::caller_arity<1u>::impl<F, Policies, Sig>::operator()(PyObject*, PyObject*) [with F = std::function<float(int)> (*)(const float*); Policies = boost::python::return_internal_reference<>; Sig = boost::mpl::vector2<std::function<float(int)>, const float*>; PyObject = _object]'
/opt/local/include/boost/python/object/py_function.hpp:38:33:   required from 'PyObject* boost::python::objects::caller_py_function_impl<Caller>::operator()(PyObject*, PyObject*) [with Caller = boost::python::detail::caller<std::function<float(int)> (*)(const float*), boost::python::return_internal_reference<>, boost::mpl::vector2<std::function<float(int)>, const float*> >; PyObject = _object]'
Test.cc:14:1:   required from here
/opt/local/include/boost/python/detail/invoke.hpp:75:82: error: no match for call to '(const boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<std::function<float(int)> >) (std::function<float(int)>)'
     return rc(f( BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, ac, () BOOST_PP_INTERCEPT) ));
                                                                                  ^
In file included from /opt/local/include/boost/python/object/function_handle.hpp:8:0,
                 from /opt/local/include/boost/python/converter/arg_to_python.hpp:19,
                 from /opt/local/include/boost/python/call.hpp:15,
                 from /opt/local/include/boost/python/object_core.hpp:14,
                 from /opt/local/include/boost/python/args.hpp:25,
                 from /opt/local/include/boost/python.hpp:11,
                 from Test.cc:2:
/opt/local/include/boost/python/detail/caller.hpp: In instantiation of 'static const PyTypeObject* boost::python::detail::converter_target_type<ResultConverter>::get_pytype() [with ResultConverter = boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<std::function<float(int)> >; PyTypeObject = _typeobject]':
/opt/local/include/boost/python/detail/caller.hpp:240:19:   required from 'static boost::python::detail::py_func_sig_info boost::python::detail::caller_arity<1u>::impl<F, Policies, Sig>::signature() [with F = std::function<float(int)> (*)(const float*); Policies = boost::python::return_internal_reference<>; Sig = boost::mpl::vector2<std::function<float(int)>, const float*>]'
/opt/local/include/boost/python/object/py_function.hpp:48:35:   required from 'boost::python::detail::py_func_sig_info boost::python::objects::caller_py_function_impl<Caller>::signature() const [with Caller = boost::python::detail::caller<std::function<float(int)> (*)(const float*), boost::python::return_internal_reference<>, boost::mpl::vector2<std::function<float(int)>, const float*> >]'
Test.cc:14:1:   required from here
/opt/local/include/boost/python/detail/caller.hpp:102:109: error: 'struct boost::python::detail::reference_existing_object_requires_a_pointer_or_reference_return_type<std::function<float(int)> >' has no member named 'get_pytype'
         return create_result_converter((PyObject*)0, (ResultConverter *)0, (ResultConverter *)0).get_pytype();
                                                                                                             ^

Answer 1

考虑使用with_custodian_and_ward_postcallCallPolicy。此策略允许按值返回返回类型，同时仍将另一个对象的生命周期延长到至少与返回对象的生命周期一样长。

BOOST_PYTHON_MODULE(test) {
  def("func", func, with_custodian_and_ward_postcall<0, 1>());
}

---

如return_internal_reference 文档中所述，返回的对象引用了现有的内部对象：

return_internal_reference [...] 允许安全地返回指向内部保存的对象的指针和引用 [...] 而无需复制所指对象。

文档还简要提到了它对with_custodian_and_ward_postcall 的使用。总之，return_internal_reference 对暴露的函数有两个值得注意的影响：

• 返回的 Python 对象既没有被引用的 C++ 对象的显式所有权，也没有共享所有权。
• 返回的 Python 对象是一个保管人，它将延长由 owner_arg 指示的 ward 对象的生命周期，至少与保管人一样长。

由于返回的 Python 对象引用了它既没有显式所有权也没有共享所有权的现有对象，Boost.Python 执行类型检查以防止创建悬空引用。在示例代码中，func() 按值返回函子，导致编译器错误提示返回类型必须是指针或引用：

struct boost::python::detail::
reference_existing_object_requires_a_pointer_or_reference_return_type

要显式控制返回对象的生命周期，应考虑使用return_value_policy 和ResultConverterGenerators 的模型。例如，如果func() 通过指针返回一个由new() 创建的仿函数，并希望将对象的所有权传递给Python，同时仍保持保管人和被监护人的关系，那么可以使用policy composition 链接策略：

BOOST_PYTHON_MODULE(test) {
  def("func", func, 
    return_value_policy<manage_new_object,
      with_custodian_and_ward_postcall<0, 1> >());
}

---

这是一个基于原始代码的完整最小示例，详细输出到demonstrate with_custodian_and_ward_postcall 行为：

#include <boost/python.hpp>
#include <iostream>

/// @brief Mockup class with verbose construction and destruction.
class foo
{
public:
  foo() { std::cout << "foo() " << this << std::endl; }
  foo(const foo&) { std::cout << "foo(const foo&) " << this << std::endl; }
  ~foo() { std::cout << "~foo() " << this << std::endl; }
};

/// @brief Mockup class with verbose construction and destruction.
class bar
{
public:
  bar() { std::cout << "bar() " << this << std::endl; }
  bar(const bar&) { std::cout << "bar(const bar&) " << this << std::endl; }
  ~bar() { std::cout << "~bar() " << this << std::endl; }
};

/// @brief Mockup factory function.
foo make_foo(bar& /* unused */)
{
  return foo();
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  // Do not allow Foo to be explicitly created from its type.
  python::class_<foo>("Foo", python::no_init);
  python::class_<bar>("Bar", python::init<>());

  // Expose make_foo, that returns a foo object when provided a
  // bar object.  The bar object's lifetime will be extended to
  // be at least as long as that of the returned foo object.
  python::def("make_foo", &make_foo,
    python::with_custodian_and_ward_postcall<
      0, // custodian = returned Foo object
      1  // ward = provided Bar object
    >());
}

互动使用：

>>> import example
>>> bar = example.Bar()
bar() 0x125ac30
>>> foo = example.make_foo(bar)
foo() 0x7fffa9b5efff
foo(const foo&) 0x7f1fcbe40090
~foo() 0x7fffa9b5efff
>>> bar = None
>>> foo = None
~foo() 0x7f1fcbe40090
~bar() 0x125ac30

请注意，尽管 bar 变量设置为 None，但实际的 ward 对象仍然存在，直到 foo 保管人被销毁。