Inheritance and shared_ptr ref parameters with Boost.Python

Question

I've run into an edge case with Boost.Python that seems like it should work but doesn't.

What I have is a Base and a Derived class that I am storing in std::shared_ptr's on the python side. What I would like to do is pass a Derived type shared_ptr to a function that accepts a Base shared_ptr by reference.

I've done some research and have learned about implicitly_convertible and have attempted to employ it to fix the problem but without success (although it does help in some other situations). Passing a Derived to function that accepts a Base& works with this but if they're wrapped in shared_ptr then it fails.

What I get currently is the message below:

Boost.Python.ArgumentError: Python argument types in
    test_bed_bindings.acceptBaseSharedPtrRef(Derived) did not match C++ signature:
    acceptBaseSharedPtrRef(std::shared_ptr<(anonymous namespace)::Base> {lvalue})

See below for example code:

C++ Binding code

#define BOOST_PYTHON_STATIC_LIB
#define BOOST_PYTHON_USE_GCC_SYMBOL_VISIBILITY 1

#include <boost/optional.hpp>
#include <boost/python.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/make_shared.hpp>

#include <iostream>
#include <memory>


namespace
{

  class Base
  {
  };

  class Derived : public Base
  {
  };

  std::shared_ptr<Base> getBaseSharedPtr()
  {
    auto retVal = std::make_shared<Base>();
    std::cout << "Creating Base shared_ptr - " << retVal.get() << std::endl;
    return retVal;
  }

  std::shared_ptr<Derived> getDerivedSharedPtr()
  {
    auto retVal = std::make_shared<Derived>();
    std::cout << "Creating Derived shared_ptr - " << retVal.get() << std::endl;
    return retVal;
  }

  void acceptBaseSharedPtrRef(std::shared_ptr<Base>& base)
  {
    std::cout << "acceptBaseSharedPtrRef() with " << base.get() << std::endl;
  }

  void acceptBaseSharedPtrConstRef(const std::shared_ptr<Base>& base)
  {
    std::cout << "acceptBaseSharedPtrConstRef() with " << base.get() << std::endl;
  }

  void acceptBaseSharedPtrCopy(std::shared_ptr<Base> base)
  {
    std::cout << "acceptBaseSharedPtrCopy() with " << base.get() << std::endl;
  }

  //

  void acceptBaseRef(Base base)
  {

  }

} // namespace

namespace bindings
{
  BOOST_PYTHON_MODULE(test_bed_bindings)
  {
    PyEval_InitThreads();
    Py_Initialize();

    using namespace boost::python;

    def("getBaseSharedPtr",            &::getBaseSharedPtr);
    def("getDerivedSharedPtr",         &::getDerivedSharedPtr);
    def("acceptBaseSharedPtrRef",      &::acceptBaseSharedPtrRef);
    def("acceptBaseSharedPtrConstRef", &::acceptBaseSharedPtrConstRef);
    def("acceptBaseSharedPtrCopy",     &::acceptBaseSharedPtrCopy);

    def("acceptBaseRef",     &::acceptBaseRef);

    class_<Base, std::shared_ptr<Base> >("Base")
        .def(init<>())
        ;

    class_<Derived, bases<Base>, std::shared_ptr<Derived> >("Derived")
        .def(init<>())
        ;

    implicitly_convertible<Derived, Base>();
    implicitly_convertible<std::shared_ptr<Derived>, std::shared_ptr<Base>>();

  } // BOOST_PYTHON

} // namespace bindings

Python execution code

import test_bed_bindings

baseObj = test_bed_bindings.Base()
derivedObj = test_bed_bindings.Derived()

test_bed_bindings.acceptBaseRef( baseObj )
test_bed_bindings.acceptBaseRef( derivedObj )

baseSharedPtr = test_bed_bindings.getBaseSharedPtr()
derivedSharedPtr = test_bed_bindings.getDerivedSharedPtr()

test_bed_bindings.acceptBaseSharedPtrCopy( baseSharedPtr )
test_bed_bindings.acceptBaseSharedPtrCopy( derivedSharedPtr )

test_bed_bindings.acceptBaseSharedPtrConstRef( baseSharedPtr )
test_bed_bindings.acceptBaseSharedPtrConstRef( derivedSharedPtr )

test_bed_bindings.acceptBaseSharedPtrRef( baseSharedPtr )
test_bed_bindings.acceptBaseSharedPtrRef( derivedSharedPtr )

Sample Output

Creating Base shared_ptr - 0x276fdb8
Creating Derived shared_ptr - 0x276fde8
acceptBaseSharedPtrCopy() with 0x276fdb8
acceptBaseSharedPtrCopy() with 0x276fde8
acceptBaseSharedPtrConstRef() with 0x276fdb8
acceptBaseSharedPtrConstRef() with 0x276fde8
acceptBaseSharedPtrRef() with 0x276fdb8
Traceback (most recent call last):
  File "test_script.py", line 21, in <module>
    test_bed_bindings.acceptBaseSharedPtrRef( derivedSharedPtr )
Boost.Python.ArgumentError: Python argument types in
    test_bed_bindings.acceptBaseSharedPtrRef(Derived)
did not match C++ signature:
    acceptBaseSharedPtrRef(std::shared_ptr<(anonymous namespace)::Base> {lvalue})

Tanner Sansbury · Accepted Answer · 2016-03-22 21:41:46Z

This is intentional. To reduce the chance of a dangling reference and provide explicit directionality between the languages, Boost.Python will pass the temporary object resulting from an rvalue conversion by const reference to functions. The implicit_convertible<Source, Target> function registers an rvalue from-Python conversion. As the result of the converter is an rvalue, one can only accept it by either value or constant reference.

When a class is registered via boost::python::class_<T, HeldType, Bases> and HeldType is wrapping T:

the resulting Python class embeds an instance of HeldType
registers to-Python converter from instances of T to an instance of the Python class
registers lvalue from-Python converter for instances of the Python class to instances of T
registers to-Python converter from instances of HeldType to Python object
registers lvalue from-Python converter for instances of the Python class to instances of HeldType
for each base in Bases, registers lvalue from-Python converter for instances of the Python class to instances of T in the base (not the base's HeldType)
for each polymorphic base in Bases, registers a to-Python converter from instances of T held by a base to the Python class

With the following setup:

class base {};
class derived: public base {};

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;

  python::class_<base, std::shared_ptr<base>>("Base");
  python::class_<derived, python::bases<base>,
    std::shared_ptr<derived>>("Derived");

  python::implicitly_convertible<std::shared_ptr<derived>,
                                 std::shared_ptr<base>>();
}

The following lvalue from-Python conversions are possible because the Python object holds an instance of the C++ object:

example.Base to base, base&, const base&, std::shared_ptr<base>, std::shared_ptr<base>&, and const std::shared_ptr<base>&
example.Derived to base, base&, const base&, derived, derived&, const derived&, std::shared_ptr<derived>, std::shared_ptr<derived>&, and const std::shared_ptr<derived>&

The following to-Python conversions are possible:

base or std::shared_ptr<base> to example.Base
derived or std::shared_ptr<derived to example.Derived

If base was polymorphic, then the following to-Python conversions would be possible:

object with dynamic type of derived* and static type of base* to example.Derived
std::shared_ptr<base> holding an instance of derived to example.Derived

The following rvalue conversions is possible due to explicit registration via implicitly_convertible:

example.Derived to std::shared_ptr<base> and const std::shared_ptr<base>&

The difference between an lvalue and an rvalue conversion is whether or not the target C++ object already exists and is being held in a Python object. For example, an lvalue conversion of example.Derived to base& is possible because example.Derived holds an instance of derived which is-a base. On the other hand, an lvalue conversion from example.Derived to std::shared_ptr<base>& is not possible because example.Derived holds an instance of std::shared_ptr<derived>, which does not inherit from std::shared_ptr<base>. Hence, a std::shared_ptr<base> with an unspecified lifetime is constructed and passed as an rvalue argument to the exposed function.

Here is a complete example demonstrating these conversions:

#include <boost/python.hpp>
#include <memory> // std::shared_ptr

class base {};
class derived: public base {};

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<base, std::shared_ptr<base>>("Base");
  python::class_<derived, python::bases<base>, 
    std::shared_ptr<derived>>("Derived");

  python::implicitly_convertible<std::shared_ptr<derived>,
                                 std::shared_ptr<base>>();

  python::def("base_value", +[](base){});
  python::def("base_ref", +[](base&){});
  python::def("base_cref", +[](const base&){});

  python::def("shared_base_value", +[](std::shared_ptr<base>){});
  python::def("shared_base_ref", +[](std::shared_ptr<base>&){});
  python::def("shared_base_cref", +[](const std::shared_ptr<base>&){});

  python::def("derived_value", +[](derived){});
  python::def("derived_ref", +[](derived&){});
  python::def("derived_cref", +[](const derived&){});

  python::def("shared_derived_value", +[](std::shared_ptr<derived>){});
  python::def("shared_derived_ref", +[](std::shared_ptr<derived>&){});
  python::def("shared_derived_cref", +[](const std::shared_ptr<derived>&){});
}

Interactive usage:

>>> base = example.Base()
>>> example.base_value(base)
>>> example.base_ref(base)
>>> example.base_cref(base)
>>> example.shared_base_value(base)
>>> example.shared_base_ref(base)
>>> example.shared_base_cref(base)
>>> 
>>> derived = example.Derived()
>>> example.base_value(derived)
>>> example.base_ref(derived)
>>> example.base_cref(derived)
>>> example.shared_base_value(derived)
>>> try:
...     got_exception = False
...     example.shared_base_ref(derived)
... except TypeError:
...     got_exception = True
... finally:
...     assert(got_exception)
...
>>> example.shared_base_cref(derived)
>>> example.derived_value(derived)
>>> example.derived_ref(derived)
>>> example.derived_cref(derived)
>>> example.shared_derived_value(derived)
>>> example.shared_derived_ref(derived)
>>> example.shared_derived_cref(derived)

Thanks for the in depth explanation of the inner workings. I'd worked out most of this from research but I wasn't sure if I was missing something. I guess from here I'll be implementing a workaround of some kind.

Collectives™ on Stack Overflow

Inheritance and shared_ptr ref parameters with Boost.Python

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related