1

I am trying to extend existing C++ objects in Python via inheritance. I can do this successfully and run virtual methods overridden in Python. When I however, try to add the python object to a list of pointers of the C++ Base object type(the Base object the python class has overridden), I get a type error: 'Attempting to append an invalid type'

I am sure this error is due to there begin no 'implicitly_convertible' functionality from derived* to base*. In C++, this would be defined as so: implicitly_convertible<[Derived_from_base],Base>();. Is it possible to define this in python?

How can I achieve this?

Here is sample code reproducing this behaviour.

C++

struct Base {
    virtual ~Base() {}
    virtual int f() = 0;
};
struct A {
    std::vector<Base*>& GetBaseList() { return m_base_List; }
    std::vector<Base*> m_base_List;
};
struct BaseWrap : Base, wrapper<Base> {
    int f() { return this->get_override("f")(); }
};

BOOST_PYTHON_MODULE(sandbox)
{
    class_<BaseWrap, Base*, boost::noncopyable>("Base", no_init)
        .def("f", pure_virtual(&Base::f));

    class_<A, A*>("A", init<>())
        .add_property("baseList", make_function(&A::GetBaseList, return_internal_reference<>()));

    //implicitly_convertible<[Derived_from_base]*,Base*>();
    class_<std::vector<Base*>>("BaseList").def(vector_indexing_suite<std::vector<Base*>>());
}

Python from sandbox import *

class derived(Base):
    def __init__(self):
        self.name = "test"
    def f(self):
        print("Hello Derived!")

d = derived()
d.f()          # Output: Hello Derived!

a = A()
a.baseList.append(d) # TypeError: Attempting to append an invalid type

Any help or ideas will be greatly appreciated.

1 Answer 1

1

The BaseList.append() function receives an argument with the right type; however, the argument has an inappropriate value. In Python, the derived initializer is not initializing the sandbox.Base part of its hierarchy. This results in the Boost.Python object not containing a C++ BaseWrap object. Hence, when BaseList.append() attempts to extract the C++ BaseWrap object, it fails and throws an error.

class derived(Base):
    def __init__(self):
        self.name = "test"
        # Base is not initialized.
    def f(self):
        print("Hello Derived!")

d = derived()
d.f() # `derived.f()` is resolved through Python's method-resolution-order.  
      # It is not invoking `BaseWrap::f()`.

a = A()
a.baseList.append(d) # d does not contain a BaseWrap object, so this throws.

To resolve the issue, explicitly invoke Base.__init__() within derived.__init__():

class derived(Base):
    def __init__(self):
        self.name = "test"
        Base.__init__(self)

However, attempting to do this will surface other problems with how BaseWrap is exposed:

  • The sandbox.Base class must be constructible from Python, so the bindings cannot provide boost::python::no_init as its initializer specification. Generally, one would only want to use boost::python::no_init when the C++ objects are being explicitly instantiated from C++ and passed to Python, such as via factory functions.
  • When T is BaseWrap, a HeldType of Base* fails to meet the requirements of HeldType. In particular, the HeldType either needs to be: BaseWrap, a class derived from BaseWrap, or a dereferenceable type for which boost::python::pointee<Base*>::type is BaseWrap or a class derived from BaseWrap. See the class_ specification for requirement details.

These can be resolved by exposing the class as follows:

namespace python = boost::python;
python::class_<BaseWrap, boost::noncopyable>("Base", python::init<>())
  .def("f", python::pure_virtual(&Base::f))
  ;

Here is a complete example demonstrating passing an object that derives from a C++ exposed class to a C++ vector exposed via the vector_indexing_suite:

#include <vector>
#include <boost/python.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>

struct base
{
  virtual ~base() {}
  virtual int perform() = 0;
};

struct base_wrap: base, boost::python::wrapper<base>
{
  int perform() { return int(this->get_override("perform")()) - 10; }
};

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<base_wrap, boost::noncopyable>("Base", python::init<>())
    .def("perform", python::pure_virtual(&base::perform))
    ;

  python::class_<std::vector<base*>>("BaseList")
    .def(python::vector_indexing_suite<std::vector<base*>>())
    ;

  python::def("do_perform", +[](base* object) {
    return object->perform();
  });
}

Interactive usage:

>>> import example
>>> class derived(example.Base):
...     def __init__(self):
...         self.name = "test"
...         example.Base.__init__(self)
...     def perform(self):
...         return 42
...       
>>> d = derived()
>>> base_list = example.BaseList()
>>> base_list.append(d)
>>> assert(len(base_list) == 1)
>>> assert(base_list[0].perform() == 42)
>>> assert(example.do_perform(base_list[0]) == 32)

With collections and pointers, there are often some caveats. In this case:

  • The BaseList object does not have shared ownership of objects to which its elements refer. Be careful to guarantee that objects referenced by the container have a lifetime at least as long as the container itself. In the above example, if object d is deleted, then invoking base_list[0].perform() can result in undefined behavior.
  • One cannot iterate over the base_list, as the iterator's value will attempt to perform a base*-to-Python conversion, which does not exists.

The above example also demonstrates the difference in function dispatching. If Python can directly invoke a method, it will do so using its own method-resolution mechanics. Note how base_list[0].perform() and example.do_perform(base_list[0]) return different values, as one gets dispatched through base_wrap::perform() which manipulates the result, and the other does not.

In the original code:

class derived(sandbox.Base):
    ...
    def f(self):
        print("Hello Derived!")

d = derived()
d.f()

As Python is aware of derived.f(), invoking d.f() will not get dispatched through BaseWrap::f(). If BaseWrap::f() had been invoked, it would have thrown because derived.f() returned None, which will fail to convert to an int:

struct BaseWrap : Base, wrapper<Base> {
    int f() { return this->get_override("f")(); }
                        // ^~~ returns a boost::python::object, faling to 
                        //     extract `int` will throw. 
};
Sign up to request clarification or add additional context in comments.

2 Comments

Thank you for your very well explained answer. I wish I could upvote the answer more. I'm struggling with the "One cannot iterate over the base_list, as the iterator's value will attempt to perform a base*-to-Python conversion, which does not exists." part now. But the rest works perfectly.
@user3035260 When iterating over a BaseList object, Python cannot safely assume that all base* are instances of base_wrap. Consider the case where a C++ type foo derives from base and a pointer to a foo instance gets pushed into the std::vector<base*> that is held within a Python object. For more details about the vector_indexing_suite, consider reading this answer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.