Calling member function through a pointer from Python with pybind11

Question

I am creating a Python module (module.so) following pybind11's tutorial on trampolines:

// module.cpp
#include <string>
#include <vector>

#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include <pybind11/stl.h>

namespace py = pybind11;

class IReader
{
  public:
  virtual std::vector<int> read(const std::string &path) = 0;
};

class PyIReader : public IReader
{
  using IReader::IReader;
  std::vector<int> read(const std::string &path) override
  {
    PYBIND11_OVERRIDE_PURE(std::vector<int>, IReader, read, path);
  }
};

class C
{
  public:
  C(IReader *reader) : reader_(reader) {}
  std::vector<int> workOn(const std::string &path) { return reader_->read(path); }

  private:
  IReader *reader_;
};

PYBIND11_MODULE(module, m)
{
  py::class_<IReader, PyIReader>(m, "IReader").def(py::init<>()).def("read", &IReader::read);
  // I need call_guard to avoid deadlocking in my real code;
  // but removing it doesn't help
  py::class_<C>(m, "C").def(py::init<IReader *>()).def("work_on", &C::workOn, py::call_guard<py::gil_scoped_release>());
}

and

# main.py
import pickle
from typing import *
from module import IReader, C


class PklReader(IReader):
    def read(self, path: str) -> List[int]:
        with open(path, "rb") as f:
            return pickle.load(f)


if __name__ == "__main__":
    c = C(PklReader())
    print(c.work_on("a.pkl"))    # a = [1, 2, 3]

The Python snippet always gives a segmentation fault. I doubt that GIL is the culprit as an interpreter is embedded to run the Python code, but I am not sure what goes wrong.

(I've read passing pointer to C++ from python using pybind11 but it seems another question for me as the pointer there is double * so no class members)

Also my CMakelists.txt if it's helpful:

cmake_minimum_required(VERSION 3.16)
project(CLASS_POINTER)

set(CMAKE_CXX_STANDARD 17)
add_compile_options(-O3 -fopenmp -fPIC)
add_link_options(-fopenmp)
find_package(Python 3 REQUIRED COMPONENTS Development NumPy)

add_subdirectory(pybind11)

pybind11_add_module(module module.cpp)
set_target_properties(module PROPERTIES CXX_STANDARD_REQUIRED ON)
target_include_directories(module PRIVATE ${pybind11_INCLUDE_DIRS} ${Python_INCLUDE_DIRS} ${Python_NumPy_INCLUDE_DIRS})
target_link_directories(module PRIVATE ${Python_LIBRARY_DIRS})
target_link_libraries(module PRIVATE ${pybind11_LIBRARIES} ${Python_LIBRARIES})

unddoch · Accepted Answer · 2022-01-29 18:27:17Z

1

Receiving raw pointers usually^* means you don't assume ownership of the object. When you receive IReader* in the constructor of C, pybind11 assumes you will still hold the temporary PklReader() object and keep it alive outside. But you don't, so it gets freed and you get a segfault.

I think

if __name__ == "__main__":
    pr = PklReader()
    c = C(pkr)
    print(c.work_on("a.pkl"))    # a = [1, 2, 3]

should work, given everything else is correct.

You can also receive and store a shared_ptr in C to receive ownership of the reader.

^* C++ Core Guidelines: Never transfer ownership by a raw pointer or reference.

answered Jan 29, 2022 at 18:27

unddoch

6,0421 gold badge28 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ziyuan Over a year ago

Yes it works, thanks.

Collectives™ on Stack Overflow

Calling member function through a pointer from Python with pybind11

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related