Interoperability between python's asyncio and C++ threads via pybind11

Question

I am trying to make an asyncio-enabled Python API for a C++ library. The idea is that the C++ library does work in the background in some std::thread, and I want Python to be able to continue running and await the result of the C++ processing. I'm using pybind11 for the C++/Python binding. Here is an example I came up with:

#include <pybind11/pybind11.h>
#include <thread>
#include <iostream>
#include <chrono>

namespace py = pybind11;

struct MyTask {

    py::object  m_future;
    std::thread m_thread;

    MyTask(py::object loop, int time) {
        m_future = loop.attr("create_future")();
        m_thread = std::thread([this, time, loop]() {
            std::cout << "Starting task" << std::endl;
            std::this_thread::sleep_for(std::chrono::seconds(time));
            std::cout << "Setting future task" << std::endl;
            auto set_result = m_future.attr("set_result");
            auto call_soon_threadsafe = loop.attr("call_soon_threadsafe");
            call_soon_threadsafe(set_result, 42*time);
            std::cout << "Task completed" << std::endl;
        });
        std::cout << "Task initialized" << std::endl;
    }

    py::object await() {
        std::cout << "Entering await" << std::endl;
        auto it = m_future.attr("__await__")();
        std::cout << "Leaving await" << std::endl;
        return it;
    }

    ~MyTask() {
        m_thread.join();
    }
};

PYBIND11_MODULE(_await_test, m) {
    py::class_<MyTask>(m, "MyTask")
        .def(py::init<py::object, int>())
        .def("__await__", &MyTask::await);
}

And an example Python code that uses it:

import asyncio
import _await_test

async def get_await_result(x):
    return await x

if __name__ == "__main__":
    loop = asyncio.new_event_loop()
    task = _await_test.MyTask(loop, 3);
    loop.run_until_complete(get_await_result(task))
    loop.close()

This code unfortunately does not work. It segfaults when calling call_soon_threadsafe on the C++ side (calling set_result directly also segfaults, an I read in the asyncio documentation that call_soon_threadsafe should be used because set_result is not thread-safe, but I must be misunderstanding how to use it).

The backtrace for the segfault in question:

#0  0x0000562d958fbbab in _PyObject_GenericGetAttrWithDict ()
#1  0x0000562d95901469 in PyObject_GetAttrString ()
#2  0x00007f34eda57400 in pybind11::getattr (name=<optimized out>, obj=...)
    at pybind11/pytypes.h:491
#3  pybind11::detail::accessor_policies::str_attr::get (key=<optimized out>, obj=...)
    at pybind11/pytypes.h:656
#4  pybind11::detail::accessor<pybind11::detail::accessor_policies::str_attr>::get_cache (this=0x7f34ed7f2de0)
    at pybind11/pytypes.h:634
#5  pybind11::detail::accessor<pybind11::detail::accessor_policies::str_attr>::operator pybind11::object (this=0x7f34ed7f2de0)
    at pybind11/pytypes.h:628
#6  pybind11::make_tuple<(pybind11::return_value_policy)1, pybind11::detail::accessor<pybind11::detail::accessor_policies::str_attr>&, int> ()
    at pybind11/cast.h:1004
#7  0x00007f34eda576f3 in pybind11::detail::simple_collector<(pybind11::return_value_policy)1>::simple_collector<pybind11::detail::accessor<pybind11::detail::accessor_policies::str_attr>&, int> (this=0x7f34ed7f2dd8) at /usr/include/c++/11/bits/move.h:77
#8  pybind11::detail::collect_arguments<(pybind11::return_value_policy)1, pybind11::detail::accessor<pybind11::detail::accessor_policies::str_attr>&, int, void> ()
    at pybind11/cast.h:1365
#9  pybind11::detail::object_api<pybind11::detail::accessor<pybind11::detail::accessor_policies::str_attr> >::operator()<(pybind11::return_value_policy)1, pybind11::detail::accessor<pybind11::detail::accessor_policies::str_attr>&, int> (this=<synthetic pointer>)
    at pybind11/cast.h:1390
#10 MyTask::MyTask(pybind11::object, int)::{lambda()#1}::operator()() const (__closure=0x562d95fb6ea8) at await_test/src/await_test.cpp:25

Any hint on how to make this work, or any pointer/example on how to use asyncio in conjunction with C++/Pybind11?

An alternative design to use my C++ library would be to leverage a Request class that it exposes, and which has a blocking wait and a non-blocking test function to check for completion. But again, I couldn't find any example using such a pattern.

unddoch · Accepted Answer · 2022-09-17 20:22:23Z

3

Spawning C++ threads and letting them access Python objects is violating the GIL (global interpreter lock).

You can only access Python objects if your thread currently has the lock. pybind11 implicitly assumes that you have it when you call m_future.attr("set_result");. The new thread doesn't have it and pybind11 doesn't know that. You should acquire the lock in the new thread with py::gil_scoped_acquire.

The other (and IMO better) way to do that is to not avoid C++ threads at all. If you call into pybind11 functions that are exported using py::call_guard<py::gil_scoped_release>() and call those functions with Python multi-threading code you get the same performance boost without the headache of using bare C++ threads.

answered Sep 17, 2022 at 20:22

unddoch

6,0421 gold badge28 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sunmat Over a year ago

Thanks! Acquiring and releasing the GIL around the call to set_result was indeed what was missing. As for not using C++ threads, unfortunately as I said I'm wrapping a C++ library that manages its own threads, so I have no control over this.

Collectives™ on Stack Overflow

Interoperability between python's asyncio and C++ threads via pybind11

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related