PyQt custom widget in c++ using pybind11

PyQt expects sip objects, so it is not possible to create Python object directly compatible with PyQt5 using pybind11.

But using sip.wrapinstance, it is possible to get a PyQt5 object that is actually bound with pybind11. sip.wrapinstance function is not documented in official SIP doc v6.7. It is hardly documented in the SIP v4 (no longer supported) archive.

To get a proper PyQt5 object from a pybind11 object:

Here is a C++ Visualizer class that is exposed to python using pybind11 and that returns a pointer to a C++ Qt Object:

py::class_<Visualizer>(m, "StreamVisualizer")
    .def(py::init<>(), "Show ")
    .def("get_quick_widget", [](const Visualizer & v)
         { return static_cast<void*>(v->getQuickWidget()); },
         py::return_value_policy::reference)
    ;

To get the PyQt5 QuickWidget handler, use ctypes in order to extract the pointer from the PyCapsule, then, use sip.wrapinstance on the pointer:

import sys
import mypybind11lib
import PyQt5.QtWidgets
import PyQt5.QtCore    
import PyQt5.QtQuickWidgets
# we must load sip module after PyQt5 so that we are sure to load 
# PyQt's sip module of the current PyQt
import sip 
#print(sip) 

import ctypes
def convert_capsule_to_int(pycapsule):
    ctypes.pythonapi.PyCapsule_GetPointer.restype = ctypes.c_void_p
    ctypes.pythonapi.PyCapsule_GetPointer.argtypes = [ctypes.py_object, ctypes.c_char_p]
    return ctypes.pythonapi.PyCapsule_GetPointer(pycapsule, None)

def convert_to_pyqt(pycapsule, qtype):
    return sip.wrapinstance(convert_capsule_to_int(pycapsule), qtype)


app = PyQt5.QtWidgets.QApplication(sys.argv)
win = PyQt5.QtWidgets.QMainWindow()
win.setFixedSize(400, 300)
    
visualizer = mypybind11lib.Visualizer()
pyQt5QuickWidget = convert_to_pyqt(visualizer.get_quick_widget(), 
                                   PyQt5.QtQuickWidgets.QQuickWidget)
win.setCentralWidget(pyQt5QuickWidget)
win.show()
app.exec_()