Cannot import .pyb / .dll file into Python after compiling C++ code with pybind11 on Windows

Question

My OS is Windows 10, I use G++ compiler, VS Code. I attempt to compile an example pybind11 file into a Python object. The C++ code looks as follows:

#include <pybind11/pybind11.h>

int add(int i, int j) {
    return i + j;
}

namespace py = pybind11;

PYBIND11_MODULE(cmake_module, m) {
    m.def("add", &add, R"pbdoc(
        Add two numbers

        Some other explanation about the add function.
    )pbdoc");
};

To compile the project I use the CMake script:

cmake_minimum_required(VERSION 3.30.2)
set(CMAKE_CXX_STANDARD 23)
set(CMAKE_CXX_FLAGS "-Wall -Wextra -pedantic -shared")
set(CMAKE_CXX_FLAGS_DEBUG "-Werror -Og")

# Specify the path to pybind11 if it's not installed globally
set(PYBIND11_DIR "C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\pybind11")

# Specify the path to the Python interpreter
set(PYBIND11_FINDPYTHON "C:\\Users\\User\\AppData\\Local\\Programs\\Python\\Python312\\python.exe")

# Find pybind11
find_package(pybind11 REQUIRED PATHS ${PYBIND11_DIR})

# Find Python components
find_package(Python COMPONENTS Interpreter Development REQUIRED)

# Include directories for Python headers
include_directories(${Python_INCLUDE_DIRS})

# Create the module
set(MODULE_NAME cmake_example)
pybind11_add_module(${MODULE_NAME} src/test.cpp)

After successfully building the file, I get a cmake_example.cp312-win_amd64.pyd file in a build folder. Here, I run import cmake_example, which lead to the following error: ImportError: DLL load failed while importing cmake_example: The specified module could not be found.

I use Python 3.12.4, if it's somehow relevant.

My attempts to solve the issue:

Provided a path to my Python interpreter in CMake
Tried os.add_dll_directory("path/to/build/dir")
Moved the compiled .pyd file to the "C: Users\User\AppData\Local\Programs\Python\Python312\DLLs" directory.

Ahmed AEK · Accepted Answer · 2024-09-28 17:22:08Z

0

When loading a .pyd file you also need to load the dependencies of the .pyd file otherwise the import will fail. (which for some reason fails with not found error instead of failed to load)

Add the folder containing MinGW standard libraries (namely libgcc and libstdc++.dll) to the end of your PATH environment variable, it is the same folder that contains gcc.exe and g++.exe

You can see the dependencies of your dll using dependency walker, those dependencies either need to be in the same folder or on your PATH, ( note: .pyd is a .dll just renamed)

It is possible to statically link only a few dependencies, but you cannot use the -static flag as python312.dll needs to be dynamically linked, so you need to be picky about what you statically link (only statically link libgcc and libstdc++ and libwinpthread, ie: -static-libgcc ... etc ...)

edited Sep 28, 2024 at 17:22

answered Sep 28, 2024 at 11:41

Ahmed AEK

23.2k3 gold badges19 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Dan Mašek Over a year ago

The old dependency walker is quite terrible on Windows 10 and newer. There's a decent rewrite of it here: github.com/lucasg/Dependencies

Iurii Ermakov Over a year ago

Thanks! The problem was indeed in the dependency linkage. I didn't manage to figure it out manually in VS Code, though using Visual Studio 22 solved the issue. These flags -shared -fPIC -static-libgcc -static-libstdc++ -lmingw32 -mwindows might've also helped.

Ahmed AEK Over a year ago

@IuriiErmakov that's because visual studio installer automatically adds the directory of its dlls to your PATH but for mingw you have to do it manually.

Collectives™ on Stack Overflow

Cannot import .pyb / .dll file into Python after compiling C++ code with pybind11 on Windows

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related