Python C_API with GCC - Python.h not found

Question

I am testing the Python C_API using the program shown below with Python 3.8 on Ubuntu 18.04. I have tried compiling with Clang-10 and GCC 7.5, but in both cases the compiler is unable to locate Python.h.

When I run this string:

sudo gcc -v -shared -fPIC -l /usr/include/python3.8 -I/usr/lib -I/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -I/usr/lib/python3.8 -lpython3.8 -lcrypt -lpthread -ldl  -lutil -lm  -Wall -o Python_Math.o Python_Math.c

I get "Python_Math.c:2:10: fatal error: Python.h: No such file or directory #include <Python.h>"

I changed the string to this:

sudo gcc Python_Math.c -o Python_Math.o -v -shared -fPIC -l/usr/lib -l/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -I/usr/include/python3.8 -l/usr/lib/python3.8 -lpython3.8 -lcrypt -lpthread -ldl  -lutil -lm -Wall

and I get:

/usr/bin/ld: cannot find -l/usr/lib /usr/bin/ld: cannot find -l/usr/lib/python3.8/config-3.8-x86_64-linux-gnu /usr/bin/ld: cannot find -l/usr/lib/python3.8

So I reduced the string to this:

sudo gcc Python_Math.c -o Python_Math.o -v -shared -fPIC -I/usr/include/python3.8 -lpython3.8 -lcrypt -lpthread -ldl  -lutil -lm -Wall

Now the program compiles and links but when I run "nm" on Python_Math.o it shows all the imported symbols (including Py_Initialize) as undefined.

Finally I reduced the string to this:

sudo gcc Python_Math.c -o Python_Math.o -v -shared -fPIC -I/usr/include/python3.8 -lpython3.8 -Wall

and again it compiles and links but nm still shows symbols as "undefined."

If I change "I" in the string above to "L" for -L/usr/include/python3.8 I again get "Python.h" not found, and a message re the search paths for includes, which shows usr/include in the path.

#include <...> search starts here:
 /usr/lib/gcc/x86_64-linux-gnu/7/include
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

The problem appears to be the path to Python.h. Locate says Python.h is located in /usr/include/python3.8/Python.h.

A question at Embedding Python in C, linking fails with undefined reference to `Py_Initialize' said to add "embed" on Ubuntu 20.04, but I haven't found a way to make that work with Ubuntu 18.04.

Here is the complete program:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

int Lib_Math(char* module_name, char* fn_name, double value_in)
{
    PyObject *pName, *pModule, *pFunc;
    PyObject *pArgs, *pValue;
    int i = 0;

    Py_Initialize();
    pName = PyUnicode_DecodeFSDefault(module_name);

    pModule = PyImport_Import(pName);
    Py_DECREF(pName);

    if (pModule != NULL) {
        pFunc = PyObject_GetAttrString(pModule, fn_name);
        /* pFunc is a new reference */

        if (pFunc && PyCallable_Check(pFunc)) {
            pArgs = PyTuple_New(value_in);
            pValue = PyLong_FromDouble(value_in);

            if (!pValue) {
                Py_DECREF(pArgs);
                Py_DECREF(pModule);
                fprintf(stderr, "Cannot convert argument\n");
                return 1;
            }

            /* pValue reference stolen here: */
            PyTuple_SetItem(pArgs, i, pValue);

            pValue = PyObject_CallObject(pFunc, pArgs);
            Py_DECREF(pArgs);
            if (pValue != NULL) {
                printf("Result of call: %ld\n", PyLong_AsLong(pValue));
                Py_DECREF(pValue);
            }
else {
                Py_DECREF(pFunc);
                Py_DECREF(pModule);
                PyErr_Print();
                fprintf(stderr,"Call failed\n");
                return 1;
            }
        }
        else {
            if (PyErr_Occurred())
                PyErr_Print();
            fprintf(stderr, "Cannot find function \"%s\"\n", fn_name);
        }    
        Py_XDECREF(pFunc);
        Py_DECREF(pModule);
    }
    else {
        PyErr_Print();
        fprintf(stderr, "Failed to load \"%s\"\n", module_name);
        return 1;
    }
    if (Py_FinalizeEx() < 0) {
        return 120;
    }
    return 0;
}

Thanks for any help with getting the right link options.

The symbols will be provided by libpython.so, use ldd to see the dependency, or you could enforce the linking against the static version, see e.g. stackoverflow.com/q/48796014/5769463 — ead
– ead, Commented Oct 9, 2021 at 20:27
Locate libpython.so finds nothing, but libpython3.8.so is found, so I'll work with that. — RTC222
– RTC222, Commented Oct 9, 2021 at 20:34

Davide Madrisan · Accepted Answer · 2021-10-09 21:45:19Z

0

You messed up the I and L/l options. The right syntax is:

gcc -v -shared -fPIC \
    -I/usr/include/python3.8 \
    -L/usr/lib -L/usr/lib/python3.8 \
    -lpython3.8 -lcrypt -lpthread -ldl -lutil -lm \
    -Wall -o Python_Math.so Python_Math.c

-I for include dirs, -L for libraries dirs and -l for library files.

The compiler and linker options can be obtained by running the commands:

python3.8-config --includes
  -I/usr/include/python3.8 -I/usr/include/python3.8

python3.8-config --ldflags
  -L/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -L/usr/lib  -lcrypt -lpthread -ldl  -lutil -lm -lm

You compiles the module as shared library object *.so, so objects like Py_Initialize will be available at run time.

edited Oct 9, 2021 at 21:45

answered Oct 9, 2021 at 19:56

Davide Madrisan

2,3502 gold badges17 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

RTC222 Over a year ago

Your string compiles, but all the symbols from Python.h still show as "Undefined" when I do nm Python_Math.o in Ubuntu.

RTC222 Over a year ago

In the verbose version /usr/include/python3.8 shows as the first entry in the "include path" section, so GCC searches that folder, but I'm mystified why the symbols show as "Undefined."

RTC222 Over a year ago

I ran "nm" on libpython3.8 and the missing symbols are there. Now just to link against it.

Collectives™ on Stack Overflow

Python C_API with GCC - Python.h not found

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related