Embed python function in C++

Question

I am experimenting with Cython to generate c code from python but there seems to be some issues with name mangling. I first generate convert the code from python to c code and then I compile the code using gcc into a .so . The reason I want to use cython instead of C/python API is because I will be later using this on more complicated classes that I would like to be a library for speed etc later on (I am having a lot of trouble finding people who go from python to C++ since it is usually the other way around). Below is all the code that I have to try to execute the code (but fails). Any input will be appreciated. Thanks!

#hello.pyx
def say_hello():
    print "Hello World!"

#generate the c code
cython -a hello.pyx

#creates the shared library
gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I/usr/include/python2.6 -o libhello.so hello.c

//temp.cpp
#include <iostream>
extern "C" {
void say_hello();
};

using namespace std;

int main(){
    say_hello();
    return 1;
};

#attempt to compile (this is where it fails)
g++ -I/usr/include/python2.6/ -lpython2.6 -L./ -lhello temp.cpp -o temp

Here is the error message:

/tmp/ccpKHOMl.o: In function main: temp.cpp:(.text+0x5): undefined reference to say_hello' /tmp/ccpKHOMl.o: 
In function __static_initialization_and_destruction_0(int, int): 
  temp.cpp:(.text+0x33): undefined reference to std::ios_base::Init::Init()  
  temp.cpp:(.text+0x38): undefined reference to std::ios_base::Init::~Init() 
collect2: ld returned 1 exit status

Perhaps you could include the error messages you get in the step that fails. — Greg Hewgill
– Greg Hewgill, Commented Dec 10, 2012 at 23:04
/tmp/ccpKHOMl.o: In function main': temp.cpp:(.text+0x5): undefined reference to say_hello' /tmp/ccpKHOMl.o: In function __static_initialization_and_destruction_0(int, int)': temp.cpp:(.text+0x33): undefined reference to std::ios_base::Init::Init()' temp.cpp:(.text+0x38): undefined reference to `std::ios_base::Init::~Init()' collect2: ld returned 1 exit status — Andrew
– Andrew, Commented Dec 10, 2012 at 23:37

Mike Sandford · Accepted Answer · 2012-12-11 04:37:21Z

3

You're not going to be able to get the interoperation you want that way. If you open and inspect hello.c you won't find "static int say_hello" anywhere in there. Cython is designed for letting Python use C libraries, not letting C libraries use python.

You can look here in the documentation, but unfortunately this support is still for a python interpreter that is "in charge" and what you're looking for is the other way around.

http://docs.python.org/release/2.5.4/ext/callingPython.html

There's also the primer on "Embedding Python in Another Application"

http://docs.python.org/2/extending/embedding.html

I don't know what your requirements are, but in some cases you can successfully write data to a file, call a Python program to chew on it, then parse the results from another file. It's a little ugly and slower than keeping things in memory but it's entirely workable in many situations.

answered Dec 11, 2012 at 4:37

Mike Sandford

1,38511 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Andrew Over a year ago

this is why I believed there was some issue with name mangling. Perhaps there is a way there is a flag for the compiler to not do this. thank you for your answer though

Andrew Over a year ago

I am working with very high frequency data so writing to a file is way too slow. Everything has to be calculated in memory.

Mike Sandford Over a year ago

Got it. If that's the case you probably don't have time to do RPC via a socket, TCP, etc either. It's too bad that there's not a way to make it work but it seems like it'd probably be too slow even if it did work.

Andrew Over a year ago

everything actually will be RPC via a socket UDP socket but we are running via a 10 gig port so the data is coming in very quickly. The python receiver actually cannot keep up with the data coming in. Therefore the solution was to recode the receiver in c++ and to compile my python into a shared dynamic library. problem is the name mangling from cython. I am considering using just the python/c api right now but this isn't as efficient as having machine code itself

Community · Accepted Answer · 2017-05-23 12:27:42Z

I encountered a similar problem. It's not exactly the same problem, but it might be related.

I posted my question here: Propagating exceptions through dlsym cython. The part that is interesting for you is the 'public' keyword:

#hello.pyx
cdef public say_hello():
    print "Hello World!"

That will create a function like this

# (in the generated C file hello.c)
__PYX_EXTERN_C DL_IMPORT(...) say_hello(...);

Edit: I have added a working temp.cpp:

#include "Python.h"
#include <iostream>

#include "hello.h"

using namespace std;

int main(){
    Py_Initialize();
    inithello();
    say_hello();
    Py_Finalize();
    return 1;
};

Compiling is done with:

g++ -I/usr/include/python2.6/ -lpython2.6 -L./ -lhello temp.cpp -c -o temp.o
g++ temp.o -L. -lhello -lpython2.6 -o temp

(interestingly, it will not link in one step, complaining about undefined references.) This will successfully print 'Hello world' upon execution.

Note: The Py_Initialize() and inithello() are necessary, otherwise your code will crash. I haven't been able to get it to work without including "Python.h" and without the initialization parts (i.e. using only extern "C" { void sayhello(); } as you mention). It fails at linking. The solution can be to use dlsym and dynamically load your function, as I demonstrate in my question. But probably another solution exists, where you try to successfully export this method (in the hello.h header): __PYX_EXTERN_C DL_IMPORT(int) say_hello(void);

linello · Accepted Answer · 2013-01-11 14:57:31Z

0

If you have CMake I suggest to take a look to my project, where I use CMake to generate and link Cython based files

https://github.com/CarloNicolini/cymake

You probably have to edit some CMakeLists.txt to find the right cython installation

answered Jan 11, 2013 at 14:57

linello

8,72119 gold badges70 silver badges113 bronze badges

Collectives™ on Stack Overflow

Embed python function in C++

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related