Python and OpenMP C Extensions

Question

I have a C extension in which I'd like to use OpenMP. When I import my module, though, I get an import error:


ImportError: /home/.../_entropysplit.so: undefined symbol: GOMP_parallel_end

I've compiled the module with -fopenmp and -lgomp. Is this because my Python installation wasn't compiled with the -fopenmp flag? Will I have to build Python from source? Or is there some other possibility? This is the only time I actually use openmp in my module:


unsigned int feature_index;
#pragma omp parallel for
for (feature_index = 0; feature_index < num_features; feature_index++) {

I'd like to stick with openmp if it's possible, just because it's so easy and the parallelization in this case suits it well.

EDIT: I bit the bullet and recompiled Python with OpenMP support. My module works perfectly now, but this isn't really a great solution. I can't really distribute this if it requires a complete recompile of Python. So does anybody know some way around this? Would ctypes work, maybe?

SOLVED! It was a simple linking issue. (I rebuilt Python for that?!) OpenMP wasn't being properly linked during the compilation of the module. So it IS possible to load a C Python extension that uses OpenMP.

You might consider copying your solution into a "real" answer posted under this question, so that it's easier to see (and can be upvoted). — Michael Ratanapintha
– Michael Ratanapintha, Commented Aug 30, 2010 at 2:18

luispedro · Accepted Answer · 2010-10-08 19:25:14Z

20

Just to make it clearer, here is what your setup.py should look like:

ext = Extension(
      'milk.unsupervised._som',
      sources = ['milk/unsupervised/_som.cpp'],
      extra_compile_args=['-fopenmp'],
      extra_link_args=['-lgomp'])


...
setup(..., ext_modules = [ext])

answered Oct 8, 2010 at 19:25

luispedro

7,0345 gold badges37 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Colonel Panic Over a year ago

Now, how to do that in a cross-platform way that works with versions of gcc, msvc and Clang that support openmp and fallsback gracefully otherwise?

luispedro Over a year ago

@ColonelPanic: I would also love to know that.

kirknorth · Accepted Answer · 2014-11-24 19:34:30Z

I know this is a dated post, but I'll share my experience as I too ran into this exact same issue, but when using f2py at the command line. I was originally compiling my OpenMP enabled Fortran 90 subroutine using

f2py --fcompiler=gfortran --f90flags='-fopenmp -lgomp' -m sub -c sub.90

which succesfully created the shared object sub.so. However, trying to import this from a Python shell produced the similar undefined symbol ImportError. However, as the original author stated it's because I was trying to pass both -fopenmp and -lgomp to the compiler, whereas only -fopenmp should be passed to it, and -lgomp should be passed to the linker.

Therefore, I should have been doing the following

f2py --fcompiler=gfortran --f90flags='-fopenmp' -lgomp -m sub -c sub.f90

And that's it, problem solved, I can now import my subroutine.

ajduff574 · Accepted Answer · 2010-08-30 03:00:06Z

3

It was a simple linking issue. OpenMP wasn't being properly linked during the compilation of the module. So it IS possible to load a C Python extension that uses OpenMP. -fopenmp has to be passed to the compiler and -lgomp to the linker -- if you're using distutils, make sure your setup.py is configured properly. Rebuilding Python also worked, I'm guessing, because I had properly linked OpenMP with Python, so when Python loaded the module the library was already properly linked to.

answered Aug 30, 2010 at 3:00

ajduff574

2,1111 gold badge21 silver badges20 bronze badges

Collectives™ on Stack Overflow

Python and OpenMP C Extensions

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related