Creating a PyCObject pointer in Cython

Question

A few SciPy functions (like scipy.ndimage.interpolation.geometric_transform) can take pointers to C functions as arguments to avoid having to call a Python callable on each point of the input array.

In a nutshell :

Define a function called my_function somewhere in the C module
Return a PyCObject with the &my_function pointer and (optionally) a void* pointer to pass some global data around

The related API method is PyCObject_FromVoidPtrAndDesc, and you can read Extending ndimage in C to see it in action.

I am very interested in using Cython to keep my code more manageable, but I'm not sure how exactly I should create such an object. Any, well,... pointers?

+1 just for the pointer joke :-p

Will
– Will

2013-05-08 14:16:55 +00:00
Commented May 8, 2013 at 14:16 — Will
– Will, Commented May 8, 2013 at 14:16
It seemed the right thing to do! :D

F.X.
– F.X.

2013-05-08 14:50:50 +00:00
Commented May 8, 2013 at 14:50 — F.X.
– F.X., Commented May 8, 2013 at 14:50

Nikita Nemkin · Accepted Answer · 2013-05-09 11:56:26Z

1

Just do in Cython the same thing you would do in C, call PyCObject_FromVoidPtrAndDesc directly. Here is an example from your link ported to Cython:

###### example.pyx ######

from libc.stdlib cimport malloc, free
from cpython.cobject cimport PyCObject_FromVoidPtrAndDesc

cdef int _shift_function(int *output_coordinates, double* input_coordinates,
            int output_rank, int input_rank, double *shift_data):
    cdef double shift = shift_data[0]
    cdef int ii
    for ii in range(input_rank):
        input_coordinates[ii] = output_coordinates[ii] - shift
    return 1

cdef void _shift_destructor(void* cobject, void *shift_data):
    free(shift_data)

def shift_function(double shift):
    """This is the function callable from python."""
    cdef double* shift_data = <double*>malloc(sizeof(shift))
    shift_data[0] = shift
    return PyCObject_FromVoidPtrAndDesc(&_shift_function,
                                        shift_data,
                                        &_shift_destructor)

Performance should be identical to pure C version.

Note that Cyhton requires operator & to get function address. Also, Cython lacks pointer dereference operator *, indexing equivalent is used instead (*ptr -> ptr[0]).

answered May 9, 2013 at 11:56

Nikita Nemkin

2,83023 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

F.X. Over a year ago

Thanks for the correct cimports! After a while, I was thinking of importing the functions from the .h file and do roughly the same, but I wasn't sure how to handle references and dereferences, so your example clears it all in one stroke ;)

Davide Albanese · Accepted Answer · 2013-05-08 19:41:03Z

0

I think that is a bad idea. Cython was created to avoid writing PyObjects also! Moreover, in this case, writing the code through Cython probably doesn't improve code maintenance... Anyway, you can import the PyObject with

from cpython.ref cimport PyObject

in your Cython code.

UPDATE

from cpython cimport *

is safer.

Cheers, Davide

edited May 8, 2013 at 19:41

answered May 8, 2013 at 18:56

Davide Albanese

5725 silver badges10 bronze badges

1 Comment

F.X. Over a year ago

You might notice that I wrote PyCObject and not PyObject. A PyObject is a normal Python object, while a PyCObject is a special PyObject used to hold opaque pointers to raw memory addresses. It is used, like I said, by a few functions in SciPy, who just retrieve the function pointer from the PyCObject and use it without having the overhead of calling a real Python method. ;)

Collectives™ on Stack Overflow

Creating a PyCObject pointer in Cython

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related