Python objects as userdata in ctypes callback functions

Question

The C function myfunc operates on a larger chunk of data. The results are returned in chunks to a callback function:

int myfunc(const char *data, int (*callback)(char *result, void *userdata), void *userdata);

Using ctypes, it's no big deal to call myfunc from Python code, and to have the results being returned to a Python callback function. This callback work fine.

myfunc = mylib.myfunc
myfunc.restype = c_int
myfuncFUNCTYPE = CFUNCTYPE(STRING, c_void_p)
myfunc.argtypes = [POINTER(c_char), callbackFUNCTYPE, c_void_p]

def mycb(result, userdata):
    print result
    return True

input="A large chunk of data."
myfunc(input, myfuncFUNCTYPE(mycb), 0)

But, is there any way to give a Python object (say a list) as userdata to the callback function? In order to store away the result chunks, I'd like to do e.g.:

def mycb(result, userdata):
    userdata.append(result)

userdata=[]

But I have no idea how to cast the Python list to a c_void_p, so that it can be used in the call to myfunc.

My current workaround is to implement a linked list as a ctypes structure, which is quite cumbersome.

Alexander · Accepted Answer · 2016-03-10 09:57:48Z

5

I guess you could use the Python C API to do that... maybe you could use a PyObject pointer.

edit: As the op pointed out in the comments, there's already a py_object type readily available in ctypes, so the solution is to create first a ctypes.py_object object from the python list and then casting it to c_void_p to pass it as an argument to the C function (I think this step might be unnecessary as a parameter typed as void* should accept any pointer, and it would be faster to pass just a byref). In the callback, the reverse steps are done (casting from the void pointer to a pointer to py_object and then getting the value of the contents).

A workaround could be to use a closure for your callback function so it already knows in which list it has to append the items...

myfunc = mylib.myfunc
myfunc.restype = c_int
myfuncFUNCTYPE = CFUNCTYPE(STRING)
myfunc.argtypes = [POINTER(c_char), callbackFUNCTYPE]


def mycb(result, userdata):
    userdata.append(result)

input="A large chunk of data."
userdata = []
myfunc(input, myfuncFUNCTYPE(lambda x: mycb(x, userdata)))

edited Mar 10, 2016 at 9:57

Alexander

23.5k11 gold badges65 silver badges74 bronze badges

answered Mar 18, 2010 at 13:16

fortran

76.5k27 gold badges143 silver badges180 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

flight Over a year ago

Well, the nice thing about ctypes is that everything can be done in plain Python. So the Python C API seems to be a step back. Or am I missing something?

flight Over a year ago

But I found the solution I was looking for: cUserdata=ctypes.cast(pointer(ctypes.py_object(pyUserdata)), c_void_p) and in the callback: pyUserdata=ctypes.cast(cUserdata,POINTER(ctypes.py_object)).contents.value. I guess that's what had in mind with the Python C API?

fortran Over a year ago

About the first question, it probably has to do with the calling conventions, by examining what happened I'd say the parameter in Linux was expected in a register and in Windows in the stack (and as it was missing, the stack was corrupted at some point). Is your Linux 64 bit and your Windows 32 bit?

fortran Over a year ago

About the second, that's more or less what I was suggesting, using the C internal representation of the Python object... Luckily enough, ctypes already had it implemented and you didn't have to write your own wrapper and some C code to get the actual pointer.

flight Over a year ago

Both 32 bit. But in fact, the Windows library uses WINFUNCTYPE, while the Linux version uses CFUNCTYPE.

|

Collectives™ on Stack Overflow

Python objects as userdata in ctypes callback functions

1 Answer 1

10 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related