Python ctypes: How to modify an existing char* array

Question

I'm working on a Python application that makes use of libupnp which is a C library. I'm using CTypes to use the library which is easy enough. The problem I'm having is when I'm registering a callback function for read requests. The function has a prototype of the following form:

int read_callback(void *pFileHandle, char *pBuf, long nBufLength);

pFileHandle is just some file handle type. pBuf is a writable memory buffer. This is where the data is output. nBufLength is the number of bytes to read from the file. A status code is returned.

I have a Python function pointer for this. That was easy enough to make but when I define a Python function to handle this callback I've found that pBuf doesn't get written to because Python strings are immutable and when you assign or modify them they create new instances. This poses a big problem because the C library expects the char pointer back when the function finishes with the requested file data. The buffer ends up being empty every time though because of the way Python strings are. Is there some way around this without modifying the C library?

The handler should modify the buffer parameter that is given which is my problem.

So what I want to have happen is that the Python function gets called to perform a read of some file (could be in memory, a file system handle, or anything in between). The pBuf parameter is populated by a read of the stream (again in Python). The callback then returns to the C code with pBuf written to.

Luke · Accepted Answer · 2013-01-14 03:56:00Z

4

The callback is invoked with pBuf and nBufLength. pBuf is already allocated with writable memory, but if you ask for pBuf.value, this is converted to an immutable python string.

Instead convert pBuf to an object that can be modified directly:

## If pBuf is c_char_p, then convert to writable object
c = ctypes.cast(pBuf, ctypes.POINTER(ctypes.c_char))
## At this point, you can set individual bytes
## with c[i] = x, but this is dangerous and there's a safer way:

## get address of string
addr = ctypes.addressof(c.contents)

## convert to char[] for safe writing
c2 = (c_char*nBufLength).from_address(addr)

## see how many bytes to write
nb = min(len(msg), nBufLength-1)

c2[:nb] = msg[:nb]
c2[nb+1] = '\0'

edited Jan 14, 2013 at 3:56

answered Jan 14, 2013 at 2:59

Luke

11.7k3 gold badges54 silver badges65 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mark Tolonen Over a year ago

Instead of cast just declare pBuf as POINTER(c_char). See my answer and example.

Stephen Diehl · Accepted Answer · 2013-01-14 02:57:53Z

3

ctypes can allocate a buffer object that your C library should be able to write to:

import ctypes
init_size = 256
pBuf = ctypes.create_string_buffer(init_size)

See: http://docs.python.org/2/library/ctypes.html#ctypes.create_string_buffer

answered Jan 14, 2013 at 2:57

Stephen Diehl

8,4795 gold badges42 silver badges57 bronze badges

1 Comment

Cadyyan Over a year ago

The memory is allocated by the library and should not be reallocated by Python. From my understanding create_string_buffer allocates new memory.

Mark Tolonen · Accepted Answer · 2022-02-02 22:50:40Z

0

Don't declare pBuf as c_char_p. ctypes converts that type to an immutable Python string and you lose access the the C pointer address. You'll want to declare it as POINTER(c_char) instead and can then use ctypes.memmove to copy data to it. Windows example:

DLL code (compiled on MSVC as cl /LD test.c)

#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

typedef int (*CALLBACK)(void *pFileHandle, char *pBuf, long nBufLength);
char g_buf[10] = "012345678";
CALLBACK g_callback;

API void set_callback(CALLBACK callback) {
    g_callback = callback;
}

API int call_callback() {
    return g_callback(0, g_buf, 10);
}

API const char* get_buf() {
    return g_buf;
}

Python 3 code:

import ctypes as ct

# Declare the callback type, argument types and return types
CALLBACK = ct.CFUNCTYPE(ct.c_int,ct.c_void_p,ct.POINTER(ct.c_char),ct.c_long)
dll = ct.CDLL('./test')
dll.set_callback.argtypes = CALLBACK,
dll.set_callback.restype = None
dll.call_callback.argtypes = ()
dll.call_callback.restype = ct.c_int
dll.get_buf.argtypes = ()
dll.get_buf.restype = ct.c_char_p

# Decorating a Python function as a callback
#  makes it usable as a ctypes parameter.
@CALLBACK
def callback(handle, buf, length):
    data = b'ABCD\0'
    if length < len(data):
        return 0
    ct.memmove(buf,data,len(data))
    return 1

dll.set_callback(callback)
print(dll.call_callback())
print(dll.get_buf())

Output. Notice that get_buf returns a c_char_p and it is a byte string. The const char* value is lost.

1
b'ABCD'

edited Feb 2, 2022 at 22:50

answered Jan 18, 2013 at 6:32

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

2 Comments

Alexander Leon VI Over a year ago

Hi @MarkTolonen, I'm working in something very similar to this, but I need to use a callback like this typedef int (CALLBACK)(char ** text_list, float * values_for_each_item_on_text_list, int number_items_in_text_list)and I can't figure out how should I work the char* element, can you help me please? cheers!

Mark Tolonen Over a year ago

@AlexanderDeLeonVI, CFUNCTYPE(c_int,POINTER(POINTER(c_char)),POINTER(c_float),c_int).

Collectives™ on Stack Overflow

Python ctypes: How to modify an existing char* array

3 Answers 3

1 Comment

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related