How to pass a string pointer-to-pointer from C to Python?

Question

I need to pass a string pointer-to-pointer from C to Python, so Python can update the pointer and C can read it later.

Steps

C set a char**
C call Python
Python allocate memory
Python update the char**
C read the string

C Code:

#include <stdio.h>

#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

typedef void (*CALLBACK)(char**);

CALLBACK g_cb;

// Expose API to register the callback
API void set_callback(CALLBACK cb) {
    g_cb = cb;
}

// Expose API to call the Python callback with a char**
API void call_python_function(char** pp) {
    if(g_cb) {
        g_cb(pp);
        printf("Python response: %s\n", *pp);
    }
}

Python Code:

import ctypes as ct

CALLBACK = ct.CFUNCTYPE(None, PPCHAR)

dll = ct.CDLL('./test')
dll.set_callback.argtypes = CALLBACK,
dll.set_callback.restype = None
dll.call_python_function.argtypes = POINTER(POINTER(ctypes.c_char)),
dll.call_python_function.restype = None
dll.set_callback(my_function)

def my_function(pp):
    buffer = ct.create_string_buffer(128)
    pp = buffer

Output:

Python response: (null)

No errors or warnings while building, C can call the Python function no issue, but Python can't update the char**. My question is How can I pass a string pointer-to-pointer from C to Python ?

Have you considered providing a function written in C that takes a Python string and performs the update, and calling that function from your Python code? — Charles Duffy
– Charles Duffy, Commented Dec 9, 2022 at 16:14
I want to avoid extra non-necessary code to avoid XY problems, but I can guarantee that the hidden code has nothing to do with the issue. — Albert Shown
– Albert Shown, Commented Dec 9, 2022 at 17:24
What's c_dll? [SO]: How to create a Minimal, Reproducible Example (reprex (mcve)). — CristiFati
– CristiFati, Commented Dec 9, 2022 at 17:51
Anyone familiar with ctypes will know the DLL calls, sorry, but I want to keep the question short to avoid the XY problem. — Albert Shown
– Albert Shown, Commented Dec 9, 2022 at 19:42

Mark Tolonen · Accepted Answer · 2022-12-09 19:21:09Z

2

Here's a working example of passing a char** from C to Python.

test.c

#include <stdio.h>

#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

typedef void (*CALLBACK)(char**);

CALLBACK g_cb;

// Expose API to register the callback
API void set_callback(CALLBACK cb) {
    g_cb = cb;
}

// Expose API to call the Python callback with a char**
API void call_python_function(char** pp) {
    if(g_cb) {
        g_cb(pp);
        printf("%s\n", *pp);
    }
}

test.py

import ctypes as ct

# Set up some types.
# Note that `c_char_p` can't be used as ctypes has special handling
# to convert it to a Python bytes object that inteferes with the
# callback working properly.
PCHAR = ct.POINTER(ct.c_char)
PPCHAR = ct.POINTER(PCHAR)
CALLBACK = ct.CFUNCTYPE(None, PPCHAR) # Note first parameter is return value

dll = ct.CDLL('./test')
# Declare function arguments and return values
dll.set_callback.argtypes = CALLBACK,
dll.set_callback.restype = None
dll.call_python_function.argtypes = PPCHAR,
dll.call_python_function.restype = None

# Set up callback function.  Note that the buffer can't go out-of-scope
# once the function returns or undefined behavior occurs, so the buffer
# is stored as an attribute of the function object so it will continue
# to exist.  A global variable would work, too.
@CALLBACK
def my_function(pp):
    my_function.buffer = ct.create_string_buffer(b'Hi From Python')
    pp[0] = my_function.buffer  # [0] dereferences char** so can assign char*

dll.set_callback(my_function)
p = PCHAR()
dll.call_python_function(ct.byref(p))
# Cast to a `c_char_p` to access `.value` and get a bytes object
# up to the terminating null.
print(ct.cast(p, ct.c_char_p).value)

Output:

Hi From Python
b'Hi From Python'

edited Dec 9, 2022 at 19:21

answered Dec 9, 2022 at 19:15

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Albert Shown Over a year ago

That's exactly what I'm looking for, the PPCHAR idea did the trick, Thank you for your answer, and thank you for your good explanations in the comments.

Albert Shown Over a year ago

Can you please edit your answer and add a comment # my_function.buffer does not need to be freed? I will appreciate it, thank you.

Mark Tolonen Over a year ago

@AlbertShown If you want to free the memory, you'd need an explicit del my_function.buffer. I was just pointing out that it will be freed automatically if it was just used as a local variable and it needs to be stored in the function object or a global variable somewhere. It's up to you if and when you want to free it in your application. It may also appear to work even as a local variable, but I proved to myself it was freed by using a debug Python build where a freed buffer is filled with a byte pattern and it is definitely freed.

Albert Shown Over a year ago

Awesome, so no need to worry about it.

Albert Shown Over a year ago

Basically after printf("%s\n", *pp); I don't need my_function.buffer anymore, so I was worried if calling the function will create a new instance & new buffer every time without freeing it (memory leak).

Collectives™ on Stack Overflow

How to pass a string pointer-to-pointer from C to Python?

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related