How to write a python string to a C struct's (char *) member in python?

Question

I have a C struct passed from C to python, and I want to change the content of a (char *)member in python.

little.c

#include <Python.h>
typedef struct little littleStruct;
struct little{

        char *memberStr;
        PyObject *(*callFunc)(littleStruct *s, char *funcName, PyObject *paraList);
};

littleStruct *createLittle();
void init_little(littleStruct *s);
PyObject *little_callFunc(littleStruct *s, char *funcName, PyObject *paraList);

littleStruct *createLittle(){
    littleStruct *little = malloc(sizeof(littleStruct));
    init_little(little);
    return little;
}
void init_little(littleStruct *s){
    s->memberStr = malloc(sizeof(char)*128);
    memset(s->memberStr, 0, 128);

    s->callFunc = little_callFunc;
}

PyObject *little_callFunc(littleStruct *s, char *funcName, PyObject *paraList){
    PyGILState_STATE st = PyGILState_Ensure();

    PyObject *module, *moduleDict, *moduleFunc, *moduleFuncArgs, *moduleFuncRet;

    // skipped error checking in this little example

    module = PyImport_ImportModule(funcName);
    moduleDict = PyModule_GetDict(module);
    moduleFunc = PyDict_GetItemString(moduleDict, funcName);

    // set arguments for littleOperation.littleOperation
    moduleFuncArgs = PyTuple_New(2);
    PyTuple_SetItem(moduleFuncArgs, 0, PyLong_FromVoidPtr(s));
    PyTuple_SetItem(moduleFuncArgs, 1, paraList);

    printf("==== In C, before call func: %s, address(memberStr): %p, memberStr: %s\n", funcName, s->memberStr, s->memberStr);
    moduleFuncRet = PyObject_CallObject(moduleFunc, moduleFuncArgs);
    printf("==== In C, after call func: %s, address(memberStr): %p, memberStr: %s\n", funcName, s->memberStr, s->memberStr);
    
    PyGILState_Release(st);
    return moduleFuncRet;
}

compile to .so file

gcc -g -fPIC -c little.c -I/usr/include/python2.7 -L/usr/lib/python2.7 -lpython2.7
gcc -shared little.o -o little.so
mv little.so /usr/lib

create the struct and call the struct member function callFunc in little.py

little.py

from ctypes import *

class littleStruct(Structure):
        pass
littleStruct._fields_ = [
            ("memberStr", c_char_p),
            ("callFunc", CFUNCTYPE(py_object, POINTER(littleStruct), c_char_p, py_object))
        ]

def main():
    try:
        littleDll = PyDLL("little.so")
        littleDll.createLittle.restype = POINTER(littleStruct)
        little = littleDll.createLittle()

        paraList = {"para0": 0, "para1": "test"}
        ret = little.contents.callFunc(little, "littleOperation", paraList)
    except Exception, e:
        print("%s"%str(e))

if __name__ == "__main__":
    main()

littleOperation.py

from little import littleStruct
from ctypes import memset
def littleOperation(little_pointer, paraList):
    try:
        little = littleStruct.from_address(little_pointer)

        # set little.memberStr
        # little.memberStr = paraList["para1"]
        for i in range(0, len(paraList["para1"])):
            memset(id(little.memberStr)+i, ord(paraList["para1"][i]), 1)
        print("After set little.memberStr in littleOperation.py")
        return {"status": 0}
    except Exception, e:
        print("%s"%str(e))
        return {"status": -1}

If I deirectly assign python string to C struct (char*) member, little.memberStr = paraList["para1"], the address will be changed.

==== In C, before call func: littleOperation, address(memberStr): 0x226a200, memberStr:
After set little.memberStr in littleOperation.py
==== In C, after call func: littleOperation, address(memberStr): 0x7fb9db196ec4, memberStr: test

I tried ctypes.memset, the address remains the same, but the (char *) member not changed at all.

==== In C, before call func: littleOperation, address(memberStr): 0x1d97200, memberStr:
After set little.memberStr in littleOperation.py
==== In C, after call func: littleOperation, address(memberStr): 0x1d97200, memberStr:

Why is ctypes.memset not working ?

What owns and manages the memory that the pointer points to? — DavidW
– DavidW, Commented Dec 15, 2021 at 9:54
Sorry about missing that part, I updated the creation flow in the post. little.py create the struct, call C function defined in little.c which will go littleOperation.py, thanks — Jack
– Jack, Commented Dec 15, 2021 at 11:59

Mark Tolonen · Accepted Answer · 2021-12-16 04:57:58Z

1

Due to ctypes special handling of c_char_p types, it returns a Python byte string object when accessed. Your memset/id code is modifying the first few bytes of the PyObject structure representing that immutable byte string object.

Instead, change the memberStr type in littleStruct._fields_ to POINTER(c_char) to preserve access to the C pointer.

Then, in littleOperation.py, you may cast the POINTER(c_char) to a POINTER(c_char * <size> where <size> is the required write size or known size of the buffer. Then you can dereference the pointer via .contents to access a fixed size array that can be assigned a byte string directly through .value:

p = cast(little.memberStr,POINTER(c_char * 128))
p.contents.value = paraList['para1']

See another answer of mine that has a smaller, standalone example.

edited Dec 16, 2021 at 4:57

answered Dec 16, 2021 at 4:38

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jack Over a year ago

Thanks! It's more intuitive in this way, I think I will get rid of all c_char_p in my code.

Mark Tolonen Over a year ago

@Jack it’s fine for a read-only null-terminated return value or parameter, but doesn’t work for writable pointers

Collectives™ on Stack Overflow

How to write a python string to a C struct's (char *) member in python?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related