1

My python code is calling a C function in a native library (also written by me). The function takes two strings as arguments and returns its response as a string. The two argument strings that python passes to C library are only read by the C library, so I am not worried about their memory management. AFAIK, their memory will be allocated by python and will be released when the python runtime seems it appropriate.

However, I understand that the memory of the string that C library returns has to be managed explicitly. One way I can implement it is: the C function callocs a character array, populates it with answer and returns from the function. The python code uses the returned string and should call libc's free on that string or call another function of the same C library which will take care of freeing the memory.

Is there any other way to do this? Does ctype offer any utility functions that simplify releasing the memory of data structures returned by native libraries?

Improve this question

2 Answers 2

Reset to default
4

I'd recommend strongly against memory allocation in a foreign function called via ctypes and freeing this memory from Python. Whenever possible, allocate the memory in Python.

In your case, if you know in advance an upper limit for the length of the returned string, use

buf = ctypes.create_string_buffer(upper_limt)

to allocate the memory in Python and pass a pointer to this memory to your function.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

This could, theoretically, lead to buffer overrun.
@Noctis: That's why I said "if you know in advance an upper limit for the length of the returned string".
@chmike I think it is, if you can guarantee an upper limit. Your C API should make sure it does not exceed this limit, as described in Jim's answer below.
@SvenMarnach A simpler solution and the right answer, if I understood correctly, is to declare the return type of the function as c_char_p. ctypes will then copy the returned string into a python string and free the returned string.
@chmike While ctypes will copy string data when converting it to a bytes object, it definitely won't free the data the character pointer pointed to. It's completely unclear whether the pointer points to a string allocated on the heap with malloc() – in the vast majority of cases it won't, so trying to free it by default would make ctypes more or less unusable. If the returned string actually is heap-allocated, going the route you suggest would leak the data.
|
3

A common way to do this is adding an additional buffer parameter and a length parameter to the C function. If the function is called with a buffer size that is smaller than the required size, the function simply returns the required buffer size and performs no other action, otherwise, the data is copied into the buffer and the size is set to the actual size of the data copied.

This allows you to manage the buffer in Python, at the expense of adding 2 additional parameters to your C function.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.