Converting between bytes and POINTER(c_ubyte)

Question

How do I convert between bytes and POINTER(c_ubyte) in Python?

I want to pass a bytes object to a C function as a POINTER(c_ubyte) argument, and I want to work with a returned POINTER(c_ubyte) as bytes.

Right now I am using:

data = b'0123'
converted_to = ctypes.cast(data, ctypes.POINTER(ctypes.c_ubyte))
converted_from = bytes(converted_to)

This doesn't seem quite right. I get a warning in PyCharm on data in the converted_to line that says:

Expected type 'Union[_CData, _CArgObject]', got 'bytes' instead

Neitsa · Accepted Answer · 2019-11-12 18:32:25Z

Here's a simple C++ function:

#define EXTERN_DLL_EXPORT extern "C" __declspec(dllexport)

EXTERN_DLL_EXPORT unsigned char* DoBytesStuff(const unsigned char* buffer, size_t buffer_size)
{
    const auto new_buffer = new unsigned char[buffer_size];
    memcpy(new_buffer, buffer, buffer_size);

    for(size_t idx = 0; idx < buffer_size; idx++)
    {
        new_buffer[idx] += 1;
    }

    return new_buffer;
}

EXTERN_DLL_EXPORT void FreeBuffer(const unsigned char* buffer)
{
    delete[] buffer;
}

So basically, it takes an unsigned char* buffer in input, copy it, add 1 to each element in the copy and returns the copied buffer.

Now for python:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import ctypes


def main():
    dll = ctypes.WinDLL(r"TestDll.dll")

    dll.DoBytesStuff.argtypes = [ctypes.POINTER(ctypes.c_ubyte), ctypes.c_size_t]
    dll.DoBytesStuff.restype = ctypes.POINTER(ctypes.c_ubyte)
    dll.FreeBuffer.argtypes = [ctypes.POINTER(ctypes.c_ubyte)]

    buffer = bytes(range(0x100))  # 0 to 0xff
    # ctypes instance that shares the buffer of the source object; use from_buffer_copy() to not share it
    # note that we must use a bytearray because bytes object are immutable (and therefore not writable).
    ubuffer = (ctypes.c_ubyte * len(buffer)).from_buffer(bytearray(buffer))

    result = dll.DoBytesStuff(ubuffer, len(buffer))
    b_result = ctypes.string_at(result, len(buffer))
    print(b_result)

    dll.FreeBuffer(result)

if __name__ == "__main__":
    main()

(ctypes.c_ubyte * len(buffer)) create an array of c_ubyte which is then initialized with the from_buffer function. from_buffer accepts only a writable object and thus we can't use bytes.

As for the return, string_at directly returns a bytes object from the pointer.

Mark Tolonen · Accepted Answer · 2025-07-21 06:33:45Z

When it comes to pointers in ctypes, sometimes the argument types don't have to match exactly. In this case, use ctypes.c_char_p as the input buffer type and pass a bytes object directly. ctypes will marshal the parameter as a pointer to the internal data of the bytes object. Make sure the C function won't modify the data (const unsigned char *, for example).

For the return value, use ctypes.POINTER(ctypes.c_char) if you want to view the contents as a bytes string. A c_char_p return value is assumed to be a null-terminated C string and ctypes will automatically convert it to a bytes string, losing the original C pointer value. If that pointer is allocated memory it won't be able to be freed and be a memory leak. A POINTER(c_char) remains a pointer value that can be freed later.

In the example below a helper function is used to demonstrate receiving the output pointer, capturing its contents, and freeing the original C buffer:

test.c

#include <stdlib.h>
#include <stdint.h>

#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

API uint8_t* buffer_inc(const uint8_t* in_buffer, size_t buffer_size) {
    uint8_t* out_buffer = malloc(buffer_size);
    for(size_t i = 0; i < buffer_size; ++i)
        out_buffer[i] = in_buffer[i] + 1;
    return out_buffer;
}

API void buffer_free(uint8_t* buffer) {
    free(buffer);
}

test.py

import ctypes as ct

dll = ct.WinDLL('./test')
dll.buffer_inc.argtypes = ct.c_char_p, ct.c_size_t
dll.buffer_inc.restype = ct.POINTER(ct.c_char)
dll.buffer_free.argtypes = ct.POINTER(ct.c_char),
dll.buffer_free.restype = None

def buffer_inc(data):
    result = dll.buffer_inc(data, len(data))
    # slice the data to the correct size (makes a copy)
    retval = result[:len(data)]
    dll.buffer_free(result)
    return retval

print(buffer_inc(b'ABC123'))

Output:

b'BCD234'

Refer to another answer of mine for other ways to handle returned pointers.

Collectives™ on Stack Overflow

Converting between bytes and POINTER(c_ubyte)

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related