Python: Access/Save memory block just given by pointer without copying

Question

Here is an example of what I try to do:

import ctypes
MEM_SIZE = 1024*1024*128

# allocate memory (this is Windows specific)
ptr = ctypes.cdll.msvcrt.malloc(MEM_SIZE)

# make memory accessible to Python calls
mem = ctypes.string_at(ptr, MEM_SIZE)
# BAD: string_at duplicates memory

# store it to disk
open(r'test.raw', 'wb').write(mem)

In a nutshell: I have a plain memory pointer, I know the size of the block and want to store it on disk or reuse it as a numpy array.

How can I do that without generating a copy of the memory block?

Related question: stackoverflow: Fill python ctypes pointer (thanks to Brian Larsen for the hint)

Janne Karila · Accepted Answer · 2012-05-09 09:40:33Z

2

ctypes_array = (ctypes.c_char * MEM_SIZE).from_address(ptr)
with open('test.raw', 'wb') as f:
    f.write(ctypes_array)

numpy_array = numpy.frombuffer(ctypes_array, dtype=numpy.byte)
numpy_array.tofile('test.raw')

edited May 9, 2012 at 9:40

answered May 9, 2012 at 9:11

Janne Karila

25.3k6 gold badges59 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Tom Pohl Over a year ago

Great, your suggestion works w/o copying the data! Thanks a lot!

Tom Pohl Over a year ago

If you just want to store it and don't need the numpy array, you can do that with: open('test.raw', 'wb').write(ctypes_array)

Janne Karila Over a year ago

@TomPohl OK, added that to my answer.

Collectives™ on Stack Overflow

Python: Access/Save memory block just given by pointer without copying

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related