Write Real Raw Binary File from Python

Question

I've tried multiple different variations, but for some reason I keep getting invalid binary digits (human readable) being output to the file:

img_array = np.asarray(imageio.imread('test.png', as_gray=True), dtype='int8')
img_array.astype('int8').tofile("test.dat")

But this doesn't produce a valid binary file. Once the file is read into a Verilog tb, it complains about invalid binary digits and when I open up the file I see a bunch of numbers and other characters. It just doesn't seem like its translating correctly.

UPDATE: After running

print(img_array)
print(img_array.tobytes())

I can see that the int value '43' is being translated to '+' whereas I would expect '2B'. It seems to only be printing certain values to ASCII. Here's a simple example:

x = np.array([[0, 9], [2, 3]], dtype='uint8')
print(x.astype('uint8'))
print(x.tobytes())

The output is:

[[0 9]

[2 3]]

b'\x00\t\x02\x03'

How Can I fix this?

Any help would be greatly appreciated.

Other Solutions that I've tried:

Write a “string” as raw binary into a file Python

Write a raw binary file with NumPy array data

naivepredictor · Accepted Answer · 2019-03-06 07:50:40Z

0

https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.tobytes.html

img_array = np.asarray(imageio.imread('test.png', as_gray=True), dtype='int8')
img_array = img_array.astype('int8')

with open('test.dat', 'wb') as f:
    f.write(img_array.tobytes())

edited Mar 6, 2019 at 7:50

answered Mar 5, 2019 at 10:19

naivepredictor

9084 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

InterestingGuy Over a year ago

Hmm.. This doesn't seem to work for me. I changed your "to_bytes()" call to "tobytes()" as you referenced in your documentation link because numpy ndarray doesn't have a to_bytes() method. Even with that, the output fill contains many numbers. There also seems to be some redundancy because in the first line, I declare " dtype='int8' " then following that you use "astype('int8')".

naivepredictor Over a year ago

Sorry for the typo. tobytes encodes your content into byte string. Now, read it back and decode and you will see your array.

InterestingGuy Over a year ago

I added an update to my question that pinpoints the problem. Unfortunately, I'm not using pythons methods to read and decode. This raw data is being transferred to a verilog tb

InterestingGuy · Accepted Answer · 2019-03-09 21:14:55Z

0

This gave a workable solution which converts to a string of hex values. It's not exactly what I wanted, but it created a valid work around since my original question has yet to be answered. Although I didn't find this solution so I can reference where it came from, I'll share it here anyways. Apparently this handles signed integers as well:

("{:0>2X}" * len(x.flatten())).format(*tuple(x.flatten() & (2**8-1)))

answered Mar 9, 2019 at 21:14

InterestingGuy

498 bronze badges

Collectives™ on Stack Overflow

Write Real Raw Binary File from Python

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related