MATLAB vs. Python Binary File Read

Question

I have a MATLAB application that reads a .bin file and parses through the data. I am trying to convert this script from MATLAB to Python but am seeing discrepancies in the values being read.

The read function utilized in the MATLAB script is:

fname = 'file.bin';
f=fopen(fname);
data = fread(f, 100);
fclose(f);

The Python conversion I attempted is: (edited)

fname = 'file.bin'
with open(fname, mode='rb') as f:
    data= list(f.read(100))

I would then print a side-by-side comparison of the read bytes with their index and found discrepancies between the two. I have confirmed that the values read in Python are correct by executing $ hexdump -n 100 -C file.bin and by viewing the file's contents on the application HexEdit.

I would appreciate any insight into the source of discrepancies between the two programs and how I may be able to resolve it.

Note: I am trying to only utilize built-in Python libraries to resolve this issue.

Solution: Utilizing incorrect file path/structure between programming languages. Implementing @juanpa.arrivillaga's suggestion cleanly reproduced the MATLAB results.

Um, this is completely redundant: int(hex(ord(i)),16) can just be ord(i), IOW, int(hex(whatever), 16) == whatever — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Nov 12, 2022 at 19:29
Also, data = [int(hex(ord(i)),16) for i in bytes] would raise a TypeError, because i is an int, and ord(i) is expecting a str of length 1. You really must provide a minimal reproducible example — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Nov 12, 2022 at 19:31

Cris Luengo · Accepted Answer · 2022-11-12 19:32:31Z

1

An exact translation of the MATLAB code, using NumPy, would be:

data = np.frombuffer(f.read(100), dtype=np.uint8).astype(np.float64)

answered Nov 12, 2022 at 19:32

Cris Luengo

61.4k10 gold badges75 silver badges135 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Alex Over a year ago

Unfortunately, this reproduced the same results as @juanpa.arrivillaga suggested, except as a float64.

Cris Luengo Over a year ago

@Alex This is exactly MATLAB’s behavior. Are you sure you get a different result in MATLAB? Are you sure you’re referencing the same file? If things really don’t match up with the same file, please include the file in your post (or at least the hex dump of the first 100 values), and show us what MATLAB’s output is and what Python’s output is.

Ahmed AEK · Accepted Answer · 2022-11-12 19:31:11Z

0

python automatically transforms single bytes into unsigned integers, as done by matlab, so you just need to do the following.

fname = 'file.bin'
with open(fname, mode='rb') as f:
    bytes_arr = f.read(100)
    # Conversion for visual comparison purposes
    data = [x for x in bytes_arr]
print(data)

also welcome to python, bytes is a built-in type, so please don't override the built-in bytes type ... or you'll run into unexpected problems.

Edit: as pointed by @juanpa.arrivillaga you could use the faster

fname = 'file.bin'
with open(fname, mode='rb') as f:
    bytes_arr = f.read(100)
    # Conversion for visual comparison purposes
    data = list(bytes_arr)

edited Nov 12, 2022 at 19:31

answered Nov 12, 2022 at 19:08

Ahmed AEK

23.2k3 gold badges19 silver badges50 bronze badges

3 Comments

juanpa.arrivillaga Over a year ago

data = [x for x in bytes_arr] -> list(bytes_arr)

Alex Over a year ago

Thank you for your suggestions. This does improve the clarity of the code but, does not resolve the underlying issue of the read discrepancies.

Ahmed AEK Over a year ago

@Alex it does solve the underlying issue of discrepancies, if you are getting different results then the problem is not in this block of code.

Collectives™ on Stack Overflow

MATLAB vs. Python Binary File Read

2 Answers 2

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related