numpy try reading multi-column binary file

Question

I have a numpy file and I want to read in 3 separate columns of data from a single file. I created the data through three separate arrays where data1 = floating point, and data2 = data3 = string. The data were saved such that:

np.save(infofile, [data1, data2, data3])

I am able to read the file and load the data with the following command:

data1, data2, data3 = np.load(infofile)

where data1 will be a floating point value, and data2 and data3 are strings. Each will be ~ 1,000 rows long. When I try to look at the data, I get:

print(data1[0])
b'0.0'
print(data2[0])
b'10000'
print(data3[0])
b'20190831.230000'

I know the data is in binary, so how do I remove the preceding 'b' from all of the data so it would look like:

print(data1[0])
0.0
print(data2[0])
'10000'
print(data3[0])
'20190831.230000'

Where did the file come from and how was it created? It seems that it contains just three rows of binary strings (not floating point data). The b'' prefix indicates a Python 3 bytes object. — Iguananaut
– Iguananaut, Commented Oct 29, 2019 at 20:10

Daniel · Accepted Answer · 2019-10-29 20:20:49Z

2

The solution is to decode the data. utf-8 is a common encoding, but if you used another encoding, decode the data using that encoding.

print(data1[0].decode("utf-8"))

will give:

'0.0'

For every element in the lists:

data1 = [float(item.decode("utf-8")) for item in data1]
data2 = [item.decode("utf-8") for item in data2]
data3 = [item.decode("utf-8") for item in data3]

which will decode each element in the list, and for data1, it will parse the elements as floats.

edited Oct 29, 2019 at 20:20

answered Oct 29, 2019 at 20:10

Daniel

813 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Iguananaut Over a year ago

possibly; but they wrote "where data1 will be a floating point value, and data2 and data3 are strings", so if they were really expecting floating point data then something deeper is wrong here

WX_M Over a year ago

So this gives me 0.0, not '0.0' which is what I was looking for. Is there a way to decode the entire array in one command?

Daniel Over a year ago

It will give you 0.0 if you print it, but it is still a string, or not?

WX_M Over a year ago

it is, indeed, class 'str'. I would need to convert to floating point afterwards.

Collectives™ on Stack Overflow

numpy try reading multi-column binary file

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related