Decoding NumPy int64 binary representation

Question

So I did a stupid thing, and forgot to explicitly type-convert some values I was putting into an SQLite database (using Python's SQLalchemy). The column was set up to store an INT, whereas the input was actually a numpy.int64 dtype.

The values I am getting back out of the database look like:

b'\x15\x00\x00\x00\x00\x00\x00\x00'

It seems that SQLite has gone and stored the binary representation for these values, rather than the integer itself.

Is there a way to decode these values in Python, or am I stuck with loading all my data again (not a trivial exercise at this point)?

mhawke · Accepted Answer · 2016-07-08 06:30:19Z

4

You can use struct.unpack():

>>> import struct
>>> value = struct.unpack('<q', b'\x15\x00\x00\x00\x00\x00\x00\x00')
>>> value
(21,)
>>> value[0]
21

That assumes that the data was stored little endian as specified by the < in the unpack() format string, and that it is a signed "long long" (8 bytes) as specified by the q. If the data is big endian:

>>> struct.unpack('>q', b'\x15\x00\x00\x00\x00\x00\x00\x00')
(1513209474796486656,)

I imagine that little endian is more likely to be correct in this case.

P.S. I have just confirmed that when a numpy.int64 is inserted into a SQLite int field it can be retrieved using struct.unpack() as shown above.

edited Jul 8, 2016 at 6:30

answered Jul 8, 2016 at 6:22

mhawke

87.5k10 gold badges122 silver badges142 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Andrew Guy Over a year ago

Thank you, that works a charm. The data was indeed little endian.

Collectives™ on Stack Overflow

Decoding NumPy int64 binary representation

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related