Combine a packed header and numpy array in Python

Question

Does anybody have any tips on how to pack a header and then append it to some chunk of data?

For example, if I have a bit-packed header of 16 bytes, and then I want to append it to a raw RGB image of about 2MB, what is the most efficient way of doing this?

So far, I've tried the following

headerVals = ( str(headerVersion), str(formatEnum), frameCount, ntpTimestamp, width, height )
packedHdr = self.imgStruct.pack( *headerVals )

return packedHdr + data

However this fails with the following error because str and numpy arrays arent concat'able:

    return packedHdr + data
TypeError: cannot concatenate 'str' and 'numpy.ndarray' objects

The only way around this that I can think of, as a Python beginner, is the following which is extraordinarily slow for obvious reasons:

# Generate the header
headerVals = ( str(headerVersion), str(formatEnum), frameCount, ntpTimestamp, width, height )

packDir = 'cchqhh{0}h'.format(width*height*3)

return pack( packDir, str(headerVersion), str(formatEnum), frameCount, ntpTimestamp, width, height, *data )

Any ideas? As a Python initiate I find myself a little stumped by this!

UPDATE:

As per seth's suggestion below, I updated my code to the following, and it is working nicely.

# Generate the header
headerVals = ( str(headerVersion), str(formatEnum), frameCount, ntpTimestamp, width, height )
packedHdr = self.imgStruct.pack( *headerVals )

# Concatanate header to the data
return numpy.concatenate( ( numpy.fromstring( packedHdr, dtype=numpy.uint8 ), data ), axis=0 )

Why not put headerVersion and formatEnum inside a numpy array, then try to concat? — seth
– seth, Commented Jul 15, 2013 at 15:27
I ended up taking this approach and it seems to be working. Adding changes to question — bgoldber
– bgoldber, Commented Jul 15, 2013 at 16:04

tiago · Accepted Answer · 2013-07-15 15:50:54Z

1

To concatenate your header with the numpy array you'll need to have the array in binary form. Assuming data is an array of ints:

import struct

raw_data = packedHdr + struct.pack('i' * data.size, *data)

If your data is of a different kind, you need to specify it in st.pack. You can then unpack the resulting raw_data in the desired form.

answered Jul 15, 2013 at 15:50

tiago

23.7k13 gold badges75 silver badges89 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

bgoldber Over a year ago

So this works, but I think the efficiency is not great because data is blitted into data.size ints behind the scenes, right?

tiago Over a year ago

@bgoldber this will indeed double your data in memory, but it should be put in the same data type as the original numpy array (otherwise you are packing data representations in wrong formats!). For a 2Mb image this is really trivial and the performance issues should be minor, unless you are doing this a lot.

seth · Accepted Answer · 2013-07-15 16:48:52Z

0

Why not put headerVersion and formatEnum inside a numpy array, then try to concat?

answered Jul 15, 2013 at 16:48

seth

1,78816 silver badges17 bronze badges

Collectives™ on Stack Overflow

Combine a packed header and numpy array in Python

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related