Python: struct.pack and struct.unpack for NumPy array

Question

I am exploring NumPy version of struct.pack and struct.unpack. My particular problem is converting an RGB value (r, g, b) into a binary.

For example:

f = lambda r, g, b: struct.unpack('I', struct.pack('BBBB', r, g, b, 255))[0]
f(155, 156, 148)
#--> 4287929499

When I apply this conversion to an image (2-d array of RGB), it becomes very slow.

For example:

import numpy as np
import struct

img_rgb = np.random.randint(0, 256, (480, 640, 3))
%%time
np.apply_along_axis(lambda rgb: struct.unpack('I', struct.pack('BBBB', rgb[0], rgb[1], rgb[2], 255))[0], 2, img_rgb)
#--> Wall time: 5.23 s

My question is that is there NumPy version of struct.pack and struct.unpack? Or, how can I make this code faster with NumPy?

Mad Physicist · Accepted Answer · 2021-03-11 02:24:58Z

3

You don't need pack and unpack operations in numpy because it uses raw binary arrays instead of individually wrapping each value in a separate object. In general, the operations you use are reshape, transpose, ndarray.view, and sometimes ndarray.astype.

In your particular case, you have and (M, N, 3) image, which you want to add a channel of 255 to and convert to np.uint8:

img_rgb = np.random.randint(0, 256, size=(480, 640, 3))
img_rgba = np.concatenate((img_rgb.astype(np.uint8), np.full((*img_rgb.shape[:-1], 1), 255, dtype=np.uint8)), axis=-1)

By setting the last dimension to size 4, you ensure that you can now view the data directly as np.uint32:

result = img_rgba.view(np.uint32)

The result will be of shape (M, N, 1). You can remove the extra dimension with np.squeeze:

result = np.squeeze(img_rgba.view(np.uint32))

This doesn't really buy you much: it removes the convenient 3-channel representation without altering the memory layout. Dumping either img_rgba or result to a raw binary file would produce identical results.

In general, functions like apply_along_axis are generally code-smell in numpy. They are no better than running an explicit for-loop. Your goal should always be to vectorize your code to perform operations on all elements at once.

edited Mar 11, 2021 at 2:24

answered Mar 10, 2021 at 19:24

Mad Physicist

116k29 gold badges202 silver badges292 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Akihiko Over a year ago

Great! This way improved the computation speed 150 times.

Collectives™ on Stack Overflow

Python: struct.pack and struct.unpack for NumPy array

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related