Numpy Array shape changes when changing dtype

Question

Why does changing the dtype of elements of a numpy array changes the shape of the array? I am new to numpy and I was trying to change the dtype to np.float16 from existing np.int32. Doing that changed the shape of the array, but changing to np.float32 doesn't modify the shape.

>>> import numpy as np
>>> arr1=np.array([1,2,3,4]
>>> arr1
array([1, 2, 3, 4])
>>> arr1.shape
(4,)
>>> arr1.dtype=np.float16
>>> arr1
array([  5.96046448e-08,   0.00000000e+00,   1.19209290e-07,
     0.00000000e+00,   1.78813934e-07,   0.00000000e+00,
     2.38418579e-07,   0.00000000e+00], dtype=float16)
>>> arr1.shape
(8,)

YSelf · Accepted Answer · 2017-10-26 09:43:50Z

Because changing the data type of the array changes what the bytes that are actually stored mean, and how many of them make up one value. Your initial array probably has the data type int32, therefore each of the four values consists of four bytes:

a = np.array([1, 2, 3, 4], dtype=np.int32)
a
>>> array([1, 2, 3, 4], dtype=int32)
bytes(a)
>>> b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00'

Changing the data type does not change the stored bytes, but just what they mean:

a.dtype = np.float16
bytes(a)
>>> b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00'
a
>>> array([  5.96046448e-08,   0.00000000e+00,   1.19209290e-07,
     0.00000000e+00,   1.78813934e-07,   0.00000000e+00,
     2.38418579e-07,   0.00000000e+00], dtype=float16)

To actually convert the values, use .astype:

a.astype(np.float32)
>>> array([ 1.,  2.,  3.,  4.], dtype=float32)

Ignacio Vergara Kausel · Accepted Answer · 2017-10-26 09:42:42Z

0

You shouldn't be using the atrtibute dtype to "change" the dtype of the array. You can also see that the new numbers don't make sense in comparison with the original ones.

Instead you should use arr1.astype(np.float16).

My guess is that by just changing the dtype diretly you just tell numpy to change how much memory it reads for each element. Thus since you reduce from float32 to float16, which is half the memory representation, you get double the size.

answered Oct 26, 2017 at 9:42

Ignacio Vergara Kausel

6,0844 gold badges35 silver badges42 bronze badges

Comments

Ken Wei · Accepted Answer · 2017-10-26 09:44:50Z

0

If you want to change the datatype of an array, you should do arr1.astype(np.float16).

I'm not completely sure about this, but I'm guessing that numpy calculates the shape by dividing the memory usage of the array by the size of the datatype.

If you modify dtype directly, it the array still thinks its elements have the same size, so you might get a different answer.

answered Oct 26, 2017 at 9:44

Ken Wei

3,1381 gold badge12 silver badges31 bronze badges

Collectives™ on Stack Overflow

Numpy Array shape changes when changing dtype

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related