Numpy square return wrong values for array

Question

I have a data set where pairs of numbers are represented by 32 bits, each number is represented as two 8 unsigned integers.

I'm trying to get the first real part in one array and the complex part in a second array. I'm then trying to square each part, add them up and take the square root of the sum. (aka taking the magnitude). When I try squaring the elements of each arrays using numpy.square I get not only negative but also inaccurate values. Any idea what's going on/what's wrong?

import matplotlib.pyplot as plt
import numpy as np
import scipy.signal as signal

data = np.fromfile(r'C:\Users\Miaou\Desktop\RAW_DATA_000066_000008.bin', dtype="int16")

print(data)

Is = data[0::2]
Qs = data[1::2]
Is_square = np.square(Is, dtype='int16')
Qs_square = np.square(Qs, dtype='int16')

print('Is',Is)
print('Qs',Qs)
print('Is square',Is_square)
print('Qs square',Qs_square)

Output: Is [  335  -720  8294 ... -3377  3878  6759]
Qs [-2735  4047  1274 ...  -279  1319  4918]
Is square [-18847  -5888 -22364 ...    865  31140   5489]
Qs square [  9121  -5791 -15324 ...  12305 -29711   3940]

Matt Messersmith · Accepted Answer · 2020-05-13 01:43:20Z

You're experiencing integer overflow. The min value of the int16 (signed) type is -32768 and the maximum value is 32767. The reason for this is because you only have 16 bits (that's what int16) means. Note that 2^16 = 65536, but since it's signed (negatives allowed), we don't have values 0 through 65536, but instead you can imagine they are shifted down such that 0 is centered (i.e. -32768 to 32767)

Let's take your first element of the Is as an example:

>>> 335**2
112225

Note that 112225 > 32767. This means you'll get overflow. It just keeps wrapping around until it lands in the valid range:

>>> x = 112225
>>> x = x - 2**16
>>> x
46689 # Still not in the valid range. Repeat.
>>> x = x - 2**16
>>> x
-18847 # Yep, now we are between -32768 and 32768

The other answer here is not quite right as leaving off the dtype does not suffice:

>>> Is = np.array([335, -720, 8294, -3377, 3878, 6759]).astype('int16')
>>> Is
array([  335,  -720,  8294, -3377,  3878,  6759], dtype=int16)
>>> Is_sq_nodtype = np.square(Is)
>>> Is_sq_nodtype
array([-18847,  -5888, -22364,    865,  31140,   5489], dtype=int16)

numpy ops keep the same dtype. You actually need to "up" the dtype to have more bits. int32 should probably do the trick for your values (you can also do int64 or float depending on how big your values are, here is a list of dtypes: https://docs.scipy.org/doc/numpy-1.10.0/user/basics.types.html)

Working example:

>>> Is = np.array([335, -720, 8294, -3377, 3878, 6759]).astype('int16')
array([  335,  -720,  8294, -3377,  3878,  6759], dtype=int16)
>>> Is_sq = np.square(Is, dtype='int32')
>>> Is_sq
array([  112225,   518400, 68790436, 11404129, 15038884, 45684081], dtype=int32)

HTH.

Noah Smith · Accepted Answer · 2020-05-13 00:40:45Z

-1

Remove dtype='int16' from your np.square() calls.

answered May 13, 2020 at 0:40

Noah Smith

1874 bronze badges

1 Comment

Meow Over a year ago

I didn't have this originally and I was getting the same issue. That was an attempt at trouble shooting it!

Collectives™ on Stack Overflow

Numpy square return wrong values for array

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related