Numpy overflow in short scalars

Question

I have a large .wav file array (200k samples) loaded in with scipy.io.wavfile. I tried to make a histogram of the data using matplotlib.pyplot hist with auto binning. It returned the error:

ValueError: Number of samples, -72, must be non-negative.

So I decided to set the bins myself using binwidth=1000:

min_bin = np.min(data[peaks])
max_bin = np.max(data[peaks])
plt.hist(data[peaks], bins=np.arange(min_bin,max_bin, binwidth))

When I do this, it gives the error:

RuntimeWarning: overflow encountered in short_scalars
from scipy.io import wavfile

Here are the type print outs of min_bin, max_bin, data:

Type min_bin: <class 'numpy.int16'> max_bin: <class 'numpy.int16'>
min_bin: -21231 max_bin: 32444
Type data <class 'numpy.ndarray'>

The problem seems to be with np.arange which fails when I provide it the bin range from the np.max and np.min .wav array values. When I manually type the max and min integer values into np.arange it has no problem. My hypothesis is that it is some sort of addressing error when referencing the .wav array but not sure how to fix it or why it is occurring.

Could you print out type(data), min_bin, max_bin, type(min_bin), and type(max_bin) and see what the output is? — nog642
– nog642, Commented Oct 21, 2019 at 2:33
Sorry for the delay, I have added the requested type print outs — pproctor
– pproctor, Commented Oct 22, 2019 at 4:20

user2357112 · Accepted Answer · 2022-12-05 05:53:11Z

3

As part of the computation of the length of the array, numpy.arange calculates stop - start, in Python object arithmetic. When stop and start are numpy.int16(32444) and numpy.int16(-21231), this subtraction overflows and produces numpy.int16(-11861). This is where the warning comes from. The nonsense value leads numpy.arange to believe that the result should be a length-0 array.

The workaround is simple; just convert the arguments to ints first. The dtype of the array itself can still be set to np.int16 to save space, since that's all you need to store the necessary data.

min_bin = int(np.min(data[peaks]))
max_bin = int(np.max(data[peaks]))
plt.hist(data[peaks], bins=np.arange(min_bin, max_bin, binwidth, dtype=np.int16))

edited Dec 5, 2022 at 5:53

user2357112

286k32 gold badges490 silver badges570 bronze badges

answered Oct 24, 2019 at 6:16

nog642

6195 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Numpy overflow in short scalars

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related