Avoiding Overflow in python 3.5

Question

I was used to Matlab, so when I want to create an array of 10**j in Python, where j is an integer, I use the following code:

import numpy as np
a=np.array(range(11))
b=10**a

However my machine gives me the following output:

array([          1,          10,         100, ...,   100000000,
    1000000000, -2147483648], dtype=int32)

The last entry is obviously wrong. Now, I know I can do the following

b=np.array([10**k for k in range(11)])

which gives the correct answer, so I guess the problem is from the numpy function array (Of course, I can avoid this problem by saving a as a int64 integer, but there will be overflow problem for larger j). It seems like there could be a lot of situations where one could fall into this pitfall of overflowed integers when doing mathematical operations on arrays. I am wondering whether there are ways to avoid this problem?

is there a reason you can't use floats? That is usually the solution when dealing with large exponents with a small number of significant digits. — TheBlackCat
– TheBlackCat, Commented Mar 2, 2017 at 15:30

MSeifert · Accepted Answer · 2017-03-02 21:37:31Z

1

No, there isn't really a way to avoid this problem because numpy deliberatly not checks for overflow in some arithmetic operations because it's faster not to check. Generally you need to pay attention to what kind of objects you represent in numpy.

However you could force an object-array, to get Python behaviour:

>>> import numpy as np
>>> a = np.array(range(15), object)   
>>> # Note: You could also use "a = np.arange(15, dtype=object)" instead! Thanks @TheBlackCat
>>> b = 10 ** a
>>> b
array([1, 10, 100, 1000, 10000, 100000, 1000000, 10000000, 100000000,
       1000000000, 10000000000, 100000000000, 1000000000000,
       10000000000000, 100000000000000], dtype=object)

Note that object arrays are slow and should only be used if you don't need the full-numpy-speed and functionality.

In your specific case you can also use the np.float_power-ufunc:

b = np.float_power(10, a)
b
array([  1.00000000e+00,   1.00000000e+01,   1.00000000e+02,
         1.00000000e+03,   1.00000000e+04,   1.00000000e+05,
         1.00000000e+06,   1.00000000e+07,   1.00000000e+08,
         1.00000000e+09,   1.00000000e+10,   1.00000000e+11,
         1.00000000e+12,   1.00000000e+13,   1.00000000e+14])

edited Mar 2, 2017 at 21:37

answered Mar 2, 2017 at 14:09

MSeifert

154k41 gold badges356 silver badges377 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

TheBlackCat Over a year ago

You can also use np.arange(15, dtype='object')

MSeifert Over a year ago

@TheBlackCat You're right! I just copied and modified the code from the question. I'll edit the answer.

Collectives™ on Stack Overflow

Avoiding Overflow in python 3.5

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related