11

Indexing one numpy array with another - both are defined as dtype='uint32'. Using numpy.take to index and get an unsafe casting error. Not come across this before. Any idea what is going on?

Python 2.7.8 |Anaconda 2.1.0 (32-bit)| (default, Jul  2 2014, 15:13:35) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.

>>> import numpy
>>> numpy.__version__
'1.9.0'

>>> a = numpy.array([9, 7, 5, 4, 3, 1], dtype=numpy.uint32)
>>> b = numpy.array([1, 3], dtype=numpy.uint32)
>>> c = a.take(b)

Traceback (most recent call last):
  File "<pyshell#12>", line 1, in <module>
    c = a.take(b)
TypeError: Cannot cast array data from dtype('uint32') to dtype('int32') according to the rule 'safe'
Improve this question
5
  • I get no warning running python 3.4.3 64-bit and numpy '1.9.2rc1' Commented Mar 4, 2015 at 11:53
  • I think is the problem is with assigning to a.take(b) to c. Try setting c =numpy.array(0,dtype=numpy.uint32) before c = a.take(b) Commented Mar 4, 2015 at 11:55
  • No warning either here with Python 2.7.5 64-bit and numpy 1.9.1. Commented Mar 4, 2015 at 11:57
  • Updated to numpy 1.9.2 but still the same error. Setting c=numpy.array(0, dtype=numpy.uint32) doesn't work either. Same result on Windows 32-bit XP and Windows 8.1. Weird! Commented Mar 4, 2015 at 12:19
  • I would interpret this problem as follows. NumPy uses signed ints for indexing (since a[-1] works like it would in a Python list, after all). When it sees that the index array has the wrong dtype, it tries to convert the index array to an array of signed ints (which would be reasonable and produce reasonable results if b were an array of, say, int16). Since this conversion isn't safe for an array of uint32 - for example, a maximal uint32 value would become -1, a rather different index - NumPy throws an error. You may want to convert the dtype of b yourself before using take. Commented Mar 4, 2015 at 12:37

1 Answer 1

Reset to default
11

This is quite common when working NumPy functions that require indexes or lengths to be specified (it's not just take, see for example here).

The problem is that, for indexing purposes, NumPy would like to treat your uint32 array as an int32 array (which is probably the "pointer" integer type on your 32 bit system, np.intp) and wants to cast it down to that type.

It can't do this safely - some of the integers in an unsigned array might not be representable as a signed 32 bit integer. The error you see reflects this.

This means that you'll get the same error if b has dtype int64 or a float dtype, but not if it is int32 or a smaller integer dtype.

For what it's worth, this isn't an immediate problem for the indexing notation a[b] which allows the unsafe casting (but will raise an error if the index is out of bounds). Try for example a[2**31] - NumPy casts to int32 but then complains that the index -2147483648 is out of bounds.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks! I normally use a[b] but that is supposed to be slower than .take. Anyway, will stick to what works!
It isn't slower anymore in 1.9.x. And this behavior should probably be reported as a bug, np.take should do the same that indexing does: use unsafe casting once it has confirmed that the index is an integer, e.g. on a 32 bit system a[2**32-1] returns a[-1], because the 2**32-1 gets converted to a np.uint32 which is then unsafely cast to a np.int32. This corner case buggy behavior is less annoying than making np.uint32 unusable as an index.
@Jaime Good to hear that the a[b] type of indexing is no longer slow. I'll drop a note on the numpy mailing list about np.take. Thanks!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.