Numpy boolean indexing issue

Question

Here is my code snippet that uses numpy

import numpy as np
n = 10
arr = np.array(range(n))
print(arr)
selection = [i % 2 == 0 for i in range(n)]
print(selection)
neg_selection = np.invert(selection)
print(neg_selection)

print(arr[selection])
print(arr[neg_selection])

The above code prints:

[0 1 2 3 4 5 6 7 8 9]
[True, False, True, False, True, False, True, False, True, False]
[False  True False  True False  True False  True False  True]
[1 0 1 0 1 0 1 0 1 0]
[1 3 5 7 9]

the last two expected prints are:

[0 2 4 6 8]
[1 3 5 7 9]

What is wrong here?

What are you trying to do? This this a convoluted way to extract odd and even numbers — Chris_Rands
– Chris_Rands, Commented Sep 30, 2016 at 6:43
No, this is not for extracting odds and evens. I have a dataset of 'N' records in ndarray. I need to separate it into test and train based on index of items ( thats the protocol, no random shuffling allowed). In my actual case, every 7th item in the array goes to test and the rest goes to train. Here I used 2 instead of 7 and a simple 1D array instead of my complex array to simplify the problem. — TG Gowda
– TG Gowda, Commented Sep 30, 2016 at 6:57

TG Gowda · Accepted Answer · 2016-09-30 06:25:34Z

3

looks like numpy has trouble dealing with list of booleans.

n = 10
arr = np.array(range(n))
selection = [i % 2 == 0 for i in range(n)]
neg_selection = np.invert(selection)

print(type(selection), arr[selection])
print(type(neg_selection), arr[neg_selection])

It produced:

<type 'list'> [1 0 1 0 1 0 1 0 1 0]
<type 'numpy.ndarray'> [1 3 5 7 9]

Noticed that the trouble here is due to <type 'list'>. So, change the selection object to numpy array

selection = np.array([i % 2 == 0 for i in range(n)])

Or, even simpler:

selection = arr % 2 == 0

Then it worked.

answered Sep 30, 2016 at 6:25

TG Gowda

12.1k5 gold badges52 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

hpaulj · Accepted Answer · 2016-09-30 16:24:57Z

In [63]: arr=np.arange(10)
In [64]: arr
Out[64]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [65]: mask = [n%2==0 for n in arr]
In [66]: mask
Out[66]: [True, False, True, False, True, False, True, False, True, False]

Trying to index with this list:

In [67]: arr[mask]
/usr/local/bin/ipython3:1: FutureWarning: in the future, boolean array-likes will be handled as a boolean array index
  #!/usr/bin/python3
Out[67]: array([1, 0, 1, 0, 1, 0, 1, 0, 1, 0])

It's treating your list as a list of index integers, not booleans; same as:

In [72]: arr[[3,0,2,0,1,0,5,0,2]]
Out[72]: array([3, 0, 2, 0, 1, 0, 5, 0, 2])

I wondered why you used np.invert, but then realized that with a list, ~ doesn't work:

In [68]: arr[~mask]
...

TypeError: bad operand type for unary ~: 'list'

invert turns the list into an array and does the not

In [69]: np.invert(mask)
Out[69]: array([False,  True, False,  True, False,  True, False,  True, False,  True], dtype=bool)
In [70]: arr[np.invert(mask)]
Out[70]: array([1, 3, 5, 7, 9])

and we can not that array:

In [71]: arr[~np.invert(mask)]
Out[71]: array([0, 2, 4, 6, 8])

Or if I create the mask array right from the start:

In [73]: mask = np.array([n%2==0 for n in arr])
In [74]: arr[mask]
Out[74]: array([0, 2, 4, 6, 8])

So basically, don't try to use a boolean list as a mask. Use a boolean array.

Guillaume S · Accepted Answer · 2016-09-30 07:32:39Z

0

Note that you can (should) write:

arr = np.arange(n) # elements of arr have the same type as n (int, float, complex, etc)
arr = np.arange(n, dtype=int)
arr = np.arange(n, dtype=float)

depending on what type you want for arr.

answered Sep 30, 2016 at 7:32

Guillaume S

1902 silver badges4 bronze badges

Collectives™ on Stack Overflow

Numpy boolean indexing issue

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related