How do I select a row from a complex numpy array?

Question

I have read 10-20 different question/answers and cannot find an example like this. I want to select rows out of a numpy array as follows:

test = [ [ [0], np.zeros((250,250)), np.array([0,0]) ],
         [ [0], np.zeros((250,250)), np.array([1,1]) ],
         [ [1], np.zeros((250,250)), np.array([2,2]) ],
         [ [2], np.zeros((250,250)), np.array([2,2]) ],
         [ [2], np.zeros((250,250)), np.array([2,2]) ]
       ]

Now convert the list to an numpy array and print the first column:

nptest = np.array(test)
print (nptest[:,0])
> output is: [[0] [0] [1] [2] [2]]

Now try to select just the row where the first element is = 1

just_1s = nptest[nptest[:,0] == 1]
print (just_1s)
> output is []

I don't understand this output.

In my actual problem set I have 100s of each with an arbitrary number of rows with values of 0-15 in the first column. Using the example data above, the desired result would be three numpy arrays as follows:

just_0s = [[ [0], np.zeros((250,250)), np.array([0,0]) ],
           [ [0], np.zeros((250,250)), np.array([1,1]) ]
          ]

just_1s = [[ [1], np.zeros((250,250)), np.array([2,2]) ]]

just_2s = [[ [2], np.zeros((250,250)), np.array([2,2]) ],
           [ [2], np.zeros((250,250)), np.array([2,2]) ]
          ]

When I print nptest[:,0] I get array([list([0]), list([0]), list([1]), list([2]), list([2])], dtype=object), None of those things equal 1, hence your empty array. — o-90
– o-90, Commented Jul 31, 2017 at 21:30
So do I just have to do a for loop through the numpy array? It seems like there should be a way to select these rows based on the value in that column... — Brian F
– Brian F, Commented Jul 31, 2017 at 21:35

keepAlive · Accepted Answer · 2017-08-24 23:53:21Z

1

Use list comprehension

just_1s = [el for el in nptest if el[0] == [1]]

But there is actually no need to work with nd.arrays, using the original list is ok

just_1s = [el for el in test if el[0] == [1]]

If your point is why does one get [] when doing nptest[nptest[:, 0] == [1]] (note that I test [1] instead of 1), I do not know. Because according to me (and Sam Marinelli) it should have worked. We are wrong.

But if one looks closer, it appears that

just_1s = nptest[ nptest[:,0].tolist().index([1]) ]

works fine but only if [1] is unique. Which is not the case of, e.g. [2].

edited Aug 24, 2017 at 23:53

answered Jul 31, 2017 at 22:03

keepAlive

6,7055 gold badges29 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Brian F Over a year ago

I left for a while, but I had already tried the [1] thought. I am going to try the use "list comprehension" approach and will come back to plus 1 if it does the trick. It looks like it will. I do want a numpy array at the end of the rainbow, but I can do that after I separate the list into its components. Back in a few...

Brian F Over a year ago

Thanks to all but, special thanks to Kanak. The list comprehension on the original list is the best of the solutions. Thanks for the rapid help and response!

hpaulj · Accepted Answer · 2017-07-31 23:10:15Z

1

This is list produces a (5,3) array of objects:

In [47]: nptest=np.array(test)
In [49]: nptest.shape
Out[49]: (5, 3)
In [50]: nptest.dtype
Out[50]: dtype('O')
In [51]: nptest[:,0]
Out[51]: array([list([0]), list([0]), list([1]), list([2]), list([2])], dtype=object)

The first column is an array (1d) of lists. In my newer numpy version that's more explicit.

Doing an equality test on an array, or even list, of lists is not easy. After trying several things I found this:

In [52]: arow = nptest[:,0]

In [56]: [x[0]==1 for x in arow]
Out[56]: [False, False, True, False, False]
In [57]: mask = [x[0]==1 for x in arow]
In [58]: nptest[mask,:]
Out[58]: 
array([[list([1]),
        array([[ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       ..., 
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.]]),
        array([2, 2])]], dtype=object)

Or we could turn the array of lists into an array of numbers:

In [60]: np.array(arow.tolist())
Out[60]: 
array([[0],
       [0],
       [1],
       [2],
       [2]])
In [61]: np.array(arow.tolist())==1
Out[61]: 
array([[False],
       [False],
       [ True],
       [False],
       [False]], dtype=bool)

Or test for [1] instead of 1. Lists match lists, not their contents.

In [64]: [x==[1] for x in arow]
Out[64]: [False, False, True, False, False]

edited Jul 31, 2017 at 23:10

answered Jul 31, 2017 at 23:03

hpaulj

233k14 gold badges260 silver badges392 bronze badges

1 Comment

keepAlive Over a year ago

This may be considered as an answer to my question ?

Sam Marinelli · Accepted Answer · 2017-07-31 21:38:03Z

0

I believe the issue is with the expression nptest[:, 0] == 1. You've already shown that nptest[:, 0] returns [[0], [0], [1], [2], [2]] as expected. You can see that none of these values is equal to 1, so nptest[nptest[:, 0] == 1] will be empty. Try nptest[nptest[:, 0] == [1]] instead.

answered Jul 31, 2017 at 21:38

Sam Marinelli

1,1391 gold badge9 silver badges18 bronze badges

Collectives™ on Stack Overflow

How do I select a row from a complex numpy array?

3 Answers 3

2 Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related