Filter Pandas Dataframe that is an array

Question

I have a data frame that I'd like to filter by a column that is of type array. What is the most effective way to do this?

df = pd.DataFrame({'a': [1,2,3,4,5], 'b': [['true','false'],['false'],['false','false','false'],['false','false','true'],[]]})
df

    a   b
0   1   [true, false]
1   2   [false]
2   3   [false, false, false]
3   4   [false, false, true]
4   5   []

I'd ideally like to only return rows that contain a true value.

array is not a dtype. There no real effective way to work with lists in pandas.DataFrame's, but you could always do something like df[df.b.apply(lambda x: 'true' in x)] — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Jan 30, 2018 at 1:14
@pault well, if they are actualy numpy.ndarray objects instead of list objects, maybe slightly, but the time sink is iterating over the rows, which is necessitated in this case. Furthermore, those arrays would be dtype=object anyway, so iteration would still be slow — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Jan 30, 2018 at 1:21

BENY · Accepted Answer · 2018-01-30 01:37:44Z

4

Without loop :-)

df[pd.DataFrame(df.b.tolist()).eq('true').any(1)]
Out[98]: 
   a                     b
0  1         [true, false]
3  4  [false, false, true]

answered Jan 30, 2018 at 1:37

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

cs95 Over a year ago

Wonderful answer... very impressive.

Collectives™ on Stack Overflow

Filter Pandas Dataframe that is an array

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related