pandas DataFrame filter regex

Question

I don't understand pandas DataFrame filter.

Setup

import pandas as pd

df = pd.DataFrame(
    [
        ['Hello', 'World'],
        ['Just', 'Wanted'],
        ['To', 'Say'],
        ['I\'m', 'Tired']
    ]
)

Problem

df.filter([0], regex=r'(Hel|Just)', axis=0)

I'd expect the [0] to specify the 1st column as the one to look at and axis=0 to specify filtering rows. What I get is this:

       0      1
0  Hello  World

I was expecting

       0       1
0  Hello   World
1   Just  Wanted

Question

What would have gotten me what I expected?

sync11 · Accepted Answer · 2017-09-14 10:29:57Z

18

Per the docs,

Arguments are mutually exclusive, but this is not checked for

So, it appears, the first optional argument, items=[0] trumps the third optional argument, regex=r'(Hel|Just)'.

In [194]: df.filter([0], regex=r'(Hel|Just)', axis=0)
Out[194]: 
       0      1
0  Hello  World

is equivalent to

In [201]: df.filter([0], axis=0)
Out[201]: 
       0      1
0  Hello  World

which is merely selecting the row(s) with index values in [0] along the 0-axis.

To get the desired result, you could use str.contains to create a boolean mask, and use df.loc to select rows:

In [210]: df.loc[df.iloc[:,0].str.contains(r'(Hel|Just)')]
Out[210]: 
       0       1
0  Hello   World
1   Just  Wanted

edited Sep 14, 2017 at 10:29

sync11

1,3102 gold badges10 silver badges23 bronze badges

answered May 6, 2016 at 20:21

unutbu

886k197 gold badges1.9k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Max · Accepted Answer · 2016-05-06 20:24:59Z

12

This should work:

df[df[0].str.contains('(Hel|Just)', regex=True)]

answered May 6, 2016 at 20:24

Max

4583 silver badges10 bronze badges

Comments

Ramin Melikov · Accepted Answer · 2020-04-20 05:32:26Z

1

Here is a chaining method:

df.loc[lambda x: x['column_name'].str.contains(regex_patern, regex = True)]

answered Apr 20, 2020 at 5:32

Ramin Melikov

1,03510 silver badges15 bronze badges

Collectives™ on Stack Overflow

pandas DataFrame filter regex

Setup

Problem

Question

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Setup

Problem

Question

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related