Pandas dataframe select rows with multiple columns' string conditions

Question

I have a similar dataframe:

df = pd.DataFrame([{'year':2017, 'text':'yes it is', 'label_one':'POSITIVE', 'label_two':'positive'}, 
{'year':2017, 'text':'it could be', 'label_one':'POSITIVE', 'label_two':'negative'},
{'year':2017, 'text':'it may be', 'label_one':'NEGATIVE', 'label_two':'positive'},
{'year':2018, 'text':'it has to be done', 'label_one':'POSITIVE', 'label_two':'positive'},
{'year':2018, 'text':'no', 'label_one':'NEGATIVE', 'label_two':'negative'},
{'year':2019, 'text':'you should be afraid of it', 'label_one':'POSITIVE', 'label_two':'negative'},
{'year':2019, 'text':'he is right', 'label_one':'POSITIVE', 'label_two':'positive'},
{'year':2020, 'text':'do not mind, I wil fix it', 'label_one':'NEGATIVE', 'label_two':'positive'},
{'year':2020, 'text':'that is a trap', 'label_one':'NEGATIVE', 'label_two':'negative'},
{'year':2021, 'text':'I am on my way', 'label_one':'POSITIVE', 'label_two':'positive'}])

How can I filter it in order to have just rows for which label_one and label_two string values are both POSITIVE/positive or NEGATIVE/negative

I tried with the following but it does not work:

ptp = df.loc[(df['label_one'].str.startswith('P') and df['label_two'].str.startswith('p')) & (df['label_one'].str.startswith('N') and df['label_two'].str.startswith('n'))]

Stefan · Accepted Answer · 2021-01-30 18:25:38Z

4

What about

df[df['label_one'].str.lower() == df['label_two'].str.lower()]

assuming that label_one and label_two only holds negative, positive, NEGATIVE or POSITIVE.

edited Jan 30, 2021 at 18:25

answered Jan 30, 2021 at 18:20

Stefan

1,96222 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Paul Brennan · Accepted Answer · 2021-01-30 18:22:44Z

2

This works. Following your pattern both start with P/p or N/n

ptp = df.loc[((df['label_one'].str.startswith('P')) &
              (df['label_two'].str.startswith('p'))) |          
             ((df['label_one'].str.startswith('N')) &        
              (df['label_two'].str.startswith('n')))]

gives

PTP
        year    text                label_one   label_two
    0   2017    yes it is           POSITIVE    positive
    3   2018    it has to be done   POSITIVE    positive
    4   2018    no                  NEGATIVE    negative
    6   2019    he is right         POSITIVE    positive
    8   2020    that is a trap      NEGATIVE    negative
    9   2021    I am on my way      POSITIVE    positive

answered Jan 30, 2021 at 18:22

Paul Brennan

2,7364 gold badges23 silver badges27 bronze badges

Collectives™ on Stack Overflow

Pandas dataframe select rows with multiple columns' string conditions

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related