How to replace values in a column in pandas using regex and a conditional

Question

Im trying to replace certain values in a pandas column (dataframe) using regex, but I want to apply the regex based on values in another column.

A basic example;

index  col1  col2
1      yes   foobar
2      yes   foo
3      no    foobar

Using the following;

df.loc[df['col1'] == 'yes', 'col2'].replace({r'(fo)o(?!bar)' :r'\1'}, inplace=True, regex=True)

I expected the following result;

index  col1  col2
1      yes   foobar
2      yes   fo
3      no    foobar

However it doesn't seem to be working? It doesn't throw any errors or a settingwithcopy warning, it just does nothing. Is there an alternative way to do this?

jezrael · Accepted Answer · 2018-08-30 07:04:23Z

4

For avoid chained assignments assign back and remove inplace=True:

mask = df['col1'] == 'yes'
df.loc[mask, 'col2'] = df.loc[mask, 'col2'].replace({r'(fo)o(?!bar)' :r'\1'}, regex=True)

print (df)
  col1    col2
1  yes  foobar
2  yes      fo
3   no  foobar

answered Aug 30, 2018 at 7:04

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Nordle Over a year ago

Thank you jez, as always. Will accept when it allows me

Jon Clements Over a year ago

Is there any reason to be using the DataFrame.replace method @Nordle? It seems that df['col2'] = df.col2[df.col1 == 'yes'].str.replace('(fo)o(?!bar)', r'\1') would suit your purposes here...

Nordle Over a year ago

@JonClements what's the difference between the 2 Jon? But no there is no particular reason I'm using df.replace

Jon Clements Over a year ago

@Nordle df.replace can be used to replace multiple columns at once with regexes or just translate one value to another using mappings... if you're just replacing a string in a single column, it's better to use Series.str.replace as above... jezrael may offer his opinion on that, but df.replace is a sledge hammer to crack a nut here...

Jon Clements Over a year ago

(and would be correctly be written along the lines of df.replace({'col2': '(fo)o(?!bar)'}, r'\1', regex=True) - might be an idea to read the docs for both and note the differences.

|

user3483203 · Accepted Answer · 2018-08-30 07:16:36Z

1

Using np.where:

df.assign(
    col2=np.where(df.col1.eq('yes'), df.col2.str.replace(r'(fo)o(?!bar)', r'\1'), df.col2)
)

  col1    col2
1  yes  foobar
2  yes      fo
3   no  foobar

edited Aug 30, 2018 at 7:16

answered Aug 30, 2018 at 7:08

user3483203

51.3k10 gold badges72 silver badges104 bronze badges

Collectives™ on Stack Overflow

How to replace values in a column in pandas using regex and a conditional

2 Answers 2

6 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related