Pandas 0.24 replace regex issue

Question

With pandas 0.19.2 python 3.6.0 DataFrame.replace with a dictionary acts on substrings (like "find"), and so does Series.replace. Pandas 0.24.0 python 3.6.8 seems to act on the entire string (like "match") for DataFrames, and still act on substrings for Series (like "find").

df = pd.DataFrame({'c1':['AD','BD'],'c2':['AD','BD']})
print(df)
print(df.replace(to_replace={'c1':{r'D': ''}, 'c2':{r'BD': ''}},regex=True))
print(df.replace(to_replace={r'D': ''},regex=True))
print(df['c1'].replace(to_replace=r'D', value='',regex=True))

Pandas 0.19.2 produces (I added some blank lines for legibility):

   c1  c2
0  AD  AD
1  BD  BD

  c1  c2
0  A  AD
1  B    

  c1 c2
0  A  A
1  B  B

0    A
1    B
Name: c1, dtype: object

With Pandas 0.24.0:

   c1  c2
0  AD  AD
1  BD  BD

   c1  c2
0  AD  AD
1  BD    

   c1  c2
0  AD  AD
1  BD  BD

0    A
1    B
Name: c1, dtype: object

Looks like a pandas bug to me, or am I missing something?

Vaishali, the thread you link is concerning a somewhat different issue, though it's clearly related. As you can see my example does not use integers at all, which is what the linked issue is about. This issue here is about regex now matching the whole string ("match") vs matching a substring ("find"). All this is for strings, no integers involved. — Uwe Mayer
– Uwe Mayer, Commented Feb 5, 2019 at 18:52

Wiktor Stribiżew · Accepted Answer · 2019-03-19 19:15:45Z

1

The bug is listed among the Fixed regressions for Pandas 0.24.2:

Fixed regression in DataFrame.replace() where regex=True was only replacing patterns matching the start of the string (GH25259)

As you see, only

print(df.replace(to_replace={'c1':{r'D': ''}, 'c2':{r'BD': ''}},regex=True))
print(df.replace(to_replace={r'D': ''},regex=True))

did not work correctly. Now, the issue is fixed.

answered Mar 19, 2019 at 19:15

Wiktor Stribiżew

631k41 gold badges502 silver badges632 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Uwe Mayer Over a year ago

Thanks Wiktor, indeed verified as fixed.

Collectives™ on Stack Overflow

Pandas 0.24 replace regex issue

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related