Multiple regex replacements in a loop with pandas

Question

I am relatively new to regex and I am trying to replace part of the string inside of the string column in Pandas DataFrame. The challenge is that I have multiple string types that I want to remove from my column while keeping the rest of the string.

I have code working for 1 type of string, but when I try to use for loop, the code is not working. I am not sure how to specify iterator inside of the regex expression.

Here is code that works when applied to 1 type of sub-string:

df = pd.DataFrame({'A': ['ba ca t', 'foo', 'bait'],'B': ['abc', 'bar', 'xyz']})
df
df=df.replace({'A': r'^ba ca'}, {'A': ''}, regex=True)
df

Here is code that is not working when I try to us For Loop:

df = pd.DataFrame({'A': ['ba ca t', 'foo', 'bait'],'B': ['abc', 'bar', 'xyz']})
replace_list=['ba ca','foo']
for i in replace_list:
    df=df.replace({'A': r'^(i)'}, {'A': ''}, regex=True)
df

I would like to iterate over a list of strings to remove them from a column in the DataFrame.

Are you trying to remove those patterns (i.e., replace with empty string '')? — cs95
– cs95, Commented Jun 20, 2019 at 21:38
yes, I am trying to remove them (replace with empty string). — Sveta
– Sveta, Commented Jun 20, 2019 at 21:41

cs95 · Accepted Answer · 2019-06-20 21:41:37Z

3

'^(i)' is not the correct method of performing string interpolation. You're looking for something along the lines of f-string formatting (rf'^{i}') or str.format (r'^{}'.format(i)).

Although a better solution here would be to ditch the loop, since replace allows you to perform multiple replacements at once.

df.replace({'A': replace_list}, '', regex=True)

      A    B
0     t  abc
1        bar
2  bait  xyz

Or, with str.replace:

df['A'].str.replace('|'.join(replace_list), '')

0       t
1        
2    bait
Name: A, dtype: object

This post by me should also be worth a read: What is the difference between Series.replace and Series.str.replace?

answered Jun 20, 2019 at 21:41

cs95

406k106 gold badges744 silver badges797 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sebastien D · Accepted Answer · 2019-06-20 21:41:02Z

2

Since you wan't i to modify your regex pattern, you should consider this change:

 df=df.replace({'A': r'^({})'.format(i)}, {'A': ''}, regex=True)

Output

+----+-------+-----+
|    |  A    |  B  |
+----+-------+-----+
| 0  | t     | abc |
| 1  |       | bar |
| 2  | bait  | xyz |
+----+-------+-----+

answered Jun 20, 2019 at 21:41

Sebastien D

4,5024 gold badges23 silver badges50 bronze badges

Collectives™ on Stack Overflow

Multiple regex replacements in a loop with pandas

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related