renaming column in dataframe for Pandas using regular expression

Question

I have a dataframe made by Pandas that I want to remove the empty space at the end of each column name. I tried something like:

raw_data.columns.values = re.sub(' $','',raw_data.columns.values)

But this is not working, anything I did wrong here?

If the empty space is at the end of the column names, shouldn't it be re.sub(' $', '')? — Marius
– Marius, Commented Oct 22, 2014 at 4:06
sorry for the wrong regex there, I tried the corrected one but python responded type error — lokheart
– lokheart, Commented Oct 22, 2014 at 4:07

Timur Shtatland · Accepted Answer · 2023-12-07 20:06:23Z

55

I should have used rename with re.sub:

import re
raw_data = raw_data.rename(columns=lambda x: re.sub(' $','',x))

Or to change in-place:

raw_data.rename(columns=lambda x: re.sub(' $','',x), inplace=True)

edited Dec 7, 2023 at 20:06

Timur Shtatland

12.8k3 gold badges41 silver badges68 bronze badges

answered Oct 22, 2014 at 4:17

lokheart

24.9k40 gold badges108 silver badges175 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

vks Over a year ago

This will only remove 1 space.Use this if you want to remove all.raw_data.columns.values = re.sub(r'[ ]*$','',raw_data.columns.values)

jvalenti Over a year ago

would this work if the resulting column names were non-unique?

Prebsus Over a year ago

I tried my hand at looping through columns [for col in df.columns], using regex to identify what I needed to remove, and then re-name the columns one-by-one, which worked just fine, but was very slow. Your solution is much more performant! (for context: I was stripping a column-counter, i.e. (1), (2), ... (300), so that's why I needed regex)

renan-eccel · Accepted Answer · 2020-04-23 20:15:16Z

10

I would recommend using pandas.Series.str.strip

df.columns = df.columns.str.strip()

edited Apr 23, 2020 at 20:15

renan-eccel

19212 bronze badges

answered Feb 27, 2018 at 16:09

Christian

6051 gold badge9 silver badges19 bronze badges

Comments

José · Accepted Answer · 2019-02-05 09:40:38Z

5

The answer from @Christian is probably right for this specific question, but to the more general question about replacing names in the columns, I would suggest to create a dictionary comprehension and pass it to the rename function:

df.rename(columns={element: re.sub(r'$ (.+)',r'\1', element, flags = re.MULTILINE) for element in df.columns.tolist()})

In my case, I wanted to add something to the beginning of each column, so:

df.rename(columns={element: re.sub(r'(.+)',r'x_\1', element) for element in df.columns.tolist()})

You can use the inplace=True parameter to actually make the change in the dataframe.

answered Feb 5, 2019 at 9:40

José

5831 gold badge5 silver badges17 bronze badges

Collectives™ on Stack Overflow

renaming column in dataframe for Pandas using regular expression

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related