Pandas lambda function works only on single column not multiple

Question

I'm trying to apply a simple function (eliminating spaces) across multiple columns of a pandas DataFrame. However, while the .apply() method works properly on a single column, it doesn't work properly over multiple columns. Example:

#Weird Pandas behavior
######
#Input
df = pd.DataFrame ({'a' : ["7  7","5 3"],
                 'b' : ['f o', 'b  r'],
                 'c' : ["77","53"]})
print(df)

      a     b   c
0  7  7   f o  77
1   5 3  b  r  53

df[["a","b"]]=df[["a","b"]].apply(lambda x: x.replace(" ",""))
print(df)

      a     b   c
0  7  7   f o  77
1   5 3  b  r  53


df2=copy.deepcopy(df)
print(df2)

      a     b   c
0  7  7   f o  77
1   5 3  b  r  53

df2["a"]=df2["a"].apply(lambda x: x.replace(" ",""))
print(df2)

    a     b   c
0  77   f o  77
1  53  b  r  53

As you can see, df doesn't change at all when I try to apply the "replace" operation to two columns, but the same dataset (or rather a copy of it) does change when I run the same operation on a single column. How can I remove spaces from two or more columns at once using the .apply() syntax?

I tried passing in the arguments '[a]' (nothing happens) and 'list(a)' (nothing happens) to df[].

use a string function instead : df[['a','b']].apply(lambda x: x.str.replace(" ", "")) you are working on replace whitespace within a string — sammywemmy
– sammywemmy, Commented Dec 10, 2022 at 1:08

ScottC · Accepted Answer · 2022-12-10 01:22:59Z

3

You can use the replace function directly:

df[['a','b']] = df[['a','b']].replace(' ','', regex=True)

Output:

edited Dec 10, 2022 at 1:22

answered Dec 10, 2022 at 1:17

ScottC

4,1251 gold badge9 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Barmar · Accepted Answer · 2022-12-10 01:10:09Z

2

When you pass multiple columns, x is a pandas series, not the individual column values. You need to use .str.replace() to operate on each column.

df[["a","b"]]=df[["a","b"]].apply(lambda x: x.str.replace(" ",""))

answered Dec 10, 2022 at 1:10

Barmar

789k57 gold badges554 silver badges669 bronze badges

1 Comment

Holt Dwyer Over a year ago

Thanks! Why is it not a pandas series when I pass in one column e.g. df[["a"]]? And is there some way to get the contents of a pandas series instead of the series itself? I'm used to R where data tables have columns that are vectors of a single type, not series.

Collectives™ on Stack Overflow

Pandas lambda function works only on single column not multiple

2 Answers 2

Output:

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Output:

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related