Pandas dataframe: creating a new column that is a custom function using 2 other columns

Question

Consider the following data set stored in a pandas DataFrame dfX:

I have a function that is:

def someThingSpecial(x,y)
  # z = do something special with x,y
  return z

I now want to create a new column in df that bears the computed z value

Looking at other SO examples, I've tried several variants including:

dfX['C'] = dfX.apply(lambda x: someThingSpecial(x=x['A'], y=x['B']), axis=1)

Which returns errors. What is the right way to do this?

Good question. In trying to answer your point, I realized my lambda was correct - the error was in someThingSpecial and I got lost in the python error logs. — user1361529
– user1361529, Commented Dec 13, 2017 at 19:37

cs95 · Accepted Answer · 2017-12-13 19:26:37Z

8

This seems to work for me on v0.21. Take a look -

df

   A  B
0  1  2
1  4  6
2  7  9

def someThingSpecial(x,y):
     return x + y


df.apply(lambda x: someThingSpecial(x.A, x.B), 1)

0     3
1    10
2    16
dtype: int64

You might want to try upgrading your pandas version to the latest stable release (0.21 as of now).

Here's another option. You can vectorise your function.

v = np.vectorize(someThingSpecial)

v now accepts arrays, but operates on each pair of elements individually. Note that this just hides the loop, as apply does, but is much cleaner. Now, you can compute C as so -

df['C'] = v(df.A, df.B)

answered Dec 13, 2017 at 19:26

cs95

406k106 gold badges744 silver badges797 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user1361529 Over a year ago

That works well, thank you! I like the elegance. As it turned out my original approach also worked - the error was in "somethingSpecial" but I changed to yours.

cs95 Over a year ago

@user1361529 You're welcome. I've also found performance improvements with vectorize. Not sure what it does, but it's a real goto of mine.

Trevor · Accepted Answer · 2019-01-13 05:38:28Z

3

if your function only needs one column's value, then do this instead of coldspeed's answer:

dfX['A'].apply(your_func)

to store it:

dfX['C'] = dfX['A'].apply(your_func)

answered Jan 13, 2019 at 5:38

Trevor

311 bronze badge

Collectives™ on Stack Overflow

Pandas dataframe: creating a new column that is a custom function using 2 other columns

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related