Pandas : Create multiple new variables by applying function to dataframe

Question

Consider the following simple function:

def Powers(x):
    return [x, x**2, x**3, x**4, x**5]

and input dataframe:

df = pd.DataFrame({ 'x':(1, 2, 3, 4, 5) })

I would like to generate new variables: ['Exp_1', 'Exp_2', 'Exp_3', 'Exp_4', 'Exp_5']

When I apply the function to the dataframe as follows:

df[['Exp_1', 'Exp_2', 'Exp_3', 'Exp_4', 'Exp_5']] = df.apply(lambda x: Powers(x.x), axis=1)

I get:

In other words, the values are transposed. That is, the 5th exponent of 1 is 1 not 5 and the 1st exponent of 5 is 5 and not 1.

I have tried axis=0, in the call above and this does not work either. I also know I have a problem because if the input dataframe is of a different length I get errors.

How do I fix this?

not sure if Powers is only an example or not but for some speed if you care about it, you can go for np.vander(df.x, len(df) + 1, increasing=True); this generates a Vandermonde matrix numpy.org/doc/stable/reference/generated/numpy.vander.html. — Mustafa Aydın
– Mustafa Aydın, Commented Aug 13, 2022 at 15:43

Ynjxsjmh · Accepted Answer · 2022-08-13 15:41:21Z

3

You can return Series in Powers function

def Powers(x):
    return pd.Series([x, x**2, x**3, x**4, x**5])

df[['Exp_1', 'Exp_2', 'Exp_3', 'Exp_4', 'Exp_5']] = df.apply(lambda x: Powers(x.x), axis=1)

print(df)

   x  Exp_1  Exp_2  Exp_3  Exp_4  Exp_5
0  1      1      1      1      1      1
1  2      2      4      8     16     32
2  3      3      9     27     81    243
3  4      4     16     64    256   1024
4  5      5     25    125    625   3125

Or use result_type in DataFrame.apply

def Powers(x):
    return [x, x**2, x**3, x**4, x**5]

df[['Exp_1', 'Exp_2', 'Exp_3', 'Exp_4', 'Exp_5']] = df.apply(lambda x: Powers(x.x), axis=1, result_type='expand')
# or
df[['Exp_1', 'Exp_2', 'Exp_3', 'Exp_4', 'Exp_5']] = df.apply(lambda x: Powers(x.x), axis=1).tolist()

edited Aug 13, 2022 at 15:41

answered Aug 13, 2022 at 15:35

Ynjxsjmh

30.3k7 gold badges43 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

brb Over a year ago

Thanks, that's great. I will experiment with both, but let's say the function will sometimes be used with a scalar and sometimes applied to a dataframe, is option 2 preferred?

Ynjxsjmh Over a year ago

@brb Don't quite know what you mean applied to a dataframe. If you mean what you do in this example, it's actually same with df['x'].apply(Powers). Both ways are ok, depending on which you like.

brb Over a year ago

Sorry, I mean sometimes I might want to do results = Powers(3) and before results would be a list. I am not sure I want to return a pd.Series in this case. So your second way of doing result_type='Expand' maybe better as it can be used with a scalar input or used by applying to a dataframe?

Ynjxsjmh Over a year ago

@brb In your results = Powers(3), I think second way is better.

brb Over a year ago

Thank you. Appreciate your help. Helping me learn.

Collectives™ on Stack Overflow

Pandas : Create multiple new variables by applying function to dataframe

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related