create a tuple from columns in a pandas DataFrame

Question

I would like to automatically create a tuple (to be passed to a scipy.stats function) from columns in a pandas dataframe, so that each row of the tuple are the values from each column of the dataframe. here is the header from my dataframe:

                     4_3-a-0    5_3-a-4    7_3-a-3
datetime_pac                                      
2015-09-03 22:00:00   -100.4 -96.857143 -55.000000
2015-09-03 22:01:00   -100.5 -91.700000 -55.600000
2015-09-03 22:02:00   -100.4 -90.875000 -55.900000
2015-09-03 22:03:00   -100.4 -94.000000 -55.555556
2015-09-03 22:04:00   -100.5 -99.500000 -55.545455

I can achieve this manually like so:

from scipy import stats
stats.f_oneway(df.ix[:,0], df.ix[:,1], df.ix[:,2])

But I would like to 'automate' it in cases where the number of columns in the dataframe is unknown. The following attempts (and many variations of) would not work:

stats.f_oneway(tuple(x) for x in xtmp.values)
stats.f_oneway((xtmp[x]) for x in xtmp.columns)

Thanks for your help!

I found the answer in another post: stats.f_oneway(*df.values) — Nicole Goebel
– Nicole Goebel, Commented Sep 11, 2015 at 10:19

EdChum · Accepted Answer · 2015-09-09 18:39:41Z

9

Just call apply and call tuple:

In [3]:
df = pd.DataFrame(np.random.randn(5,3))
df

Out[3]:
          0         1         2
0  0.785562 -0.263813  2.239865
1  1.083918  0.035746  0.429111
2  1.422599 -0.818151  0.765725
3  1.022289  0.098561 -2.393095
4 -0.548451 -0.345796  0.298237

In [4]:
df.apply(tuple, axis=1)

Out[4]:
0     (0.785562108573, -0.263813112223, 2.23986497964)
1     (1.08391788685, 0.0357457180803, 0.429110675053)
2      (1.4225989372, -0.818150896781, 0.765724984713)
3     (1.02228880387, 0.0985610274998, -2.39309469576)
4    (-0.548450748411, -0.345796089243, 0.298237353...
dtype: object

answered Sep 9, 2015 at 18:39

EdChum

397k204 gold badges836 silver badges583 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Nicole Goebel Over a year ago

Thanks for the new approach. While this certainly puts columns into a tuple (with axis=0), the stats.f_oneway() throws the error: 'ValueError: setting an array element with a sequence.' This may be because the output is actually a series and not a tuple.

EdChum Over a year ago

Sorry but I can't coding problems based on the errors, you need to show your code, also doesn't new_df = df.apply(tuple) for col in new_df: print(stats.f_oneway(col)) do what you want?

Nicole Goebel Over a year ago

IN the end, I needed to unpack the variables and pass them to the function. Thanks again for your input!

hellpanderr · Accepted Answer · 2015-09-09 17:31:19Z

0

What about

tuple([tuple(df[col]) for col in df])

answered Sep 9, 2015 at 17:31

hellpanderr

5,9563 gold badges42 silver badges50 bronze badges

4 Comments

Nicole Goebel Over a year ago

Thanks for this suggestion. It does create a tuple in the correct format but when fed to stats.f_oneway() it returns f and p values of the same length as each row of the tuple. This is the case if I were to pass a tuple (tup=df.ix[:,0], df.ix[:,1], df.ix[:,2]) to the function as stats.f_oneway(tup) instead of stats.f_oneway(df.ix[:,0], df.ix[:,1], df.ix[:,2]) - the latter provides the correct f and p values.

hellpanderr Over a year ago

That's because you need to unpack the tuple into positional arguments: stats.f_oneway(*tuple([tuple(df[col]) for col in df])) or stats.f_oneway(*df.apply(tuple, axis=0))

hellpanderr Over a year ago

Or even shorter stats.f_oneway(*df.T.values)

Nicole Goebel Over a year ago

Thanks very much for that. As it turns out I did the last of your suggestions where I feed in the dataframe by unpacking with '*', which was something I did not understand previously. No need to convert my dataframe to a tuple!

Collectives™ on Stack Overflow

create a tuple from columns in a pandas DataFrame

2 Answers 2

3 Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related