How to Convert Pandas Dataframe to Single List

Question

Suppose I have a dataframe:

    col1    col2    col3
0    1       5       2
1    7       13
2    9       1
3            7

How do I convert to a single list such as:

[1, 7, 9, 5, 13, 1, 7]

I have tried:

df.values.tolist()

However this returns a list of lists rather than a single list:

[[1.0, 5.0, 2.0], [7.0, 13.0, nan], [9.0, 1.0, nan], [nan, 7.0, nan]]

Note the dataframe will contain an unknown number of columns. The order of the values is not important so long as the list contains all values in the dataframe.

I imagine I could write a function to unpack the values, however I'm wondering if there is a simple built-in way of converting a dataframe to a series/list?

The desired outcome list is missing 2 from col3. Is that desired, or an oversight? — Trenton McKinney
– Trenton McKinney, Commented Aug 23, 2019 at 3:42
Sorry, yes that's a typo. Should be [1, 7, 9, 5, 13, 1, 7, 2] as you pointed out. — Alan
– Alan, Commented Aug 23, 2019 at 6:25

busybear · Accepted Answer · 2019-08-23 03:37:18Z

16

Following your current approach, you can flatten your array before converting it to a list. If you need to drop nan values, you can do that after flattening as well:

arr = df.to_numpy().flatten()
list(arr[~np.isnan(arr)])

Also, future versions of Pandas seem to prefer to_numpy over values

An alternate, perhaps cleaner, approach is to 'stack' your dataframe:

df.stack().tolist()

edited Aug 23, 2019 at 3:37

answered Aug 23, 2019 at 3:28

busybear

10.7k1 gold badge29 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Roushan · Accepted Answer · 2019-08-23 03:47:34Z

2

you can use dataframe stack

In [12]: df = pd.DataFrame({"col1":[np.nan,3,4,np.nan], "col2":['test',np.nan,45,3]})

In [13]: df.stack().tolist()
Out[13]: ['test', 3.0, 4.0, 45, 3]

edited Aug 23, 2019 at 3:47

answered Aug 23, 2019 at 3:36

Roushan

4,4914 gold badges25 silver badges39 bronze badges

Comments

Learner13 · Accepted Answer · 2019-08-23 07:48:29Z

1

For Ordered list (As per problem statement):
Only if your data contains integer values:

Firstly get all items in data frame and then remove the nan from the list.

items = [item for sublist in [df[cols].tolist() for cols in df.columns] for item in sublist]
items = [int(x) for x in items if str(x) != 'nan']

For Un-Ordered list:
Only if your data contains integer values:

items = [int(x) for x in sum(df.values.tolist(),[]) if str(x) != 'nan']

answered Aug 23, 2019 at 7:48

Learner13

539 bronze badges

Collectives™ on Stack Overflow

How to Convert Pandas Dataframe to Single List

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related