Pandas: create a Dataframe from lists with similar-size lists inside

Question

I am trying to construct a Dataframe from lists, which have similar-size lists inside.

In the example below I have 3 lists (with similar-size lists inside), which I want to use to construct an specific Dataframe.

Here are the lists:

list_type = [['A1', 'A2'], ['A2'], ['A1', 'A3', 'A4']]

list_xx = [['1', '5'], ['3'], ['2', '7', '3']]

list_date = [['Jan', 'Jan'], ['Feb'], ['Mar', 'Mar', 'Mar']]

And the Dataframe I want to construct is:

   date type    xx
0   Jan   A1     1
1   Jan   A2     5
2   Feb   A2     3
3   Mar   A1     2
4   Mar   A3     7
5   Mar   A4     3

As you can see, the structure of the Dateframe is based on the inner-lists.

I would really value an efficient way of doing this in Python.

Quang Hoang · Accepted Answer · 2020-02-16 13:55:43Z

2

You can make do with concat and list comprehension:

pd.concat(pd.DataFrame({'date':d,'type':t,'xx':xx })
          for d,t,xx in zip(list_date, list_type, list_xx))

Or np.concatenate with transpose:

pd.DataFrame([np.concatenate(l) for l in (list_date,list_type,list_xx)],
             index=['date','type','xx']).T

Output:

  date type xx
0  Jan   A1  1
1  Jan   A2  5
0  Feb   A2  3
0  Mar   A1  2
1  Mar   A3  7
2  Mar   A4  3

answered Feb 16, 2020 at 13:55

Quang Hoang

151k11 gold badges64 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

totalhack · Accepted Answer · 2020-02-16 14:11:26Z

2

An alternative would be to flatten the lists first with itertools.chain.from_iterable, instead of concatenating DataFrames as in this answer.

import itertools
import pandas as pd

list_type = [['A1', 'A2'], ['A2'], ['A1', 'A3', 'A4']]
list_xx = [['1', '5'], ['3'], ['2', '7', '3']]
list_date = [['Jan', 'Jan'], ['Feb'], ['Mar', 'Mar', 'Mar']]

df = pd.DataFrame({
    "date": itertools.chain.from_iterable(list_date),
    "type": itertools.chain.from_iterable(list_type),
    "xx": itertools.chain.from_iterable(list_xx)
})

This gives your desired output. I would expect this to be more efficient than building and concatenating sub DataFrames, but haven't explicitly tested the two.

answered Feb 16, 2020 at 14:11

totalhack

2,6781 gold badge24 silver badges28 bronze badges

1 Comment

totalhack Over a year ago

Excellent, please accept my answer if you end up going this route.

Collectives™ on Stack Overflow

Pandas: create a Dataframe from lists with similar-size lists inside

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related