How to iterate through a list of lists in Python and add values from other lists?

Question

I have a list:

my_list = [['ga:date'], ['ga:country', 'ga:date'], ['ga:country', 'ga:date']]

And try to iterate through it in order to get the values and its' positions, like so:

date    1
country 1
date    2
country 1
date    2

And store it all in a pandas DF.

As it was suggested I can do it like that, and it works perfectly:

Use list comprehension with enumerate and flattening for list of tuples:

my_list = [['ga:date'], ['ga:country', 'ga:date'], ['ga:country', 'ga:date']]

x = [(b, a) for i in my_list for (a, b) in enumerate(i, 1)]
print (x)
[('ga:date', 1), ('ga:country', 1), ('ga:date', 2), ('ga:country', 1), ('ga:date', 2)]

df = pd.DataFrame(x, columns = ['field','listIndex'])
print (df)
        field  listIndex
0     ga:date          1
1  ga:country          1
2     ga:date          2
3  ga:country          1
4     ga:date          2

Or if possible change position of columns:

x1 = [z for i in my_list for z in enumerate(i, 1)]
print (x1)
[(1, 'ga:date'), (1, 'ga:country'), (2, 'ga:date'), (1, 'ga:country'), (2, 'ga:date')]

df = pd.DataFrame(x1, columns = ['listIndex','field'])
print (df)
   listIndex       field
0          1     ga:date
1          1  ga:country
2          2     ga:date
3          1  ga:country
4          2     ga:date

But there are also 3 other lists, which I have to add to the resulted df.

my_id_list = ['01', '02', '03']

start_dates = ['2019-01-01', '2019-01-03', '2019-01-10']

end_dates = ['2019-01-02', '2019-01-05', '2019-01-11']

So it needs to look like that:

        field  listIndex   id start_date end_date
0     ga:date          1   01 2019-01-01 2019-01-02
1  ga:country          1   02 2019-01-03 2019-01-03
2     ga:date          2   02 2019-01-03 2019-01-03
3  ga:country          1   03 2019-01-10 2019-01-11
4     ga:date          2   03 2019-01-10 2019-01-11

Values can be different, there is no fix.

Would appreciate any help, I just want to end a project at work and forget it.

update

My id list contains of different int numbers. And they can differ, I mean, these 3 below are not the only ones.

my_id_list = ['115126931', '199714437', '197531387']

So it needs to look like that:

        field  listIndex   id        start_ date  end_date
0     ga:date          1   115126931 2019-01-01   2019-01-02
1  ga:country          1   199714437 2019-01-03   2019-01-03
2     ga:date          2   199714437 2019-01-03   2019-01-03
3  ga:country          1   197531387 2019-01-10   2019-01-11
4     ga:date          2   197531387 2019-01-10   2019-01-11

consider storing the index of each list while enumerating as a column called id . create another df with the 3 lists you have and merge on id which is the id_list — anky
– anky, Commented Aug 25, 2019 at 10:56

anky · Accepted Answer · 2019-08-26 06:58:54Z

1

You can try:

df=pd.DataFrame([(a,b,e) for e,i in enumerate(my_list) for (a, b) in enumerate(i, 1)],
                       columns=['list_index','feild','index_list_of_list'])
df1=pd.DataFrame(zip(map(int,my_id_list),start_dates,end_dates)
             ,columns=['id','startdate','enddate'])

df.merge(df1,left_on='index_list_of_list',right_index=True).drop('index_list_of_list',1)

   list_index       feild         id   startdate     enddate
0           1     ga:date  115126931  2019-01-01  2019-01-02
1           1  ga:country  199714437  2019-01-03  2019-01-05
2           2     ga:date  199714437  2019-01-03  2019-01-05
3           1  ga:country  197531387  2019-01-10  2019-01-11
4           2     ga:date  197531387  2019-01-10  2019-01-11

Note: also consider changing the dates to datetime by pd.to_datetime()

edited Aug 26, 2019 at 6:58

answered Aug 25, 2019 at 11:04

anky

75.3k11 gold badges46 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Anna Over a year ago

Cool, thanks, but there is a moment. Ids are not just numbers, they are pre defined already. I can't take any number as id.

anky Over a year ago

@AnnaDmitrieva so my_id_list is your id right? and it looks you are looking yo to the list index under the list of list for mapping,can you update your question with more details about how do you do the join?

Anna Over a year ago

Yeah, no problem I'll update it. my_id_list are ids, but they look like '115126931' this.

anky Over a year ago

@AnnaDmitrieva i have updated my answer, the idea is same, only the merge method changes

anky Over a year ago

@AnnaDmitrieva no problem, and the enumerator also changes for mylist, it should start from 0 since default index assigned to a dataframe is 0, so merging on right_index works :)

Collectives™ on Stack Overflow

How to iterate through a list of lists in Python and add values from other lists?

update

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

update

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related