extract pandas object into list of lists and extract unique values

Question

So I've got a dictionary of files stored as pandas dataframe objects, and I'm accessing each file through a for loop to extract the 'Country' column. What I'm looking to do is extract each of these into a list and then take the set of the entire list of lists. Here is the code and my predicament:

    country_setter = []
        for file in files_list:
        country_setter.append(all_comps[file]['Country'].tolist())

    uni_country_setter = ?

The resulting output is a list of lists, with each pandas df ['Country'] column taking a list within the parent list. It looks like this:

[['France',
  'United States',
  'Poland',
  'Poland',
  'Poland',
  'Poland',
  'Hungary',
  'Poland',
  'France',
  'United Kingdom',
    ....
  'Namibia',
  'China',
  'China',
  'Ireland'],
 ['Netherlands',
  'Canada',
  'United States',
  'Canada',
  'Canada',
  'United States',
  'Sweden',
  'Sweden',
  'United Kingdom',
   ....
  'Ireland',
  'Netherlands',
  'Netherlands',
  'France',
  'Hong Kong',
  'France',
  'France',
  'United States',
  'France',
  'United States']]

It's a list with 40 individual lists within it. I can take the set(country_setter[0]) and that works fine in getting me the unique values of the first list, but I need to know the unique values of all files in conjunction.

Let me know if any of you can help. I've pored through stackoverflow and only found one question slightly similar, but they're goal was to maintain the list structure in the unique extraction and used itertools. I want the unique individual values across all of the lists here.

Thank you in advance!

Can you add data sample?

jezrael
– jezrael

2017-10-14 19:08:00 +00:00
Commented Oct 14, 2017 at 19:08 — jezrael
– jezrael, Commented Oct 14, 2017 at 19:08
Sure, I'll give the structure.

fattmagan
– fattmagan

2017-10-14 19:16:00 +00:00
Commented Oct 14, 2017 at 19:16 — fattmagan
– fattmagan, Commented Oct 14, 2017 at 19:16
@jezrael does that help?

fattmagan
– fattmagan

2017-10-14 19:21:56 +00:00
Commented Oct 14, 2017 at 19:21 — fattmagan
– fattmagan, Commented Oct 14, 2017 at 19:21
Not 100% sure, but you need unique values form all lists?

jezrael
– jezrael

2017-10-14 19:27:28 +00:00
Commented Oct 14, 2017 at 19:27 — jezrael
– jezrael, Commented Oct 14, 2017 at 19:27
yes - I need unique values across all of the lists.

fattmagan
– fattmagan

2017-10-14 19:32:14 +00:00
Commented Oct 14, 2017 at 19:32 — fattmagan
– fattmagan, Commented Oct 14, 2017 at 19:32

jezrael · Accepted Answer · 2017-10-14 19:43:26Z

1

I think you need flatten lists and then create unique list by set:

uni_country_setter = list(set([item for sublist in country_setter for item in sublist]))

EDIT:

First loop is not necessary, is possible use:

uni_country_setter = list(set([item for file in files_list 
                               for item in all_comps[file]['Country'].tolist()]))

edited Oct 14, 2017 at 19:43

answered Oct 14, 2017 at 19:23

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

fattmagan Over a year ago

Thank you! I don't think I could have figured that on my own. Can you explain the logic behind that double "for" call? Are you defining each sublist and then iterating through them?

jezrael Over a year ago

Maybe better explanation is here for flatenning.

Collectives™ on Stack Overflow

extract pandas object into list of lists and extract unique values

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related