Python List of list, remove duplicates

Question

Trying to remove duplicates in list of list and print same without duplicates.

Original List

a = [['country',['America_1','America_2','America_3','America_4','England_5','England_6'],['apple_1_more','orange_1_more']],['country',['Brazil_2','Brazil_3','Brazil_1','Brazil_4','Mexico_1','Mexico_3','Mexico_2'],['grapes_1_less','banana_1_more']]]

looking for output:

[['country', ['America', 'England'], ['orange_more', 'apple_more']], ['country', ['Mexico', 'Brazil'], ['grapes_less', 'banana_more']]]

but getting:

[['country', ['America', 'England'], ['orange_more', 'apple_more']], ['country', ['America', 'England', 'Mexico', 'Brazil'], ['orange_more', 'grapes_less', 'banana_more', 'apple_more']]]

code::

 a = [['country',['America_1','America_2','America_3','America_4','England_5','England_6'],['apple_1_more','orange_1_more']],['country',['Brazil_2','Brazil_3','Brazil_1','Brazil_4','Mexico_1','Mexico_3','Mexico_2'],['grapes_1_less','banana_1_more']]]
aa ={}
aaa=[]
aaaa=[]
aaaaa=[]
for i in a:
    for j in i[1]:
        j=j.split('_',1)[0]
        aaa.append(j)
    for k in i[2]:
        k=k.split('_',2)[0]+'_'+k.split('_',2)[2]
        aaaa.append(k)
    aa['country'] = [i[0],list(set(aaa)),list(set(aaaa))]
    aaaaa.append(aa['country'])
print (aaaaa)

You'll have a much easier time if you use meaningful variable names. — John Ellmore
– John Ellmore, Commented Apr 12, 2018 at 4:34

user3483203 · Accepted Answer · 2018-04-12 04:35:17Z

4

Using a list comprehension, converting the second item in each sublist to and from a set():

a = [['country',['America','America','America','America','England','England']],['country',['Brazil','Brazil','Brazil','Brazil','Mexico','Mexico','Mexico']]]

a = [[i, list(set(j))] for i, j in a]
print(a)

Output:

[['country', ['England', 'America']], ['country', ['Brazil', 'Mexico']]]

This may not preserve the order of the inner list, as sets are unordered, so you may need to account for this.

answered Apr 12, 2018 at 4:35

user3483203

51.3k10 gold badges72 silver badges104 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Mehrdad Pedramfar · Accepted Answer · 2018-04-12 05:48:12Z

1

Use this recursive function to remove duplicate item in multi level array:

def dup(input_):
    if isinstance(input_, list):
        try:
            input_ = list(set([i.split('_')[0] if not isinstance(i, list) else i for i in input_]))
        except TypeError:
            pass
        for child in input_:
            input_[input_.index(child)] = dup(child)

    return input_

edited Apr 12, 2018 at 5:48

answered Apr 12, 2018 at 5:26

Mehrdad Pedramfar

11.1k4 gold badges43 silver badges61 bronze badges

1 Comment

GRNearth Over a year ago

Traceback (most recent call last): dup(a) input_ = list(set([i.split('')[0] for i in input])) input_ = list(set([i.split('')[0] for i in input])) AttributeError: 'list' object has no attribute 'split'

Krafty Coder · Accepted Answer · 2018-04-12 06:18:37Z

0

This is how I would go about it.

country_list1 = [a[0[0]]]
country_list2 = [a[1[0]]]
duplicates = [country for country in country_list1 in country_list2]
non_duplicates = [country for country in country_list1 not in country_list2]

This will give you both the duplicated ones and non-duplicated This is considering case sensitiveness of the names in both

answered Apr 12, 2018 at 6:18

Krafty Coder

711 silver badge2 bronze badges

Comments

Community · Accepted Answer · 2020-06-20 09:12:55Z

0

You can try this approach :

a = [['country',['America','America','America','America','England','England']],['country',['Brazil','Brazil','Brazil','Brazil','Mexico','Mexico','Mexico']]]



print(list(map(lambda x:[x[0],list(set(x[1:][0]))],a)))

output:

[['country', ['England', 'America']], ['country', ['Mexico', 'Brazil']]]

Your variables names are very confusing , Still i tried new approach , you can try this:

a = [['country',['America_1','America_2','America_3','America_4','England_5','England_6'],['apple_1_more','orange_1_more']],['country',['Brazil_2','Brazil_3','Brazil_1','Brazil_4','Mexico_1','Mexico_3','Mexico_2'],['grapes_1_less','banana_1_more']]]


final_data=[]
for i in a:
    sub_data=[]

    for j in i[1:]:
        d = {}

        for m in j:
            data=m.split('_')[0]
            d[data]=data

        sub_data.append(list(d.keys()))
    final_data.append(['country',*sub_data])
print(final_data)

output:

[['country', ['America', 'England'], ['orange', 'apple']], ['country', ['Brazil', 'Mexico'], ['banana', 'grapes']]]

If your data format is always like this then you can try this:

update

a = [['country',['America_1','America_2','America_3','America_4','England_5','England_6'],['apple_1_more','orange_1_more']],['country',['Brazil_2','Brazil_3','Brazil_1','Brazil_4','Mexico_1','Mexico_3','Mexico_2'],['grapes_1_less','banana_1_more']]]


final_data=[]
for i in a:
    sub_data=[]
    sub_extra=[]

    for j in i[1:2]:
        sub_extra.append(i[2])
        d = {}

        for m in j:
            data=m.split('_')[0]
            d[data]=data

        sub_data.extend([list(d.keys()),*sub_extra])
    final_data.append(['country',*sub_data])
print(final_data)

output:

[['country', ['America', 'England'], ['apple_1_more', 'orange_1_more']], ['country', ['Mexico', 'Brazil'], ['grapes_1_less', 'banana_1_more']]]

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Apr 12, 2018 at 4:59

Aaditya Ura

12.8k7 gold badges60 silver badges96 bronze badges

4 Comments

GRNearth Over a year ago

Thanks, this worked perfectly...Thank @Ayodhyankit Paul... updated list of list but unable to use lambda

GRNearth Over a year ago

updated.. with split for updated list of list ..getting error..print(list(map(lambda x:[x[0],list(set(x[1:][0].split('',1)[0])),list(set(x[1:][1].split('',2)[0]+'_'+[2]))],a)))

GRNearth Over a year ago

AttributeError: 'list' object has no attribute 'split'

GRNearth Over a year ago

- [['country', ['America', 'England'], ['apple', 'orange']], ['country', ['Brazil', 'Mexico'], ['grapes', 'banana']]] but looking for [['country', ['America', 'England'], ['orange_more', 'apple_more']], ['country', ['Mexico', 'Brazil'], ['grapes_less', 'banana_more']]]

Collectives™ on Stack Overflow

Python List of list, remove duplicates

4 Answers 4

Comments

1 Comment

Comments

update

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

1 Comment

Comments

update

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related