Python - Remove duplicate key from a list of dictionaries

Question

I have a list of dictionaries in Python:

[{u'value': u'firstname', u'key': u'Coach'}, 
 {u'value': u'lastname', u'key': u'Coach'}, 
 {u'value': u'age', u'key': u'Coach'}, 
 {u'value': u'firstname', u'key': u'Player'}, 
 {u'value': u'league', u'key': u'Player'}]

How can I remove duplicate key from a list of dictionaries and merge their values into a list as shown below:

l = [('Coach', ['firstname', 'lastname', 'age']), 
     ('Player', ['firstname', 'league'])]

Iterate over list items

for k, v in l:
    print k, v

awesoon · Accepted Answer · 2016-05-10 12:04:43Z

4

You could use defaultdict with list as value type in order to collect values:

In [5]: from collections import defaultdict

In [6]: data = [{u'value': u'firstname', u'key': u'Coach'}, 
 {u'value': u'lastname', u'key': u'Coach'}, 
 {u'value': u'age', u'key': u'Coach'}, 
 {u'value': u'firstname', u'key': u'Player'}, 
 {u'value': u'league', u'key': u'Player'}]

In [7]: l = defaultdict(list)

In [8]: for row in data:
   ...:     l[row['key']].append(row['value'])
   ...:     

In [9]: l
Out[9]: 
defaultdict(list,
            {'Coach': ['firstname', 'lastname', 'age'],
             'Player': ['firstname', 'league']})

And then you could easily convert the dictionary into a list of tuples using items method:

In [10]: list(l.items())
Out[10]: 
[('Coach', ['firstname', 'lastname', 'age']),
 ('Player', ['firstname', 'league'])]

answered May 10, 2016 at 12:04

awesoon

33.8k14 gold badges81 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Lifu Huang · Accepted Answer · 2016-05-10 12:31:30Z

3

[(k, [x['value'] for x in v]) for k, v in itertools.groupby(sorted(data, key=lambda x: x['key']), lambda d: d['key'])]

groupby is in the standard module itertools.

EDIT: Thank @soon for pointing out that itertools.groupby requires input sequence being consecutive, thus need to sort beforehand.

edited May 10, 2016 at 12:31

answered May 10, 2016 at 12:20

Lifu Huang

13k15 gold badges59 silver badges80 bronze badges

2 Comments

awesoon Over a year ago

groupby groups consecutive elements only

awesoon Over a year ago

operator.itemgetter could also be useful.

Collectives™ on Stack Overflow

Python - Remove duplicate key from a list of dictionaries

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related