How to remove duplicate dict from list in Python

Question

I have list of dicts where sometime there are duplicate dict. For ex:

objList = 
[{
    'Name': 'plate',
    'StartTime': '2022-05-17T10:26:05.738101',

}, {
    'Name': 'bezel',
    'StartTime': '2022-05-17T10:26:09.922667',

}, {
    'Name': 'chrome',
    'StartTime': '2022-05-17T10:26:23.283304',

}, {
    'Name': 'plate placement',
    'StartTime': '2022-05-17T10:26:25.570845',

}, {
    'Name': 'plate placement',
    'StartTime': '2022-05-17T10:26:39.3390',
}]

In above data, plate placement is duplicated. Similarly, any dict can be duplicated but I have delete any of the duplicate data and just keep one. For this, first I thought of checking if in the list we have duplicate dicts or not:

obj_names = []
for obj in objList:
    obj_names.append(obj['Name'])

Now obj_names contains ['plate', 'bezel', 'chrome', 'plate placement', 'plate placement']. So this way we know that which dict is duplicated. We now have to delete any one of its occurrences. How can we delete that occurrence from the list?

@MechanicPig Can you please explain this in an answer?

S Andrew
– S Andrew

2022-05-17 11:01:04 +00:00
Commented May 17, 2022 at 11:01 — S Andrew
– S Andrew, Commented May 17, 2022 at 11:01

Mechanic Pig · Accepted Answer · 2022-05-17 11:08:35Z

1

As @mugiseyebrows said, we use the 'Name' of each dictionary (this statement is not very rigorous.) as the key and the dictionary itself as the value to create a new dictionary so that you can ensure that a dictionary with the same 'Name' appears once, and then use its values to create a new list:

>>> new_dict = {dct['Name']: dct for dct in objList}
>>> new_list = list(new_dict.values())
>>> print('},\n'.join(str(new_list).split('},')))
[{'Name': 'plate', 'StartTime': '2022-05-17T10:26:05.738101'},
 {'Name': 'bezel', 'StartTime': '2022-05-17T10:26:09.922667'},
 {'Name': 'chrome', 'StartTime': '2022-05-17T10:26:23.283304'},
 {'Name': 'plate placement', 'StartTime': '2022-05-17T10:26:39.3390'}]

answered May 17, 2022 at 11:08

Mechanic Pig

7,7813 gold badges17 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

mugiseyebrows · Accepted Answer · 2022-05-17 10:51:12Z

0

Instead of list of dictionaries you can use dictionary of dictionaries using name as a key. This way if you insert new object with same name it will replace previous object.

If you need to preserve order you can use OrderedDict.

answered May 17, 2022 at 10:51

mugiseyebrows

4,9491 gold badge16 silver badges22 bronze badges

Collectives™ on Stack Overflow

How to remove duplicate dict from list in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related