Removing duplicate output from list of Dictionary in JSON Python

Question

I have following JSON file that I am trying to parse and encountering some issues.

[
{
"ballot_name": "LAPP, David",
"office": "MAYOR",
"votes": "7",
"voting_station": "3",
"voting_station_id": "703",
"voting_station_name": "Branton JR High School",
"voting_station_type": "Regular",
"ward": "7"
},
{
"ballot_name": "SMITH, Bill",
"office": "MAYOR",
"votes": "683",
"voting_station": "1",
"voting_station_id": "1101",
"voting_station_name": "St. Mary's Parish Hall",
"voting_station_type": "Regular",
"ward": "11"
},
{
"ballot_name": "HEATHER, Larry R",
"office": "MAYOR",
"votes": "1",
"voting_station": "37",
"voting_station_id": "737",
"voting_station_name": "Clover Living",
"voting_station_type": "Special",
"ward": "7"
},
{
"ballot_name": "OLSON, Curtis",
"office": "MAYOR",
"votes": "0",
"voting_station": "32",
"voting_station_id": "1432",
"voting_station_name": "Lake Bonavista Village",
"voting_station_type": "Special",
"ward": "14"
},
{
"ballot_name": "LIN, Jun",
"office": "COUNCILLOR",
"votes": "2",
"voting_station": "66",
"voting_station_id": "366",
"voting_station_name": "Memorial Park Library",
"voting_station_type": "Advance",
"ward": "3"
},
{
"ballot_name": "HEJDUK, Marek",
"office": "COUNCILLOR",
"votes": "0",
"voting_station": "67",
"voting_station_id": "767",
"voting_station_name": "Saddletowne Library",
"voting_station_type": "Advance",
"ward": "7"
},

My objectives so far to do the following

1> Print the list of voting_station_name removing all the duplicates - Which I can print but not able to remove duplicates?

Below is the code I have tried so far.

import json
import urllib

print "This is Json Data Parser Program \nThis program will download the Election Results from 2017 file from OpenData Portal"


_url_= "https://data.cityname.ca/resource/kqmd-3dsq.json"
_response_ = urllib.urlopen(_url_)
_data_= json.loads(_response_.read())

#with open('data.json', 'w') as outfile:
#    json.dump(_data_,outfile,indent=4,sort_keys=True)
def _ward_(_no_):
    print "Your choosen ward number is" , _no_
    for _i_ in _data_:
        result = []
        if (_i_["ward"] == _no_ and  _i_["voting_station_name"] not in result):
                    result.append(_i_["voting_station_name"])
                    print result

_ward_("12")

I am able to get the output as following but as we can see it has some duplicates "voting_station_name"

How can I remove the duplicates in my output?

This is Json Data Parser Program
This program will download the CoC Election Results from 2017 file from OpenData Portal
Your choosen ward number is 12
Cranston School
McKenzie Towne Care Centre
Millican/Ogden Community Association
Age Care - Seton Seniors Community
Auburn Heights Retirement Residence
University of Calgary Taylor Family Digital Librar
McKenzie Towne Church
Age Care - Seton Seniors Community
Christ the King Catholic School
Auburn Heights Retirement Residence

Poshi · Accepted Answer · 2018-07-22 09:02:18Z

1

You are reinitializing the list in each iteration, hence it is always empty when you perform the check:

def _ward_(_no_):
    print "Your choosen ward number is" , _no_
    result = []
    for _i_ in _data_:
        if (_i_["ward"] == _no_ and  _i_["voting_station_name"] not in result):
                    result.append(_i_["voting_station_name"])
    print result

EDIT:

You ask me for improvements on the code structure. I'm not sure if it is an improvement, you should try and benchmark the result, but my development would had been something like:

def _ward_(_no_):
    print "Your choosen ward number is" , _no_

    print set([e["voting_station_name"] for e in _data_ if e["ward"]==_no_])

In this code, I generate a list comprenhension that extract the "voting_station_name" from all elements of _data_ that have a "ward" equals to _no_. I convert this list to a set to remove the duplicates and print the result.

edited Jul 22, 2018 at 9:02

answered Jul 21, 2018 at 22:27

Poshi

5,8533 gold badges22 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

qräbnö Over a year ago

Ha, you got the first commentators;) The set is not necessary, because the if checks whether the element is already in the list. But I don't know what is more elegant and/or faster: list or set?

aashitvyas Over a year ago

Thank You !! I was not realizing that part at all. I was banging my head to this since few hours. Any Improvements to the code structure you can recommend ? I am newbie so working Python and JSON first time

Poshi Over a year ago

@qräbnö you are right, I commented on using set instead of list because it is the common way to remove duplicates if order is nor important... until I realized the problem was a different one. To me, a set is more elegant, because you are gathering a set of voting stations. Also, this simplifies the code, because you don't need to check for previous existence.

Collectives™ on Stack Overflow

Removing duplicate output from list of Dictionary in JSON Python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related