Python - Extract multiple json with the same key

Question

So I am loading a JSON file containing a proxies,

my JSON Object .

 {  
   "http":{  
      "http://":"64.90.50.38:45876/",
      "http://":"89.250.220.40:54687/",
      "http://":"89.207.92.146:37766/",
      "http://":"89.23.194.174:8080/",
      "http://":"82.208.111.100:52480/"
   }
}

I want to access each proxy but I keep getting the last proxy which is "http://":"82.208.111.100:52480/

my code :

import json
x = open('proxy.json', 'r')
data = json.load(x)
print data['http']

My Question is : how can I access these values with the same key ?

MrFregg · Accepted Answer · 2025-07-30 07:22:23Z

As per the documentation

The RFC specifies that the names within a JSON object should be unique, but does not mandate how repeated names in JSON objects should be handled. By default, this module does not raise an exception; instead, it ignores all but the last name-value pair for a given name:

>>> weird_json = '{"x": 1, "x": 2, "x": 3}'
>>> json.loads(weird_json)
{'x': 3}

The object_pairs_hook parameter can be used to alter this behavior.

Documentation also state that object_pairs_hook is an optional function that will be called with the result of any object literal decoded with an ordered list of pairs. The return value of object_pairs_hook will be used instead of the dict. This feature can be used to implement custom decoders. If object_hook is also defined, the object_pairs_hook takes priority.

For example,

Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 19:29:22) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> from collections import defaultdict
>>> from pprint import pprint
>>>
>>> s = """
... {
...    "http":{
...       "http://":"64.90.50.38:45876/",
...       "http://":"89.250.220.40:54687/",
...       "http://":"89.207.92.146:37766/",
...       "http://":"89.23.194.174:8080/",
...       "http://":"82.208.111.100:52480/"
...    }
... }
... """
>>>
>>> def custom_hook(obj):
...     # Identify dictionary with duplicate keys...
...     # If found create a separate dict with single key and val and as list.
...     if len(obj) != len(set(i for i, j in obj)):
...         data_dict = defaultdict(list)
...         for i, j in obj:
...             data_dict[i].append(j)
...         return dict(data_dict)
...     return dict(obj)
...
>>> data = json.loads(s, object_pairs_hook=custom_hook)
>>> pprint(data)
{'http': {'http://': ['64.90.50.38:45876/',
                      '89.250.220.40:54687/',
                      '89.207.92.146:37766/',
                      '89.23.194.174:8080/',
                      '82.208.111.100:52480/']}}
>>>
>>> pprint(data['http'])
{'http://': ['64.90.50.38:45876/',
             '89.250.220.40:54687/',
             '89.207.92.146:37766/',
             '89.23.194.174:8080/',
             '82.208.111.100:52480/']}

ForceBru · Accepted Answer · 2019-07-12 13:52:44Z

1

Python dictionaries cannot have duplicate keys. If there are duplicate keys in the definition of the dictionary, the last key:value pair will be used.

The JSON object is converted to a Python dictionary by json.load.

answered Jul 12, 2019 at 13:52

ForceBru

45k10 gold badges71 silver badges104 bronze badges

Comments

Tim · Accepted Answer · 2019-07-12 14:13:43Z

1

The usual approach is to have a data structure that supports holding multiple values. For example

{  
   "http": [
      "64.90.50.38:45876",
      "89.250.220.40:54687",
      "89.207.92.146:37766",
      "89.23.194.174:8080",
      "82.208.111.100:52480"
   ]
}

Your code would then print ["64.90.50.38:45876", "89.250.220.40:54687", ...]

Django, for example, has MultiValueDict that you can then load your data into to provide a nicer API. The source can be found here.

edited Jul 12, 2019 at 14:13

answered Jul 12, 2019 at 13:54

Tim

2,6831 gold badge23 silver badges27 bronze badges

Comments

Moritz · Accepted Answer · 2019-07-12 14:00:13Z

0

Too slow, but if you can change the JSON, why not make it a list?

{  
   "http": [
       "64.90.50.38:45876",
       "89.250.220.40:54687",
       "89.207.92.146:37766",
       "89.23.194.174:8080",
       "82.208.111.100:52480"
   ]
}

You can then parse the data as follows:

with open('proxy.json', 'r) as json_data:
    proxy_data = json.load(json_data)

And access it:

proxy_data['http'][0]

>>> 'http://64.90.50.38:45876/'

edited Jul 12, 2019 at 14:00

answered Jul 12, 2019 at 13:54

Moritz

971 silver badge11 bronze badges

Collectives™ on Stack Overflow

Python - Extract multiple json with the same key

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related