Python Top Level JSON Index

Question

I have the following JSON file from an API I am calling:

{
  "14500": [
    {
      "5": {
        "versionName": "VersionOne",
        "expand": "executionSummaries",
        "name": "Regression",
        "projectId": 15006,
        "startDate": "",
        "executionSummaries": {
          "executionSummary": []
        }
      },
      "7": {
        "versionName": "VersionOne",
        "expand": "executionSummaries",
        "versionId": 14500,
        "projectId": 15006,
        "startDate": "19/Sep/16",
        "executionSummaries": {
          "executionSummary": []
        }
      },
      "-1": {
        "versionName": "VersionOne",
        "expand": "executionSummaries",
        "name": "Ad hoc",
        "modifiedBy": "",
        "projectId": 15006,
        "startDate": "",
        "executionSummaries": {
          "executionSummary": []
        }
      },
      "recordsCount": 3
    }
  ],
  "14501": [
    {
      "-1": {
        "versionName": "Version 2",
        "expand": "executionSummaries",
        "projectId": 15006,
        "startDate": "",
        "executionSummaries": {
          "executionSummary": []
        }
      },
      "recordsCount": 1
    }
  ],
}

I need to iterate through the top level, and the next level (ex. "14500", and "5", "7", etc..) to find keys and values. So for instance I need to search through the entire JSON file to find the name that matches "Regression" and locate that set of datas values for "ProjectID" and possibly other strings. I had previously done this by using data["level1"][0]["level2"] etc., but in this case the numbers will never be the same so I don't know how to call them. I wrote the following after looking through some posts on here, but it only works for one level, not the next level down in JSON.

request = requests.get(getCyclesURL, headers=headers)
requestData = json.loads(request.text)
requestDataKeys = requestData.keys

for k in requestDataKeys():
    dictionaryIDs = requestData[k]
    for m in requestDataKeys:
        newDictionaryIDs = requestData[k][m]
        for dict in newDictionaryIDs:
            if dict['versionName'] == versionName:
                versionID = dict['versionID']
                print '%s: %s'%(versionName, versionID)

@MMF it does, as they didn't call the method on assignment.

jonrsharpe
– jonrsharpe

2016-09-30 18:38:40 +00:00
Commented Sep 30, 2016 at 18:38 — jonrsharpe
– jonrsharpe, Commented Sep 30, 2016 at 18:38

skovorodkin · Accepted Answer · 2016-09-30 18:59:36Z

1

Check out amazing boltons library! It has a remap function that may be an overkill in your case, but is a good thing to know about. Here's an elegant way to extract all dicts with 'name': 'Regression' from your nested data structure:

from boltons.iterutils import remap

# Copy your actual data here
data = {'14500': [{'5': {'name': 'Regression'}, '7': {'name': 'Ad hoc'}}]}

regressions = []

def visit(path, key, value):
    if isinstance(value, dict) and value.get('name') == 'Regression':
        # You can do whatever you want here!
        # If you're here then `value` is a dict
        # and its `name` field equals to 'Regression'.
        regressions.append(value)
    return key, value

remap(data, visit=visit, reraise_visit=False)

assert regressions == [{'name': 'Regression'}]

If you only need dicts on a certain level, you can also check length of path in the visit function.

edited Sep 30, 2016 at 18:59

answered Sep 30, 2016 at 18:53

skovorodkin

10.5k2 gold badges41 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Robᵩ · Accepted Answer · 2016-09-30 18:31:33Z

0

Here is a partial script tailored to your exact input. If it finds name: regression at the appropriate level, it prints a few of the related values.

for k, list_of_dicts in requestData.items():
    for d in list_of_dicts:
        for k, v in d.items():
            if isinstance(v, dict) and v.get('name') == "Regression":
                print '%s: %s %s'%(v.get('projectId'),
                                   v.get('versionName'),
                                   v.get('versionId'))

answered Sep 30, 2016 at 18:31

Robᵩ

170k20 gold badges251 silver badges323 bronze badges

Comments

Patrick Haugh · Accepted Answer · 2016-09-30 18:26:04Z

0

def find_in_dicts(d, key, value): #This finds all dictionaries that has the given key and value
    for k, v in d.items():
        if k == key:
            if v == value:
                yield d
        else:
            if isinstance(v, dict):
               for i in find_in_dicts(v, key, value):
                   yield i
            elif isinstance(v, list) and isinstance(v[0], dict):
                for item in v:
                    for i in find_in_dicts(item, key, value):
                        yield i

This should work recursivly no matter how deep your data structure gets. I'm not able to test it at the moment, but I hope it at elast gives you some idea.

answered Sep 30, 2016 at 18:26

Patrick Haugh

61.3k13 gold badges94 silver badges101 bronze badges

Collectives™ on Stack Overflow

Python Top Level JSON Index

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related