Most effective way to parse JSON Objects [duplicate]

Question

I'm currently trying to access a specific value within a JSON object, I am successfully doing so using the Try / except statement.

As I am still learning Python, I am wondering if there is a more efficient way of achieving this. I often find that for different sets of JSON data the key / value pair moves, and I have to change the index value manually in my code from [0] [1] [2] or [3]

JSON data:

{
 "notifications": [
  {
   "timestamp": "1619847600000000000",
   "path_elements": [
    "Devices",
    "NME39488200",
    "versioned-data",
    "interfaces",
    "data",
    "Ethernet1",
    "aggregate",
    "rates",
    "1m"
   ],
   "updates": {
    "alignmentErrors": {
     "key": "alignmentErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "fcsErrors": {
     "key": "fcsErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "frameTooLongs": {
     "key": "frameTooLongs",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "frameTooShorts": {
     "key": "frameTooShorts",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "inErrors": {
     "key": "inErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "outDiscards": {
     "key": "outDiscards",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "outErrors": {
     "key": "outErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "symbolErrors": {
     "key": "symbolErrors",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    }
   }
  },
  {
   "timestamp": "1622629140000000000",
   "path_elements": [
    "Devices",
    "NME39488200",
    "versioned-data",
    "interfaces",
    "data",
    "Ethernet11",
    "aggregate",
    "rates",
    "1m"
   ],
   "updates": {
    "inMulticastPkts": {
     "key": "inMulticastPkts",
     "value": {
      "avg": {
       "float": 0.03333333333333334
      },
      "max": {
       "float": 0.1
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0.047140452079103175
      },
      "weight": {
       "float": 1
      }
     }
    },
    "inOctets": {
     "key": "inOctets",
     "value": {
      "avg": {
       "float": 269348.4166666667
      },
      "max": {
       "float": 341898.5
      },
      "min": {
       "float": 223613.3
      },
      "stddev": {
       "float": 45132.897905054306
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    },
    "inUcastPkts": {
     "key": "inUcastPkts",
     "value": {
      "avg": {
       "float": 300.3333333333333
      },
      "max": {
       "float": 364
      },
      "min": {
       "float": 258.8
      },
      "stddev": {
       "float": 44.179847089921985
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    },
    "outMulticastPkts": {
     "key": "outMulticastPkts",
     "value": {
      "avg": {
       "float": 0.03333333333333334
      },
      "max": {
       "float": 0.1
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0.047140452079103175
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    },
    "outOctets": {
     "key": "outOctets",
     "value": {
      "avg": {
       "float": 28700.35
      },
      "max": {
       "float": 42799.9
      },
      "min": {
       "float": 17571.2
      },
      "stddev": {
       "float": 9737.234046475074
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    },
    "outUcastPkts": {
     "key": "outUcastPkts",
     "value": {
      "avg": {
       "float": 196.13333333333333
      },
      "max": {
       "float": 237
      },
      "min": {
       "float": 164.4
      },
      "stddev": {
       "float": 28.93957305989306
      },
      "weight": {
       "float": 0.9999999999999999
      }
     }
    }
   }
  },
  {
   "timestamp": "1620649440000000000",
   "path_elements": [
    "Devices",
    "NME39488200",
    "versioned-data",
    "interfaces",
    "data",
    "Ethernet11",
    "aggregate",
    "rates",
    "1m"
   ],
   "updates": {
    "inDiscards": {
     "key": "inDiscards",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    }
   }
  },
  {
   "timestamp": "1621745580000000000",
   "path_elements": [
    "Devices",
    "JPE13031399",
    "versioned-data",
    "interfaces",
    "data",
    "Ethernet11",
    "aggregate",
    "rates",
    "1m"
   ],
   "updates": {
    "inBroadcastPkts": {
     "key": "inBroadcastPkts",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    },
    "outBroadcastPkts": {
     "key": "outBroadcastPkts",
     "value": {
      "avg": {
       "float": 0
      },
      "max": {
       "float": 0
      },
      "min": {
       "float": 0
      },
      "stddev": {
       "float": 0
      },
      "weight": {
       "float": 1
      }
     }
    }
   }
  }
 ]
}

I am taking the JSON data from multiple URLs. The code I am using to extract the value 'Avg' from the Key: 'InDiscards' or 'OutDiscards' is as follows:

for url in URLs:
    JSONdata = requests.get(url, cookies=cookies, verify=False).json()
    JSONkey = JSONdata["notifications"]
    name = JSONkey[0]["path_elements"][1]
    interface = JSONkey[0]["path_elements"][5]
    try:
        outDiscards = JSONkey[0]["updates"]["outDiscards"]["value"]["avg"]["float"]
    except KeyError:
        outDiscards = JSONkey[2]["updates"]["outDiscards"]["value"]["avg"]["float"]
    try:
        inDiscards = JSONkey[0]["updates"]["inDiscards"]["value"]["avg"]["float"]
    except KeyError:
        inDiscards = JSONkey[2]["updates"]["inDiscards"]["value"]["avg"]["float"]

My understanding here is that we check index [0] for the avg value of OutDiscards (or InDiscards) , if we fail to find a value, we then check index [2] , however surely there must be a better way of achieving this given that sometimes the index may be 1/3/4/5 etc...

Does this answer your question? How to convert JSON data into a Python object — JMA
– JMA, Commented Jun 2, 2021 at 10:41

vinzenz · Accepted Answer · 2021-06-02 12:47:24Z

0

I recommend searching with a for loop.

First initialize your variables, I use None so I can later determine if my search routine found something

outDiscards = None
inDiscards = None

Then iterate over all elements in the notifications list

for notification in JSONdata["notifications"]:

only try to access the value when it hasn't been found yet

    if outDiscards is None:

try to to access the value and ignore the KeyError

        try:
            outDiscards = notification["updates"]["outDiscards"]["value"]["avg"]["float"]
        except KeyError:
            pass
    if inDiscards is None:
        try:
            inDiscards = notification["updates"]["inDiscards"]["value"]["avg"]["float"]
        except KeyError:
            pass

after the search routine there should be values in outDiscards and inDiscards, you can test for that with if outDiscards is not None

print(outDiscards)
print(inDiscards)

edited Jun 2, 2021 at 12:47

answered Jun 2, 2021 at 10:49

vinzenz

6893 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

netcode995 Over a year ago

This worked perfectly, however i do not understand why and nor could i come to the solution myself, would it be possible to explain why this works?

vinzenz Over a year ago

what explanation do you need? should i clarify for notification in JSONdata["notifications"] ?

netcode995 Over a year ago

Regarding the = none, and then looping on that variable

vinzenz Over a year ago

i edited my answer to clarify

Collectives™ on Stack Overflow

Most effective way to parse JSON Objects [duplicate]

1 Answer 1

4 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Linked

Related