How to parse JSON data from API response in Python?

Question

I am trying to write a script that will pull current status from our monitoring tools and update them in MS SQL DB. When I call the API I get a HUGE response in JSON format:

{
  "hoststatuslist": {
    "recordcount": "1084",
    "hoststatus": [
      {
        "@attributes": {
          "id": "XXXX"
        },
        "host_id": "XXX",
        "name": "XXXXX",
        "display_name": "XXXXXXX",
        "address": "XXXXXX",
        "alias": "XXXXXX",
        "status_text": "XXXXXXXXXXXXXXXXXXXXXXX",
        etc.
      },
      {
        "@attributes": {
          "id": "XXXX"
        },
        "host_id": "XXX",
        "name": "XXXXX",
        "display_name": "XXXXXXX",
        "address": "XXXXXX",
        "alias": "XXXXXX",
        "status_text": "XXXXXXXXXXXXXXXXXXXXXXX",
        etc.
      },
      etc.
    ]
  }
}

As you can see I get over 1000 host objects with attributes. I want to parse the response, so that I can add/update the MS SQL DB. I'm trying to parse out the host_id, name, and status_text for each host.

I tried to do something like this Python - Parsing JSON Data Set but I keep getting errors that the response object has no attribute read or decode.

Here is my current code:

import requests
import json

response = requests.get('url with API Key')
decoded_response = response.read().decode("UTF-8")
data = json.loads(decoded_response)
jsonData = data["hoststatus"]

for host in jsonData:
    Name = host.get("name")
    StatusText = host.get("status_text")

If anyone has a suggestion to do this with another language or tool I am open. I need to call about 20 APIs and put all the status/other information into a DB so that it's all in one location.

Any help is appreciated.

j-bin · Accepted Answer · 2017-04-18 20:43:03Z

14

Like @danil-kondratiev said, you can use response.json() and you don't need to encode/decode. Will this work for you?

import requests

response = requests.get('url with keys')

json_data = response.json() if response and response.status_code == 200 else None

if json_data and 'hoststatuslist' in json_data:
    if 'hoststatus' in json_data['hoststatuslist']:
        for hoststatus in json_data['hoststatuslist']['hoststatus']:
            host_name = hoststatus.get('name')
            status_text = hoststatus.get('status_text')

answered Apr 18, 2017 at 20:43

j-bin

6917 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sunderam Dubey · Accepted Answer · 2022-06-29 02:44:31Z

In my opinion, you don't necessarily need to call response.decode(...). This should suffice:

import requests

response = requests.get('url with API Key') # Note: the timeout parameter is very useful for requests!
data = response.json()

if 'hoststatus' in data and isinstance(data['hoststatus'], list):
    list_to_return = list()  # we'll put the filtered results in this
    for element in data['hoststatus']
        current_item = {  # create a dict of the 3 required values
            host_id = element['host_id'],
            name = element['name'],
            status_text = element['status_text']
        }
        list_to_return.append(current_item)  # add it to the return list

# list_to_return now has all of the required 3 properties stored for all items in an easily-accessible format

This could be done much faster (generators) / in a much more compact format (list comprehension), but I tried to write something which is easy to understand.

Danil Kondratiev · Accepted Answer · 2017-04-18 19:59:50Z

-1

Try. Requests docs

requests.get('https://api.github.com/user', auth=('user', 'pass'))
>>>r.status_code
200
>>> r.encoding
'utf-8'
>>> r.json()
{json}

answered Apr 18, 2017 at 19:59

Danil Kondratiev

963 silver badges7 bronze badges

1 Comment

Wish I Knew this stuff Over a year ago

I'm not sure if I didn't explain myself correctly or if I misunderstand your post, but this post is simple giving me the json data. I already have the data I need to pull out the Host name/status from the json data. As a side note when I try r.encoding I receive no response even with a 200 status code

Ojasvi Monga · Accepted Answer · 2019-08-29 15:50:47Z

-1

If the response is from a Flask app, you will need to use response.get_json().

Also, ensure that you have an updated version of Flask.

answered Aug 29, 2019 at 15:50

Ojasvi Monga

5,2092 gold badges23 silver badges37 bronze badges

Collectives™ on Stack Overflow

How to parse JSON data from API response in Python?

4 Answers 4

Comments

Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related