Issues parsing JSON from API with Python and Requests

Question

Got a problem with this API request. Actually two problems. The JSON file of the type I'm accessing can be seen here: https://api.nfz.gov.pl/app-umw-api/plans/55d957ea-3640-44e7-7998-4dc35c3d0c23?page=1&limit=15&format=json&api-version=1.2 My plan_list1 is just a file with a long list of codes like this: 55d957ea-3640-44e7-7998-4dc35c3d0c23?page=1&limit=15&format=json&api-version=1.2

I'm trying to get info from branches of this JSON:

from 'meta' I'd like to get the 'count'
from 'data/header' - all of it
from 'data/agreement plan' - all the 'attributes' and crucially the 'id'

I'm trying to use a version of a code that worked on other similar API methods from the same source, but there we had dictionaries within lists within dictionaries, here it should be more straightforward.

However, I don't really understand this and various attempts like this one here yield errors saying: "all_data.append({**a["header"], "plan-number": a["agreement-plan"]['id']}) TypeError: string indices must be integers"

import requests
import pandas as pd
from plans_list1 import plans

baseurl = 'https://api.nfz.gov.pl/app-umw-api/plans/'
all_data = []
for plan in plans:
    api_url = ''.join([baseurl, plan])

    def main_request(api_url):
        r = requests.get(api_url)
        return r.json()

    for page in range(0, 1):
        data = requests.get(api_url.format(page)).json()

        for a in data["data"]:
            all_data.append({**a["header"], "plan-number": a["agreement-plan"]['id']})

df = pd.DataFrame(all_data)

df.to_csv('2018-plans-detail.csv', encoding='utf-8-sig', index=False)
print(get_pages(p_number))

for a in data["data"] Iterating over a dictionary gives you the keys, which are strings. So in this loop, a is a string. But it seems you are expecting it to be a dictionary. — John Gordon
– John Gordon, Commented Jan 17, 2023 at 1:52
And if you already know the specific dictionary keys you want (which seems to be the case), why are you iterating at all? Just refer to them directly. data["header"], data["agreement-plan"], etc. — John Gordon
– John Gordon, Commented Jan 17, 2023 at 1:55

Kaleb Fenley · Accepted Answer · 2023-01-17 17:42:07Z

2

I don't see the problem with doing it how John Gordon suggested in the comment above.

import requests

r = requests.get('https://api.nfz.gov.pl/app-umw-api/plans/55d957ea-3640-44e7-7998-4dc35c3d0c23?page=1&limit=15&format=json&api-version=1.2')
content = r.json()
count = content['meta']["count"]
header = content['data']['header']
agreement_plan = content['data']['agreement-plan']

print(f'count: {count}\n')
print(f'header: {header}\n')
print(f'agreement_plan: {agreement_plan}\n')

Output:

count: 12

header: {'year': 2018, 'branch': '01', 'provider-code': '3101057', 'provider-name': 'WOJEWÓDZKI SZPITAL SPECJALISTYCZNY WE WROCŁAWIU', 'agreement-code': '03/8/3101057/01/2018/01', 'service-type-name': 'Leczenie szpitalne'}

agreement_plan: {'id': '55d957ea-3640-44e7-7998-4dc35c3d0c23', 'type': 'agreement-plan', 'attributes': {'product-code': '00.9999.000.02', 'product-name': 'KOSZTY Ĺ\x9aWIADCZEĹ\x83 WYNIKAJÄ\x84CE Z ROZPORZÄ\x84DZENIA ZMIENIAJÄ\x84CEGO OWU - W SYSTEMIE PODSTAWOWEGO SZPITALNEGO ZABEZPIECZENIA Ĺ\x9aWIADCZEĹ\x83 OPIEKI ZDROWOTNEJ', 'order': 1, 'unit-count': 7197.08, 'price': 9575168.0, 'avg-price': 1330.42, 'date-from': '2018-01-01T00:00:00', 'date-to': '2018-12-31T00:00:00'}, 'links': None}

Updated

I must admit I'm not that great with pandas. But this will get you the data exported to a csv like you are asking in the most recent comment

import requests
import pandas as pd


plan_list = ['55d957ea-3640-44e7-7998-4dc35c3d0c23']

df_list = [pd.read_json(requests.get(f'https://api.nfz.gov.pl/app-umw-api/plans/{plan}?page=1&limit=15&format=json&api-version=1.2').text) for plan in plan_list]
df = pd.concat(df_list)
pd.DataFrame.to_csv(df)

edited Jan 17, 2023 at 17:42

answered Jan 17, 2023 at 5:08

Kaleb Fenley

2262 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Michael Wiz Over a year ago

Kaleb, thank you very much for your help! This works w' one plan. Can u help send to csv? I'm iterating a long list of plans. This puts all data into 2 csv cells, not a table: imports... url1 = 'api.nfz.gov.pl/app-umw-api/plans' for plan in plans: url2 = ''.join([url1, plan]) r = requests.get(url2) cont = r.json() count = cont['meta']["count"] header = cont['data']['header'] agr_plan = cont['data']['agreement-plan'] dict = [f'count: {count}\n', f'header: {header}\n', f'agreement_plan: {agreement_plan}\n'] df = pd.DataFrame(dict) df.to_csv('pl.csv')

Kaleb Fenley Over a year ago

Updated my answer to include writing to csv.

Collectives™ on Stack Overflow

Issues parsing JSON from API with Python and Requests

1 Answer 1

I don't see the problem with doing it how John Gordon suggested in the comment above.

Output:

Updated

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

I don't see the problem with doing it how John Gordon suggested in the comment above.

Output:

Updated

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related