Load JSON data from Google GitHub repo

Question

I am trying to load the following JSON file (from the Google Github repo) in Python as follows:

import json
import requests

url = "https://raw.githubusercontent.com/google/vsaq/master/questionnaires/webapp.json"
r = requests.get(url)
data = r.text.splitlines(True)
#remove first n lines which is not JSON (commented license)
data = ''.join(data[14:])

When I use json.loads(data) I get the following error:

JSONDecodeError: Expecting ',' delimiter: line 725 column 543 (char 54975)

As this has been saved as a json file by the GitHub repo owner (Google) I'm wondering what Im doing wrong here.

I was too hasty in my last comment. What on Earth is this? Why does it give invalid JSON? Putting a commented license at the start is nonsense :/ — roganjosh
– roganjosh, Commented Nov 10, 2018 at 10:56
I've passed it through jsonlint and it's invalid. I don't think there's anything you can do generically to fix it, it's a custom fix needed — roganjosh
– roganjosh, Commented Nov 10, 2018 at 11:00

hygull · Accepted Answer · 2018-11-10 13:49:25Z

2

I found the obtained text from API call is like a simple text, not a valid JSON (I checked at https://jsonformatter.curiousconcept.com/).

Here is my code that I used to filter the valid JSON part from the response.

I have used re module to extract the JSON part.

import json
import requests
import re

url = "https://raw.githubusercontent.com/google/vsaq/master/questionnaires/webapp.json"
r = requests.get(url)
text = r.text.strip()

m = re.search(r'\{(.|\s)*\}',  text) # It is for finding a valid JSON part from obtained text
s = m.group(0).replace('false', 'False') # Python has 'False/True' not 'false/true' (Replacement)
d = eval(s)

print(d) # {...}
print(type(d)) # <class 'dict'>

References »

edited Nov 10, 2018 at 13:49

answered Nov 10, 2018 at 12:08

hygull

8,7282 gold badges47 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Load JSON data from Google GitHub repo

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related