Various errors while parsing JSON in Python

Question

Attempting to parse json from a url requiring login. Including all my code here as I'm not sure where the error is.

try: import simplejson as json
except ImportError: import json
import urllib2

username = 'user'
password = '1234'
url = "https://www.blah.com/someplace"

# set up the username/password/url request
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_mgr.add_password(None, "https://www.blah.com", username, password)
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(handler)
urllib2.install_opener(opener)
request = urllib2.Request(url)
response = opener.open(request)

# option 1
json_object = json.loads(str(response))

#option 2
json_object = json.loads(response)

If I run the code with option 1 (commenting out option 2), I get this error:

Traceback (most recent call last):
  File "jsontest.py", line 22, in <module>
    json_object = json.loads(str(request))
  File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 413, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 402, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 420, in raw_decode
    raise JSONDecodeError("No JSON object could be decoded", s, idx)
simplejson.decoder.JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)

If I run option 2:

Traceback (most recent call last):
  File "jsontest.py", line 23, in <module>
    json_object = json.loads(request)
  File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 413, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 402, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer

My sample JSON is valid as far as I can tell:

{"set1":[{"data1":"411","data2":"2033","data3":"1","data4":"43968077","data5":"217","data6":"106828","data7":[]}], "set2":{"data8":"411","data9":"2033","data10":"43968077","data11":"217223360","data12":"106828"}}

simplejson version = 2.3.2, Python 2.7.3

Very new to all this so any pointers would be very helpful.

why are you trying to read the request object? Don't you want the response? response = opener.open(req);data = response.read() — monkut
– monkut, Commented Sep 7, 2012 at 22:53
Don't you mean json_object = json.loads(response)? (not request?) Just a typo? — ron rothman
– ron rothman, Commented Sep 7, 2012 at 22:54

Martijn Pieters · Accepted Answer · 2012-09-07 23:06:44Z

8

You want to decode the response, not the request:

json_object = json.load(response)

The response is a file-like object, so you can use .load() to have the json library read it directly.

Alternatively (at the cost of some temporary memory use), use the .loads() function with the fully read response:

json_object = json.loads(response.read())

Note that python 2.7 already includes the simplejson library, renamed to json:

import json

edited Sep 7, 2012 at 23:06

answered Sep 7, 2012 at 22:54

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

acpigeon Over a year ago

Correct, I needed to use .read() or load(). The 'response' was a typo yes. I now have a ready-to-go py dict, thanks!

ron rothman · Accepted Answer · 2012-09-07 22:57:00Z

1

You need to use response, not request (maybe just a typo?), but furthermore you need to use response.read() to get the body of the HTTP response:

json_object = json.loads(response.read())

answered Sep 7, 2012 at 22:57

ron rothman

18.4k7 gold badges47 silver badges45 bronze badges

Collectives™ on Stack Overflow

Various errors while parsing JSON in Python

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related