Python json.loads not able to parse json string

Question

I'm just working on a side project and having trouble getting the json module to parse a json object I've stored in a text file. The text file contains a list of newline separated json objects.

Thus far, I have this code which I've confirmed is retrieving each full json line, then feeding it to json.loads():

def load_save_game(file_name):
    save_game = []
    with open(file_name) as f:
        for line in f.readline():
            save_game.append(json.loads(line))
    return save_game

When I run this, I get a rather long traceback:

Traceback (most recent call last):
  File ____, line 70, in <module> 
    main()
  File ____, line 66, in main
    view = ViewerWindow(load_save_game('played/20_05_2016 16-04-31.txt'))
  File ____, line 60, in load_save_game
    save_game.append(json.loads(line))
  File "C:\Python27\lib\json\__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "C:\Python27\lib\json\decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Python27\lib\json\decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting object: line 1 column 1 (char 0)

I suspect that the problem may be related to encoding, but upon looking into the matter, I've seen that the JSON library can be picky about things such as the capitalization of truth values, dangling commas and single quotes rather than double quotes.

Python 2.7 appears to use the RFC 7159 and ECMA 404 specifications for json, so I used a free online json validator to see if the json was badly formed. It was successfully validated for all standards of json, so python should be quite happy with the input.

I've hosted one line of the json on pastebin, the only difference being that there is no whitespace in the file, which I've also uploaded.

I've looked for solutions online and tried a few different ones, such as capitalizing truth values, replacing all single quotes, and decoding the text from ascii.

The json.loads() function accepts any valid JSON. Yes, that means that True is not accepted, because that is not valid JSON, that's Python syntax instead. However, that's not your problem here; any online JSON validator will tell you it is valid JSON. — Martijn Pieters
– Martijn Pieters, Commented May 23, 2016 at 9:26

Martijn Pieters · Accepted Answer · 2016-05-23 09:39:15Z

3

You are looping over the characters of the first line:

for line in f.readline():

f.readline() returns one string, the first line of the file. You don't need to call readline() at all here, simply iterate over the file object directly:

with open(file_name) as f:
    for line in f:
        if line.strip():
            save_game.append(json.loads(line))

I've added in an extra test to skip empty lines (the end of a file can easily contain one).

You can turn the above into a list comprehension too:

def load_save_game(file_name):
    with open(file_name) as f:
        return [json.loads(l) for l in f if l.strip()]

Note that the above only works if your JSON documents do not by themselves contain newlines. Use json.load(f) (no looping) if you have just one JSON document in the file, or use a different technique to parse multiple JSON documents with newlines in the documents themselves.

The above works fine on your supplied sample file, with or without the line.strip() call:

>>> len(load_save_game(os.path.expanduser('~/Downloads/20_05_2016_16-04-31.txt')))
301

edited May 23, 2016 at 9:39

answered May 23, 2016 at 9:24

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Aaron3468 Over a year ago

Thanks. Naturally, I should have caught the bad iteration and I'm not exactly sure why it's such a common mistake.

Lifu Huang · Accepted Answer · 2016-05-23 09:29:35Z

1

As Martijn Mentioned, you should iterate the opened file directly.

I would write code like this:

save_game = map(json.loads, f)

EDIT:

That works for Python2, since map returns a list. If you are working with Python3, you need to surround map with list.

answered May 23, 2016 at 9:29

Lifu Huang

13k15 gold badges59 silver badges80 bronze badges

1 Comment

Aaron3468 Over a year ago

Indeed. map is a really useful function. The challenging part is learning to think of it as an option when you already have visible tools to solve the problem.

Collectives™ on Stack Overflow

Python json.loads not able to parse json string

2 Answers 2

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related