Read json files in python

Question

I am trying to read a file which has the contents in the below format:

{"op":"mcm","clk":"6394474220","pt":1523095339090,"mc":[{"id":"1.141299528","rc":[{"atb":[[10,5.56]],"id":30246}],"con":true,"img":false}]}
{"op":"mcm","clk":"6394627886","pt":1523096762118,"mc":[{"id":"1.141299528","rc":[{"atb":[[10.5,20.78]],"id":30246}],"con":true,"img":false}]}
{"op":"mcm","clk":"6394647672","pt":1523096790720,"mc":[{"id":"1.141299528","rc":[{"atb":[[10,22.23]],"id":30246}],"con":true,"img":false}]}

I am trying to read it as json but it seems these are multiple jsons in one file. When I trying to read this json file using:

connection_file = open(filepath, 'r')
conn_string = json.load(connection_file)

It gives an error:

json.decoder.JSONDecodeError: Extra data: line 2 column 1

Please let me know how to read such files.

Take your JSON and paste it into jsonlint.com and click "Validate json". It will tell you whether your JSON is valid — Daniel
– Daniel, Commented Jul 30, 2018 at 9:35
It seems you have a file where each line is a JSON string, but the file as a whole is not valid JSON. — Mike Scotty
– Mike Scotty, Commented Jul 30, 2018 at 9:36
Following on from @Mike's observation that means you should loop over the file line by line and use json.loads on each line instead of json.load for the entire lot. — Jon Clements
– Jon Clements, Commented Jul 30, 2018 at 9:36
It says: Error: Parse error on line 16: ... "img": false }]} { "op": "mcm", "cl ---------------------^ Expecting 'EOF', '}', ',', ']', got '{' But how to fix this? — star_kid
– star_kid, Commented Jul 30, 2018 at 9:37
For the sake of completion: you need to make sure / agree with whoever produces this JSON that the delimiter between JSON "documents" in the input will be a new line. That is, that new lines are not allowed within a single JSON document as they usually would be. — millimoose
– millimoose, Commented Jul 30, 2018 at 9:39

Reut Sharabani · Accepted Answer · 2018-07-30 09:39:03Z

1

Your file isn't a valid json file. Each line is a json but they are delimited by newlines. You can use this to get them all as a list:

with open(filepath) as f:
    jsons = list(map(json.loads, f))
# jsons is now a list of all jsons in your file.

answered Jul 30, 2018 at 9:39

Reut Sharabani

31.5k7 gold badges76 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sunitha · Accepted Answer · 2018-07-30 09:39:14Z

Looks like file is not a proper json file, but contains json data in each line. So read the file line by line and convert it to json

>>> import json
>>> with open('tmp.txt') as f:
...     json_list = [json.loads(line) for line in f]
... 
>>> json_list
[{'op': 'mcm', 'clk': '6394474220', 'pt': 1523095339090, 'mc': [{'id': '1.141299528', 'rc': [{'atb': [[10, 5.56]], 'id': 30246}], 'con': True, 'img': False}]}, {'op': 'mcm', 'clk': '6394627886', 'pt': 1523096762118, 'mc': [{'id': '1.141299528', 'rc': [{'atb': [[10.5, 20.78]], 'id': 30246}], 'con': True, 'img': False}]}, {'op': 'mcm', 'clk': '6394647672', 'pt': 1523096790720, 'mc': [{'id': '1.141299528', 'rc': [{'atb': [[10, 22.23]], 'id': 30246}], 'con': True, 'img': False}]}]

Collectives™ on Stack Overflow

Read json files in python

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related