open invalid json file in python

Question

I have a file with such non-valid json data (it's cut for clarity):

[
{
  "orderID": 90,
  "orderDate": '2017-05-10',  #issue №1
  "clientName": "Mr. Bean",
  "clientPhoneN": "123-4567",
  "orderContents": [
    {
      "productID": 05,        #issue №2
      "productName": "Bicycle",
      "quantity": 1,
      "price": 8000
    },
    {
      "productID": 23,
      "productName": "helmet",
      "quantity": 2,
      "price": 1000
    }
  ],
  "orderCompleted": true
}
]

I tried to open it in python and transform it to list of dictionaries, but with no success. Depending on the case I get different errors. It will take too much space to outline all my attempts and their ending errors.

I have two issues here with the file:

issue №1 - single quotes in orderDate value. it results with :

JSONDecodeError: Expecting value

issue №2 - zero leading productID. It results with:

JSONDecodeError: Expecting ',' delimiter

I can hardcode these issues, but I feel that it's not true pythonic way.

Is there an option of "pretty" opening and converting this data file to list of dictionaries?

Most probably I want to keep productID data typa as integer, but if it's impossible, str is ok too.

I know that it's invalid, I wrote it in the heading of my question. Anybody can open valid json. The very question is how to process such json which is broken. — Dmitriy Fialkovskiy
– Dmitriy Fialkovskiy, Commented Aug 10, 2017 at 19:07
you are contradicting this as json, it is close to being JSON format, but needs to be modified before it is JSON, as it is currently simply "data". If you cannot modify the contents of the "data", you will need to use a custom parser. — chickity china chinese chicken
– chickity china chinese chicken, Commented Aug 10, 2017 at 19:15
If you can modify the contents of the data, you could create valid JSON using double quotes for strings fix issue #1 stackoverflow.com/a/4162651/1248974 — chickity china chinese chicken
– chickity china chinese chicken, Commented Aug 10, 2017 at 19:17

Huang · Accepted Answer · 2017-08-10 19:15:07Z

2

Try demjson package:

from demjson import decode
decode("""[
{
  "orderID": 90,
  "orderDate": '2017-05-10',
  "clientName": "Mr. Bean",
  "clientPhoneN": "123-4567",
  "orderContents": [
    {
      "productID": 05,
      "productName": "Bicycle",
      "quantity": 1,
      "price": 8000
    },
    {
      "productID": 23,
      "productName": "helmet",
      "quantity": 2,
      "price": 1000
    }
  ],
  "orderCompleted": true
}
]""")

You'll get:

[{'clientName': 'Mr. Bean',
  'clientPhoneN': '123-4567',
  'orderCompleted': True,
  'orderContents': [{'price': 8000,
    'productID': 5,
    'productName': 'Bicycle',
    'quantity': 1},
   {'price': 1000, 'productID': 23, 'productName': 'helmet', 'quantity': 2}],
  'orderDate': '2017-05-10',
  'orderID': 90}]

answered Aug 10, 2017 at 19:15

Huang

6093 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Dmitriy Fialkovskiy Over a year ago

I knew that there's something easy to apply=) thanks, Huang!

Alex Over a year ago

For those curious why this module works as opposed to the standard library json: "In non-strict mode it can also deal with slightly non-conforming input that is more JavaScript than JSON (such as allowing comments)."

Collectives™ on Stack Overflow

open invalid json file in python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related