How to decode unicode-encoded JSON object in Python

Question

I have a JSON object that was saved to a file using the repr function:

f = open('file', 'w')
f.write(repr(my_json))
f.close()

The encoded JSON now has leading 'u' characters indicating unicode encoding.

{u'messages': 'messages'}

This is expected Python 2.x behavior, but now I'm trying to figure out how to decode the string stored in that file. Everything I've tried has given me the same error:

ValueError: Expecting property name: line 1 column 2 (char 1)

Help?

@kid_drew I think Rafael is saying that you should not use f.write(repr(my_json)) to write it to file at the beginning. You should use json.dump instead. — ljk321
– ljk321, Commented Jan 22, 2015 at 0:44
I don't have any problem even with reading after an f.write which python version are you using? — user3467349
– user3467349, Commented Jan 22, 2015 at 0:46
@skyline75489 - I get that, and I already know about using json.dumps. My question isn't about json.dumps, it's about how to undo an existing screwup. The data can't be regenerated. — kid_drew
– kid_drew, Commented Jan 22, 2015 at 0:47

mgilson · Accepted Answer · 2015-01-22 00:55:14Z

1

Assuming that the data that got written to the file was json(able), that also means that it's representation should be a python literal. So... You can try to "decode" it using ast.literal_eval.

e.g.

import ast
with open('file', 'r') as datafile:
    literal_data = datafile.read()

json_data_decoded = ast.literal_eval(literal_data)

demo that ast.literal_eval works with the u prefix:

>>> repr(data)
"{u'foo': u'bar'}"
>>> ast.literal_eval(repr(data))
{u'foo': u'bar'}
>>> ast.literal_eval(repr(data))['foo']
u'bar'

Note, there might be a few python versions (3.0, 3.1 and 3.2) where this doesn't work. I don't really have a good workaround for those versions -- hopefully you don't need to support them.

edited Jan 22, 2015 at 0:55

answered Jan 22, 2015 at 0:49

mgilson

312k70 gold badges656 silver badges722 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user3467349 Over a year ago

btw just regular eval works fine for me in 2.7 and 3.4

kid_drew Over a year ago

json_data_decoded is still a string with the leading 'u' characters for me.

kid_drew Over a year ago

Scratch that. I screwed something up. It worked. =)

mgilson Over a year ago

@user3467349 -- Yeah, eval should work but also opens you up to a world of pain if the data you are evaluating happens to be malicious :-)

kid_drew Over a year ago

Luckily, this was just a 1-off. We pulled a huge amount of data from a third party and accidentally stored it in a file incorrectly. The file was like 100MB, so manually making changes would have been painful.

Collectives™ on Stack Overflow

How to decode unicode-encoded JSON object in Python

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related