How to load unicode string into json in Python?

Question

I am trying to read file from a compressed file and convert data into json/ dictionary. But there is unicode issue that I have been struggling for a while. Can anyone help ?

exfile_obj = tar.extractfile(member)
data = exfile_obj.read()
print(type(data)) ## shows str
print(data)  ## it is something like: "{u'building': False, u'displayName': u'Tam\\xe1s Kosztol\\xe1nczi', u'changeSet': {u'items': u'comment'}}"
json_obj = json.loads(data) # it is a unicode object.

PM 2Ring · Accepted Answer · 2017-02-11 14:01:44Z

9

That data is a string representation of a Python dictionary. You can convert it to a dictionary using ast.literal_eval, and you can convert that dict to a JSON string using json.dumps.

import ast
import json

src = "{u'building': False, u'displayName': u'Tam\\xe1s Kosztol\\xe1nczi', u'changeSet': {u'items': u'comment'}}"
data = ast.literal_eval(src)
print(data)
j = json.dumps(data)
print(j)

output

{'building': False, 'displayName': 'Tamás Kosztolánczi', 'changeSet': {'items': 'comment'}}
{"building": false, "displayName": "Tam\u00e1s Kosztol\u00e1nczi", "changeSet": {"items": "comment"}}

answered Feb 11, 2017 at 14:01

PM 2Ring

55.6k6 gold badges96 silver badges201 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Luffy Cyliu Over a year ago

Your example returns a dict (data = ast.literal_eval(src)). But it is strange that it did not return as I expected when I modified my code accordingly. It returned a str still. My changed codes are : exfile_obj = tar.extractfile(member) data = exfile_obj.read() print(type(data)) json_obj = ast.literal_eval(data) print(type(json_obj)) ## it is a str

Luffy Cyliu Over a year ago

What else I found was that when ast.literal_eval gets input from file using with open("example.json") as file: src = f.read() ( NOT from variable) it returns str. But if ast.literal_eval gets input from variable, it returns a dict.

Luffy Cyliu Over a year ago

When I read from file. The data look like this: '"{u\'building\': False, u\'displayName\': u\'Tam\\\\xe1s Kosztol\\\\xe1nczi\', u\'changeSet\': {u\'items\': u\'comment\'}}"\n' @PM 2Ring

Luffy Cyliu Over a year ago

I think I found the reason. The json file is not formatted properly. It is a str type not a json type. Once quotes are removed from the file, this solution worked. thanks.

Collectives™ on Stack Overflow

How to load unicode string into json in Python?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest