JSON Decoding String

Question

I am trying to decode a JSON string using

json.loads(request.POST.get('d'))

where d is a POST parameter containing a JSON string.

I get the following error in the stacktrace:

ValueError: Unterminated string starting at: line 1 column 22 (char 22)

This is the JSON string:

{"data":{"40":{"html":"<span style=\"color:#ffffff;\">test</span>","background":"transparent"},"41":{"html":"","background":"transparent"},"42":{"html":"","background":"transparent"}},"action":"save"}

However it works if I dont apply the span tag in data->40->html

{"data":{"40":{"html":"test","background":"transparent"},"41":{"html":"","background":"transparent"},"42":{"html":"","background":"transparent"}},"action":"save"}

What is the problem here?

Where did you get the json string from? is it exactly what you get out of request.POST.get('d')) ? — Keith
– Keith, Commented Nov 15, 2012 at 7:48
The string comes from a Javascript array. The 'html' element of the array comes from a CKEditor instance. — Michael Bates
– Michael Bates, Commented Nov 15, 2012 at 7:50
I mean the string that you posted here. How did you get it? Because what you posted works fine, so I suspect that what comes out of the POST may be a little different. — Keith
– Keith, Commented Nov 15, 2012 at 7:51
And yes, the string is exactly what comes out of request.POST.get('d') — Michael Bates
– Michael Bates, Commented Nov 15, 2012 at 7:51

Tchotchke · Accepted Answer · 2016-08-05 13:22:53Z

1

I suppose there is something with backslashes in the source string.

When I parse

"""{"data":{"40":{"html":"<span style=\"color:#ffffff;\">test</span>","background":"transparent"},"41":{"html":"","background":"transparent"},"42":{"html":"","background":"transparent"}},"action":"save"}"""

with json.loads(), it fails with a similar error.

However, when I disable escape sequences (r'' string literal), it works:

r"""{"data":{"40":{"html":"<span style=\"color:#ffffff;\">test</span>","background":"transparent"},"41":{"html":"","background":"transparent"},"42":{"html":"","background":"transparent"}},"action":"save"}"""

Obviously, '\"' in your string is being escaped and results in '"' when you construct the string, probably in JS(?). Haven't seen the code that builds it, but try adding an extra backslash: '\\"'

UPDATE: You may replace r'\' with r'\\' in a string. But it is better to understand how does the string look to begin with. When you inserted the string body into your message, where did you get it from?

edited Aug 5, 2016 at 13:22

Tchotchke

3,1213 gold badges27 silver badges38 bronze badges

answered Nov 15, 2012 at 5:55

Ellioh

5,3503 gold badges23 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Michael Bates Over a year ago

The string comes from a html wysiwyg textarea

Ellioh Over a year ago

Check what happens to backslash symbols if they are entered by user. Do they become double backslashes now, without the replacement?

Michael Bates Over a year ago

No they don't, but I don't think this is the issue. It appears request.POST.get('d') is being cutoff where there is a semicolon in the string.

Ellioh Over a year ago

Then try to replace a backslash with a pair of backslashes, and look at the result. Probably it will satisfy you.

Michael Bates Over a year ago

Backslashes are being escaped into double backslashes, I just checked. As I said the problem seems to be semicolons. If I send {"data":"te;st"} all thats in request.POST.get('d') is: {"data":"te

|

mgilson · Accepted Answer · 2012-11-15 05:55:55Z

0

How do you know that is the string you're getting? It works for me:

>>> ss = r'{"data":{"40":{"html":"<span style=\"color:#ffffff;\">test</span>","background":"transparent"},"41":{"html":"","background":"transparent"},"42":{"html":"","background":"transparent"}},"action":"save"}'
>>> json.loads(ss)
{u'action': u'save', u'data': {u'42': {u'html': u'', u'background': u'transparent'}, u'40': {u'html': u'<span style="color:#ffffff;">test</span>', u'background': u'transparent'}, u'41': {u'html': u'', u'background': u'transparent'}}}

Notice that I used a raw string for ss because otherwise \" will just be replaced by " in the string resulting in '"<span style="color:#ffffff;">test</span>"' which doesn't work for obvious reasons.

answered Nov 15, 2012 at 5:55

mgilson

312k70 gold badges656 silver badges722 bronze badges

4 Comments

Michael Bates Over a year ago

Ah ok, that sounds like the problem. How can I convert a variable into a raw string?

mgilson Over a year ago

To make a string "raw" you just prefix it with r: r"this is a raw string". "this is not a raw string". This doesn't work with "variables" if that's what your asking though: foo = "bar"; rbar #doesn't make bar a raw string

Ellioh Over a year ago

Have you printed request.POST.get('d') before inserting it into your message? It is necessary to understand what exactly it contains. Probably it is enough to replace r'\' with r'\\' in it.

Michael Bates Over a year ago

The html comes from a user input textarea so the problem with replacing r'\' with r'\\' is that if a user explicitly types '\' it will get replaced with '\\'..

Erwin Julius · Accepted Answer · 2015-11-16 14:15:14Z

0

This worked for us:

json.loads(request.POST.get('d').encode('string-escape'))

answered Nov 16, 2015 at 14:15

Erwin Julius

5395 silver badges14 bronze badges

Collectives™ on Stack Overflow

JSON Decoding String

3 Answers 3

6 Comments

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related