Decoding unicode from Javascript in Python & Django

Question

On a website I have the word pluș sent via POST to a Django view. It is sent as plu%25C8%2599. So I took that string and tried to figure out a way how to make %25C8%2599 back into ș.

I tried decoding the string like this:

from urllib import unquote_plus
s = "plu%25C8%2599"
print unquote_plus(unquote_plus(s).decode('utf-8'))

The result i get is pluÈ which actually has a length of 5, not 4.

How can I get the original string pluș after it's encoded ?

edit:

I managed to do it like this

def js_unquote(quoted):
  quoted = quoted.encode('utf-8')
  quoted = unquote_plus(unquote_plus(quoted)).decode('utf-8')
  return quoted

It looks weird but works the way I needed it.

Which unicode character is that supposed to be? U+0219 (Latin small s with comma below) or U+015F (latin small S with cedilla)? — bgporter
– bgporter, Commented Dec 2, 2010 at 14:57
@bgporter: S-comma, U+0219. It's irrelevant which letter though, as I'm having this problem for other unicode letters as ț, â, etc. — Virgiliu
– Virgiliu, Commented Dec 2, 2010 at 15:19

Ignacio Vazquez-Abrams · Accepted Answer · 2010-12-02 14:48:39Z

2

URL-decode twice, then decode as UTF-8.

answered Dec 2, 2010 at 14:48

Ignacio Vazquez-Abrams

804k160 gold badges1.4k silver badges1.4k bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Virgiliu Over a year ago

If I URL-decode twice then decode as UTF-8 I get 'ascii' codec can't encode characters in position 3-4: ordinal not in range(128) :(

Ignacio Vazquez-Abrams Over a year ago

@yoshi: That's your console being unable to output it. The result is fine.

Virgiliu Over a year ago

print unquote_plus(unquote_plus('plu%25C8%2599')).decode('utf-8') gives me pluș. But when I put it inside the application it spews that ascii error :| It's on the same machine, same environment.

mkelley33 Over a year ago

Is the decoded result being saved to a database? If so, I'm guessing it might be the database not configured with utf-8 as the encoding.

Virgiliu Over a year ago

The result is saved in a MySQL database but it has utf8 encoding. The problem occurs before querying the database.

|

mkelley33 · Accepted Answer · 2010-12-02 14:49:08Z

1

You can't unless you know what the encoding is. Unicode itself is not an encoding. You might try BeautifulSoup or UnicodeDammit, which might help you get the result you were hoping for.

http://www.crummy.com/software/BeautifulSoup/

I hope this helps!

Also take a look at:

http://www.joelonsoftware.com/articles/Unicode.html

answered Dec 2, 2010 at 14:49

mkelley33

5,63910 gold badges51 silver badges76 bronze badges

2 Comments

mkelley33 Over a year ago

I should note that Ignacio's answer is the solution for you specific problem, but keep in mind that you won't always know the encoding of what your user is posting.

mkelley33 Over a year ago

Also, please excuse my suggestion if you were already familiar those links and how Unicode and encoding works :)

Sencer H. · Accepted Answer · 2012-12-11 07:42:09Z

0

unquote_plus(s).encode('your_lang_encoding')

I was try like that. I was tried to sent a json POST request by HTML form to directly a django URI, which is included unicode characters like "şğüöçı+" and it works. I have used iso_8859-9 encoder in encode() function.

answered Dec 11, 2012 at 7:42

Sencer H.

1,2711 gold badge13 silver badges36 bronze badges

Collectives™ on Stack Overflow

Decoding unicode from Javascript in Python & Django

3 Answers 3

6 Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related