Python URL decoding?

Question

In javascript I do the following:

encodeURIComponent(comments)

while in Python i do the following:

urllib2.unquote(comments)

For some reason, when I do the following:

encodedURIComponents('ø')

I get %C3%B8, but when I decode

urllib2.unquote('%C3%B8')

I get Ã¸ instead of ø, which is the original character.

What gives?

I'm on a platform that uses jQuery on client side, and Python/Django server side.

The reason you're seeing this is that one side is using UTF-8, the other is using Latin-1. This is why you should never just use the default encodings and cross your fingers; if you have any non-ASCII characters, keep them in unicode strings internally, and encode and decode strings as appropriate at each boundary. (Python 3 makes this easier by giving you an error instead of mojibake when you get it wrong.) — abarnert
– abarnert, Commented Dec 8, 2012 at 2:04
The form on client side types in, I'm assuming UTF-8, while python/django uses unicode? I'm on Python 2.x. — user1886965
– user1886965, Commented Dec 8, 2012 at 16:26
Web forms have a way to specify the charset in each direction (in the HTTP headers, or the response body), and default to Latin-1. Python 2 has a way to encode and decode between str and unicode, and if you just use str you're dealing with bytes whose meaning is unspecified. (See what sys.getdefaultencoding() returns.) So get the charset the form uses, and decode the string into unicode to work with; when sending it back, encode to UTF-8 and set the charset (or, better, let Django take care of it, in case the browser sends an Accept-Charset for some reason). — abarnert
– abarnert, Commented Dec 10, 2012 at 0:27

VisioN · Accepted Answer · 2012-12-08 01:03:49Z

7

Simply try to decode it:

urllib2.unquote('%C3%B8').decode('utf-8')   # --> 'ø'

answered Dec 8, 2012 at 1:03

VisioN

146k35 gold badges287 silver badges291 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

jfs Over a year ago

note: it is actually urllib.unquote()

user1886965 Over a year ago

If I do that, I get u'\xf8'...?

abarnert Over a year ago

@user1886965: u'\xf8' is the exact same string as u'ø', because ø is the 248th Unicode character.

user1886965 Over a year ago

@abarnert: So how do I prepare it for web viewing?

abarnert Over a year ago

@user1886965: I'm not sure what you mean by that. I'm believe Django lets you pass unicode strings everywhere, and does any conversions necessary for talking to the browser/client, and to the database, and the log files, etc., so you don't need to do anything. I'm not sure why you ever need the line urllib.unquote('%C3%B8') in your code, so if there's a problem, and VisioN's answer doesn't solve it, maybe explain your actual use case?

|

Collectives™ on Stack Overflow

Python URL decoding?

1 Answer 1

10 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Related