How to encode Python 3 string using \u escape code?

Question

In Python 3, suppose I have

>>> thai_string = 'สีเ'

Using encode gives

>>> thai_string.encode('utf-8')
b'\xe0\xb8\xaa\xe0\xb8\xb5'

My question: how can I get encode() to return a bytes sequence using \u instead of \x? And how can I decode them back to a Python 3 str type?

I tried using the ascii builtin, which gives

>>> ascii(thai_string)
"'\\u0e2a\\u0e35'"

But this doesn't seem quite right, as I can't decode it back to obtain thai_string.

Python documentation tells me that

\xhh escapes the character with the hex value hh while
\uxxxx escapes the character with the 16-bit hex value xxxx

The documentation says that \u is only used in string literals, but I'm not sure what that means. Is this a hint that my question has a flawed premise?

What about .decode('utf-8')? Aren't strings in Python unicode anyway? — Zizouz212
– Zizouz212, Commented Aug 28, 2015 at 22:45
@Zizouz212, neither thai_string nor ascii(thai_string) have a decode method, and thai_string.encode('utf-8').decode('utf-8') brings me back to where I started, thai_string, which is not the desired output. — Michael B. Currie
– Michael B. Currie, Commented Aug 28, 2015 at 23:00
Python documentation relevant to the escape sequence \u: docs.python.org/3/reference/lexical_analysis.html and docs.python.org/3/library/codecs.html#encodings-and-unicode — 0 _
– 0 _, Commented Apr 8, 2021 at 2:56
Does this answer your question? How to work with surrogate pairs in Python? — ti7
– ti7, Commented Jul 26, 2021 at 15:45

Simeon Visser · Accepted Answer · 2015-08-28 22:46:33Z

13

You can use unicode_escape:

>>> thai_string.encode('unicode_escape')
b'\\u0e2a\\u0e35\\u0e40'

Note that encode() will always return a byte string (bytes) and the unicode_escape encoding is intended to:

Produce a string that is suitable as Unicode literal in Python source code

answered Aug 28, 2015 at 22:46

Simeon Visser

123k19 gold badges192 silver badges185 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Michael B. Currie Over a year ago

Perfect. But why does this string have two slashes before the "u" while the "x" only has one?

tripleee Over a year ago

This is simply how Python displays a literal backslash inside a quoted string. Compare '\\n' (literal backslash, literal n) to '\n' (newline character).

tripleee Over a year ago

If you want the result as a string, you can tack on .decode('ascii')

Collectives™ on Stack Overflow

How to encode Python 3 string using \u escape code?

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related