I am wondering how is binary encoding for a string is given in Python.
For example,
>>> b'\x25'
b'%'
or
>>>b'\xe2\x82\xac'.decode()
'€'
but
>>> b'\xy9'
File "<stdin>", line 1
SyntaxError: (value error) invalid \x escape at position 0
Please, could you explain what \xe2 stand for and how this binary encoding works.
\x is used to introduce a hexadecimal value, and must be followed by exactly two hexadecimal digits. For example, \xe2 represents the byte (in decimal) 226 (= 14 * 16 + 2).
In the first case, the two strings b'\x25' and b'%' are identical; Python displays values using ASCII equivalents where possible.
\x25 would be ambiguous: is it the single character % or is it a two-character sequence consisting of the ASCII control character START TRANSMISSION followed by the character 5?bytes value, \x is used to specify a single byte by its hexadecimal value. In a str value, you can use \x (primarily for historical reasons) to specify a Unicode code point using a single value between 0 and 255, \u to specify one using a 16-bit value, or \U using a 32-bit value.I assume that you use a Python 3 version. In Python 3 the default encoding is UTF-8, so b'\xe2\x82\xac'.decode() is in fact b'\xe2\x82\xac'.decode('UTF-8).
It gives the character '€' which is U+20AC in unicode and the UTF8 encoding of U+20AC is indeed `b'\xe2\x82\xac' in 3 bytes.
So all ascii characters (code below 128) are encoded into one single byte with same value as the unicode code. Non ascii characters corresponding to one single 16 bit unicode value are utf8 encoded into 2 or 3 bytes (this is known as the Basic Multilingual Plane).
