Different base64 encoding between python versions

Question

I'm having trouble sending an html code through JSON.

I'm noticing my string values are different between python versions (2.7 and 3.5)

My string being something like: <html>PAÇOCA</html>

on Python 2.7:

x = '<html><p>PAÇOCA</p></html>'
base64.b64encode(x)
=> PGh0bWw+PHA+UEGAT0NBPC9wPjwvaHRtbD4=

on Python 3.5:

x = '<html><p>PAÇOCA</p></html>'
base64.b64encode(x)
=> b'PGh0bWw+PHA+UEHDh09DQTwvcD48L2h0bWw+'

Why are these values different? How can I make the 3.5 string equal to the 2.7?

This is causing me troubles with receiving e-mails due to the accents being lost.

PGh0bWw+PHA+UEGAT0NBPC9wPjwvaHRtbD4= is the [ cp437, cp850, cp852, cp857, cp858, cp860, cp861, cp863 or cp865 ] + base64 encoding of <html>PAÇOCA</html> — ikegami
– ikegami, Commented Jun 23, 2017 at 19:31
PGh0bWw+PHA+UEHDh09DQTwvcD48L2h0bWw+ is the UTF-8 + base64 encoding of <html>PAÇOCA</html>. — ikegami
– ikegami, Commented Jun 23, 2017 at 19:37
(I can't help further since I don't know how Python handle character encodings.) — ikegami
– ikegami, Commented Jun 23, 2017 at 19:37

Mark Tolonen · Accepted Answer · 2017-06-24 06:30:29Z

2

Your example x values are not valid Python so it is difficult to tell where the code went wrong, but the answer is to use Unicode strings and explicitly encode them to get consistent answers. The below code gives the same answer in Python 2 and 3, although Python 3 decorates byte strings with b'' when printed. Save the source file in the encoding declared via #coding. The source code encoding can be any encoding that supports the characters used in the source file. Typically UTF-8 is used for non-ASCII source code, but I made it deliberately different to show it doesn't matter.

#coding:cp1252
from __future__ import print_function
import base64
x = u'<html><p>PAÇOCA</p></html>'.encode('utf8')
enc = base64.b64encode(x)
print(enc)

Output using Pylauncher to choose the major Python version:

C:\>py -2 test.py
PGh0bWw+PHA+UEHDh09DQTwvcD48L2h0bWw+

C:\>py -3 test.py
b'PGh0bWw+PHA+UEHDh09DQTwvcD48L2h0bWw+'

answered Jun 24, 2017 at 6:30

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Farees Hussain Over a year ago

I think this answer is partially correct. There might be some cases where the base64 encoding used in python2 is saved in the database. In that case we need a way to get the same base64 encoding as python3. Is there any way to do so?

Mark Tolonen Over a year ago

@FareesHussain They are the same. Python 3 just displays byte strings with b'' as an indicator that it is a byte string, but the results in both Python versions are byte strings.

Farees Hussain Over a year ago

What I mean is to get the PGh0bWw+PHA+UEGAT0NBPC9wPjwvaHRtbD4= base64 encoded value using python3

Mark Tolonen Over a year ago

@FareesHussain .decode() will convert it to a Unicode string, which will print without the bytes notation on Python 3.

Collectives™ on Stack Overflow

Different base64 encoding between python versions

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related