Setting query results encoding in cx_Oracle / UnicodeDecodeError with Chinese characters

Question

I'm working with a database containing a lot of Chinese characters. My code goes something like this:

connection = cx_Oracle.connect("%s/%s@%s:%s/%s" % (username, password, host, port, service_name))
cursor = connection.cursor()
cursor.execute('SELECT HOTEL_ID,CREATE_TIME,SOURCE,CONTENT,TITLE,RATE,UPDATE_TIME FROM T_FX_COMMENTS')

for row in cursor:
    # Stuff goes here
    pass

But I get this error:

Traceback (most recent call last):
  File "test.py", line 17, in <module>
    for row in cursor:
UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 26: illegal multibyte sequence

It seems GBK is not enough. I want to make cx-oracle give me GB18030 encoded results, instead of GBK. How do I do this?

cx_Oracle.Connection.encoding is read-only... I haven't found anything in the cx-oracle documentation that suggests I can do this.

I'm on Python 3.3.2 and cx-oracle 5.1.2. There must be something I'm missing here. Help is appreciated!

Are you using Python 2 or 3?

Maciek
– Maciek

2013-08-16 09:05:14 +00:00
Commented Aug 16, 2013 at 9:05 — Maciek
– Maciek, Commented Aug 16, 2013 at 9:05
Same scenario for me... did you find a solution?

daveoncode
– daveoncode

2016-01-13 13:48:20 +00:00
Commented Jan 13, 2016 at 13:48 — daveoncode
– daveoncode, Commented Jan 13, 2016 at 13:48

daveoncode · Accepted Answer · 2016-01-13 14:33:41Z

7

I was facing the same issue and I solved by setting the environment variable NLS_LANG to .AL32UTF8 (it seems a sort of "wildcard" that says "use utf-8 for any language")

answered Jan 13, 2016 at 14:33

daveoncode

19.8k19 gold badges108 silver badges163 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jay Over a year ago

3.5 years later, this also still works. Python 3.6.3 + cx_Oracle 7.1.2

Maciek · Accepted Answer · 2013-08-16 09:13:50Z

2

Try setting the NLS_LANG environment variable at the beginning of your program:

import os
os.environ["NLS_LANG"] = ".GB18030"

answered Aug 16, 2013 at 9:13

Maciek

3,2342 gold badges24 silver badges27 bronze badges

1 Comment

midrare Over a year ago

It's giving me this error: cx_Oracle.DatabaseError: ORA-12705: Cannot access NLS data files or invalid environment specified. I'm on Python 3.3.2 by the way.

Hector · Accepted Answer · 2016-07-11 09:06:52Z

0

Use this:

import os
os.environ["NLS_LANG"] = ".zhs16gbk"

os.environ["NLS_LANG"] is for Oracle.So use the format of Oracle. I solved with this with my python 2.6.8 and Oracle 11g.

answered Jul 11, 2016 at 9:06

Hector

1

Collectives™ on Stack Overflow

Setting query results encoding in cx_Oracle / UnicodeDecodeError with Chinese characters

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related