Python convert file content to unicode form

Question

For example, I have a file a.js whose content is:

Hello, 你好, bye.

Which contains two Chinese characters whose unicode form is \u4f60\u597d
I want to write a python program which convert the Chinese characters in a.js to its unicode form to output b.js, whose content should be: Hello, \u4f60\u597d, bye.

My code:

fp = open("a.js")
content = fp.read()
fp.close()

fp2 = open("b.js", "w")
result = content.decode("utf-8")
fp2.write(result)
fp2.close()

but it seems that the Chinese characters are still one character , not an ASCII string like I want.

Ignacio Vazquez-Abrams · Accepted Answer · 2011-09-07 05:45:05Z

5

>>> print u'Hello, 你好, bye.'.encode('unicode-escape')
Hello, \u4f60\u597d, bye.

But you should consider using JSON, via json.

answered Sep 7, 2011 at 5:45

Ignacio Vazquez-Abrams

804k160 gold badges1.4k silver badges1.4k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

steabert Over a year ago

in python3, encoding a string yields a bytes, so you have to encode back to ascii to get the string out of it.

ibear · Accepted Answer · 2011-09-07 05:56:37Z

1

You can try codecs module

codecs.open(filename, mode[, encoding[, errors[, buffering]]])

a = codecs.open("a.js", "r", "cp936").read() # a is a unicode object

codecs.open("b.js", "w", "utf16").write(a)

answered Sep 7, 2011 at 5:56

ibear

1577 bronze badges

Comments

hitrust · Accepted Answer · 2022-08-06 10:08:22Z

0

There two ways you can use. first one, use 'encode' method

str1 = "Hello, 你好, bye. "
print(str1.encode("raw_unicode_escape"))
print(str1.encode("unicode_escape"))

Also you can use 'codecs' module：

import codecs
print(codecs.raw_unicode_escape_encode(str1))

edited Aug 6, 2022 at 10:08

answered Aug 6, 2022 at 10:06

hitrust

1011 silver badge6 bronze badges

Comments

wong2 · Accepted Answer · 2011-09-07 05:41:08Z

-1

I found that repr(content.decode("utf-8")) will return "u'Hello, \u4f60\u597d, bye'"
so repr(content.decode("utf-8"))[2:-1] will do the job

answered Sep 7, 2011 at 5:41

wong2

36.1k51 gold badges138 silver badges182 bronze badges

1 Comment

Eric O. Lebigot Over a year ago

Warning: This does not work in Python 3. Furthermore, it is not guaranteed that the various Python 2.x version escape Unicode strings the way you want: for instance, on my machine, repr(u"é") is "u'\\xe9'"

HYRY · Accepted Answer · 2011-09-07 05:44:35Z

-1

you can use repr:

a = u"Hello, 你好, bye. "
print repr(a)[2:-1]

or you can use encode method:

print a.encode("raw_unicode_escape")
print a.encode("unicode_escape")

answered Sep 7, 2011 at 5:44

HYRY

97.8k28 gold badges197 silver badges192 bronze badges

1 Comment

Eric O. Lebigot Over a year ago

There are problems with this: see my comment to wong2's suggestion. :)

Collectives™ on Stack Overflow

Python convert file content to unicode form

5 Answers 5

1 Comment

Comments

Comments

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

Comments

Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related