Problems with unicode in Python

Question

I was having some trouble with using unicode in python, so I wrote this program, and I am confused by the results. Whenever I run it, different characters give me error #2, which means that utf32, utf16 and utf8 all gave errors when I tried to write a unicode character to my test file. Never the same ones. Is it a problem with my program, or am I doing somthing python is not designed to handle?

for a in range(65535):
    try:
        open('test_text.txt','w').write(unichr(a).encode("utf32"))
        if len(open('test_text.txt','r').read()) == 0:
            print  unichr(a) + ' Error #1 #' + str(a)
    except IOError:
        try:
            open('test_text.txt','w').write(unichr(a).encode("utf16"))
        except IOError:
            try:
                open('test_text.txt','w').write(unichr(a).encode("utf8"))
            except IOError:
                print unichr(a) + ' Error #2 #' + str(a)
    except UnicodeEncodeError:
        print unichr(a) + ' Error #3 #' + str(a)
raw_input('\n\nEnter char to end:')

What sorts of errors are you seeing? Can you give some examples? — ethan
– ethan, Commented Jul 21, 2013 at 1:18
What trouble were you having that caused you to write this program? It may make sense to open another SO question asking about the problems you were having earlier. — user2357112
– user2357112, Commented Jul 21, 2013 at 1:20
I suppose you could have trouble because you are opening the same file for reading at least 65535 times, without closing it, and simultaneously opening it for read. All kinds of buffering problems lie in this way. And no error here... — Jade Amora Lua
– Jade Amora Lua, Commented Jul 21, 2013 at 2:33

Matthew Wesly · Accepted Answer · 2013-07-21 02:05:07Z

1

Your code did not throw any errors when I tried it. Also, you're overriding the file every time through the loop. You could try changing the mode to 'a' instead of 'w' to append to the file. Or you could simply do the following:

f = open('test_text.txt','wb')
for a in range(65535):
    f.write(unichr(a).encode("utf32"))
f.close()

There is more information about reading/writing to files in python here: http://docs.python.org/2/tutorial/inputoutput.html

answered Jul 21, 2013 at 2:05

Matthew Wesly

1,2381 gold badge13 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Problems with unicode in Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related