0

I am currently developing an application relating to unicode characters.

As the unicode characters would have to be read in python to determine the language before passing on to Java for processing. However, currently I am reading the file first using python to determine the language before calling upon the corresponding Java engine to process it.

This method takes too long as there is too much I/O cost involved, but directly passing the unicode characters as an argument does not work, it throws an error:

'charmap' codec cant encode characters in position xx - xx: character maps to <undefined>.

What I would like to do (excerpt of my code):

#reads in the unicode char 
str = "some unicode words"
command = "java -jar unicodeProcessor.jar " + str
subprocess.Popen(command, stdout = PIPE, stderr = PIPE)

Java processes it and writes it to a file.

Currently,

#determines what is the language. 
filepath = "filepath of text file"
command = "java -jar unicodeProcessor.jar " + filepath
subprocess.Popen(command, stdout = PIPE, stderr = PIPE)
#in this method I am taking the parameter to be a file instead of a string

This method is too slow.

Current code :

unic = open("unicode_words.txt")
words = unic.read()
if ininstance(words, str):
    convert = unicode(words, 'utf-8')
else: 
    convert = words

command = "java -jar unicodeProcessor.jar " + convert
subprocess.Popen(command, stdout = PIPE, stderr = PIPE)
Improve this question
16
  • 1
    What is your question? Explain "does not work." Commented Mar 2, 2015 at 9:23
  • @Lutz Horn updated my question Commented Mar 2, 2015 at 9:26
  • So what codec does your Java application expect? You cannot just write Unicode strings to it; implicit encoding failed. Commented Mar 2, 2015 at 9:27
  • Where do you use charmap? Show us the input that gives this error and the code that throws it. Commented Mar 2, 2015 at 9:27
  • @MartijnPieters What do you mean by it. I can encode them in utf-8. But before i passed them to the stdout. I already have converted them to utf-8. Commented Mar 2, 2015 at 9:28
Related questions
8
Java modified UTF-8 strings in Python
2
Python STILL won't allow Japanese Characters despite specifying the encoding
1
Print unicode characters encoded
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.