UnicodeDecodeError: 'ascii' codec can't decode byte with reading CSV

Question

Trying to read from a CSV file and write the data into an XML file. I am encountering:

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8a in position 87: ordinal not in range(128)

My question is, what is the best way to ignore this kind of error and continue processing the data set. After reading other similar questions, I did add: # -*- coding: utf-8 -*- to my file but it didn't help

Properly decode the input, e.g. read as bytes and then do input.decode("utf-8") (if your input is utf-8). — syntonym
– syntonym, Commented Sep 8, 2016 at 15:01

zipa · Accepted Answer · 2016-09-08 15:36:14Z

1

You can try opening csv with codecs:

import codecs
codecs.open(file_name, 'r', 'utf8')

Given that each line will contain '\n' string you will need to apply line.rstrip() when looping trough lines.

Note: Please don't try to convert values to str as you will encounter another error there.

edited Sep 8, 2016 at 15:36

answered Sep 8, 2016 at 15:33

zipa

28k6 gold badges45 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

user1195192 Over a year ago

Thanks @Boris, going to try your suggestion

zipa Over a year ago

Please read the edit, that '\n' gave me headaches more than once :)

user1195192 Over a year ago

This is how I am reading at the moment: with open('myFile.csv', 'rb') as ifile: reader = csv.reader(ifile) for rownum, row in enumerate(reader):

zipa Over a year ago

You can first replace open inside your with statement with codecs.open as i suggested in answer. Then just use for rownum, row in enumerate(ifile): row = row.rstrip() on the first line of your iteration and it should replace your csv.reader() method.

Shital Shah · Accepted Answer · 2020-02-09 00:07:34Z

1

I was getting this error while reading readme ad long description in setup.py. If you are using open, you can use the encoding parameter:

with open("README.md", "r", encoding='utf_8') as f:
    long_description = f.read()

answered Feb 9, 2020 at 0:07

Shital Shah

69.9k21 gold badges258 silver badges202 bronze badges

1 Comment

Alexandru R Over a year ago

integrated python 2.7 open does not have encoding param.

Collectives™ on Stack Overflow

UnicodeDecodeError: 'ascii' codec can't decode byte with reading CSV

2 Answers 2

4 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related