Python 2.7 encoding from csv file

Question

I have a problem with Python 2.7 encoding I have a csv file with some french characters (mangé, parlé, prêtre ...), the code I'm using is the following:

import pandas as pd
path_dataset = 'P:\\Version_python\\Dataset\\data_set - Copy.csv'
dataset = pd.read_csv(path_dataset, sep=';')

for lab, row in dataset.iterrows():
    print(row['Summary'])

I tried to add encoding to read_csv(), it didn't work. I tried unicode, decode(UTF-8) ... Nothing worked.

Then I tried to concatenate those extracted words with some text, and I got a utf-8 error, I don't know how to deal with that. Thanks

'ascii' codec can't decode byte 0xe9 in position 39: ordinal not in range(128) — geilelou
– geilelou, Commented Oct 18, 2016 at 9:34

Neill Herbst · Accepted Answer · 2016-10-18 08:42:48Z

2

Here is a list of standard python encodings

Standard Python 2.7 encodings

utf-8 does not work but you can try some other encodings on the link above.

Just tested latin_1 works. So the code should be:

dataset = pd.read_csv(path_dataset, sep=';', encoding='latin_1')

edited Oct 18, 2016 at 8:42

answered Oct 18, 2016 at 8:37

Neill Herbst

2,1421 gold badge15 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

MaxU - stand with Ukraine Over a year ago

yep, i think OP just has to find the correct encoding for his CSV file...

geilelou Over a year ago

Ok, actually I think I found my problem, latin_1 is working fine, but later in my code I use this: df_content_mail = df_content_mail.replace('/', ' ') which send an error 'ascii' codec can't decode byte 0xe9 in position 1: ordinal not in range(128)

Neill Herbst Over a year ago

@geilelou Not sure if ascii will work though. I tested your words with read_csv and ascii did not work for me

geilelou Over a year ago

I found one solution, it is to use re.sub(), df.replace() doesn't deal well with latin_1

Shekhar Samanta · Accepted Answer · 2016-10-18 08:32:02Z

0

You can use codecs in python2.7

import codecs
file = codecs.open(filename, encoding="utf-8")

answered Oct 18, 2016 at 8:32

Shekhar Samanta

9152 gold badges12 silver badges26 bronze badges

Collectives™ on Stack Overflow

Python 2.7 encoding from csv file

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related