How to parse a string using a CSV parser in Python?

Question

I need to parse a string using a CSV parser. I've found this solution in many places, but it doesn't work for me. I was using Python 3.4, now I changed it to 2.7.9 and still nothing...

import csv
import StringIO

csv_file = StringIO.StringIO(line)
csv_reader = csv.reader(csv_file)
for data in csv_reader:
      # do something

Could anyone please suggest me another way to parse this string using a CSV parser? Or how can I make this work?

Obs: I have a string in a CSV format, with fields that have commas inside, that's why I can't parse it in the standard way.

Please read the guide on how to construct an MCVE; you didn't even explain what isn't working. — DSM
– DSM, Commented Mar 21, 2015 at 3:28

Community · Accepted Answer · 2021-10-07 06:01:06Z

You need to put double quotes around elements that contain commas.

The CSV format implements RFC 4180, which states:

Fields containing line breaks (CRLF), double quotes, and commas should be enclosed in double-quotes.

So for instance (run code here.):

import StringIO
import csv

# the text between double quotes will be treated 
# as a single element and not parsed by commas
line = '1,2,3,"1,2,3",4'

csv_file = StringIO.StringIO(line)
csv_reader = csv.reader(csv_file)
for data in csv_reader:
    # output: ['1', '2', '3', '1,2,3', '4']
    print data

As another option, you can change the delimiter. The default for csv.reader is delimiter=',' and quotechar='"' but both of these can be changed depending on your needs.

Semicolon Delimiter:

line = '1;2;3;1,2,3;4'

csv_file = StringIO.StringIO(line)
csv_reader = csv.reader(csv_file, delimiter=';')
for data in csv_reader:
    # output: ['1', '2', '3', '1,2,3', '4']
    print data

Vertical Bar Quotechar

line = '1,2,3,|1,2,3|,4'

csv_file = StringIO.StringIO(line)
csv_reader = csv.reader(csv_file, quotechar='|')
for data in csv_reader:
    # output: ['1', '2', '3', '1,2,3', '4']
    print data

Also, the python csv module works on python 2.6 - 3.x, so that shouldn't be the problem.

Ron Thompson · Accepted Answer · 2015-03-21 04:12:18Z

0

The obvious solution that jumps out of the page, rather than reimplementing CSV parsing, is to preprocess the data first and replace all of the commas within strings by some never used token character (or even the word COMMA), then feeding that into the CSV parser, and then going back through the data and replacing the tokens back with commas.

Sorry, I've not tried this myself in Python, but I had issues with quotes in my data in another language, and that's how I solved it.

Also, Bcorso's answer is much more complete. Mine is just a quick hack to get around a common limitation.

edited Mar 21, 2015 at 4:12

answered Mar 21, 2015 at 3:29

Ron Thompson

1,0966 silver badges13 bronze badges

Collectives™ on Stack Overflow

How to parse a string using a CSV parser in Python?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related