How to use Python's CSV DictReader to parse a CSV output?

Question

I have integrated Salesforce's Bulk API for fetching records in my Python project. With 'Content-Type': 'text/csv; charset=UTF-8' header, it returns CSV Output.

"Id","Name","CreatedById","Salary","Base_Salary","Type","Pay_cycle","Description","Code"
"a0u90000003R4Y9AAK","Freelance Hadoop developer","005900000039GMdAAM","","","","","",""
"a0u90000003R5UPAA0","Senior Hadoop developer","005900000039GMdAAM","","","","","",""
"a0u90000003R5V3AAK","Freelance Webmaster","005900000039GMdAAM","","","","","",""
"a0u90000004ZMUwAAO","Full-Stack Web Developer - PHP and Python","005900000039GMdAAM","","","","","",""

Now, without storing this in a CSV file, I want to put these records this in my Db.

The DictReader is a really nice and straightforward solution for getting the field values, But I guess it only works with a (CSV) file.

import csv
with open('output.csv') as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:
        print(row['Id'], row['Name'])

I am trying something but I don't like it. See

objects = []
for line in iter(csv_data.splitlines()):
    # Split by separator
    data = line.split(",")
    # Remove double quotes around the field value
    objects.append([i[1:-1] for i in data])

What will be the most elegant way of doing this?

hiro protagonist · Accepted Answer · 2015-11-03 11:45:50Z

5

you can wrap the csv string into a io.StringIO object. this will then work perfectly fine with the csv module:

data_str = '''"Id","Name","CreatedById","Salary","Base_Salary","Type","Pay_cycle","Description","Code"
"a0u90000003R4Y9AAK","Freelance Hadoop developer","005900000039GMdAAM","","","","","",""
"a0u90000003R5UPAA0","Senior Hadoop developer","005900000039GMdAAM","","","","","",""
"a0u90000003R5V3AAK","Freelance Webmaster","005900000039GMdAAM","","","","","",""
"a0u90000004ZMUwAAO","Full-Stack Web Developer - PHP and Python","005900000039GMdAAM","","","","","",""'''

# in python 2 you need to decode the string before passing it to StringIO
data_io = io.StringIO(data_str.decode('utf-8'))
## this is not needed in python 3
# data_io = io.StringIO(data_str)

reader = csv.DictReader(data_io)
for row in reader:
    print row['Id'], row['Name']

edited Nov 3, 2015 at 11:45

answered Nov 3, 2015 at 11:17

hiro protagonist

47.3k17 gold badges93 silver badges119 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Hussain Over a year ago

This is great. But I am getting an exception TypeError: initial_value must be unicode or None, not str in data = io.StringIO(csv_output)

hiro protagonist Over a year ago

oh, you are on python 2.7, right? or what is you python version?

hiro protagonist Over a year ago

then you need to decode the string appending a .decode('utf-8') before you pass it to io.StringIO.

Hussain Over a year ago

Yes. I see that here too stackoverflow.com/questions/22316333/…

Hussain Over a year ago

Now it works perfect! Thanks! Upvoted and accepted your answer. You should put .decode('utf-8') thing in your it.

|

Collectives™ on Stack Overflow

How to use Python's CSV DictReader to parse a CSV output?

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related