Storing and retrieving lists from csv file in python

Question

Simple question: Can I store lists in a CSV file? If so, how do I retrieve it?

Here is the file content that I wrote to the file, and I'm now having a hell of a time trying to retrieve it.

40;['5', '0'];['15', '0'] ; ['25', '0'] ; ['30', '0'] ; ['40', '0'] ; ['50', '0'].....   
50; ['0', '0'] ; ['10', '0'] ; ['15', '0'] ; ['25', '0'] ; ['30', '0'] ; ['40', '0'].....
60; ['0', '0'] ; ['10', '0'] ; ['15', '0'] ; ['20', '0'] ; ['25', '0'] ; ['30', '0'].....
70; ['0', '0'] ; ['5', '0'] ; ['10', '0'] ; ['15', '0'] ; ['20', '0'] ; ['30', '0'].....

And this is some code I've played with to try and retrieve it:

import csv
reader = csv.reader(open("NewTableInfo", "r"), delimiter=';')
for row in reader:
    print (row)

This results in a list in which the values that are in brackets are type string. I want them to be lists (In another file I make each row a dictionary item).

['40', "['5', '0']", "['15', '0'] ", " ['25', '0'] ", " ['30', '0'] "......

Any ideas? Is it possible to write CSV files, in the way that I did above, and have them be usable?

No, I'm just learning programming, and this is more of a test of what I can do with it. What I'm finding out is that it is better to use JSON or cPickle, both of which I'm reading up on now. Any suggestions on which would be better for this sort of data? — shteeven
– shteeven, Commented Oct 8, 2012 at 5:35
JSON if you're planning on reading the file with another language or want it to be human-readable/editable. Pickle for everything else. — Blender
– Blender, Commented Oct 8, 2012 at 5:36

jamylak · Accepted Answer · 2012-09-29 06:13:38Z

3

Doing what you have said is possible:

import csv
import ast

with open("NewTableInfo", "r") as f:
    reader = csv.reader(f, delimiter=';')
    for row in reader:
        print [ast.literal_eval(x.strip()) for x in row]

However I would recommend using pickle instead, this is not a good way of saving data.

answered Sep 29, 2012 at 6:13

jamylak

134k30 gold badges238 silver badges240 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

shteeven Over a year ago

Oh nice! I've done something similar, but with just eval. I just read up on ast.literal_eval, and apparently it's secure, unlike eval. Though I'll either do it in cPickle or JSON next, I'll keep the ast.literal_eval for future use.

BrenBarn · Accepted Answer · 2012-09-29 06:08:12Z

2

In general CSV does not handle lists well. It is possible to get your data out, but you'll have to parse the string representation of the list yourself. An alernative is to use a format that does support list data, like JSON.

The quick-and-dirty but insecure way to get your lists out of the CSV is to use eval on the strings you get back. If you have the string "['5', '0']", then eval-ing it will give you the list back:

>>> eval("['5', '0']")
['5', '0']

However, as I said, this is insecure, so you should only use it if you're dealing with CSV files you created yourself, so you know they only contain lists of numbers and the like. Don't eval strings you don't control; people could make malicious strings that, when eval-ed, delete files on your hard drive or do other evil things.

If you can't use eval because of security concerns, you will have to parse the list format yourself. This could be a fairly onerous task. If you need to do this, you might be better of choosing a different format for your data, like perhaps JSON.

answered Sep 29, 2012 at 6:08

BrenBarn

253k39 gold badges421 silver badges392 bronze badges

2 Comments

shteeven Over a year ago

Thank you so much. I'll do eval for this particular project, and probably focus on learning JSON for the future. Thanks again.

jamylak Over a year ago

In this case ast.literal_eval can be used and it's safe unlike eval although I still agree that JSON or pickle would be better

monkut · Accepted Answer · 2012-09-29 08:37:50Z

0

you could also use json. is there a reason for csv?

import json

# write
out = open("datafile.json", "w")
data = [['5', '0'],['15', '0'] ,['25', '0'] ,['30', '0'] ,['40', '0'], ['50', '0']]
json_out_data = json.dumps(data)
out.write(json_out_data)
out.close()

input = open("datafile.json", "r")
data = json.loads(input.read())

answered Sep 29, 2012 at 8:37

monkut

44.1k26 gold badges133 silver badges161 bronze badges

Collectives™ on Stack Overflow

Storing and retrieving lists from csv file in python

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related