Any simple library or function to parse a csv encoded string and turn it into an array or dictionary?
All the built in csv module examples take filepaths, not strings.
Add a comment
|
11 Answers
You can convert a string to a file object using io.StringIO and then pass that to the csv module:
from io import StringIO
import csv
scsv = """text,with,Polish,non-Latin,letters
1,2,3,4,5,6
a,b,c,d,e,f
gęś,zółty,wąż,idzie,wąską,dróżką,
"""
f = StringIO(scsv)
reader = csv.reader(f, delimiter=',')
for row in reader:
print('\t'.join(row))
simpler version with split() on newlines:
reader = csv.reader(scsv.split('\n'), delimiter=',')
for row in reader:
print('\t'.join(row))
Or you can simply split() this string into lines using \n as separator, and then split() each line into values, but this way you must be aware of quoting, so using csv module is preferred.
On Python 2 you have to import StringIO as
from StringIO import StringIO
instead.
8 Comments
Carson Myers
the split method wouldn't work if his csv file contained strings which contained commas
adamk
or quoted strings as values (with or without commas)
JStrahl
Python 3 now uses io.StringIO. (Hopefully save Python 3 users a little time). so import io and io.StringIO.
Denilson Sá Maia
Instead of
.split('\n'), you can use .splitlines().
Michał Niklas
No, it works very well with Polish letters with ogonki :-)
|
Simple - the csv module works with lists, too:
>>> a=["1,2,3","4,5,6"] # or a = "1,2,3\n4,5,6".split('\n')
>>> import csv
>>> x = csv.reader(a)
>>> list(x)
[['1', '2', '3'], ['4', '5', '6']]
4 Comments
Inaimathi
Good to know, but keep in mind that
.split('\n') will do odd things if your fields contain newlines.
John La Rooy
@Inaimathi, If it's csv, the newlines inside should be escaped.
Jonathan Stray
Newlines don't need to be escaped if the field is quoted.
piotrekkr
Also works nice with csv line from
stdin like echo '"test 1","test 2"' | python -c 'import sys, csv; print(list(csv.reader(sys.stdin))[0])'The official doc for csv.reader() https://docs.python.org/2/library/csv.html is very helpful, which says
file objects and list objects are both suitable
import csv
text = """1,2,3
a,b,c
d,e,f"""
lines = text.splitlines()
reader = csv.reader(lines, delimiter=',')
for row in reader:
print('\t'.join(row))
Comments
And while the module doesn’t directly support parsing strings, it can easily be done:
import csv
for row in csv.reader(['one,two,three']):
print row
Just turn your string into a single element list.
Importing StringIO seems a bit excessive to me when this example is explicitly in the docs.
1 Comment
gowthz
The whole of the string cannot be a single element. It has to be split and then passed as multiple element list.
As others have already pointed out, Python includes a module to read and write CSV files. It works pretty well as long as the input characters stay within ASCII limits. In case you want to process other encodings, more work is needed.
The Python documentation for the csv module implements an extension of csv.reader, which uses the same interface but can handle other encodings and returns unicode strings. Just copy and paste the code from the documentation. After that, you can process a CSV file like this:
with open("some.csv", "rb") as csvFile:
for row in UnicodeReader(csvFile, encoding="iso-8859-15"):
print row
Not a generic CSV parser but usable for simple strings with commas.
>>> a = "1,2"
>>> a
'1,2'
>>> b = a.split(",")
>>> b
['1', '2']
To parse a CSV file:
f = open(file.csv, "r")
lines = f.read().split("\n") # "\r\n" if needed
for line in lines:
if line != "": # add other needed checks to skip titles
cols = line.split(",")
print cols
3 Comments
Abdelouahab
'Simple is better than complex!'
Zz'Rot
-1 The issue with this solution is that it doesn't take into account of "string escaping," i.e.
3, "4,5,6, 6 shall be treated as three fields instead of five.
Christophe Roussy
Simple but only works in some specific cases, this is not generic CSV parsing code
https://docs.python.org/2/library/csv.html?highlight=csv#csv.reader
csvfile can be any object which supports the iterator protocol and returns a string each time its next() method is called
Thus, a StringIO.StringIO(), str.splitlines() or even a generator are all good.
Comments
Use this to have a csv loaded into a list
import csv
csvfile = open(myfile, 'r')
reader = csv.reader(csvfile, delimiter='\t')
my_list = list(reader)
print my_list
>>>[['1st_line', '0'],
['2nd_line', '0']]
Comments
Here's an alternative solution:
>>> import pyexcel as pe
>>> text="""1,2,3
... a,b,c
... d,e,f"""
>>> s = pe.load_from_memory('csv', text)
>>> s
Sheet Name: csv
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| a | b | c |
+---+---+---+
| d | e | f |
+---+---+---+
>>> s.to_array()
[[u'1', u'2', u'3'], [u'a', u'b', u'c'], [u'd', u'e', u'f']]
Here's the documentation
Comments
For anyone still looking for a reliable way of converting a standard CSV str to a list[str] as well as in reverse, here are two functions I put together from some of the answers in this and other SO threads:
def to_line(row: list[str]) -> str:
with StringIO() as line:
csv.writer(line).writerow(row)
return line.getvalue().strip()
def from_line(line: str) -> list[str]:
return next(csv.reader([line]))
Comments
For csv files:
data = blob.download_as_text()
pd.DataFrame(i.split(",") for i in data.split("\n"))
