This is sample from large csv file:
6.1;6.1;7.2;8.9;5.0;
8.9;10.0;8.9;6.1;5.0;
If I try to read it to numpy array with np.loadtxt('test.csv', delimiter=';') I get:
ValueError: could not convert string to float:
and don't understand why?
Add a comment
|
2 Answers
You need to strip off the trailing ';' from the lines.
A possible workaround if you know you have 5 columns is:
np.loadtxt('test.csv', delimiter=';', usecols=range(5))
Or, use genfromtext instead which handles missing values
np.genfromtxt('test.csv', delimiter=';')[:,:-1]
5 Comments
theta
You are right, but I can't strip trailing
; for some lines as there are missing values. Using usecols=range(5) if line ends with ;; (i.e. last value is missing) yields to error againtheta
Maybe I should use other module to read CSV. I tried
pandas right now, and it reads just fine as anyone would have expected, but OTOH I don't want to load huge package just to read CSV to array, and don't use it again...
wim
Nah I think numpy is a good choice, there is
csv module but it's more for strings ..theta
I just got a tip to use
np.genfromtxt('test.csv', delimiter=';') instead - works like a charm. I must have overlooked your last line
themadmax
Under Windows I have problem with no-ANSI encoded files
So in my case the csv file had column names written in the first row. e.g.
Column1,Column2,Column3
5.4,2.3,2.4
6.7,3.6,9.3
So as given in the docs, all I had to do was use the skiprows parameter
So the API call became,
np.loadtxt('test.csv', delimiter=',', skiprows=1)
