numpy reading a csv file to an numpy array

Question

I am new to python and using numpy to read a csv into an array .So I used two methods:

Approach 1

train = np.asarray(np.genfromtxt(open("/Users/mac/train.csv","rb"),delimiter=","))

Approach 2

with open('/Users/mac/train.csv') as csvfile:
        rows = csv.reader(csvfile)
        for row in rows:
            newrow = np.array(row).astype(np.int)
            train.append(newrow)

I am not sure what is the difference between these two approaches? What is recommended to use?

I am not concerned which is faster since my data size is small but instead concerned more about differences in the resulting data type.

Aside from @Lucas great suggestion, the use case also depends on whether your data contains a mixture of different data types, or is more heterogeneous. — dennlinger
– dennlinger, Commented Sep 10, 2018 at 6:56
Possible duplicate of The fastest way to read input in Python — Daniel F
– Daniel F, Commented Sep 10, 2018 at 7:30
What is recommended to use? This is a broad question. What specifically are you concerned about? If it's not performance, is it readability, or something else? — jpp
– jpp, Commented Sep 10, 2018 at 8:58

Rachit Tayal · Accepted Answer · 2018-09-10 11:34:50Z

2

You can use pandas also, it is better and simple to use.

import pandas as pd
import numpy as np

dataset = pd.read_csv('file.csv')
# get all headers in csv
values = list(dataset.columns.values)

# get the labels, assuming last row is labels in csv
y = dataset[values[-1:]]
y = np.array(y, dtype='float32')
X = dataset[values[0:-1]]
X = np.array(X, dtype='float32')

answered Sep 10, 2018 at 11:34

Rachit Tayal

1,29217 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

hpaulj · Accepted Answer · 2018-09-10 11:07:35Z

1

So what is the difference in the result?

genfromtxt is the numpy csv reader. It returns an array. No need for an extra asarray.

The second expression is incomplete, looks like would produce a list of arrays, one for each line of the file. It uses the generic python csv reader which doesn't do much other than read a line and split it into strings.

answered Sep 10, 2018 at 11:07

hpaulj

233k14 gold badges260 silver badges392 bronze badges

3 Comments

Ricky Over a year ago

But when I return it is a list instead of an array

hpaulj Over a year ago

That's what I was trying to point out. The 2 methods return different things.

Ricky Over a year ago

So to convert the list to array I used np.asarray

Collectives™ on Stack Overflow

numpy reading a csv file to an numpy array

2 Answers 2

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related