assigning arrays from CSV with pandas module

Question

If I have a file of 100+ columns, how can I make each column into an array, referenced by the column header, without having to do header1 = [1,2,3], header2 = ['a','b','c'] , and so on..?

Here is what I have so far, where headers is a list of the header names:

import pandas as pd

data = []
df = pd.read_csv('outtest.csv')
for i in headers:
    data.append(getattr(df, i).values)

I want each element of the array headers to be the variable name of the corresponding data array in data (they are in order). Somehow I want one line that does this so that the next line I can say, for example, test = headername1*headername2.

Could you provide an example input and expected output?

R. Max
– R. Max

2014-06-19 23:15:28 +00:00
Commented Jun 19, 2014 at 23:15 — R. Max
– R. Max, Commented Jun 19, 2014 at 23:15

user41047 · Accepted Answer · 2014-06-22 13:33:10Z

2

import pandas as pd

If the headers are in the csv file, we can simply use:

df = pd.read_csv('outtest.csv')

If the headers are not present in the csv file:

headers = ['list', 'of', 'headers']
df = pd.read_csv('outtest.csv', header=None, names=headers)

Assuming headername1 and headername2 are constants:

test = df.headername1 * df.headername2

Or

test = df['headername1'] * df['headername2']

Assuming they are variable:

test = df[headername1] * df[headername2]

By default this form of access returns a pd.Series, which is generally interoperable with numpy. You can fetch the values explicitly using .values:

df[headername1].values

But you seem to already know this.

edited Jun 22, 2014 at 13:33

answered Jun 21, 2014 at 19:30

user41047

3261 silver badge4 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Aaron Hall · Accepted Answer · 2014-06-19 18:25:28Z

1

I think I see what you're going for, so using a StringIO object to simulate a file object as the setup:

import pandas as pd 
import StringIO

txt = '''foo,bar,baz
1, 2, 3
3, 2, 1'''
fileobj = StringIO.StringIO(txt)

Here's the approximate code you want:

data = []
df = pd.read_csv(fileobj)
for i in df.columns:
    data.append(df[i])

for i in data: 
    print i

prints

0    1
1    3
Name: foo
0    2
1    2
Name: bar
0    3
1    1
Name: baz

answered Jun 19, 2014 at 18:25

Aaron Hall♦

400k93 gold badges416 silver badges342 bronze badges

Collectives™ on Stack Overflow

assigning arrays from CSV with pandas module

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related