Read an undefined number of columns in Python and plot data with matplotlib

Question

In a python code I do the following:

x1=[]
y1=[]
y2=[]
y3=[]
y4=[]



for line in res_file_1.readlines():
    ll=string.split(line)
    x1.append(float(ll[0]))
    y1.append(float(ll[1]))
    y2.append(float(ll[2]))
    y3.append(float(ll[3]))
    y4.append(float(ll[4]))

So it reads 4 columns of a file and stores values in the different "y" arrays. I wonder how could I do this more general, i.e., when I do not know whether the input file will contain 4, or 16 or 2000 columns

UPDATE:

Later I need to plot data with matplotlib, so I usually do:

ax.plot(x1,y1,'ks',color='red')
ax.plot(x1,y2,'ks',color='green')
ax.plot(x1,y3,'ks',color='blue')

How can I do the plot for the undefined number of columns (and colors)?

bmu · Accepted Answer · 2013-03-25 19:36:26Z

3

If you have a data file input.csv like this:

x1, y1, y2, y3, y4
A, 3, 6, 7, 1
B, 4, 5, 9, 0
X, 7, 2, 8, 5
C, 4, 3, 2, 1

With pandas you would do something like this:

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('input.csv', delimiter=',', index_col='x1')
df.plot(marker='s')
plt.show()

and you will get this plot:

enter image description here

edited Mar 25, 2013 at 19:36

answered Jul 4, 2012 at 18:05

bmu

36.4k13 gold badges93 silver badges108 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Open the way Over a year ago

ok, and what can you do in pandas when the x1 column is not filled with numbers but just letters? and the rest of y columns are numbers

Open the way Over a year ago

Sorry, I ran exactly that code on Mac OS X 10.7, there is no error but no output at all. I installed pandas using macports and run it using python2.7 plot.py, why the error?

bmu Over a year ago

I think you should ask this as another question or open an issue on github. It should work on MAC OS (I'm on linux).

bmu Over a year ago

Sorry, I was running this in ipython --pylab. Added the show call.

georg · Accepted Answer · 2012-07-04 15:04:58Z

2

from itertools import izip_longest
with open(filename, 'r') as f:
    data = list(izip_longest(*[x.split() for x in f]))

This creates a list of tuples, where each tuple contains values from one column.

answered Jul 4, 2012 at 15:04

georg

216k57 gold badges324 silver badges401 bronze badges

Comments

nye17 · Accepted Answer · 2012-07-04 19:09:07Z

2

use numpy.genfromtxt and read the columns into recarrays, and when you want to loop through the columns for plotting, use recarray.dtype.names to retrieve each column.

edited Jul 4, 2012 at 19:09

answered Jul 4, 2012 at 15:28

nye17

13.5k11 gold badges62 silver badges69 bronze badges

1 Comment

nye17 Over a year ago

@flow paste me the data file (a few lines would be fine) and I can come up with some code snippets.

Jordan Dimov · Accepted Answer · 2012-07-04 15:04:51Z

1

Use a dictionary, where they keys are x and the values are lists of y's:

data = {}
for line in res_file_1.readlines():
    ll = line.split()
    data[ll[0]] = ll[1:]

answered Jul 4, 2012 at 15:04

Jordan Dimov

1,32812 silver badges28 bronze badges

3 Comments

Jordan Dimov Over a year ago

this is assuming that ordering of rows isn't important

Open the way Over a year ago

how can one access the different elements, for instance, for x=1, the element of column3? I am trying to do it

Jordan Dimov Over a year ago

data[1][2] will give you the value of x=1, y=3

Collectives™ on Stack Overflow

Read an undefined number of columns in Python and plot data with matplotlib

4 Answers 4

4 Comments

Comments

1 Comment

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

4 Comments

Comments

1 Comment

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related