Python/numpy append arrays, transpose, and save as csv

Question

What I am doing: generating a series of long 1D arrays.

What I want to do: append/concatentate/vstack/? these into a 2D array, then save the rows as columns in a csv file.

The following works, but it's not elegant:

rlist=[]                        # create empty list
for i in range(nnn)             # nnn is typically 2000
    (calculate an array "r")
    rlist.append(r)             # append f.p. array to rlist
rarr = array(rlist)             # turn it back into array so I can transpose
numpy.savetxt('test.csv',rarr.T,delimiter=',')  # save rows as columns in csv file

Is there a more elegant or pythonesque way to do it?

Why do you say this is not elegant? Sure, you could use numpy.vstack and a transpose -- But in the end, it will be the same number of lines of code and you'll either be growing the same array at every iteration of the loop (which is likely to be inefficient) or waiting until the end to stack them all together (which is what you're doing now). I don't see much difference ... — mgilson
– mgilson, Commented Aug 2, 2012 at 12:24

mtrw · Accepted Answer · 2012-08-02 12:56:02Z

1

If you know the length of r and nnn in advance, you can do:

rarr = np.zeros((r_len, nnn)) # r_len rows, nnn columns
for i in range(nnn):
    rarr[:,i] = r_calc()
numpy.savetxt('test.csv', rarr, delimiter=',')

This puts the data vectors directly into rows of rarr, saving you the conversion to array and transpose.

answered Aug 2, 2012 at 12:56

mtrw

35.3k7 gold badges66 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

senderle Over a year ago

Preallocating the array is a good approach. I'm not sure it is any more elegant, but it is likely to be a bit faster.

Nicolas Barbey · Accepted Answer · 2012-08-02 13:14:03Z

0

rarr = np.concatenate([get_array(r) for r in xrange(nnn)])
np.savetxt('test.cvs', rarr.T, delimiter=',')

answered Aug 2, 2012 at 13:14

Nicolas Barbey

6,8274 gold badges30 silver badges35 bronze badges

1 Comment

dcnicholls Over a year ago

Thanks for those suggestions. I'm a real Python newbie, so I expect my code to be ugly. Maybe not as bad as I thought! I'll work through the ideas. Excellent way to learn Python, but (coming from FORTRAN) Python's methods aren't alway obvious.

Collectives™ on Stack Overflow

Python/numpy append arrays, transpose, and save as csv

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related