Write ascii output from numpy arrays

Question

I want to write some random numbers into an ascii output file. I generate the numbers with numpy, so the numbers are stored in numpy.array

import numpy as np
random1=np.random.uniform(-1.2,1.2,7e6)
random2=...
random3=...

All three array are of the same size. I used standard file output, but this is really slow. Just about 8000 lines per 30 min. This may because I loop over three large arrays though.

fout1 = open("output.dat","w")

for i in range(len(random1)):
  fout1.write(str(random1[i])+"\t"+ str(random2[i])+"\t"+ str(random3[i])+"\n")
fout1.close()

I also just used print str(random1[i])+"\t"+ str(random2[i])+"\t"+ str(random3[i]) and dumped everything in a file usind shell ./myprog.py > output.dat which seems a bit faster but still I am not satisfied with the output speed.

Any recommendations are really welcome.

Michael Wild · Accepted Answer · 2013-02-11 14:36:45Z

4

Have you tried

random = np.vstack((random1, random2, random3)).T
random.savetxt("output.dat", delimiter="\t")

edited Feb 11, 2013 at 14:36

answered Feb 11, 2013 at 9:26

Michael Wild

26.7k3 gold badges46 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

madzone Over a year ago

thanks for the good hint. Any recommendation how i delimit random1,random2,random3 in between by '\t' as in my example?

Michael Wild Over a year ago

Have you tried asking uncle google?

Jaime Over a year ago

@madzone Use the delimiter argument: savetext docs

madzone Over a year ago

Hi. This works fine but I want to delimit the entries in between the random numbers by '\t' not after random1,random2,random3 anyways thanks for the help.

Michael Wild Over a year ago

That's what the newline argument does. delimiter separates the columns, just as you want.

JensB · Accepted Answer · 2013-02-11 09:24:59Z

0

Im guessing the disk io is the most expensive operation you are doing.. You could try to create your own buffer to deal with this, instead of writing every line every loop buffer up say 100 lines and write them in one big block. Then experiment with this and see what the most benficial buffer size is

answered Feb 11, 2013 at 9:24

JensB

6,9324 gold badges63 silver badges110 bronze badges

Collectives™ on Stack Overflow

Write ascii output from numpy arrays

2 Answers 2

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related