Save array data in one output file using for loop Python

Question

I need someones opinion on how to extract and save the array data in one output file from this script. Files3 contain set number dataset files in Latitude, Longitude and Rainfall. Each dataset is arranged just like this:

Latitude    Longitude   Rainfall
-5.000    95.000     0.000
-4.900    95.000     0.000
-4.800    95.000     0.000
-4.700    95.000     0.000
-4.600    95.000     0.000

Objective : How to save all the dataset especially the rainfall and saved in file into this array. The problem is the output only picked up only one dataset.

Latitude  Longitude  Rainfall1  Rainfall2  Rainfall3 Rainfall4 . . 
   -5.000    95.000     0.000       0.000       0.000     0.000
   -4.900    95.000     5.7 0       0.000       0.000     23.9
   -4.800    95.000     0.000       10.4        0.000     0.000
   -4.700    95.000     0.000       0.000       0.000     0.000
   -4.600    95.000     0.000       0.000       0.000     0.000

The halfway script that I am using is just as follows:

import numpy as np
import matplotlib.pyplot as plt

files=np.loadtxt('files3',dtype=str)

for x in files:
    data=np.loadtxt(x)
    lat=data[:,0]
    lon=data[:,1]
    rain=data[:,2]
    rain=[lat,lon,rain]
    data1=np.array(rain)
    data1=data1.T
    np.savetxt( 'build-mean1.dat2',data1,fmt='%9.3f')

You are calling np.savetxt inside the loop, so at each iteration it will just replace the file with a new dataset, and you will end up only with the last one in the file — user2261062
– user2261062, Commented Nov 23, 2017 at 10:35
yes they are identical for all files....how to dump all the data for each data into saved files?... — Azam
– Azam, Commented Nov 23, 2017 at 11:33
I have removed np.savetxt outside the loop but remain the same. I guess the mistake must have been somewhere during declaration of rain=[lat,lon,rain] or may be another loop need to be imposed before it.. — Azam
– Azam, Commented Nov 23, 2017 at 11:43

Dlamini · Accepted Answer · 2017-11-24 13:12:06Z

1

The following works, I believe. Change the names of the files in my 'files3 declaration' to the names of your files, and the the number of files you like, or use glob.glob to automatically make them into a list called 'files3'. You can delete all of the print statements after you confirm the evolution of the array as data is added.

import numpy as np

files3 = ['brain-mean1.dat2','brain-mean2.dat2','brain-mean3.dat2']

for x in range(len(files3)):
    file = files3[x]
    data = np.loadtxt(file)
    lat = data[:,0]
    lon = data[:,1]
    rainR = data[:,2]
    rain = [lat,lon,rainR]
    rain = np.array(rain)
    print('new data')
    print(rain)
    print('rainfall only')
    print(rainR)
    if x==0:
        agRain = rain
        print(type(agRain))
    else:
        rainR = np.array(rainR)
        agRain = np.vstack((agRain, rainR))
    print('agRain')
    print(agRain)

data1 = agRain.T
print('data1')
print(data1)
np.savetxt('build-mean.dat2',data1,fmt='%9.3f')

answered Nov 24, 2017 at 13:12

Dlamini

2851 silver badge9 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Azam Over a year ago

Dear @Dlamini, oh my goodness, I really works. I'm so happy with these solution and I can continue my next step finding the statistics value. I really want to say thanks in millions for your brainy idea..

Dlamini Over a year ago

You are most welcome. Good luck with the rest of the project. Please click on the tick - I think it's just below the valuation arrows to show that the answer was useful. I have voted +1 for the question because I learned something while answering it.

Azam Over a year ago

I managed to impose files=np.loadtxt('files3',dtype=str) into the script as the dataset contain 48 files representing half hourly rainfall for 24 hours..the result is amazing although the iteration took quite sometimes..

Azam Over a year ago

im still new here in this platform..how to make vote for all the answers?..

Dlamini Over a year ago

@NoorAzam you can vote on questions after your reputation has reached at least 15. You should get there soon. There is a guide if you click the question mark next to your reputation etc on the top left corner of your page when logged into stackoverflow.

Collectives™ on Stack Overflow

Save array data in one output file using for loop Python

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related