How to run a script python on multiple files

Question

I have multiple .txt files in a folder (file1, file2, file3, file4, file5,....) and I need to run on a large number of files.

I've created this code to reorder and make some changes ... I would like to apply it on multiple txt files from a original folder(path) and save them separately into another folder (inside the last folder) or another path

Can someone please help?

    import pandas as pd
    import numpy as np
    import os
    import glob
    import datetime
    #from datetime import datetime

    df1=pd.read_csv("D:\\Spyder2019\\38A.txt", sep='\s+') 

#############################################33#
#to change name of column and add Temp ms column

    df1=df1.rename(index=str, columns={"Temps":"Heure", "Force":"Force(N)", "Vitesse":"Vitesse(RPM)", "Puissance":"Puissance(w)","Torque":"Torque(N/m)","Angle":"Angle(deg)"})#to change name Temps to Heure

    dtemp=df1['Heure']#to change datetime values into seconds and microseconds in Heure

    dtemp=pd.to_datetime(df1.Heure) #change to datetime values float 

    dtemp1=dtemp.dt.microsecond #dtemp1 object in microseconds with 6 decimals

    dtempms=dtemp1 / 1000000 #dtemp1 object into 2 decimals

    dtemp2=dtemp.dt.second #dtemp2 object in seconds

    df1['Temps_ms']= dtempms + dtemp2 #add second and microseconds to the object

    #################################################
    # to transform the Heure data into sec and microsec

    df1=df1[['Date','NoBille','Heure','Temps_ms','Force(N)','Vitesse(RPM)','Puissance(w)','Torque(N/m)','Angle(deg)']]# reorder the dataframe

    io=df1.iat[0,3]
    #to transform the Heure data into seconds ans microseconds

    df1['Temps(ms)'] = np.where(df1['Temps_ms'] - io <0, df1['Temps_ms'] + 60 - io, df1['Temps_ms'] -io)

    df1=df1.drop(columns=['Temps_ms'])#to eliminate column Temps_ms

    df1=df1[['Date','NoBille','Heure','Temps(ms)','Force(N)','Vitesse(RPM)','Puissance(w)','Torque(N/m)','Angle(deg)']]# to reorder the final dataframe

    ##############################################################################
    print(df1.head())

    df1.to_csv('data1.txt', index = False)

cccnrc · Accepted Answer · 2019-06-19 04:02:51Z

1

If this script can be applied to all databases that you have in your "input folder" you can pass the DB name as argument to python script:

import sys

#...your other imports...

### take the first argument passed in commmand line as db_name and second as output_file name
db_name = sys.argv[1]
output_filename = sys.argv[2]
df1 = pd.read_csv(db_name, sep='\s+') 

#...rest of your script...

df1.to_csv(output_filename, index = False)

and then yuo can easily call the python script over the files in the input directory with an external loop, as ex. in bash:

mkdir <output-dir>
ls <input-dir> | while read file; do
    output_filename="<output-dir>/$file.output.txt"
    python <your-script-name> $file $output_filename
done

and it will automatically take the files, operate the python script and write them down in output-dir with the specified extension

answered Jun 19, 2019 at 4:02

cccnrc

1,2492 gold badges16 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Master03 Skywalker Over a year ago

Thanks!! but I still have problems. Any another option?

cccnrc Over a year ago

write what you tried and exactly the error you encountered (stackoverflow.com/help/how-to-ask)

Collectives™ on Stack Overflow

How to run a script python on multiple files

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related