parsing from a file in python

Question

When I write a date in a terminal, I execute my code like this:

python code.py " date"

For example python code.py 2017-05-14, but I have more than 1500 dates. I have saved them in a file.txt.

But I don't know how I can adapt parsing in my code to recuperate all the variables from the file

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    with open(...) as f:
    for line in f:
        <do something with line>
    parser.add_argument("date", help="date format YYYY-MM-DD", type=str)
    parser.add_argument("--output", help="csv output filepath",type=str)
    args = parser.parse_args()

    region = {
        'France':'France',
        'ACA':'Grand-Est',
        'ALP':'Nouvelle-Aquitaine',
        'ARA':'Auvergne-Rhônes-Alpes',
        'BFC':'Bourgogne-Franche-Comté',
        'BRE':'Bretagne',
        'CEN':'Centre-Val de Loire',
        'IDF':'Ile-de-France',
        'LRM':'Occitanie',
        'NPP':'Hauts-de-France',
        'NOR':'Normandie',
        'PLO':'Pays-de-Loire',
        'PAC':'PACA',
    }

    # french date format...
    datefr = args.date[-2:] + '/' + args.date[5:7] + '/' + args.date[:4]

    if args.output:
        output = args.output
    else:
        output = 'eco2mix-' + args.date + '.csv'

    # grab all regions...
    df = [grab_params({'region':k,'date':datefr}) for k in region.keys()]
    df = pd.concat(df)
    assert len(df) == 96*len(region.keys())

    # NOTE : patching malformed date...
    df['Date'] = [i if '-' in i else i[-4:]+'-'+i[3:5]+'-'+i[:2] for i in df['Date']]

    assert len(set(df.Date)) == 1

    # NOTE : remove - empty values by NaN
    df.replace(to_replace = '-', value = np.NaN, inplace=True)
    df.replace(to_replace = 'ND', value = np.NaN, inplace=True)

df.to_csv(output,index = False, encoding = 'UTF8')

Colwin · Accepted Answer · 2017-05-17 14:22:58Z

1

From what I understand you want to parse the file with dates and pass each one to your script correct? You can write a quick bash script to do that

while read p; do
   python code.py $p
done <file.txt

You can save this as a script.sh file and run it to iterate over each date in your file

answered May 17, 2017 at 14:22

Colwin

2,6913 gold badges25 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Amal Kostali Targhi Over a year ago

thank you but it does not work i get this error AttributeError: 'Namespace' object has no attribute

Colwin Over a year ago

Can you update your question with the traceback? That looks like a python error in your code.py script. The bash script is only calling your script

Amal Kostali Targhi Over a year ago

i have not error just the code abose if for a date my i have more than 1000 dates so i cannot in every time write my date so i have all the dates in a file and in need to browse them

Colwin Over a year ago

Yes I understand. That bash script I've provided you with does exactly that it reads your file with the dates and then passes it to your code.py script

Amal Kostali Targhi Over a year ago

i did it but i get an error Traceback (most recent call last): File "rte.py", line 74, in <module> assert len(set(df.Date)) == 1 AssertionError

|

aquil.abdullah · Accepted Answer · 2017-05-17 14:28:22Z

1

Why not just add a date_file option to your argument parser?

parser.add_argument("-f", "--date-file", dest="date_file", help="file with dates in format YYYY-MM-DD", type=str, default=None)

You could then modify your code such that if you enter a date file you do something like:

dates = [args.date]
if args.date_file is not None:
    dates = list()
    with open(args.date_file) as handle:
        dates = [x.strip() for x in handle.read()]

Now you have all the dates and you can:

for date in dates:
    # processing code goes here.

You can now execute your code like:

python code.py -f dates.txt

answered May 17, 2017 at 14:28

aquil.abdullah

3,1673 gold badges24 silver badges40 bronze badges

2 Comments

Amal Kostali Targhi Over a year ago

thaaank you but i did not undrstand this part for date in dates: (what should i do here)

aquil.abdullah Over a year ago

It wasn't clear to me exactly what you were trying to do with the dates, so I just put them in a for loop. If you are using a pandas DataFrame then you can use a boolean expression to index into it using the dates that you obtain from the file. It may be that I misunderstand your question.

whodini9 · Accepted Answer · 2017-05-17 14:28:39Z

1

Here's a quick example of parsing a file of dates into a List

dates.py

from datetime import datetime
import sys

dates = []
date_file = open(sys.argv[1])
for line in date_file:
    formatted_date = datetime.strptime(line.rstrip(), "%Y-%m-%d")
    dates.append(formatted_date)
    # Implement code here, or loop through dates later

print(dates)

dates.txt

python dates.py dates.txt returns

[datetime.datetime(2017, 5, 14, 0, 0), datetime.datetime(2017, 5, 14, 0, 0), datetime.datetime(2017, 5, 14, 0, 0), datetime.datetime(2017, 5, 14, 0, 0)]

answered May 17, 2017 at 14:28

whodini9

1,43413 silver badges18 bronze badges

Collectives™ on Stack Overflow

parsing from a file in python

3 Answers 3

8 Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

8 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related