Reformatting Dates after strptime Python

Question

I am trying to read in two columns from a .txt file and reformat them. The first column is the date, with the format: %Y-%m-%d %H:%M:%S. The second column is a precipitation value, which I need to manipulate. Ultimately, I will need to make another tab delimited .txt file with a column containing the day, a column containing the hour, a column containing the minute, and a column containing the precipitation*0.2. My code is below:

#read in file
def readfiles(file_list):
    data = []
    for fname in file_list:
        data.append(
                    np.genfromtxt(fname,
                               usecols=(0,5),
                               comments='#',    # skip comment lines
                               delimiter='\t',
                               dtype="|S", autostrip=True).T)
    return data
data = readfiles(['soundTransit1_remote_rawMeasurements_15m.txt'])

np.set_printoptions(threshold=np.nan)

#create array containing desired precipitation values
precip = np.array(data, dtype='|S4')[:,1]
precip = precip.astype(np.float)
precip_mm = precip * 0.2

#strip date and time
for i,d in enumerate(data):
    x = [dt.datetime.strptime(date,'%Y-%m-%d %H:%M:%S') for date in d[0]]

I've gotten this far, but the variable 'x' looks like this when printed:

[datetime.datetime(2015, 7, 11, 13, 30), datetime.datetime(2015, 7, 11, 13, 45), datetime.datetime(2015, 7, 11, 14, 0),

I'm not sure how to use this output and create the .txt file that I need. I also may be completely overcomplicating this, so I would be open to suggestions on how to restart the whole process from scratch.

An example of input and expected output could help clarifying what you're trying to achieve. — Nir Alfasi
– Nir Alfasi, Commented Jul 28, 2015 at 19:14
Instead of reading ALL the data and then create ALL the output, you can read one line of input and create one line of output with the manipulated data at a time. — Pedro Walter
– Pedro Walter, Commented Jul 28, 2015 at 19:18

Anand S Kumar · Accepted Answer · 2015-07-28 19:26:43Z

datetime.datetime objects have day , hour and minute attributes , which you can use to get the corresponding information -

Demo -

>>> import datetime
>>> d = datetime.datetime(2015, 7, 11, 13, 30)
>>> d.day
11
>>> d.hour
13
>>> d.minute
30

I think using numpy module for this may really be overkill, you can easily use csv module for this.

Code -

import csv
import datetime
with open('test.txt','r') as infile, open('output.txt','w') as outfile:
    inr = csv.reader(infile,delimiter='\t')
    ouw = csv.writer(outfile,delimiter='\t')
    for row in inr:
            d = datetime.datetime.strptime(row[0],'%Y-%m-%d %H:%M:%S')
            p = float(row[1])
            nr = [d.day, d.hour, d.minute, p*0.2]
            ouw.writerow(nr)

Demo -

test.txt looks like -

2015-07-29 12:40:22    1
2015-07-28 17:40:22    2
2015-07-27 08:22:22    3
2015-07-24 12:40:22    4

Above code on this csv produces output.txt as -

29    12    40    0.2

28    17    40    0.4

27    8    22    0.6000000000000001

24    12    40    0.8

Ethan Furman · Accepted Answer · 2015-07-28 19:15:30Z

0

The p in strptime stands for parse -- you want strftime. And good luck in remembering that -- I have to look it up every time. :(

answered Jul 28, 2015 at 19:15

Ethan Furman

70.1k21 gold badges174 silver badges251 bronze badges

Comments

Padraic Cunningham · Accepted Answer · 2015-07-28 19:32:26Z

If you just want to write the data to another file, you can do it all with datetime and the csv module:

import csv
from datetime import datetime

with open("in.txt") as f, open("out.txt", "w") as out:
    wr = csv.writer(out,delimiter="\t")
    r = csv.reader(f,delimiter="\t")
    # write header
    wr.writerow(["Day", "Hour", "Min", "Prec"])
    for row in r:
        # unpack the row getting date and precip value from input file
        tme, pre = row
        # create datetime object 
        dt = datetime.strptime(tme, "%Y-%m-%d %H:%M:%S")
        # get the day, hour, minute from the datetime object
        # multiply the precip value by .2 and write the row
        wr.writerow([dt.day, dt.hour, dt.minute,float(pre)*.2])

If you want the full weekday name use dt.strftime("%A"), abbreviated weekday is "%a", as is the code will write the day as a decimal. If you input file has a header don't forget to call next on the file object to skip it.

All the strftime options are listed here

Collectives™ on Stack Overflow

Reformatting Dates after strptime Python

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related