parse date-time while reading 'csv' file with pandas

Question

I am trying to parse dates while I am reading my data from cvs file. The command that I use is

df  = pd.read_csv('/Users/n....', names=names, parse_dates=['date']) )

And it is working on my files generally. But I have couple of data sets which has variety in date formats. I mean it has date format is like that (09/20/15 09:59 ) while it has another format in other lines is like that ( 2015-09-20 10:22:01.013 ) in the same file. And the command that I wrote above doesn't work on these file. It is working when I delete (parse_dates=['date']), but that time I can't use date column as datetime format, it reads that column as integer . I would be appreciate anyone could answer that!

try to do it this way: df = pd.read_csv('/Users/n....', names=names); df['date'] = pd.to_datetime(df['date']) — MaxU - stand with Ukraine
– MaxU - stand with Ukraine, Commented Aug 9, 2016 at 11:50

Anzel · Accepted Answer · 2016-08-09 12:31:27Z

5

Pandas read_csv accepts date_parser argument which you can define your own date parsing function. So for example in your case you have 2 different datetime formats you can simply do:

import datetime

def date_parser(d):
    try:
        d = datetime.datetime.strptime("format 1")
    except ValueError:
        try:
            d = datetime.datetime.strptime("format 2")
        except:
            # both formats not match, do something about it
    return d

df = pd.read_csv('/Users/n....', 
                 names=names, 
                 parse_dates=['date1', 'date2']),
                 date_parser=date_parser)

You can then parse those dates in different formats in those columns.

answered Aug 9, 2016 at 12:31

Anzel

20.6k5 gold badges54 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

manecosta · Accepted Answer · 2017-11-06 15:50:11Z

3

Like this:

df = pd.read_csv(file, names=names)
df['date'] = pd.to_datetime(df['date'])

edited Nov 6, 2017 at 15:50

manecosta

8,7823 gold badges28 silver badges40 bronze badges

answered Aug 9, 2016 at 11:52

waniz

533 bronze badges

Collectives™ on Stack Overflow

parse date-time while reading 'csv' file with pandas

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related