Python data-frame using pandas

Question

I have a dataset which looks like below

  [25/May/2015:23:11:15  000]
  [25/May/2015:23:11:15  000]
  [25/May/2015:23:11:16  000]
  [25/May/2015:23:11:16  000]

Now i have made this into a DF and df[0] has [25/May/2015:23:11:15 and df[1] has 000]. I want to send all the data which ends with same seconds to a file. in the above example they end with 15 and 16 as seconds. So all ending with 15 seconds into one and the other into a different one and many more

I have tried the below code

   import pandas as pd
   data = pd.read_csv('apache-access-log.txt', sep=" ", header=None)
   df = pd.DataFrame(data)
   print(df[0],df[1].str[-2:])

AChampion · Accepted Answer · 2018-05-24 04:55:17Z

2

Converting that column to a datetime would make it easier to work on, e.g.:

df['date'] = pd.to_datetime(df['date'], format='%d/%B/%Y:%H:%m:%S')

The you can simply iterate over a groupby(), e.g.:

In []:
for k, frame in df.groupby(df['date'].dt.second):
     #frame.to_csv('file{}.csv'.format(k))
     print('{}\n{}\n'.format(k, frame))

Out[]: 
15
                 date  value
0 2015-11-25 23:00:15      0
1 2015-11-25 23:00:15      0

16
                 date  value
2 2015-11-25 23:00:16      0
3 2015-11-25 23:00:16      0

answered May 24, 2018 at 4:55

AChampion

30.4k4 gold badges63 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

AChampion Over a year ago

See commented out code... this would create a file called file15.csv for all the 15 second data and file16.csv for all the 16 second data.

Angelo Cardellicchio · Accepted Answer · 2018-05-24 04:57:05Z

0

You can set your datetime as the index for the dataframe, and then use loc and to_csv Pandas' functions. Obviously, as other answers points out, you should convert your date to datetime while reading your dataframe.

Example:

df = df.set_index(['date'])
df.loc['25/05/2018 23:11:15':'25/05/2018 23:11:15'].to_csv('df_data.csv')

answered May 24, 2018 at 4:57

Angelo Cardellicchio

4167 silver badges8 bronze badges

Comments

Narendra Prasath · Accepted Answer · 2018-05-24 04:58:53Z

0

Try out this,

## Convert a new column with seconds value    
df['seconds'] = df.apply(lambda row: row[0].split(":")[3].split(" ")[0], axis=1)

for sec in df['seconds'].unique():  
    ## filter by seconds
    print("Resutl ",df[df['seconds'] == sec])

answered May 24, 2018 at 4:58

Narendra Prasath

1,5311 gold badge10 silver badges20 bronze badges

Collectives™ on Stack Overflow

Python data-frame using pandas

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related