Conditional statements on time series data python

Question

I'm trying to execute conditional statements on time series data. Is there a way to set the "t_value" to zero if the time is between 1) 00:00:00 and 02:00:00 2) 04:00:00 & 06:00:00

                   t_value
2019-11-24 00:00:00  4.0
2019-11-24 01:00:00  7.8
2019-11-24 02:00:00  95.1
2019-11-24 03:00:00  78.4
2019-11-24 04:00:00  8.0
2019-11-24 05:00:00  17.50
2019-11-24 06:00:00  55.00
2019-11-24 07:00:00  66.00
2019-11-25 00:00:00  21.00
2019-11-25 01:00:00  12.40

if-else & np.where are probable options but I'm unsure on how to implement the conditions on hours.

Yes, but this is only an example. I’ll edit it to a more suitable example. — sklal
– sklal, Commented May 18, 2020 at 1:29

sammywemmy · Accepted Answer · 2020-05-18 02:40:53Z

1

use between_time to get the datetimes between the specified times, then use loc to assign the new values :

I'll use @Ben.T's sample data :

df = pd.DataFrame({'t_value':range(1,11)}, 
              index=pd.date_range('2020-05-17 00:00:00', periods=10, freq='1H'))

#get the time indices for the different ranges
m1 = df.between_time('00:00:00','02:00:00').index
m2 = df.between_time('04:00:00','06:00:00').index

#assign 0 to the t_value column matches : 
df.loc[m1|m2] = 0

print(df)

            t_value
2020-05-17 00:00:00 0
2020-05-17 01:00:00 0
2020-05-17 02:00:00 0
2020-05-17 03:00:00 4
2020-05-17 04:00:00 0
2020-05-17 05:00:00 0
2020-05-17 06:00:00 0
2020-05-17 07:00:00 8
2020-05-17 08:00:00 9
2020-05-17 09:00:00 10

answered May 18, 2020 at 2:40

sammywemmy

28.9k4 gold badges21 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

sklal Over a year ago

what if I want to use the time range (00:00:00,03:00:00) and t_value>0 to calculate a new column which classifies them as correct(>0) & incorrect(=0)?

sammywemmy Over a year ago

off the back of my head, sounds like sth u can put into loc, with the m1|m2 part . u could edit ur question for the addition, although it is much better to ask a new question altogether

sklal Over a year ago

Posted a new question, stackoverflow.com/questions/61869990/… If you want to give it a try

Ben.T · Accepted Answer · 2020-05-18 01:47:10Z

you can acces the time from your datetime index with time and create mask depending on your condition. Then use loc and | to concatenate your mask as or.

#sample data
df = pd.DataFrame({'t_value':range(1,11)}, 
              index=pd.date_range('2020-05-17 00:00:00', periods=10, freq='1H'))

# masks
m1 = ((df.index.time>=pd.to_datetime('00:00:00').time()) 
      & (df.index.time<=pd.to_datetime('02:00:00').time()))
m2 = ((df.index.time>=pd.to_datetime('04:00:00').time())
      & (df.index.time<=pd.to_datetime('06:00:00').time()))

#set the value to 0
df.loc[m1|m2, 't_value'] = 0

print (df)
                     t_value
2020-05-17 00:00:00        0
2020-05-17 01:00:00        0
2020-05-17 02:00:00        0
2020-05-17 03:00:00        4
2020-05-17 04:00:00        0
2020-05-17 05:00:00        0
2020-05-17 06:00:00        0
2020-05-17 07:00:00        8
2020-05-17 08:00:00        9
2020-05-17 09:00:00       10

Collectives™ on Stack Overflow

Conditional statements on time series data python

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related