How Python threading Timer work internally?

Question

I want to know how python threading.Timer works.

In more detail, When i run a couple of threading.Timer, does it run separate thread for counting a time and running the handler ?

Or one thread manages and counts a couple of timer together ?

I am asking because my application need to schedule many event, But

If threading.Timer runs separate each thread for counting a timer, and i run many timers, it may affect performance so much.

So i am worry that if i have to implement a scheduler running only one thread if it has big effect in performance.

freakish · Accepted Answer · 2015-03-23 12:43:59Z

threading.Timer class is a subclass of threading.Thread and basically it just runs a separate thread in which it sleeps for the specified amount of time and runs the corresponding function.

It is definitely not an efficient way to schedule events. Better way is to do the scheduling in a single thread by using Queue.PriorityQueue in which you would put your events where "priority" actually means "next fire date". Similar to how cron works.

Or even better: use something that already exists, do not reinvent the wheel: Cron, Celery, whatever...

A very simplified example of making a scheduler via Queue.PriorityQueue:

import time
from Queue import PriorityQueue

class Task(object):
    def __init__(self, fn, crontab):
        # TODO: it should be possible to pass args, kwargs
        # so that fn can be called with fn(*args, **kwargs)
        self.fn = fn
        self.crontab = crontab

    def get_next_fire_date(self):
        # TODO: evaluate next fire date based on self.crontab
        pass

class Scheduler(object):
    def __init__(self):
        self.event_queue = PriorityQueue()
        self.new_task = False

    def schedule_task(self, fn, crontab):
        # TODO: add scheduling language, crontab or something
        task = Task(fn, crontab)
        next_fire = task.get_next_fire_date()
        if next_fire:
            self.new_task = True
            self.event_queue.put((next_fire, task))

    def run(self):
        self.new_task = False

        # TODO: do we really want an infinite loop?
        while True:
            # TODO: actually we want .get() with timeout and to handle
            # the case when the queue is empty
            next_fire, task = self.event_queue.get()

            # incremental sleep so that we can check
            # if new tasks arrived in the meantime
            sleep_for = int(next_fire - time.time())
            for _ in xrange(sleep_for):
                time.sleep(1)
                if self.new_task:
                    self.new_task = False
                    self.event_queue.put((next_fire, task))
                    continue

            # TODO: run in separate thread?
            task.fn()

            time.sleep(1)
            next_fire = task.get_next_fire_date()

            if next_fire:
                event_queue.put((next_fire, task))

def test():
    return 'hello world'

sch = Scheduler()
sch.schedule_task(test, '5 * * * *')
sch.schedule_task(test, '0 22 * * 1-5')
sch.schedule_task(test, '1 1 * * *')
sch.run()

It's just an idea. You would have to properly implement both Task and Scheduler classes, i.e. get_next_fire_date method plus some kind of scheduling language (crontab?) and error handling. I still strongly suggest to use one of the existing libraries.

Could you show a simple example how to implement a scheduler using Queue.PriorityQueue ? it doesn't make sense for me.

Tom Dalton · Accepted Answer · 2015-03-23 12:00:00Z

From the CPython 2.7 source:

def Timer(*args, **kwargs):
    """Factory function to create a Timer object.

    Timers call a function after a specified number of seconds:

        t = Timer(30.0, f, args=[], kwargs={})
        t.start()
        t.cancel()     # stop the timer's action if it's still waiting

    """
    return _Timer(*args, **kwargs)

class _Timer(Thread):
    """Call a function after a specified number of seconds:

            t = Timer(30.0, f, args=[], kwargs={})
            t.start()
            t.cancel()     # stop the timer's action if it's still waiting

    """

    def __init__(self, interval, function, args=[], kwargs={}):
        Thread.__init__(self)
        self.interval = interval
        self.function = function
        self.args = args
        self.kwargs = kwargs
        self.finished = Event()

    def cancel(self):
        """Stop the timer if it hasn't finished yet"""
        self.finished.set()

    def run(self):
        self.finished.wait(self.interval)
        if not self.finished.is_set():
            self.function(*self.args, **self.kwargs)
        self.finished.set()

As said in another answer, it is a separate thread (since it subclasses Thread). The callback function when the timer runs out is called from the new thread.

Collectives™ on Stack Overflow

How Python threading Timer work internally?

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related