Monitor Management Command Execution in Django

Question

I'm writing a Django app that uses a management command to pull data from various sources. The plan is to run this command hourly with cron, and also have it run on user command from a view (i.e. when they add a new item that needs data, I don't want them to wait for the next hour to roll around to see results). The question is:

How can I set up this command such that if it is already currently running, it won't execute? Is there some place where I can stash a variable that can be checked by the script before execution? My current best idea is to have the command monitor stdout for a while to make sure nothing else is executing, but that seems like a hack at best. This is the only task that will be running in the background.

I'm basically trying to avoid using Celery here.

Don't sweat Celery. It's simple to set up and easy to run. I used Redis as a MQ with minimal problems if RabbitMQ is your headache. — Árni St. Steinunnarson
– Árni St. Steinunnarson, Commented Nov 17, 2014 at 13:06

Árni St. Steinunnarson · Accepted Answer · 2014-11-17 09:02:17Z

1

Wrap your read function in a try/except block marking some external memoization for execution control. I'm using Redis as an example:

from redis import Redis

try:
    r_client = Redis() # assuming standard settings
    sentinel = r_client.incr("my_sentinel")
    if sentinel == 1:
        run_command()
    else:
        r_client.decr("my_sentinel")
except Exception as e:
    r_client.decr("my_sentinel")
    raise e

answered Nov 17, 2014 at 9:02

Árni St. Steinunnarson

1,62012 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

dakota Over a year ago

I was hoping to avoid adding another data store. According to this question, atomic operations are doable in Django. Thoughts on using that instead?

Árni St. Steinunnarson Over a year ago

Yes, that's something you can do but it requires a roundtrip to the database. You can also do it with the filesystem, store the value in a filelock. You can also use memcached (almost all the code is the same).

Artur Barseghyan · Accepted Answer · 2014-11-17 11:30:28Z

0

You could create a file named "i_am_running.log" at the beginning of your management command and remove it at the end if it. When running same management command, check for its' presence. In case of no exist - go further. Otherwise - abort.

answered Nov 17, 2014 at 11:30

Artur Barseghyan

14.5k5 gold badges57 silver badges47 bronze badges

1 Comment

Árni St. Steinunnarson Over a year ago

This is too simple if it is vital to have only one running instance. It's too vulnerable to race conditions. You need something atomic to insure a single running instance.

Collectives™ on Stack Overflow

Monitor Management Command Execution in Django

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related