Update database fields hourly with Python/Django

Question

Suppose I have 1000 user_ids in a table and I would run every hour to get from Google API info and update 3 fields in that table. How would the impact be and how can it be done efficiently?

I've seen this variant:

m = Module.objects.get(user_id=1).update(field_one=100, field_two=200, field_three=300)

And this one:

m = Module.objects.get(user_id=1)
m.field_one = 100
m.field_two = 200
m.field_three = 300
m.save()

Also how can it be done so that it will run every hour and grab that information? Never done something like this.

Why every hour? Your 1000 users won't be on your site 24 hours a day 365 days a year. Most sites have many users, but only a few regular users. The rest lost interest. So you might think of a rule that checks less often. Kickoff a task when a user logs in? — allcaps
– allcaps, Commented Jul 9, 2016 at 21:27

Community · Accepted Answer · 2017-05-23 12:00:29Z

2

Use Redis, Celery to setup asynchronous task queue every hour. Look here https://realpython.com/blog/python/asynchronous-tasks-with-django-and-celery/ for more info on how to setup asych task queue system for django.

Here is the code for tasks.py

from celery.task import periodic_task
from celery.schedules import crontab  


@periodic_task(run_every=crontab(minute=0, hour='*/1'))
def get_data_from_google_api():
    data_from_google =ping_google_api() # ping google api to get data
    return Module.objects.get(user_id=1).update(field_one= data_from_google['field_one'], field_two= data_from_google['field_two'], field_three= data_from_google['field_three'])

Look here for more info :

edited May 23, 2017 at 12:00

CommunityBot

11 silver badge

answered Jul 9, 2016 at 20:50

rrmerugu

1,9062 gold badges20 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ivan Semochkin · Accepted Answer · 2016-07-09 20:58:51Z

1

Fof this purpose you need to run background queries with periodic taks.
Here is most popular in django task-queue-libs
For example, if you decide use celery, you can write simple periodic task:

from celery.schedules import crontab
from celery.task import periodic_task

@periodic_task(
name='UPDATE_USER',
run_every=crontab(
    minute='1',
    hour='1,4,7,10,13,16,19,22'))
def update_user():
    #get some value from api
    Module.objects.filter(user_id=1).update(
        field_one=value, field_two=value, field_three=value)

All settings for django you can look in celery docs

answered Jul 9, 2016 at 20:58

Ivan Semochkin

8,9173 gold badges49 silver badges78 bronze badges

Collectives™ on Stack Overflow

Update database fields hourly with Python/Django

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related