Django Db routing

Question

I am trying to run my Django application with two db's (1 master, 1 read replica). My problem is if I try to read right after a write the code explodes. For example:

p = Product.objects.create()
1. Product.objects.get(id=p.id)

OR

1. If user is redirected to Product's details page

The code runs way faster than the read replica. And if the read operation uses the replica the code crashes, because it didn't update in time.

Is there any way to avoid this? For example, the db to read being chosen by request instead of by operation?

My Router is identical to Django's documentation:

import random

class PrimaryReplicaRouter(object):
    def db_for_read(self, model, **hints):
        """
        Reads go to a randomly-chosen replica.
        """
        return random.choice(['replica1', 'replica2'])

    def db_for_write(self, model, **hints):
        """
        Writes always go to primary.
        """
        return 'primary'

    def allow_relation(self, obj1, obj2, **hints):
        """
        Relations between objects are allowed if both objects are
        in the primary/replica pool.
        """
        db_list = ('primary', 'replica1', 'replica2')
        if obj1._state.db in db_list and obj2._state.db in db_list:
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        All non-auth models end up in this pool.
        """
        return True

pedrotorres · Accepted Answer · 2016-09-26 09:17:13Z

3

Solved it with :

class Model(models.Model): 
    objects = models.Manager()  -> objects only access master
    sobjects = ReplicasManager() -> sobjects access either master and replicas

    class Meta: 
        abstract = True  -> so django doesn't create a table

make every model extend this one instead of models.Model, and then use objects or sobjects whether I want to access only master or if want to access either master or replicas

answered Sep 26, 2016 at 9:17

pedrotorres

1,2422 gold badges13 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Hevlastka · Accepted Answer · 2020-03-22 15:45:31Z

3

Depending on the size of the data and the application I'd tackle this with either of the following methods:

Database pinning:

Extend your database router to allow pinning functions to specific databases. For example:

from customrouter.pinning import use_master

@use_master
def save_and_fetch_foo():
    ...

A good example of that can be seen in django-multidb-router. Of course you could just use this package as well.

Use a model manager to route queries to specific databases.

class MyManager(models.Manager):
    def get_queryset(self):
        qs = CustomQuerySet(self.model)
        if self._db is not None:
            qs = qs.using(self._db)
        return qs

Write a middleware that'd route your requests to master/slave automatically. Basically same as the pinning method but you wouldn't specify when to run GET requests against master.

edited Mar 22, 2020 at 15:45

answered Sep 23, 2016 at 12:54

Hevlastka

1,9682 gold badges18 silver badges29 bronze badges

1 Comment

pedrotorres Over a year ago

Solved it with : class Model(models.Model): objects = models.Manager() sobjects = ReplicasManager() class Meta: abstract = True is i want to use master i do User.objects... if it doesnt matter whether maste or replica i use sobjects

sachin · Accepted Answer · 2016-09-29 07:33:45Z

1

IN master replica conf the new data will take few millisecond to replicate the data on all other replica server/database.

so whenever u tried to read after write it wont gives you correct result.

Instead of reading from replica you can use master to read immediately after write by using using('primary') keyword with your get query.

answered Sep 29, 2016 at 7:33

sachin

3793 silver badges16 bronze badges

Collectives™ on Stack Overflow

Django Db routing

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related