7

I am using django restframework and want to handle multiple databases. I am using django function using(alias) and switch_db(alias) for manually switching between database whenever I want to Get, Post or update data.

I am facing problem while posting and updating data.i.e whenever serializer.is_valid() will be called.

serializer.is_valid() will go and first check for db_alias in model.py file. If I have not specified db_alias under meta it will select default database for validation. If I am specifying db_alias in model it will select that database for validation.

But I do not want to specify db_alias in model since my usecase is to store data on different database based on some logic in my view file. So dynamically I want to select database from view and want to store data in it.

Almost I have implemented but I am facing problem when my model is having Reference Field. In this case serializer.is_valid is going to default database for validating that reference field.

Required Details: I am using mongoengine(0.9.0), document, document serializer.

My files are as below:

model.py:

class ngroup(Document):

    groupname = StringField(max_length=100, required=True)
    description = StringField(max_length=100, required=False)
    parent = ReferenceField('ngroup',null=True)
    created_ts = DateTimeField(default=datetime.now())
    modified_ts = DateTimeField(default=datetime.now())
    is_deleted = BooleanField(default=False)

serializer.py:

from device_management.models import ngroup
from rest_framework_mongoengine.serializers import DocumentSerializer
from mongoengine import EmbeddedDocumentField, ReferenceField, StringField, ObjectIdField, IntField, BooleanField, FloatField, DateTimeField,ListField


class ngroupSerializer(DocumentSerializer):

    class Meta:
        model = ngroup

    def setOrgId(self, orgid):
        self.orgid = orgid

    def create(self, validated_data):
        ngroup_data = ngroup(**validated_data).switch_db(self.orgid)
        ngroup_data.save()
        return ngroup_data

    def update(self, instance, validated_data):
        ngroup_data = ngroup.objects.using(self.orgid).get(id = instance.id)
        ngroup_data = ngroup_data.switch_db(self.orgid)
        ngroup_data = ngroup_data.update(**validated_data)
        return validated_data

    def to_internal_value(self, data):
        print "data:" , data
        return super(DocumentSerializer, self).to_internal_value(data)

view.py:

def create(self, request, format=None):
    orgid = str(request.user.orgid.id)
    data=request.data

    serializer = ngroupSerializer(data=data)
    if serializer.is_valid():
        try:
            serializer.save()
        except Exception as e:
            log.error("create" , extra={'extra':{'error': str(e),'message' :strings.DATA_VALIDATION_ERROR }})
            return response.errorResponse(message=strings.SERIALIZATION_ERROR_MSG,error=str(e),rstatus=status.HTTP_400_BAD_REQUEST)
        return response.successResponse(res_data=serializer.data, message=strings.POST_SUCCESS_MSG, rstatus=status.HTTP_201_CREATED)
    log.error("create" , extra={'extra':{'error': serializer.errors,'message' :strings.DATA_VALIDATION_ERROR }})
    return response.errorResponse(message=strings.DATA_VALIDATION_ERROR,error=serializer.errors,rstatus=status.HTTP_400_BAD_REQUEST)

settings.py:

DATABASES = {
     'default': {
        'ENGINE': 'django_mongodb_engine',
        'NAME': 'mydb',
        'USER': 'admin',
        'PASSWORD':'admin123',
        'HOST': '127.0.0.1',
        'PORT': 27017,
        'DBTYPE' : "mongo",
    },
    '586e47c784413825f2b5bc49': {
        'ENGINE': 'django_mongodb_engine',
        'NAME': 'mydb1',
        'USER': 'admin',
        'PASSWORD':'admin123',
        'HOST': '127.0.0.1',
        'PORT': 27017,
        'DBTYPE' : "mongo",
    },
    # Enter super_user organisation here. This DB will be same as default db only always
    '58996fb28441384430dc8ae6': {
        'ENGINE': 'django_mongodb_engine',
        'NAME': 'mydb',
        'USER': 'admin',
        'PASSWORD':'admin123',
        'HOST': '127.0.0.1',
        'PORT': 27017,
        'DBTYPE' : "mongo",
    },
}

pip freeze(Installation versions):

Django==1.5.11
django-browserid==2.0.2
django-classy-tags==0.8.0
django-missing==0.1.18
django-mongo-auth==0.1.3
django-mongodb-engine==0.6.0
django-mongoengine==0.2.1
django-redis-sessions==0.5.6
django-rest-framework-mongoengine==3.3.0
django-sekizai==0.10.0
django-websocket-redis==0.4.7
djangorestframework==3.1.2
djangorestframework-jwt==1.9.0
djangotoolbox==1.8.0
gevent==1.1.2
greenlet==0.4.10
httplib2==0.9.2
mongoengine==0.9.0
oauthlib==2.0.1
pika==0.10.0
Pygments==2.1.3
PyJWT==1.4.2
pymongo==2.8
python-dateutil==2.6.0
python-openid==2.2.5
pytz==2016.10
redis==2.10.5
requests==2.12.3
requests-oauthlib==0.7.0
rest-condition==1.0.3
six==1.10.0
tweepy==3.5.0
twilio==5.7.0

I have overide create in serializer to take care of database while calling serializer.save() but how to handle serializer.is_valid().

My project has been stuck at this point. Any help will be greatly appreciated...

Improve this question

2 Answers 2

Reset to default
2
+50

This is not the exact solution to above problem but we have 2 options.

1) Do not go for serializer.is_valid() or serializer.save(). Directly create ngroup:

def my_create(self, validated_data):
    gateway = Gateway(**validated_data).switch_db(self.orgid)
    gateway.save()
    return gateway

2) Another solution is to use django-mogodb-engine and django models and modelserializers instead of documents and documents serializers.

I have tried following this with Django-mongodb-engine and are working well:

-> JWT authentication
-> custom user
-> foreign key
-> embedded model
-> list of embedded model
-> dict field
-> **Routers for switching between databases.(Manual switching DB is not required)**

I can also use middleware_classes to specify runtime in each request which database to use. Reference Link: Django Authenticate Backend Multiple Databases

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

Husain,

Unfortunately, you're mixing incompatible projects together. Mongoengine, django-mongoengine and Django-REST-Framework-Mongoengine projects are alternative to django-mongodb-engine, they are not meant to be used together.

As far as I know, django-mongodb-engine project's been dead for 2 years, or even longer, to be honest. At the same time, Mongoengine stack is working in production, though, development is not too active. I really want to create a proper django database backend out of Mongoengine to make it a first-class citizen in Django world, and it seems like Django guys are looking in that direction, too.

You might also want to look into this post.

This is my second attempt. I tried to switch the database connection in view create(). Didn't work for me:

settings.py

# We define 2 Mongo databases - default (project) and project2
MONGODB_DATABASES = {
    "project": {
        "name": "project",
        "host": "localhost",
        "port": 27017,
        "tz_aware": True,  # if you use timezones in django (USE_TZ = True)
    },

    "project2": {
        "name": "project2",
        "host": "localhost",
        "port": 27017,
        "tz_aware": True,  # if you use timezones in django (USE_TZ = True)
    }
}


mongoengine.register_connection(alias='default', name=MONGODB_DATABASES["project"]["name"], host="local")
mongoengine.register_connection(alias='project2', name=MONGODB_DATABASES["project2"]["name"], host="local")

connection = mongoengine.connect(db="project", alias='default')

views.py

class AuthorViewSet(MongoModelViewSet):
    lookup_field = 'id'
    serializer_class = AuthorSerializer

    def create(self, request, format=None):
        global Author
        mongoengine.connection.disconnect(alias='project')
        mongoengine.connect('project2', alias='project2')
        return super(AuthorViewSet, self).create(request, format)

    def get_queryset(self):
        return Author.objects.all()
Improve this answer

9 Comments

Hi Boris, Can I use django-mongodb-engine as database backend and model and modelserializer instead of document and document serializer.
continue... i.e If I start using django-mongodb-engine instead of mongoengine and rest-framework-mongoengine, will that can have any problems. Because I have checked following things with django-mongodb-engine + Modeld.model + Modelserializer and Mongodb as database. 1) Database routers for database switching. 2) Embedded Model field 3) DictField and things are working really nice
continue... and things are working really nice So if I migrate my project to models from documents, will that can stop me anywhere in future. Are there any limitations of django-mongodb-engine ??? (Since you mention that "django-mongodb-engine project's been dead for 2 years, or even longer,")
@HusainPansoliwala That's actually a very interesting question - I really like this consistency with the rest of django in django-mongodb-engine. I've never really worked with django-mongodb-engine - I started using Mongo with Django in 2014 and it was already abandoned at that point. I'm not sure about how stable that implementation is and what's the level of support of Mongo-specific features in it, like EmbeddedDocuments and stuff. But if you managed to give it a try, can you share your experience with this approach?
I have replaced serializers like below: class ngroupserializer(DocumentSerializer): class Meta: def _db_switch(self): print "_db_switch......\n" return switch_db(ngroup, 'org') model = property(_db_switch) But it has no effect. It does not works.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.