2

Background

I'm using Django with multiple databases: I have a Postgres database and a mysql database (filled with legacy data that I'm slowly converting over to Postgres).

I recently brought in GeoDjango (in the form of django.contrib.gis), with the postgis extension for storing lat/lng information as a point. The application itself seems to be running fine. I can access information from the mysql database and the postgres database just fine. I can even leverage the new point field, which means (to me) that the 'gis' stuff is working and configured.

Problem

The problem occurs when I try to run tests. I get the following error:

AttributeError: 'DatabaseOperations' object has no attribute 'geo_db_type'

I believe this is occurring on the mysql database because I first get the prompt stating that the test_db postgress database exists and needs to be dropped and recreated. I enter 'yes', and it does it's thing, then gives me the same prompt about the test_mysql database. When I answer 'yes', I get the error.

My legacy app's models don't use any of the new fields; the import is django.db. So I can't figure out why the test thinks that I need geospatial operations on the mysql database.

What I've Tried

1) Commenting Out the MySQL Database

This actually works; tests run fine. But, this isn't a good solution, for obvious reasons...

2) Changing the MySQL's database engine to use 'django.contrib.gis.db.backends.mysql'

I get a different error when I do this:

django.db.utils.OperationalError: (1071, 'Specified key was too long; max key length is 767 bytes')

I'm guessing this is due to a composite key somewhere in my legacy models. This really shouldn't be an issue, I don't think, since I'm actually running MariaDB 10.0.31 in the background. Also, django.db.backends.mysql doesn't complain about it. So I'm wondering if django.contrib.gis.db.backends.mysql is a little out of date (since MySQL seems to have limited spatial support anyway).

I can't figure out why the test databases complain but the application seems to work fine normally.

Thank you in advance for any help you can provide! I'm totally stumped.

Supporting Information

My database configuration looks like this:

DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.postgis',
        'NAME': os.environ.get('PG_DB_NAME'),
        'USER': os.environ.get('PG_DB_USER'),
        'PASSWORD': os.environ.get('PG_DB_PASS'),
        'HOST': os.environ.get('PG_DB_HOST'),
        'PORT': '',
    },
    'legacy': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': os.environ.get('MY_DB_NAME'),
        'USER': os.environ.get('MY_DB_USER'),
        'PASSWORD': os.environ.get('MY_DB_PASS'),
        'HOST': os.environ.get('MY_DB_HOST'),
        'PORT': '',
    },  
}

I have a database router:

class LegacyRouter(object):
    def db_for_read(self, model, **hints):

        if model._meta.app_label == 'legacy':
            return 'legacy'
        return 'default'

Edit: Hacky Workaround

So, after some trial and error, I've discovered that I can work around this by using a single postgresql database for testing purposes. I accomplished this by adding a second settings file, and referencing it when I test.

from .settings import *

DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.postgis',
        'NAME': os.environ.get('TEST_DB_NAME'),
        'USER': os.environ.get('TEST_DB_USER'),
        'PASSWORD': os.environ.get('TEST_DB_PASS'),
        'HOST': os.environ.get('TEST_DB_HOST'),
    },
}

DATABASE_ROUTERS = []

That gets called at test time like:

python manage.py test --settings=<module>.test-settings

This works for me because there is nothing database-specific about my legacy data, so there is no reason I can't use postgres to test. Hopefully this helps someone else.

Edit: The Fix

The problem was in the database router. All apps except the 'legacy' app were being routed to the 'default' database (expected), while all apps plus the 'legacy' app were getting routed to the 'legacy' database.

To fix, I needed an allow_migrate function. Here is how I made mine:

def allow_migrate(self, db, app_label, model_name=None, **hints):

    # Legacy app should only be in legacy database
    # Return true if app is legacy and database is legacy

    if app_label == 'legacy':
        return db == 'legacy'

    # Legacy database should only contain legacy app
    # Return true if app is not legacy and database not legacy

    elif app_label != 'legacy':
        return db != 'legacy'

    else:
        return None
3
  • Please add your fix as a separate answer rather than including it at the end of the question. That way it's clear that your problem has been solved. Commented Nov 6, 2017 at 16:58
  • @Alasdair I already created and accepted an answer that explains why I had the problem and what I needed to do to fix it. The answer refers to the necessary configuration in the question for brevity and reduced redundancy. Commented Nov 7, 2017 at 18:36
  • Sorry, I didn't spot that you had already answered. My point was that your section "The fix" belongs in the answers section, not the question. Commented Nov 7, 2017 at 18:42

1 Answer 1

1

Answer

The problem was in the database router. All apps except the 'legacy' app were being routed to the 'default' database (expected), while all apps plus the 'legacy' app were getting routed to the 'legacy' database.

To fix, I needed an allow_migrate function that prevented all but the 'legacy' app from being migrated in the 'legacy' database. See the edit in my question for more information.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.