Different db for testing in Django?

Question

DATABASES = {
#    'default': {
#        'ENGINE': 'postgresql_psycopg2',
#        ...
#    }

    # for unit tests
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'mydatabase'
    }
}

I have two databases: one I'd like to use for unit tests, and one for everything else. Is it possible to configure this in Django 1.2.4?

(The reason I ask is because with postgresql I'm getting the following error:

foo@bar:~/path/$ python manage.py test
Creating test database 'default'...
Got an error creating the test database: permission denied to create database

Type 'yes' if you would like to try deleting the test database 'test_baz', or 'no' to cancel: yes
Destroying old test database...
Got an error recreating the test database: database "test_baz" does not exist

Why could I be getting this error? I guess I don't really care if I can always use SQLite for unit tests, as that works fine.)

Have you tried whether really the postgres user has database creation rights? — Carles Barrobés
– Carles Barrobés, Commented Jan 10, 2011 at 19:34
That postgres user works for browsing the site normally, but I guess I'm not sure if it can create the db. — Nick Heiner
– Nick Heiner, Commented Jan 10, 2011 at 19:36
Like @CarlesBarrobés said, it's you probably don't have CREAETE permissions. This comes straight from django's documentation: Note that to use this feature, the database user Django is connecting as must have CREATE DATABASE rights. — Jonas Geiregat
– Jonas Geiregat, Commented Mar 8, 2012 at 17:36

Sam Dolan · Accepted Answer · 2011-01-10 21:07:30Z

109

In your settings.py (or local_settings.py):

import sys
if 'test' in sys.argv:
    DATABASES['default'] = {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'mydatabase'
    }

edited Jan 10, 2011 at 21:07

answered Jan 10, 2011 at 19:29

Sam Dolan

32.6k10 gold badges93 silver badges84 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

David S Over a year ago

@Rosarch... yes; it is a bit "hacky", but you are right! It works. :) But, IMHO, having to have any database for UNIT testing is hackish. I wish Django had a different philosophy when it came to this.

KhoPhi Over a year ago

@sleepycal Why is that?

HuLu ViCa Over a year ago

I am trying to use this solution but I want to use Postgres as test database engine, and I get an error saying that tables don't exist. I thought test created the database. Is it so?

Gonzalo Over a year ago

@HuLuViCa are you using managed or unmanaged models? are you using a runner script?

Masum Over a year ago

@DavidS then you are completely wrong. It IS the right approach to not do unit testing on actual data from your production database, that is not what unit tests are done for. Django essentially has the best approach here, not only that, it even takes care of that headache for the developer. People who want best practice for unit testing wants to use a different database for testing so it doesn't affect the original database in any way, specially if you need APIs for POST/PUT.

Geoffrey Hing · Accepted Answer · 2012-03-21 01:33:38Z

49

The way I handle this is through having multiple settings files, since I use that to maintain a set of common settings with modifications for each instance. It's a little more complicated to set up than some of the other solutions, but I needed to do it anyway because I was managing slightly different settings for local development, remote development, staging and production.

https://code.djangoproject.com/wiki/SplitSettings has a number of options for managing settings, and I've chosen a practice similar to the one described at https://code.djangoproject.com/wiki/SplitSettings#SimplePackageOrganizationforEnvironments

So, in my Django project directory, I have a settings folder that looks like this:

$ tree settings
settings
├── defaults.py
├── dev.py
├── dev.pyc
├── __init__.py
├── lettuce.py
├── travis.py
├── unittest.py

The common settings are in settings/defaults.py and I import these in my instance settings files. So settings/unittest.py looks like this:

from defaults import *

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'my_database',
    }
}

Then, when I want to run tests, I just execute:

$ ./manage.py test --settings=settings.unittest

to use sqlite for testing. I'll use a different settings module if I want to use a different test runner or database configuration.

answered Mar 21, 2012 at 1:33

Geoffrey Hing

1,8252 gold badges19 silver badges23 bronze badges

1 Comment

Lym Over a year ago

As one with a Rails background, I found this answer to be the best. Rails uses the same kind of convention, with the slight difference that in Rails the settings directory is named config.

Leon Mak · Accepted Answer · 2020-02-06 02:40:21Z

34

You can specify test database in settings.py. See https://docs.djangoproject.com/en/3.0/topics/testing/overview/#the-test-database

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'USER': 'mydatabaseuser',
        'NAME': 'mydatabase',
        'TEST': {
            'NAME': 'mytestdatabase',
        },
    },
}

edited Feb 6, 2020 at 2:40

answered Jan 4, 2018 at 6:18

Leon Mak

5055 silver badges8 bronze badges

8 Comments

Jeff Bluemel Over a year ago

this was a simple solution that worked perfectly. I am using the pythonanywhere website and this allowed me to run the tutorial.

ilse2005 Over a year ago

Unfortunately, you can't change the HOST.

jeffmaher Over a year ago

This works, but note that the default still has to resolve. For example, if the URL of the database isn't resolvable when tests run, tests will crash.

NiMeDia Over a year ago

@Lantern You cannot switch the engine nor the most of the other settings.

T.H. Over a year ago

As of Django 3.1 , It cannot change database user credential USER and PASSWORD.

|

Evyde · Accepted Answer · 2021-03-29 02:55:05Z

10

You can mirror your db by editing settings.py:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'USER': 'mydatabaseuser',
        'NAME': 'mydatabase',
        'TEST': {
            'MIRROR': 'default',
        },
    },
}

answered Mar 29, 2021 at 2:55

Evyde

1011 silver badge6 bronze badges

3 Comments

Maganna Dev Over a year ago

This should be the accepted answer, not only for this question but for most questions concerning performing unit tests that involve database access in Django. Today I spent 5 hours googling and reading so many blog posts. They were all hacky and obscure. This is the only one which is really Pythonic, leverages Django's default settings and covers most use cases. Thanks @ForeverOpp

Bret Hess Over a year ago

Is the mirror a copy of the database with data, or an empty database with a copy of the schema?

AJJ Over a year ago

I agree, this should be the accepted answer.

laffuste · Accepted Answer · 2014-11-12 04:19:59Z

9

This accelerated dramatically test execution.

import sys

if 'test' in sys.argv:
    DATABASES['default'] = {
        'ENGINE': 'django.db.backends.sqlite3',
        'TEST_CHARSET': 'UTF8', # if your normal db is utf8
        'NAME': ':memory:', # in memory
        'TEST_NAME': ':memory:', # in memory
    }

    DEBUG = False # might accelerate a bit
    TEMPLATE_DEBUG = False

    from django.core.management import call_command
    call_command('syncdb', migrate=True) # tables don't get created automatically for me

answered Nov 12, 2014 at 4:19

laffuste

17.2k8 gold badges90 silver badges94 bronze badges

Comments

Caco · Accepted Answer · 2018-02-09 11:11:00Z

I solved this issue simply creating other settings constant DATABASES_AVAILABLE.

DATABASES_AVAILABLE = {
    'main': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'nep',
        'USER': 'user',
        'PASSWORD': 'passwd',
        'HOST': 'localhost',
    },
    'remote': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'nes_dev',
        'USER': 'usr',
        'PASSWORD': 'passwd',
        'HOST': '200.144.254.136',
    },
    'sqlite': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    },
}

# This solves the problem with tests
# Define a system variable called DJANGO_DATABASE_TEST and set it to the
# the database you want
database = os.environ.get('DJANGO_DATABASE_TEST', 'main')
DATABASES = {
    'default': DATABASES_AVAILABLE[database]
}

Mario Corchero · Accepted Answer · 2013-05-04 19:34:10Z

5

Though this is already solved...

If your database for tests is just a normal DB:

I think you are not doing unit test since you rely in the database. Anyway, django contains a test type for that (not unitary): django.test.TestCase

You need to derive from django.test.TestCase instead of unittest.TestCase that will create a fresh rehershal database for you that will be destroyed when the test end.

There are interesting explanations/tips about testing with db in the following link
Testing Django Applications

answered May 4, 2013 at 19:34

Mario Corchero

5,6256 gold badges39 silver badges62 bronze badges

2 Comments

user1969453 Over a year ago

The problem I sometimes run into is when live data is fairly complex and came from some external source. For testing I want a subset. Yes I could create that subset is my test class setup. But it can be non-trivial. I especially find it better if my tests are read-only as far as the DB is concerned.

Mario Corchero Over a year ago

You then will potentially have non reproducible test :$ and running your tests will require connection to the open world

mhost · Accepted Answer · 2012-03-20 01:08:13Z

3

If you have access to manually create the database, you could use django-nose as your TEST_RUNNER. Once installed, if you pass the following environment variable, it will not delete and re-create the database.

REUSE_DB=1 ./manage.py test

You can also add the following to settings.py so you don't have to write REUSE_DB=1 every time you want to run tests:

os.environ['REUSE_DB'] = "1"

Note: this will also leave all your tables in the databases which means test setup will be a little quicker, but you will have to manually update the tables (or delete and re-create the database yourself) when you change your models.

answered Mar 20, 2012 at 1:08

mhost

7,3205 gold badges41 silver badges45 bronze badges

Comments

Cristian Gusella · Accepted Answer · 2022-04-05 10:34:07Z

2

I had your same issue. I resolved it just adding in settings.py the TEST value in the database, available in my Django version 4.0.2, in this way:

DATABASES = {
     'default': {
            'ENGINE': 'django.db.backends.postgresql',
            'HOST': host,
            'NAME': name,
            'TEST': {'NAME': 'test_db'},
            'USER': user,
            'PASSWORD': your_password,
    }
}.

This temporary creates db with different name and the conflict is resolved.

edited Apr 5, 2022 at 10:34

answered Apr 5, 2022 at 10:16

Cristian Gusella

212 bronze badges

Comments

Samkit Jain · Accepted Answer · 2017-10-26 09:37:35Z

1

Why could I be getting this error?

Because of insufficient permissions. You can alter the user permissions by ALTER USER username CREATEDB; after running psql with superuser priviliges.

Example,

$ sudo su - postgres
$ psql
psql (9.3.18)
Type "help" for help.

postgres=# ALTER USER username CREATEDB;
ALTER ROLE

answered Oct 26, 2017 at 9:37

Samkit Jain

1,7482 gold badges18 silver badges37 bronze badges

Collectives™ on Stack Overflow

Different db for testing in Django?

10 Answers 10

5 Comments

1 Comment

8 Comments

3 Comments

Comments

Comments

2 Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

10 Answers 10

5 Comments

1 Comment

8 Comments

3 Comments

Comments

Comments

2 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related