Running a standalone script doing a model query in Django with `settings/dev.py` instead of `settings.py`

Question

Note the settings/dev.py instead of one settings.py file and the script.py in my_app in the following Django(1.4.3) project:

.
├── my_project
│   ├── my_app
│   │   ├── __init__.py
│   │   ├── models.py
│   │   ├── tests.py
│   │   ├── views.py
│   │   └── script.py
│   ├── __init__.py
│   ├── settings
│   │   ├── dev.py
│   │   ├── __init__.py
│   │   └── prod.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
└── requirements.txt

When I only had one settings.py file in place of the settings folder I was able to run the following script without any errors:

script.py:

###################################################################
# set up for making it possible to run a model query from my script. 
###################################################################
import os
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)))
from django.core.management import setup_environ
import settings
setup_environ(settings)
####################################################################  
from my_app.models import MyModel

all_entries = MyModel.objects.all()

Btw, I got that from the second approach in this article, my setup is a few lines longer because my script.py is in the my_app folder instead of directly under the my_project folder.

Because I am now using settings/dev.py instead of settings.py I changed the last two lines of my setup in my script to the following:

import settings.dev
import setup_environ(settings.dev)

But when I run my script now I get this error:

Traceback (most recent call last):
  File "my_script.py", line 12, in <module>
    all_entries = MyModel.objects.all()
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/manager.py", line 131, in get
    return self.get_query_set().get(*args, **kwargs)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/query.py", line 358, in get
    clone = self.filter(*args, **kwargs)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/query.py", line 624, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/query.py", line 642, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1250, in add_q
    can_reuse=used_aliases, force_having=force_having)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1122, in add_filter
    process_extras=process_extras)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1304, in setup_joins
    field, model, direct, m2m = opts.get_field_by_name(name)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/options.py", line 311, in get_field_by_name
    cache = self.init_name_map()
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/options.py", line 341, in init_name_map
    for f, model in self.get_all_related_m2m_objects_with_model():
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/options.py", line 429, in get_all_related_m2m_objects_with_model
    cache = self._fill_related_many_to_many_cache()
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/options.py", line 443, in _fill_related_many_to_many_cache
    for klass in get_models(only_installed=False):
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/loading.py", line 181, in get_models
    self._populate()
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/loading.py", line 64, in _populate
    self.load_app(app_name, True)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/loading.py", line 86, in load_app
    app_module = import_module(app_name)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
ImportError: No module named my_project.my_app

Why is this error occurring? How can I run my script in Django with settings/dev.py instead of settings.py?

danodonovan · Accepted Answer · 2013-01-30 14:51:43Z

9

If you're looking to just run a script in the django environment, then the simplest way to accomplish this is to create a ./manage.py subcommand, like this

from django.core.management.base import BaseCommand
from my_app.models import MyModel

class Command(BaseCommand):
    help = 'runs your code in the django environment'

    def handle(self, *args, **options):
        all_entries = MyModel.objects.all()
        for entry in all_entries:
            self.stdout.write('entry "%s"' % entry)

The docs are quite helpful with explaining this.

However, you can specify a settings file to run with using

$ django-admin.py runserver --settings=settings.dev

which will run the test server using the settings in dev however, I fear your problems are more deep seated than simply that. I wouldn't recommend ever changing the manage.py file as this can lead to inconsistencies and future headaches.

Note also that dev.py should be a complete settings file if you are to do this. I would personally recommend a structure like this:

|-settings
|    |- __init__.py
|    |- base.py
|    |- dev.py
|    |- prod.py

and keep all the general settings in your base.py and change the first line of your dev.py etc to something like

# settings/dev.py
from .base import *

DEBUG = True
...

EDIT

If you're just looking to test things out, why not try

$ ./manage.py shell

or with your dev settings

$ django-admin.py shell --settings=settings.dev

as this will set all the OS environment variables, settings.py for you, and then you can test / debug with

>>> from my_app.models import MyModel
>>> all_entries = MyModel.objects.all()
>>> for entry in all_entries:
...   print entry

edited Jan 30, 2013 at 14:51

answered Jan 30, 2013 at 12:06

danodonovan

20.5k10 gold badges74 silver badges78 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Bentley4 Over a year ago

I used from .common import * in dev.py and it still produces the same error(btw my structure was already as you suggested, I just wanted the problem to be presented as simple as possible). I also ran the script with a new virtual environment and the same error occurred. Doing a query from python manage.py shell --settings=my_project.settings.dev works perfectly fine.

danodonovan Over a year ago

I can't see a common module in your dir structure - is that a typo?

Bentley4 Over a year ago

My mistake, it's named common.py in my real project, according to your example It's the same as base.py. If I remove settings_environ I get

django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.

Btw my settings.DATABASES is identical to the one in my old settings.py file, which ran without a problem.

danodonovan Over a year ago

How are you running your script and what is its purpose? To avoid the settings difficulties, run everything from ./manage.py - hence my recommendation to use the shell. If you're writing a script that needs to be executed by the django instance, use a manage.py subcommand. If that's overkill, go back to debugging in the shell - it's much easier.

Bentley4 Over a year ago

Thanks, Just got the same answer from Irc. I think using custom django-admin.py commands is the least messy solution. Could you include this in your answer so I can accept it?

Collectives™ on Stack Overflow

Running a standalone script doing a model query in Django with `settings/dev.py` instead of `settings.py`

1 Answer 1

EDIT

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

EDIT

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related