2

I'm making a Django app and I have a view that display both sides of an object_set (a reverse many to many). Because of this, I want to query all of the objects on both sides at the same time. Specifically speaking, I want to have all of the Signup objects that are associated with each Event.

(The view page format should look like this.)

Event (0)
-- Signup (0.0)
-- Signup (0.1)
-- Signup (0.2)
-- Signup (0.3)
Event (1)
-- Signup (1.0)
-- Signup (1.1)
Event (3)
-- Signup (3.0)
-- Signup (3.1)
-- Signup (3.2)
-- Signup (3.3)
...

The code is as follows:

class TournamentDetailView(DetailView):
    model = Tournament

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        tournament_id = self.get_object().pk
        events = Event.objects.annotate(
            cached_signups=(
                Signup.objects
                    .filter(event_id=OuterRef('pk'), tournament_id=tournament_id, dropped=False)
                    .order_by('created')
                    .defer('tournament')
            )
        ).all()
        context['events'] = events
        return context

Here's the traceback:

Traceback:    

File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\exception.py" in inner
  35.             response = get_response(request)    

File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\base.py" in _get_response
  128.                 response = self.process_exception_by_middleware(e, request)    

File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\base.py" in _get_response
  126.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)    

File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\views\generic\base.py" in view
  69.             return self.dispatch(request, *args, **kwargs)    

File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\views\generic\base.py" in dispatch
  89.         return handler(request, *args, **kwargs)    

File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\views\generic\detail.py" in get
  106.         context = self.get_context_data(object=self.object)    

File "C:\Users\werdn\PycharmProjects\gwspo-signups-website\gwhs_speech_and_debate\tournament_signups\views.py" in get_context_data
  171.                     .defer('tournament')    

File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\db\models\manager.py" in manager_method
  82.                 return getattr(self.get_queryset(), name)(*args, **kwargs)    

File "C:\Users\werdn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\db\models\query.py" in annotate
  1000.             if alias in annotations and annotation.contains_aggregate:    

Exception Type: AttributeError at /tournaments/detail/lobo-howl/
Exception Value: 'Query' object has no attribute 'contains_aggregate'

I'm not sure why this is happening and it seems to be happening on the Signups.objects query but even with Signups.objects.all(), this Exception seems to be triggered. That leads me to believe that this is not an issue with the use of OuterRef('pk').

1
  • That's not possible afaik without using raw SQL. You could wrap your Signup query in a Subquery() (see documentation here) but that only works for specific values, not for the entire objects. If you just want to optimise your database fetches, you should use prefetch_related Commented Sep 15, 2018 at 16:06

3 Answers 3

4

You can't just put a Query inside an annotation, since an annotation is like adding a column to the row you're fetching. Django supports the concept of a Subquery in an annotation, but that only works if you're fetching one aggregated value of the related model. This would work for example:

signups = Signup.objects
                .filter(event_id=OuterRef('pk'), tournament_id=tournament_id, dropped=False)
                .order_by('created')
                .defer('tournament')
events = Event.objects.annotate(latest_signup=Subquery(signups.values('date')[:-1]))

If you just want to optimise database access so that you don't make a database query for each Event to fetch the related Signups, you should use prefetch_related:

events = Event.objects.all().prefetch_related('signups')

Since you didn't show how your models are defined, I'm assuming this is a reverse M2M relationship:

class Signup(models.Model):
    events = models.ManyToManyField(to='Event', related_name='signups')

If you don't specify a related_name, the attribute to use for the prefetch is signup_set (which is not documented anywhere and very confusing since for aggregations it's the lowercase name of the model):

events = Event.objects.all().prefetch_related('signup_set')

This will make two queries: One for the Event objects, and only one extra for all the related Signup objects (instead of Event.objects.count() queries). The documentation for prefetch_related contains some useful information on how this works.

Sign up to request clarification or add additional context in comments.

1 Comment

prefetch_related seems to fail: Cannot find 'signup' on Event object, 'signup' is an invalid parameter to prefetch_related() (this also happens for 'signups'). I think that's since the M2M exists on the Signup model.
1

See @dirkgroten's as to why the question produces the error. Read on for an alternate method that solves the issue.

The reason why @dirkgroten's answer wouldn't work is that the model definition for Event doesn't include a value signups. However, since the ManyToMany relationship is defined on the Signup model, I can get the prefetch_related to work off of the Signup query, as we see below.

signups = Signup.objects.filter(
    tournament_id=tournament_id
).prefetch_related(
    'event',
    ...
)
context['signups'] = signups
context['events'] = signups.values('event', 'event__name', ...).distinct().order_by('event__name')

(Note that order_by is required in order for distinct to work and that values() returns a dict, not a queryset.)

6 Comments

I've added some information to show how to do the prefetch for a reverse relationship.
@dirkgroten I see that -- the reverse with signup_set doesn't throw any errors but also I think that it doesn't prefetch. The .query value is: SELECT "tournament_signups_event"."id" FROM "tournament_signups_event" ORDER BY "tournament_signups_event"."created" DESC (when just selecting the id)
You mean when adding .values('id') to it? Django is smart enough to only query what’s needed when the query is actually executed. Also prefetch_related actually does two queries (Django can’t do it in one), the second one is executed when you’re actually accessing any of the related objects.
Ah. Good call. So when querying Event.objects.all().prefetch_related('signup_set'), then iterating over the result in the view, each event's signup_set prints as tournament_signups.Signup.None. Also, if I try to iterate (like: {% for signup in event.signup_set %}<p>{{signup}}</p>{% endfor %}), I get a 'RelatedManager' object is not iterable error. It doesn't seem like this method is actually returning any values in the signup_set.
It is. You need to loop through event.signup_set.all
|
0

If you want to query all Event/Signup pairs in your ManyToMany relationship, the most straightforward approach would be to query the helper table that stores just those pairs (as two ForeignKeys).

To get easy access to that table, you can make it a Django model by using the through option of ManyToManyField, see https://docs.djangoproject.com/en/stable/ref/models/fields/#manytomanyfield

Such a through model always exists implicitly for any Django model by using the through option of ManyToManyField m2nfield and can be accessed via Model.m2nfield.through.objects.

Or you don't use a ManyToManyField at all and just create a separate model with two ForeignKeyFields to represent the pairs.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.