Django annotate a queryset with objects

Question

I have models named Product, Material, and Tag. Each product has many materials, each material has many tags. Each product has tags (not an actual relation) , the tags are defined as all distinct tags of the product's materials. I would like to pass Product queryset with tags attribute to django rest serializer without N+1 problem.

I used subquery, but it only return a row and a column. I tried to operate it at python level, but it produces N+1 problem, and even if I somehow manage to avoid N+1 problem it will be slower than using ORM because it's python. I tried to add tags relation to Product, and update it everytime the material and/or material's tags changed, and query it with prefetch_related so it has no N+1, it works great, but adding more complexities to my code for something that could be so simple. So I prefer to not add tags relation in product.

class Product(models.Model):
    name = models.CharField(max_length=255)

class Material(models.Model):
    name = models.CharField(max_length=255)
    product = models.ForeignKey(Product, related_name='materials', on_delete=models.CASCADE)

class Tag(models.Model):
    name = models.CharField(max_length=255)
    material = models.ForeignKey(Material, related_name='tags', on_delete=models.CASCADE)

class TagSerializer(serializers.ModelSerializer):
    class Meta:
        model = Tag
        fields = '__all__'

class ProductSerializer(serializers.ModelSerializer):
    tags = TagSerializer(many=True)
    class Meta:
        model = Product
        fields = '__all__'

I expect to have queryset that can be passed to ProductSerializer without N+1 problem and without using signals.

You should use .prefetch_related(..) here: docs.djangoproject.com/en/2.2/ref/models/querysets/… — willeM_ Van Onsem
– willeM_ Van Onsem, Commented Sep 2, 2019 at 11:58

willeM_ Van Onsem · Accepted Answer · 2019-09-02 14:00:23Z

3

You don't need to load related objects yourself. Django has tooling to do that for you. You can make use of .prefetch_related(..) [Django-doc]. This will make queries to retrieve the related objects with an additional query, and then link the objects to the related object. It thus basically performs a JOIN at the Python/Django side and only uses a constant number of queries for these JOINs:

products = Product.objects.prefetch_related('materials__tags')
serializer = ProductSerializer(products, many=True)

Here we thus avoid fetching the tags for each Product with another query.

edited Sep 2, 2019 at 14:00

answered Sep 2, 2019 at 12:03

willeM_ Van Onsem

482k33 gold badges483 silver badges624 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

linux newbie Over a year ago

Yes. However the product doesn't have tags field. I could add it, but it will add complexities to my code because I have to manage the product's tags each time its material and material's tags are changed.

willeM_ Van Onsem Over a year ago

@linuxnewbie: you do not need a tags field. Django knows that this is the relation, since the related_name is 'tags' and thus it will properly trigger the join table. So you do not need a tags field at all :)

linux newbie Over a year ago

The tags related name is for material not product.

willeM_ Van Onsem Over a year ago

@linuxnewbie: ah, then it is materials__tags.

Collectives™ on Stack Overflow

Django annotate a queryset with objects

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related