How can I optimize loops within a Django template?

Question

I have a database setup where I have the following schema:

The main model is the Group. Under each group are an arbitrary number of Rules. Each Rule contains an arbitrary number of Regex's and Contacts.

I'm displaying this whole set up under the main page so it looks something like this:

Group A
 - Rule A
   - Regex 1, Contact 1, Contact 2
   - Regex 2, Regex 3, Contact 3

 - Rule B
   - Regex 5, Regex 6,  Regex 6, Contact 4
   - Regex 7, Contact 7, Contact 6


Group B
 - ...

The db query itself is not terribly complex. It's basically:

    for group in groups:
        r = Rule.objects.filter(group=group)
        rules[group]= r

But the template rendering is what is slowing things down. It's taking about 7 seconds to render the template because of the number of nested loops:

{% for group, rules in rules.items %}
        {% for rule in rules %}
                {% for regex in rule.regexes.all %}
                    {{regex.name}}
                    {{regex.type}}
                    {{...}}
                {% endfor %}

                {% for contacts in rule.contacts.all %}
                    {{contacts.name}}
                    {{contacts.email}}
                    {{...}}
                {% endfor %}

        {% endfor %}

{% endfor %}

There's obviously more HTML in here but even without it, this rendering takes about 3-4 seconds (full template is about 7 seconds). For reference, there are roughly 400 rules and each rule has between 1-5 regexes and 1-5 contacts. Is there anyway I can increase the speed of the rendering without having to do pagination?

Daniel Roseman · Accepted Answer · 2019-08-14 17:22:51Z

2

The problem is not the nested loops themselves, but the database lookups that those loops trigger. You can optimise by using prefetch_related in your initial queries.

r = Rule.objects.filter(group=group).prefetch_related('regexes', 'contacts')

Although you also might like to optimise that initial loop; you should be able to do the whole thing in one go. Something like:

groups = Group.objects.all().prefetch_related('rules__regexes', 'rules__contacts')

answered Aug 14, 2019 at 17:22

Daniel Roseman

602k68 gold badges910 silver badges923 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

pnus Over a year ago

Do you know of anyway to prefetch all related foreign keys? For example, if the Regex was made up of another object so we'd have something like: Group -> Rule -> Regex -> Filter is there a way to say Rule.fetch_all_related() so it grabs all associated regexes and then associated Filters as well?

pnus Over a year ago

I ended up just explicitly following the chain which end up working well. A significant speed improvement. Thanks!

Collectives™ on Stack Overflow

How can I optimize loops within a Django template?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related