Getting data from related tables using Django ORM

Question

Using the Django ORM, how does one access data from related tables without effectively making a separate call for each record (or redundantly denormalizing data to make it more easily accessible)?

Say I have 3 Models:

class Tournament(models.Model):
    name = models.CharField(max_length=250) 
    active = models.BooleanField(null=True,default=1)
    
class Team(models.Model):
    name = models.CharField(max_length=250)
    coach_name = models.CharField(max_length=250)
    active = models.BooleanField(null=True,default=1)
    
class Player(models.Model):
    user = models.ForeignKey(
        settings.AUTH_USER_MODEL,
        on_delete=models.DO_NOTHING
    )
    number = models.PositiveIntegerField()
    age = models.PositiveIntegerField()
    active = models.BooleanField(null=True,default=1)

Note that this Player model is important in the application as it's a major connection to most of the models - from registration to teams to stats to results to prizes. But this Player model doesn't actually contain the person's name as the model contains a user field which is the foreign key to a custom AUTH_USER_MODEL ('user') model which contains first/last name information. This allows the player to log in to the application and perform certain actions.

In addition to these base models, say that since a player can play on different teams in different tournaments, I also have a connecting ManyToMany model:

class PlayerToTeam(models.Model):
    player = models.ForeignKey(
        Player,
        on_delete=models.DO_NOTHING
    )
    team = models.ForeignKey(
        Team,
        on_delete=models.DO_NOTHING
    )
    tournament = models.ForeignKey(
        Tournament,
        on_delete=models.DO_NOTHING
    )

As an example of one of the challenges I'm encountering, let's say I'm trying to create a form that allows coaches to select their starting lineup. So I need my form to list the names of the Players on a particular Team at a particular Tournament.

Given the tournament and team IDs, I can easily pull back the necessary QuerySet to describe the initial records I'm interested in.

playersOnTeam = PlayerToTeam.objects.filter(tournament=[Tournament_id]).filter(team=[Team_id])

This returns the QuerySet of the IDs (but only the IDs) of the team, the tournament, and the players. However, the name data is two models away:

PlayerToTeam->[player_id]->Player->[user_id]->User->[first_name] [last_name]

Now, if I pull back only a single record, I could simply do

onlyPlayerOnTeam = PlayerToTeam.objects.filter(tournament=[Tournament_id]).filter(team=[Team_id]).filter(player=[Player_id]).get()

onlyPlayerOnTeam.player.user.first_name

So if I was only needing to display the names, I believe I could pass the QuerySet in the view return and loop through it in the template and display what I need. But I can't figure out if you can do something similar when I need the names to be displayed as part of a form.

To populate the form, I believe I could loop through the initial QuerySet and build a new datastructure:

playersOnTeam = PlayerToTeam.objects.filter(tournament=[Tournament_id]).filter(team=[Team_id])

allPlayersData= []
for nextPlayer in playersOnTeam:
    playerDetails= {
        "player_id": nextPlayer.player.id,
        "first_name": nextPlayer.player.user.first_name, 
        "last_name": nextPlayer.player.user.last_name,
    }
    allPlayersData.append(playerDetails)

form = StartingLineupForm(allPlayersData)

However, I fear that would result in a separate database call for every player/user!

And while that may be tolerable for 6-10 players, for larger datasets, that seems less than ideal. Looping through performing a query for every user seems completely wrong.

Furthermore, what's frustrating is that this would be simple enough with a straight SQL query:

SELECT User.first_name, User.last_name 
FROM PlayerToTeam 
INNER JOIN Player ON PlayerToTeam.player_id = Player.id 
INNER JOIN User ON Player.user_id = User.id 
WHERE PlayerToTeam.tournament_id=[tourney_id] AND PlayerToTeam.team_id=[team_id]

But I'm trying to stick to the Django ORM best practices as much as I can and avoid just dropping to SQL queries when I can't immediately figure something out, and I'm all but certain that this isn't so complicated of a situation that I can't accomplish this without resorting to direct SQL queries.

I'm starting to look at select_related and prefetch_related, but I'm having trouble wrapping my head around how those work for relations more than a single table connection away. Like it seems like I could access the Player.age data using the prefetch, but I don't know how to get to User.first_name from that.

Any help would be appreciated.

tarasinf · Accepted Answer · 2021-12-22 12:59:05Z

5

I would suggest two approaches:

A) select related (one DB query):

objects = PlayerToTeam.objects.filter(
    ...
).select_related(
    'player__user',
).only('player__user__name')

name = objects.first().user.name

B) annotate (one DB query):

objects = PlayerToTeam.objects.filter(
    ...
).annotate(
    player_name=F('player__user__name'),
)

name = objects.first().player_name

To be sure you have only one object for specific player, team and tournament, I would suggest adding unique_together:

class PlayerToTeam(models.Model):
    ...

    class Meta:
        unique_together = ('player', 'team', 'tournament', )

edited Dec 22, 2021 at 12:59

answered Nov 14, 2020 at 10:04

tarasinf

8886 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user29021 Over a year ago

Thank you for the response. I do have that unique_together setting. So I appreciate you confirming that. So then I would just need to loop through the QuerySet and build the datastructure to pass to the form class, right? And by using select_related, it won't execute an individual query for each record to get the name? If I may ask one more (slightly related) question, is there a relatively straightforward way to determine how many actual queries are being called per request?

tarasinf Over a year ago

I recommend writing a unit test for it using with self.assertNumQueries(10):, when this test fails it prints the whole list of DB queries.

user29021 Over a year ago

I found this article that has a decorator that appears to output time and number of queries: medium.com/better-programming/… Now I just need to figure out how to use decorators. :)

Collectives™ on Stack Overflow

Getting data from related tables using Django ORM

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related