Django filtering by list of model objects

Question

I'm trying to filter a queryset by checking that the object is in a list of those objects.

employee_list = [<Employee: A>, <Employee: B>, <Employee: C>]
qs = Employee.objects.filter(id__in=employee_list, [other_filters])

After running above, qs is an empty list. I was thinking that I could make a new list such as

employee_ids = [emp.id for emp in employee_list]
qs = Employee.objects.filter(id__in=employee_ids, [other_filters])

I haven't done benchmarking on this method, but I imagine performance would probably take a hit. Alternatively I could intersect the lists afterwards like:

qs = Employee.objects.filter([other_filters])
filtered_qs = [emp for emp in employee_lids if emp in qs]

However, I think the performance hit would be even worse this way.

What's the best/fastest way to do this? Thanks.

Martol1ni · Accepted Answer · 2014-07-01 14:01:45Z

2

Rule of thumb is to filter as much as possible through SQL, so I would go for

qs = Employee.objects.filter(id__in=[emp.id for emp in employee_list], [other_filters])

I do not have any performance testing to back this up with though.

answered Jul 1, 2014 at 14:01

Martol1ni

4,7122 gold badges33 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alvaro Over a year ago

I may note this greatly depends on the size of both the queryset, the list and the use case

Jmills · Accepted Answer · 2014-07-01 14:18:47Z

1

As Martol1ni noted you want to filter at the SQL level whenever possible so I think your methods do get progressively slower, but another issue...

Based on the Django Docs: https://docs.djangoproject.com/en/dev/ref/models/querysets/ I think yourid__in should be a list of integer ids, not a list of models.

Edit: Oh I see he covered this in his answer, but it was not explicit that it was incorrect in your question.

Edit2: But yes, if you want to know for sure, what really matters is real world performance, which you can do with django-debug-toolbar. It seems to me though the real issue was the id__in misuse which lead you to find trickier ways to do what you wanted to do.

edited Jul 1, 2014 at 14:18

answered Jul 1, 2014 at 14:04

Jmills

2,4322 gold badges20 silver badges22 bronze badges

1 Comment

SirDeimos Over a year ago

I ran a profiler on each of the different methods, but didn't see too much a difference between them. My test data set is much smaller than production, obviously, but constructing a list of id's from the list of objects and then filtering id__in id_list seemed to be the fastest.

Collectives™ on Stack Overflow

Django filtering by list of model objects

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related