Why python For in Loop is so slow?

Question

I tested performance for in loop in python. It contains just loop and plus operations. But it takes about 0.5secs. How can I do it more faster?

import time

start_time = time.time()

val = -1000000
for i in range(2000000):
    val += 1

elapsed_time = time.time() - start_time

print(elapsed_time) # 0.46402716636657715

Use the timeit module to do time trials; it uses more accurate timings and tries to avoid garbage collection and OS scheduler pitfalls (by repeating the experiment). — Martijn Pieters
– Martijn Pieters, Commented Jun 3, 2015 at 9:45
For high performance, factor out the loop: just assign val = 1000000. This is probably the worst case of premature optimisation I've ever seen! — jonrsharpe
– jonrsharpe, Commented Jun 3, 2015 at 9:49
@Shashank: pypy is not a silver bullet. Don't use it without testing and only when you are using a longer running process; for command-line scripts pypy is only going to be slower. — Martijn Pieters
– Martijn Pieters, Commented Jun 3, 2015 at 10:04
You're not testing "performance of the for in loop" (which doesn't mean much - depends on the iterable...), you're testing the performance of a call to range() with a big number + performance of the iteration on what range() returned for your Python version + performance of integer addition. FWIW in Python 2.x, range() returns a list, so you're testing the performance of creating a list of 2000000 integers... — bruno desthuilliers
– bruno desthuilliers, Commented Jun 3, 2015 at 10:20

Damien · Accepted Answer · 2018-09-14 03:57:03Z

2

Here are some optimizations:

(Python 2) Use xrange() - this will return an iterator and will not need to first generate the list to allow you to iterate it. In Python 3, range() is essentially xrange()
~~Wrap range(2000000) in an iter() function. I am not sure why, but I saw improvements during my tests~~

edited Sep 14, 2018 at 3:57

answered Sep 14, 2018 at 3:38

Damien

6747 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mad Physicist Over a year ago

for implicitly callsiter as it is, no? Can you show some example timings with a clear improvement?

Damien Over a year ago

I think I am falling victim to the "garbage collection and OS scheduler pitfalls" mentioned above. Times for both variants range from 0.12s to 0.14s. Adding iter() makes no odds

Collectives™ on Stack Overflow

Why python For in Loop is so slow?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related