Lazy Python behavior with generators

Question

OK, right off the bat: I know Python generators are lazy. That said, this is blowing my mind; I don't get it. Pythonic friends, please don't crucify me for the ignorance of this question. This was not encountered in a real-world development situation, just a curiosity that has been bothering me.

def gen1(n):
    print('gen1')
    return 1

def gen2(n):
    print('gen2')
    return 2

print(gen2(gen1(0)))

This sample does exactly what I expect it to do, it prints gen1 then gen2 then 2.

The following code sample does not:

def gen1(n):
    print('gen1')
    yield 1

def gen2(n):
    print('gen2')
    yield 2

print(list(gen2(gen1(0))))

Instead, it just prints 'gen2' and [2]. So wait a hot sec, it calls gen2 first? It never evaluates gen1? WHAT?! Under the hood, how does it pull that devil magic off? What is actually taking place here? So the outermost thing is identified as a generator and it triggers some introspections that identifies that evaluation of the inner generator is unnecessary? What is a more informative way of tracing this behavior?

Inside gen2, you are not using n, so the code inside gen1 is not executed. A generator "starts" execution only when you call next on it for the first time — BlackBear
– BlackBear, Commented Jan 28, 2018 at 0:11

user2390182 · Accepted Answer · 2018-01-28 00:26:55Z

4

The code in the generator is not executed when the generator is created, but when it is consumed by a driving construct like a loop, a comprehension or another generator. The following might clear this up:

>>> def gen1(n):
...     print('gen1')
...     yield 1
...     print('gen1')
...     yield 2
... 
>>> g = gen1(5)  
# this returns the generator object, 
# but does not execute any code in it, hence 'lazy'
>>> for x in g:  
        # each 'x' assignment executes code til the next 'yield'
...     print x
... 
gen1
1
gen1
2

You might wonder how the program knows it is a genrator before the first yield statement, but the interpreter checks the entire function body first, and if there is any yield the function is a generator function whose body is not executed when called.

edited Jan 28, 2018 at 0:26

answered Jan 28, 2018 at 0:13

user2390182

73.7k6 gold badges71 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Lazy Python behavior with generators

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related