Evaluating all nested generators in Python

Question

I can make a nested group of iterators using a recursive function like this:

def rprint(n):
    for i in range(n):
        print('n = %d' % n)
        yield rprint(n-1)

Then for a simple example I can evaluate the nested generators manually.

>>> p3 = rprint(3)
>>> p3
<generator object rprint at 0x1043b7048>
>>> p2_0 = next(p3)
>>> p2_1 = next(p3)
>>> p2_2 = next(p3)
n = 3
n = 3
n = 3
>>> next(p3) # will raise an error
StopIteration
>>> p1_0_0 = next(p2_0)
>>> p1_0_1 = next(p2_0)
n = 2
n = 2
>>> next(p2_0)# will raise an error
StopIteration
>>> p0_0_0_0 = next(p1_0_0)
n = 1
>>> next(p1_0_0)# will raise an error
StopIteration
>>> p0_0_1_0 = next(p1_0_1)
n = 1
>>> next(p1_0_1)# will raise an error
StopIteration

and this goes on as follows ...

>>> p1_1_0 = next(p2_1)
>>> p1_1_1 = next(p2_1)
n = 2
n = 2
>>> next(p2_1)# will raise an error
StopIteration

... etc.

How can I do this in an automated way for any value of n in rprint? I am not interested in making variable references to the intermediate generators (as I did in the example to illustrate the objects structure).

No @PaulPanzer I don't care about order in my actual use case — Alex
– Alex, Commented Mar 23, 2017 at 22:20
You say "intermediate generators". Does your actual use case involve the subgenerators eventually yielding something that isn't a generator? That never happens here; "flattening" these generators recursively into a non-nested generator just produces an empty generator that prints a whole bunch before reporting that it's empty. — user2357112
– user2357112, Commented Mar 23, 2017 at 22:32

user2357112 · Accepted Answer · 2017-03-23 22:20:45Z

5

While you can do that, using generators purely for side effects this way is a really weird thing to do, and you may want to reconsider your design. That said:

def do_the_thing_you_want(generator):
    # Call list on the generator to force all side effects before running
    # subgenerators, as done in the question.
    for subgenerator in list(generator):
        do_the_thing_you_want(subgenerator)

answered Mar 23, 2017 at 22:20

user2357112

286k32 gold badges490 silver badges571 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Alex Over a year ago

Yeah I am probably going down a crappy path to accomplish what I want. I am using recursion to traverse a nested tree and want to return multiple times for each function call (i.e. the recursive return is in a loop). The way I decided I could do this was by yielding the recursion function inside the loop. And hence where this question came from

user2357112 Over a year ago

@AlexG: That sounds like you should consider rewriting your original generator to use yield from instead of yield. yield from is Python's mechanism for writing recursive generators, as well as for any case where one generator should delegate to another.

user2357112 Over a year ago

Alternatively, it might not actually make sense to have generators at all. Why do you say you want to return multiple times for each call? Why do you want to return at all?

Alex Over a year ago

Yes. Just pure yes. Your yield from suggestion is great because then I just call next(rprint(3)) and bam - problem solved. But your second suggestion is the real winner. Thank you. I am dumb :)

Nils Werner · Accepted Answer · 2017-03-23 22:23:41Z

3

By recursively flattening your generators "depth first":

def flatten(nested):
    for sublist in nested:
        for element in flatten(sublist):
            yield element

yields

n = 3
n = 2
n = 1
n = 2
n = 1
n = 3
n = 2
n = 1
n = 2
n = 1
n = 3
n = 2
n = 1
n = 2
n = 1

answered Mar 23, 2017 at 22:23

Nils Werner

37.2k7 gold badges85 silver badges108 bronze badges

2 Comments

user2357112 Over a year ago

Note that this produces a different output order from the order in the question.

Nils Werner Over a year ago

Yes, order of execution vs memory consumption is a tradeoff to make. OP said in comments that order of execution does not matter though.

Paul Panzer · Accepted Answer · 2017-03-23 22:24:53Z

2

Not entirely sure that's what you want but the following code flattens your nested generator:

def rprint(n):
    for i in range(n):
        print('n = %d' % n)
        yield rprint(n-1)

def flatten(nested):
    for j in nested:
        if hasattr(j, '__iter__') or hasattr(j, '__getitem__'):
            yield from flatten(j)
        else:
            yield j

list(flatten(rprint(3)))

prints:

# n = 3
# n = 2
# n = 1
# n = 2
# n = 1
# n = 3
# n = 2
# n = 1
# n = 2
# n = 1
# n = 3
# n = 2
# n = 1
# n = 2
# n = 1

answered Mar 23, 2017 at 22:24

Paul Panzer

53.3k3 gold badges60 silver badges103 bronze badges

1 Comment

user2357112 Over a year ago

Note that this produces a different output order from the order in the question.

Collectives™ on Stack Overflow

Evaluating all nested generators in Python

3 Answers 3

4 Comments

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related