13

Let's say I have a generator like this:

def a(data):
    for val in data:
        yield val

And let's say I want to wrap this generator in another generator, b, that only yields some of the values from a, depending on their value. b should be able to forward values sent back from the caller to a. I know that the most current way of wrapping a generator in another generator is to use the yield from statement. Something like:

def b(data):
    yield from val = a(data) if val == "foo"

I know that syntax is wrong (it's just to get across the idea), so I'm wondering if there is a correct way to make yield from work with a conditional statement. Or is there some other construct I should use?

Improve this question
4
  • In your example, a only yields a single value. Is that what you're really asking about, or are you envisioning a situation in which a yields many values? Commented Dec 3, 2015 at 20:34
  • You're right - example updated. Commented Dec 3, 2015 at 20:35
  • I hope this will get added at some point. It feels just like the list comprehensions if. Commented Jun 6, 2022 at 21:13
  • You could yield from (val from a(data) if val == 'foo'). You could also conceivably return (val from a(data) if val == 'foo') but unfortunately ONLY if there are no yield statements elsewhere in the function. Return behaves spooky different in a function with a yield statement anywhere in it. (Even AFTER the return statement!) Commented Oct 9, 2024 at 16:07

3 Answers 3

Reset to default
6

As you said, yield from is only for the case where you want to pass everything from the wrapped generator through. If you don't want that, you need to manually iterate over the wrapped generator and do what you want:

def b(data):
    for value in a(data):
        if some_condition(value):
            yield value
        # otherwise don't yield it.

If you want to handle send, you need to handle that yourself too. Here is a simple example that you can play around with to see what's going on:

def a():
    sent = yield "Begin"
    for x in [1, 2, 3]:
        if sent:
            sent = yield "You sent {0}".format(sent)
        else:
            sent = yield x

def b():
    gen = a()
    val = yield next(gen)
    while True:
        wrappedVal = gen.send(val)
        if wrappedVal == 2:
            continue
        val = yield wrappedVal

Basically in this example b "hides" the value 2 if it is yielded by a. (I adapted this from my answer to this question, which is similarly about wrapping generators, but in a slightly different way. You may find discussion there useful, though.)

However, you would need to be very careful when doing this. Since b may skip values from a, any caller that tries to send values into b may not get the expected results, because b may skip values in unexpected places. In cases where you're using send, it often means the caller watches the generator for certain yielded values that communicate some information, then sends values back in response. With b in the middle, this communication may get out of sync. For instance, suppose a yields a "message" which b yields through to the caller, and the caller sends a response back, which b then sends to a. But suppose b swallows a's response. This will cause problems for the caller. You may still be able to make it work, but it just means you have to be very careful in writing b to main any communication channels that are expected by the API of a.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

How would I pass values that are sent into b back to a, though?
@LateCoder: Things get even more manual; you have to manually call send and catch StopIteration. Also, you need to decide what to send to a when b skips a value.
@LateCoder: I updated my answer. But you need to be careful about using send if b can skip values, because it may mean that values sent by the caller to b are not passed to a at "the right time", or that values from a do not reach b's caller at "the right time".
Thank you! I put up the final answer I came to, which seems to be working, inspired by your answer.
6

Assuming function a returns an Iterator[str], how about this?

yield from filter(lambda x: x == "foo" , a(data))
Improve this answer

5 Comments

This is a great answer for cases where you don't need for all the features of yield from (forwarding .send, .throw, and .close calls).
@mtraceur based on the question posted, is there any reason to assume any of the other features you mentioned are relevant to the discussion?
Yes. 1st of all that should be the default assumption - if someone asks "is there a way to get yield from+{other behavior}", the presumably want yield from behavior, not just parts of yield from behavior. 2nd, the asker both posted their own answer and asked a comment on another answer about forwarding sent values. 3rd, a question like this is very likely to be found by people who would benefit from having their attention drawn to the absence of send/throw/close forwarding.
@mtraceur The way stackoverflow works, you’re free to write your own answer if you think you’ve a better one than lecturing on others’ answers about supposed faults. So, go ahead, and do share your knowledge.
The way StackOverflow works, we also add comments to suggest improvements or provide relevant clarifying information (and this is helpful independently of posting answers, which often take more time and preparation - for example, in this case, the best answer I could give only becomes possible after I finish a certain open source library which would include a filter implementation that does full generator delegation). Anyway, I encourage you to try re-reading my comments less negatively - my first comment was mostly a compliment and my second comment was answering a question you asked.
0

Since it looks like I can't incorporate a conditional onto a yield from, I solved this problem by handling the cases I cared about manually. This is what I ended up with: 

def a():
    data = get_data_somewhere()
    for val in data:
        yield val

def b():
    a_gen = a()
    val = next(a_gen)
    try:
        while True:
            if val == "foo":
                received = yield val
                val = a_gen.send(received)
            else:
                val = a_gen.send(None)
     except StopIteration:
         pass
     finally:
         del a_gen
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.