Nesting generator expressions in the argument list for a python function call

Question

I like to use the following idiom for combining lists together, sometimes:

>>> list(itertools.chain(*[[(e, n) for e in l] for n, l in (('a', [1,2]),('b',[3,4]))]))
[(1, 'a'), (2, 'a'), (3, 'b'), (4, 'b')]

(I know there are easier ways to get this particular result, but it comes comes in handy when you want to iterate over the elements in lists of lists of lists, or something like that. The trouble is, when you use generator expressions, this becomes error prone. E.g.

>>> list(itertools.chain(*(((e, n) for e in l) for n, l in (('a', [1,2]),('b',[3,4])))))
[(1, 'b'), (2, 'b'), (3, 'b'), (4, 'b')]

What's happening here is that the inner generator expressions get passed as arguments to itertools.chain, so at the the time they're evaluated, the outer generator expression has finished, and n is fixed at its final value, 'b'. I'm wondering whether anyone has thought of ways to avoid this kind of error, beyond "don't do that."

SilentGhost · Accepted Answer · 2009-10-25 14:12:29Z

6

wouldn't a nested list comprehension be more appropriate?

>>> tt = (('a', [1,2]),('b',[3,4]))
>>> [(s, i) for i, l in tt for s in l]
[(1, 'a'), (2, 'a'), (3, 'b'), (4, 'b')]

answered Oct 25, 2009 at 14:12

SilentGhost

322k67 gold badges312 silver badges294 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Alex Coventry Over a year ago

You can stick it all in the one list comprehension, in this case, but it's more complicated to do that in some cases. I simplified the example for clarity.

SilentGhost Over a year ago

why do stick it into a single line? why not just nested for loops? that would be both more clear and won't require any intermediate lists to be built.

Anurag Uniyal Over a year ago

but isn't question about 'generator expressions' not working ?

SilentGhost Over a year ago

but to me it seems it's not a question at all. OP is quite clear why generators don't work. the only unclear thing here is why is he trying to use them?

John La Rooy · Accepted Answer · 2009-10-25 15:15:33Z

2

Your approach almost works, you just need to flatten the generators. See how the for e in l is moved to the very right

>>> list(itertools.chain((e, n) for n, l in (('a', [1,2]),('b',[3,4])) for e in l ))
[(1, 'a'), (2, 'a'), (3, 'b'), (4, 'b')]

Here is how to do the same thing using itertools.product

>>> X=itertools.chain(*(itertools.product(*i[::-1]) for i in (('a', [1,2]),('b',[3,4]))))
>>> print list(X)
[(1, 'a'), (2, 'a'), (3, 'b'), (4, 'b')]

or if you are allowed to switch the tuples around

>>> X=itertools.chain(*(itertools.product(*i) for i in (([1,2],'a'),([3,4],'b'))))
>>> print list(X)
[(1, 'a'), (2, 'a'), (3, 'b'), (4, 'b')]

edited Oct 25, 2009 at 15:15

answered Oct 25, 2009 at 14:49

John La Rooy

306k54 gold badges378 silver badges514 bronze badges

1 Comment

Alex Coventry Over a year ago

Thanks, the product function looks like it ought to be safer and lead to clearer expressions.

Mike Graham · Accepted Answer · 2009-10-25 15:47:19Z

0

I'm going to suggest

data = (('a', [1,2]), ('b', [3,4]))

result = []
for letter, numbers in data:
     for number in numbers:
         result.append((number, letter))

It's a lot more readable than your solution family. Fancy is not a good thing.

answered Oct 25, 2009 at 15:47

Mike Graham

77.2k16 gold badges105 silver badges131 bronze badges

Collectives™ on Stack Overflow

Nesting generator expressions in the argument list for a python function call

3 Answers 3

4 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related