Python bug? Lazy objects have hidden state

Question

Consider this (Python 3.3):

a=enumerate([2,3,5])
print(list(a))
print(list(a))

Do you really expect two print calls to print different things? Neither did I.

The same thing happens if you replace list with set, tuple or dict. It also happens if you replace enumerate object with map or filter, but curiously, not if you replace it with range.

Maybe it is a feature. But it's very surprising, not documented (at least I haven't been able to find anything about it), and not consistent (range works differently). What do you think?

I really expect the two print calls to print different things. — user395760
– user395760, Commented Jan 3, 2013 at 10:20

Martijn Pieters · Accepted Answer · 2013-01-03 10:35:48Z

enumerate() returns an iterator, as do the other calls.. You can only loop through an iterator once; it is then exhausted.

You can create such an iterator yourself with a generator function:

def somelist_generator():
    somelist = [1, 2, 3]
    while somelist:
        yield somelist.pop()

If you were to loop over somelist_generator(), the list somelist would be emptied. You can only do that once, since .pop() removes elements:

>>> it = somelist_generator()
>>> for i in it:
...     print(i)
... 
3
2
1
>>> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

The next() call tries to get another value from the it iterator; it is already empty so the StopIteration exception is raised. That exception signals there are no more elements to be gotten, and that's why you end up with an empty list the second time you try and get anything from an iterator:

>>> list(it)
[]

range() does not return an iterator. It returns a range object instead, which represents a memory-efficient series of numbers; only the start, end and stride need to be stored and everything else can be derived from those 3 points.

Duncan · Accepted Answer · 2013-01-03 10:26:26Z

2

The behaviour is documented at http://docs.python.org/3/glossary.html#term-iterator

One notable exception is code which attempts multiple iteration passes. ... Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container.

answered Jan 3, 2013 at 10:26

Duncan

96.4k15 gold badges129 silver badges160 bronze badges

3 Comments

Veky Over a year ago

Ah, I see. I have confused iterators with iterables. Iterators are kinda iterables, but only once. :-) That's too bad for the code I'm writing... I suppose there is no nice way iterators can be "rewound"?

Duncan Over a year ago

If you want to rewind an iterator then you should convert it to a list and use that instead. If the iterator is too large (or even infinite) to convert to a list then there is no general way to rewind it.

rtpg Over a year ago

You could probably build a structure that allows you to "step back" only once or twice. A limited rewind if you will

Collectives™ on Stack Overflow

Python bug? Lazy objects have hidden state

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related