Python generator with numpy array

Question

I'd like to create a generator that returns a array on fly. For example:

import numpy as np
def my_gen():
    c = np.ones(5)
    j = 0
    t = 10
    while j < t:
        c[0] = j
        yield c
        j += 1

With a simple for loop:

for g in my_gen():
    print (g)

I got what I want. But with list(my_gen()), I got a list which contains always the same thing.

I digged a little deeper and I find when I yield c.tolist() instead of yield c, everything went ok...

I just cannot explain myself how come this strange behaviour...

Anand S Kumar · Accepted Answer · 2015-10-07 09:56:42Z

6

That is because c is always pointing to the same numpy array reference, you are just changing the element inside c in the generator function.

When simply printing, it prints the complete c array at that particular moment , hence you correctly get the values printed.

But when you are using list(my_gen()) , you keep adding the same reference to c numpy array into the list, and hence any changes to that numpy array also reflect in the previously added elements in the list.

It works for you when you do yield c.tolist() , because that creates a new list from the numpy array, hence you keep adding new list objects to the list and hence changes in the future to c does not reflect in the previously added lists.

answered Oct 7, 2015 at 9:56

Anand S Kumar

91.4k18 gold badges196 silver badges179 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

XXXXXL Over a year ago

Great! Thank you for your answer :)

hpaulj · Accepted Answer · 2015-10-07 17:57:50Z

An alternative generator returns a copy of a list. I'm retaining the np.ones() as a convenient way of creating the numbers, but converting it to a list right away (just once) (array.tolist() is relatively expensive).

I yield c[:] to avoid that 'current version' problem.

def gen_c():
        c = np.ones(5,dtype=int).tolist()
        j = 0
        t = 10
        while j < t:
                c[0] = j
                yield c[:]
                j += 1


In [54]: list(gen_c())
Out[54]: 
[[0, 1, 1, 1, 1],
 [1, 1, 1, 1, 1],
 [2, 1, 1, 1, 1],
 [3, 1, 1, 1, 1],
 [4, 1, 1, 1, 1],
 [5, 1, 1, 1, 1],
 [6, 1, 1, 1, 1],
 [7, 1, 1, 1, 1],
 [8, 1, 1, 1, 1],
 [9, 1, 1, 1, 1]]
In [55]: np.array(list(gen_c()))
Out[55]: 
array([[0, 1, 1, 1, 1],
       [1, 1, 1, 1, 1],
       [2, 1, 1, 1, 1],
       [3, 1, 1, 1, 1],
       [4, 1, 1, 1, 1],
       [5, 1, 1, 1, 1],
       [6, 1, 1, 1, 1],
       [7, 1, 1, 1, 1],
       [8, 1, 1, 1, 1],
       [9, 1, 1, 1, 1]])

XXXXXL · Accepted Answer · 2015-10-07 09:57:16Z

0

Ok, I think because in this generator, since I'm returning the same reference, generator yield always the same thing. If I yield np.array(c), that'll work...

answered Oct 7, 2015 at 9:57

XXXXXL

1372 silver badges7 bronze badges

Collectives™ on Stack Overflow

Python generator with numpy array

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related