Iterating over the columns of a numpy array and a vector

Question

I have a matrix of dimensions n x d and a vector of labels 1 x n.

n = 3
d = 2
data = np.random.rand((n,d))
labels = np.random.choice(1, n)

I want to iterate over the ith column of data and the ith element of labels at the same time. So far, I have:

for i in range(n):
  x = data[i]
  y = labels[i]

  ... do something with them ..

I've tried to use np.nditer to do this, but have trouble getting the vector and matrix to work together nicely:

for x, y in np.nditer([data, labels]):
  ...

ValueError: operands could not be broadcast together with shapes (3,2) (3,)

Forget about nditer; it's too complicated for this. How about zip? — hpaulj
– hpaulj, Commented Feb 21, 2022 at 2:51
@hpaulj zip seems a little slow as well, so I guess it must not be the iteration causing the bottleneck. Thanks! — AndW
– AndW, Commented Feb 21, 2022 at 2:57
Repeat after me "iteration is slow, iteration is slow, ...". nditer does not speed it up. In most cases, the really slow part of iteration is the "do something". The iteration mechanism is a minor part of the overall time. — hpaulj
– hpaulj, Commented Feb 21, 2022 at 3:04
Do you know the "rules of broadcasting"? A (3,1) array will broadcast with a (3,2). A (3,) will not. — hpaulj
– hpaulj, Commented Feb 21, 2022 at 3:10

hpaulj · Accepted Answer · 2022-02-21 03:16:36Z

1

In [15]: n = 3
    ...: d = 2
    ...: data = np.random.rand(n, d)
    ...: labels = np.random.choice(10, n)
In [16]: data, labels
Out[16]: 
(array([[0.87013539, 0.66778321],
        [0.63311902, 0.74640742],
        [0.76874321, 0.43470357]]),
 array([6, 8, 1]))

The straight forward zip:

In [17]: for i, j in zip(data, labels):
    ...:     print(i, j)
[0.87013539 0.66778321] 6
[0.63311902 0.74640742] 8
[0.76874321 0.43470357] 1

a working nditer (not that I recommend it):

In [18]: for x, y in np.nditer([data, labels[:, None]]):
    ...:     print(x, y)
0.8701353861218606 6
0.6677832101171755 6
0.6331190219218099 8
0.7464074205732978 8
0.7687432095639312 1
0.43470357108767144 1

answered Feb 21, 2022 at 3:16

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Iterating over the columns of a numpy array and a vector

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related