How can I vectorize this (numpy) operation in python?

Question

I have two vectors of shape (batch, dim), which I'm trying to subtract from one another. Currently I am using a simple loop to subtract a specific entry in a vector (i.e. error) based on the second vector(i.e. label),from 1 :

per_ts_loss=0
for i, idx in enumerate(np.argmax(label, axis=1)):
    error[i, idx] -=1
    per_ts_loss += error[i, idx]

How Can I vectorize this?

For an example, error and label can look like this :

error :
array([[ 0.5488135   0.71518937  0.60276338  0.54488318  0.4236548 ]
       [ 0.64589411  0.43758721  0.891773    0.96366276  0.38344152]])
label:
    array([[0, 0, 0, 1, 0 ],
           [0, 1, 0, 0, 0]])

for this example, running the code below results in the following outcomes:

for i, idx in enumerate(np.argmax(label,axis=1)):
    error[i,idx] -=1
    ls_loss += error[i,idx]

result :

error: 
 [[ 0.5488135   0.71518937  0.60276338  0.54488318  0.4236548 ]
 [ 0.64589411  0.43758721  0.891773    0.96366276  0.38344152]]
label: 
 [[ 0.  0.  0.  1.  0.]
 [ 0.  1.  0.  0.  0.]]

error(indexes 3 and 1 are changed): 
[[ 0.5488135   0.71518937  0.60276338 -0.45511682  0.4236548 ]
 [ 0.64589411 -0.56241279  0.891773    0.96366276  0.38344152]]
per_ts_loss: 
 -1.01752960574

Here is the code itself : https://ideone.com/e1k8ra

I get stuck on how to use the result of np.argmax, since the result is a new vector of indexes, and it cant simply be used like :

 error[:, np.argmax(label, axis=1)] -=1

So I'm stuck here!

Could you share a sample of the arrays, and the expected output? — yatu
– yatu, Commented Mar 17, 2019 at 11:14
@yatu: that was a typo, I fixed it. the example is also provided — Hossein
– Hossein, Commented Mar 17, 2019 at 11:20

hpaulj · Accepted Answer · 2019-03-17 16:40:20Z

1

Replace:

error[:, np.argmax(label, axis=1)] -=1

with:

error[np.arange(error.shape[0]), np.argmax(label, axis=1)] -=1

and of course

loss = error[np.arange(error.shape[0]), np.argmax(label, axis=1)].sum()

In your example you are changing, and summing, error[0,3] and error[1,1], or in short error[[0,1],[3,1]].

answered Mar 17, 2019 at 16:40

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Tomas Giro · Accepted Answer · 2019-03-17 11:53:48Z

0

Maybe this:

import numpy as np


error = np.array([[0.32783139, 0.29204386, 0.0572163 , 0.96162543, 0.8343454 ],
       [0.67308787, 0.27715222, 0.11738748, 0.091061  , 0.51806117]])

label= np.array([[0, 0, 0, 1, 0 ],
           [0, 1, 0, 0, 0]])



def f(error, label):
    per_ts_loss=0
    t=np.zeros(error.shape)
    argma=np.argmax(label, axis=1)
    t[[i for i in range(error.shape[0])],argma]=-1
    print(t)
    error+=t
    per_ts_loss += error[[i for i in range(error.shape[0])],argma]


f(error, label)

Ouput:

[[ 0.  0.  0. -1.  0.]
 [ 0. -1.  0.  0.  0.]]

answered Mar 17, 2019 at 11:53

Tomas Giro

4,28527 silver badges31 bronze badges

1 Comment

Hossein Over a year ago

Thanks for the effort, but no. O dont subtract the label, the label is used to subtract 1 from the error (this is to show, how much we are away from the correct answer (i.e 1 ) and this is not a vectorized implementation either.

Collectives™ on Stack Overflow

How can I vectorize this (numpy) operation in python?

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related