Vectorize this function in Numpy Python

Question

I have an array of 60,000 numbers from 0-9:

In [1]: trainY
Out[1]: 
array([[5],
       [0],
       [4],
       ..., 
       [5],
       [6],
       [8]], dtype=int8)

And I have a function to transform each element in trainY into a 10 element vector as per below:

0 -> [1,0,0,0,0,0,0,0,0,0]
1 -> [0,1,0,0,0,0,0,0,0,0]
2 -> [0,0,1,0,0,0,0,0,0,0]
3 -> [0,0,0,1,0,0,0,0,0,0]
...
9 -> [0,0,0,0,0,0,0,0,0,1]

The function:

def transform_y(y):
    new_y = np.zeros(10)
    new_y[y] = 1
    return new_y

My code only works 1 element at a time. What's the best way to transform my trainY array all at once (other than a for loop)? Should I use map? Can someone also show me how to re-write the function so that's it's vectorised?

Thank you.

Saullo G. P. Castro · Accepted Answer · 2013-11-07 09:02:04Z

4

You can considerably improve your code speed creating an 2-D array with ones along the diagonal and then extract the right rows based on the input array:

a = array([[5],
           [0],
           [4],
           ..., 
           [5],
           [6],
           [8]], dtype=int8)

new_y = np.eye(a.max()+1)[a.ravel()]

An even faster solution would be to create the output array with zeros and then populate it according to the indices from a:

new_y = np.zeros((a.shape[0], a.max()+1))
new_y[np.indices(a.ravel().shape)[0], a.ravel()] = 1.

edited Nov 7, 2013 at 9:02

answered Nov 7, 2013 at 8:24

Saullo G. P. Castro

59.4k28 gold badges191 silver badges244 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Bruce Over a year ago

True, I didn't even read the code ;-) Your answer presents a better solution for his present case but I keep mine as a more generic answer.

Bruce · Accepted Answer · 2013-11-07 08:04:44Z

3

You can use the vectorizedecorator

@np.vectorize
def transform_y(y):
    new_y = np.zeros(10)
    new_y[y] = 1
    return new_y

see http://telliott99.blogspot.ch/2010/03/vectorize-in-numpy.html

answered Nov 7, 2013 at 8:04

Bruce

7,1321 gold badge27 silver badges42 bronze badges

Collectives™ on Stack Overflow

Vectorize this function in Numpy Python

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related