Gradient Descent implementation in Python

Question

I got stucked at a point where I am implementing gradient descent in python.

The formula for gradient descent is:

for iter in range(1, num_iters):
    hypo_function = np.sum(np.dot(np.dot(theta.T, X)-y, X[:,iter]))

    theta_0 = theta[0] - alpha * (1.0 / m) * hypo_function
    theta_1 = theta[1] - alpha * (1.0 / m) * hypo_function

Got an error:

---> hypo_function = np.sum(np.dot(np.dot(theta.T, X)-y, X[:,iter])) ValueError: shapes (1,97) and (2,) not aligned: 97 (dim 1) != 2 (dim 0)

PS: Here my X is (2L, 97L), y is (97L,) theta is (2L,).

paisanco · Accepted Answer · 2015-09-26 01:56:33Z

1

np.dot(a,b) takes the inner product of a and b if a and b are vectors (1-D arrays) If a and b are 2D arrays, np.dot(a,b) does matrix multiplication.

It will throw ValueError if there is a mismatch between the size of the last dimension of a and the second to last dimension of b. They have to match.

In your case you are trying to multiply a something by 97 array by a 2 by something array in one of your dot products, so there is mismatch. So you need to fix your input data so the dot product/matrix multiply is computable.

edited Sep 26, 2015 at 1:56

answered Sep 26, 2015 at 1:49

paisanco

4,3286 gold badges32 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Vamsi Over a year ago

Hi Paisanco, could you help me in correcting the code?

paisanco Over a year ago

Well for instance the transpose of X would be 97 x 2, that could be multiplied by theta (2x1) successfully. You could use numpy.transpose to do the trick.

Vamsi Over a year ago

But according to gradient descent formula I cannot do a transpose of X but a transpose of theta can be done

paisanco Over a year ago

can you do np.dot(theta_transpose,X) that would be a 1x2 multiplied by a 2 x 97, yielding a 1x 97 result

Vamsi Over a year ago

The formula that I have to implement is: (θ_transpose*x−y)x. I do not know if we can do that with the dimensions. Now I guess this will give a better picture on what I should do in this

Collectives™ on Stack Overflow

Gradient Descent implementation in Python

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related