Gradient descent algorithm giving incorrect answer in matlab

Question

I'm taking a course in machine learning and trying to implement the gradient descent algorithm in matlab. The function computeCost works fine, as I have tested it separately. I use it to see the cost at every iteration, and it doesn't seem to be decreasing at all. It just fluctuates randomly. The value of alpha was given to be 0.01, so I know it's not an issue with the learning rate being too big. The answers I get for theta are very off from the expected output. Where am I going wrong? Thanks in advance!

function theta = gradientDescent(X, y, theta, alpha, num_iters)
%GRADIENTDESCENT Performs gradient descent to learn theta

% Initialize some useful values
m = length(y); % number of training examples

temp1=0;
temp2=0;
for iter = 1:num_iters
for k = 1:m
    temp1 = temp1 + (theta(1) + theta(2)*X(k, 2) - y(k));
    temp2 = temp2 + ((theta(1) + theta(2)*X(k, 2) - y(k))*X(k, 2));

end 

theta(1) = theta(1)-(1/m)*alpha*temp1;
theta(2) = theta(2)-(1/m)*alpha*temp2;



computeCost(X, y, theta)

end

end

Edit: here is computeCost as well

function J = computeCost(X, y, theta)
m = length(y); % number of training examples


J = 0;
temp = 0;
for index = 1:m
    temp = temp + (theta(1) + theta(2)*X(index, 2)-y(index))^2;

end 
J = temp/(2*m); 
end

It would be useful to see computeCost - also is it right that your gradient doesn't depend on X(k,1) at all? — Peter de Rivaz
– Peter de Rivaz, Commented Aug 2, 2017 at 19:55
X is a matrix with ones in the first column, as a coefficient to theta1. I don't think the gradient depends on it. I have added computeCost as well. — may
– may, Commented Aug 2, 2017 at 20:01

Peter de Rivaz · Accepted Answer · 2017-08-02 20:05:53Z

3

Try changing:

temp1=0;
temp2=0;
for iter = 1:num_iters

to

for iter = 1:num_iters
  temp1=0;
  temp2=0;

The gradient needs to be computed fresh for each iteration (or you are effectively building in a momentum term).

answered Aug 2, 2017 at 20:05

Peter de Rivaz

33.6k4 gold badges51 silver badges80 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Gradient descent algorithm giving incorrect answer in matlab

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related