Gradient Descent in python implementation issue

Question

Hey I am trying to understand this algorithm for a linear hypothesis. I can't figure out if my implementation is correct or not. I think it is not correct but I can't figure out what am I missing.

theta0 = 1
theta1 = 1
alpha = 0.01
for i in range(0,le*10): 
    for j in range(0,le):
        temp0 = theta0 - alpha * (theta1 * x[j] + theta0 - y[j])
        temp1 = theta1 - alpha * (theta1 * x[j] + theta0 - y[j]) * x[j]
        theta0 = temp0 
        theta1 = temp1

print ("Values of slope and y intercept derived using gradient descent ",theta1, theta0)

It is giving me the correct answer to the 4th degree of precision. but when I compare it to other programs on the net I am getting confused by it.

Thanks in advance!

typically you iterate until e certain error measure is smaller than a prescribed maximal error. I don't see it here. — Quickbeam2k1
– Quickbeam2k1, Commented Aug 19, 2017 at 10:32
I'm aware of that but I was trying to implement it quickly. I assumed running the loop for over 10 times or 100 times would do it. — slayer
– slayer, Commented Aug 19, 2017 at 18:42
the progress you obtain also depends on alpha. There might be alpha values and problems where you could achieve convergence within the first 10 or 100 steps. But in general this might be difficult to achieve — Quickbeam2k1
– Quickbeam2k1, Commented Aug 19, 2017 at 18:58

mforpe · Accepted Answer · 2017-08-19 12:21:50Z

1

Implementation of the Gradient Descent algorithm:

import numpy as np

cur_x = 1 # Initial value
gamma = 1e-2 # step size multiplier
precision = 1e-10
prev_step_size = cur_x

# test function
def foo_func(x):
    y = (np.sin(x) + x**2)**2
    return y

# Iteration loop until a certain error measure
# is smaller than a maximal error
while (prev_step_size > precision):
    prev_x = cur_x
    cur_x += -gamma * foo_func(prev_x)
    prev_step_size = abs(cur_x - prev_x)

print("The local minimum occurs at %f" % cur_x)

answered Aug 19, 2017 at 12:21

mforpe

1,5991 gold badge13 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

slayer Over a year ago

Thanks I'll run it and get back to you.

Collectives™ on Stack Overflow

Gradient Descent in python implementation issue

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related