1

I am trying to follow the course here http://cs231n.github.io/optimization-1/ , in the section Computing the gradient numerically with finite differences, they have provided a code snippet that should computer the gradient given a function and an array. I tried to run this using my own function and numpy array as an input and I get the following error:

ValueError                                Traceback (most recent call last)
<ipython-input-18-31c1f1d6169c> in <module>()
      2     return a
      3 
----> 4 eval_numerical_gradient(f,np.array([1,2,3,4,5]))

<ipython-input-12-d6bea4220895> in eval_numerical_gradient(f, x)
     28     print(x[ix])
     29     # compute the partial derivative
---> 30     grad[ix] = (fxh - fx) / h # the slope
     31     it.iternext() # step to next dimension
     32 

ValueError: setting an array element with a sequence.

I understand the error is because it cannot assign grad[ix] a sequence, I also tried with a column array and got the same error.

Here is the code:

def eval_numerical_gradient(f, x):
  """ 
  a naive implementation of numerical gradient of f at x 
  - f should be a function that takes a single argument
  - x is the point (numpy array) to evaluate the gradient at
  """ 

  fx = f(x) # evaluate function value at original point
  print(x)
  print(fx)
  grad = np.zeros(x.shape)
  h = 0.00001

  # iterate over all indexes in x
  it = np.nditer(x, flags=['multi_index'], op_flags=['readwrite'])
  while not it.finished:
    print(it)
    # evaluate function at x+h
    ix = it.multi_index
    print(ix)
    old_value = x[ix]
    print(old_value)
    x[ix] = old_value + h # increment by h
    print(x)
    fxh = f(x) # evalute f(x + h)
    print(fxh)
    x[ix] = old_value # restore to previous value (very important!)
    print(x[ix])
    # compute the partial derivative
    grad[ix] = (fxh - fx) / h # the slope
    it.iternext() # step to next dimension

  return grad

My question is: is my input of a numpy array (row and column) wrong? Can somebody explain why this is happening?

Sample input :

def f(a):
    return a

eval_numerical_gradient(f,np.array([[1],[2],[3]]))

and

def f(a):
    return a

eval_numerical_gradient(f,np.array([1,2,3]))

1 Answer 1

2

I suggest the following fix for eval_numerical_gradient(f, x):

  • Line #25: replace fxh = f(x) with fxh = f(x[ix])
  • Line #30: replace grad[ix] = (fxh - fx) / h with grad[ix] = (fxh - fx[ix]) / h

And make your input matrix with float number entries, e.g.,

eval_numerical_gradient(f,np.array([[1],[2],[3]], dtype=np.float))
Sign up to request clarification or add additional context in comments.

2 Comments

Could you explain these suggestions please? I used your suggestions, and I get output grad = 1,1,1 for your input, also if I change the input to 1,2,4 I still get same output, is that correct?
Surely correct because your input function is just f(x)=x and its derivative is just 1. Try to change it to f(x)=x^2 or others.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.