python deeplearning softmax with numpy

Question

I implemented softmax with numpy. As you can see in the code, we have a matrix and we want to get the softmax for the row. For example, the soft max for row 1 is calculated by dividing np.sum(np.exp([1,3,6,-3,1]) by 1,3,5,-3,1 The soft max for line 2 is to find the soft max for np.sum(np.exp([5,2,1,4,3]). How do I do this?

def softmax(x):
    return np.exp(x)/np.sum(np.exp(x),axis=1)
x = np.array([[1,3,6,-3,1],
              [5,2,1,4,3]])


print(softmax(x))
print(f"1:{softmax(x)[0]} sum : {np.sum(softmax(x)[0])}")
print(f"2:{softmax(x)[1]} sum : {np.sum(softmax(x)[1])}")

ValueError                                Traceback (most recent call last)
<ipython-input-261-eb8c9feae03f> in <module>
      5 
      6 
----> 7 print(softmax(x))
      8 print(f"1:{softmax(x)[0]} sum : {np.sum(softmax(x)[0])}")
      9 print(f"2:{softmax(x)[1]} sum : {np.sum(softmax(x)[1])}")

<ipython-input-261-eb8c9feae03f> in softmax(x)
      1 def softmax(x):
----> 2     return np.exp(x)/np.sum(np.exp(x),axis=1)
      3 x = np.array([[1,3,6,-3,1],
      4               [5,2,1,4,3]])
      5 

ValueError: operands could not be broadcast together with shapes (2,5) (2,) 
>

FYI: SciPy has the softmax function since version 1.2.0: scipy.special.softmax — Warren Weckesser
– Warren Weckesser, Commented Dec 26, 2020 at 12:37

Ashwin Rajesh · Accepted Answer · 2020-12-26 12:49:30Z

3

The problem here is that sum(exp(x), axis=1) returns a 1-D numpy array. Change it to sum(esp(x), axis=1, keepdims=True) to avoid numpy from automatically dropping one dimension.

def softmax(x):
    return np.exp(x)/np.sum(np.exp(x),axis=1, keepdims=True)

x = np.array([[1,3,6,-3,1],
              [5,2,1,4,3]])


print(softmax(x))
print(f"1:{softmax(x)[0]} sum : {np.sum(softmax(x)[0])}")
print(f"2:{softmax(x)[1]} sum : {np.sum(softmax(x)[1])}")

answered Dec 26, 2020 at 12:49

Ashwin Rajesh

562 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ismail Durmaz · Accepted Answer · 2020-12-26 12:38:58Z

0

You should reshape np.sum result.

np.exp(x) / np.sum(np.exp(x), axis=1).reshape(2,1)

instead of

np.exp(x) / np.sum(np.exp(x), axis=1)

More generic way, you can use below statement.

return np.exp(x) / np.sum(np.exp(x),axis=1).reshape(x.shape[0], -1)

answered Dec 26, 2020 at 12:38

Ismail Durmaz

2,6411 gold badge8 silver badges19 bronze badges

Collectives™ on Stack Overflow

python deeplearning softmax with numpy

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related