Why there are different output between model.forward(input) and model(input)

Question

I'm using pytorch to build a simple model like VGG16,and I have overloaded the function forward in my model.

I found everyone tends to use model(input) to get the output rather than model.forward(input), and I am interested in the difference between them. I try to input the same data, but the result is different. I'm confused.

I have output the layer_weight before I input data, the weight not be changed, and I know when we using model(input) it using __call__ function, and this function will call model.forward.

   vgg = VGG()
   vgg.double()
   for layer in vgg.modules():
      if isinstance(layer,torch.nn.Linear):
         print(layer.weight)
   print("   use model.forward(input)     ")
   result = vgg.forward(array)

   for layer in vgg.modules():
     if isinstance(layer,torch.nn.Linear):
       print(layer.weight) 
   print("   use model(input)     ")
   result_2 = vgg(array)
   print(result)
   print(result_2)

output:

    Variable containing:1.00000e-02 *
    -0.2931  0.6716 -0.3497 -2.0217 -0.0764  1.2162  1.4983 -1.2881
    [torch.DoubleTensor of size 1x8]

    Variable containing:
    1.00000e-02 *
    0.5302  0.4494 -0.6866 -2.1657 -0.9504  1.0211  0.8308 -1.1665
    [torch.DoubleTensor of size 1x8]

Umang Gupta · Accepted Answer · 2019-03-25 17:54:38Z

23

model.forward just calls the forward operations as you mention but __call__ does a little extra.

If you dig into the code of nn.Module class you will see __call__ ultimately calls forward but internally handles the forward or backward hooks and manages some states that pytorch allows. When calling a simple model like just an MLP, it may not be really needed but more complex model like spectral normalization layers have hooks and therefore you should use model(.) signature as much as possible unless you explicitly just want to call model.forward

Also see Calling forward function without .forward()

In this case, however, the difference may be due to some dropout layer, you should call vgg.eval() to make sure all the stochasticity in network is turned off before comparing the outputs.

answered Mar 25, 2019 at 17:54

Umang Gupta

16.6k7 gold badges51 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Why there are different output between model.forward(input) and model(input)

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related