What Does the purpose of “train/eval” methods in nn.Modules?

Hi folks!.
I have one doubt about model train/eval methodology you have just used in the examples. Correct me if I am wrong:

  • Shouldn’t we use model.train() or model.eval() to indicates in which stage we stayed? Should we?
  • What Does the purpose of “train/eval” methods in nn.Modules?

If I were ok, we need these methods or the “torch.no_grad()” instruction in some places in the method. “train”.

I meant if we execute all instructions but optimizer.step(), the result will be the same, Is that ok?

(although internally I don’t know what has been happened.)

What I know from this discussion is:

  • model.eval() indicates to some layers that the model is in “evaluation stage”, so them won’t be executed.
  • with torch.no_grad() indicates that model doesn’t have to calculate the gradients so we optimize performance in evaluation/testing.

We will use model.train and model.eval in future lectures, we don’t need them yet. They turn on/off certain regularization features like batch normalization, dropout etc.

Also, torch.no_grad is automatically taken care of by optimizer.step() (as you can see here in the source code - it has a decorator @torch.no_grad() )

Courtesy- @aakashns