I am doing some research in the PyTorch source code. In this file the authors actually delete the modules.
from .adadelta import Adadelta # noqa: F401
from .adagrad import Adagrad # noqa: F401
from .adam import Adam # noqa: F401
del adadelta
del adagrad
del adam
What is the rationale for this?
It's a way to factor out functionality into submodules without making the submodules part of the public API. A quirk of the import system is that this sort of relative import ends up putting the submodules in the parent's namespace, so a user could do something like:
import torch.optim
then follow up by accessing attributes of torch.optim.adadelta without ever having explicitly imported torch.optim.adadelta. While some Python built-in packages work this way, and it won't be undone because too many people depend on it by accident (e.g. people who only import os, then use os.path APIs), in general it's best to avoid this sort of data leakage. Importing torch.optim should only provide access to the specific names listed, without providing any guarantees about which submodules will be imported automatically.
By doing this, people can't accidentally take a dependency on being able to use torch.optim.adadelta after importing only torch.optim, so the developers are free to refactor it to move implementations of specific classes and other APIs around, without making special efforts to ensure import torch.optim also imports all those submodules just to preserve behaviors from code that performed improper/incomplete imports.
