Errors while dynamic imports using importlib in python3

Question

I have been trying to use importlib with python3 (3.6).

Directory structure

main.py

#Note: I will only modify line 4 that uses importlib
import importlib
if __name__ == '__main__':
    print("In main.py")
    hello = importlib.import_module('hello', package='./')
    print("Loaded hello.py")
    hello.hello()

hello.py

def hello():
    print('Hello world')

folder/hello.py

def hello():
    print('Hello world in folder')

Observations

If I do

hello = importlib.import_module('hello', package='./') or

hello = importlib.import_module('hello')

It imports hello.py from the root folder and prints hello world.

If I do

hello = importlib.import_module('folder.hello')

It imports folder/hello.py from the root folder and prints hello world in folder.

But if I do

hello = importlib.import_module('hello', package='folder') or

hello = importlib.import_module('hello', package='./folder')

It gives error

Traceback (most recent call last):
  File "main.py", line 4, in <module>
    hello = importlib.import_module('hello', package='./folder')
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'hello'

Problem

I am not sure what is going on here. I am pretty sure there is something wrong with my understanding of python modules and packages. Can someone explain why this is the expected behavior?

Mahesh · Accepted Answer · 2018-05-12 05:15:41Z

10

+50

If the first argument, the module to be imported is an absolute module reference ( has no leading .), the seond argument is completely ignored.

To import a module a relative to another module b, you have to use

a = importlib.import_module('.a', package='b')

In your case, this should work

hello = importlib.import_module('.hello', package='folder')

As a rule of thumb, import package should work if you want to use package as second argument.

from package import module

then becomes

importlib.import_module(module, package)

answered May 12, 2018 at 5:15

Mahesh

3704 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Umang Gupta Over a year ago

importlib is beyond what I can understand I guess. I upvoted your answer but now importlib.import_module('math.sin') or importlib.import_module('.sin', package='math') doesn't work.

Tarun Lalwani Over a year ago

@UmangGupta, that is because sin is a function in math module and and it is not a module itself. You need to do math . = importlib.import_module('math') and then math.sin will work. The name is import_module and not import_function

Umang Gupta Over a year ago

@TarunLalwani I think that makes sense. Thanks for pointing out. Just FYI, this works importlib.import_module('random', package='numpy') So in the actual question why would hello = importlib.import_module('hello', package='folder') this not work? why do I need .hello

Tarun Lalwani Over a year ago

Does hello = importlib.import_module('hello', package='folder') give you an error or does it just loads the base module?

Umang Gupta Over a year ago

@TarunLalwani If I remove the hello.py in . folder, it gives No module named 'hello' error

|

Tarun Lalwani · Accepted Answer · 2018-05-15 06:45:41Z

4

@Mahesh's answer is 100% correct and spot-on, but I guess we need to go one level deep to make you understand it better

Below is the code for import_module

def import_module(name, package=None):
    """Import a module.

    The 'package' argument is required when performing a relative import. It
    specifies the package to use as the anchor point from which to resolve the
    relative import to an absolute import.

    """
    level = 0
    if name.startswith('.'):
        if not package:
            msg = ("the 'package' argument is required to perform a relative "
                   "import for {!r}")
            raise TypeError(msg.format(name))
        for character in name:
            if character != '.':
                break
            level += 1
    return _bootstrap._gcd_import(name[level:], package, level)

You can see if the name doesn't start with . then if part doesn't get executed. You just have return _bootstrap._gcd_import(name[level:], package, level) which gets executed with level=0 as the value

Now let's get into that function, which has below code

def _gcd_import(name, package=None, level=0):
    """Import and return the module based on its name, the package the call is
    being made from, and the level adjustment.

    This function represents the greatest common denominator of functionality
    between import_module and __import__. This includes setting __package__ if
    the loader did not.

    """
    _sanity_check(name, package, level)
    if level > 0:
        name = _resolve_name(name, package, level)
    return _find_and_load(name, _gcd_import)

Again in this it just executes _find_and_load(name, _gcd_import), now because level is 0 from our previous code, the package parameter is not being passed or used at all by _find_and_load method. Now you can easily verify this by running below

import importlib
hello = importlib.import_module('hello', package='IAmNotAfolder')
hello.hello()

And it will print Hello World from the base hello.py

So as you can see the package parameter is not used at all when the name doesn't start with ., which is for relative imports. That is why you get an error No module named 'hello' because it is trying to import hello.py from base folder irrespective of what you have in package.

Hope this answer make it easier for you to understand what is happening behind the scenes

answered May 15, 2018 at 6:45

Tarun Lalwani

147k11 gold badges217 silver badges278 bronze badges

5 Comments

Umang Gupta Over a year ago

1. Yeah I just realized importlib.import_module('random', package='numpy') imported the random module from python and not numpy.random. I feel dumb about this now. 2. I guess find_and_load does not search in system.path but only loaded packages right? 3. Thanks for putting the code out. I agree with Mahesh's observation, I got confused with numpy and sys examples :/

Tarun Lalwani Over a year ago

@UmangGupta, it is loading the package if it is not loaded and returning you the object. But it doesn't add it to your local/global scope. So if you don't take the return value into the variable you will have to call it using something like sys.modules['hello'].hello()

Umang Gupta Over a year ago

I am not sure about that. If I do linalg=importlib.import_module('scipy.linalg') It works, but linalg = importlib.import_module('.linalg', package='scipy') this won't. This import scipy; linalg = importlib.import_module('.linalg', package='scipy') works.

Umang Gupta Over a year ago

Not sure what you meant by contradiction :-|

Umang Gupta Over a year ago

I think it is the version issue.. works in 3.6 but not in 3.4. Anyways thanks very much for all the help.

Collectives™ on Stack Overflow

Errors while dynamic imports using importlib in python3

2 Answers 2

8 Comments

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

8 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related