I have a trusted remote server that stores many custom Python modules. I can fetch them via HTTP (e.g. using urllib2.urlopen) as text/plain, but I cannot save the fetched module code to the local hard disk. How can I import the code as a fully operable Python module, including its global variables and imports?
I suppose I have to use some combination of exec and imp module's functions, but I've been unable to make it work yet.
-
are you downloading them over ssl or is every intermediate router trusted as well? :Paaronasterling– aaronasterling2010-09-26 19:58:53 +00:00Commented Sep 26, 2010 at 19:58
-
1Actually, yes - both the module storage and the front-end server are parts of a single system (and are even located in the same server room). For now, suppose there are no security implications: even if I download the code over SSL, the question of how to interpret it still stands.dpq– dpq2010-09-26 20:04:30 +00:00Commented Sep 26, 2010 at 20:04
Add a comment
|
4 Answers
It looks like this should do the trick: importing a dynamically generated module
>>> import imp
>>> foo = imp.new_module("foo")
>>> foo_code = """
... class Foo:
... pass
... """
>>> exec foo_code in foo.__dict__
>>> foo.Foo.__module__
'foo'
>>>
Also, as suggested in the ActiveState article, you might want to add your new module to sys.modules:
>>> import sys >>> sys.modules["foo"] = foo >>> from foo import Foo <class 'Foo' …> >>>
3 Comments
D. A.
Is there a trick so that I could allow the user to "from foo import Foo"? In this example, I get "ImportError: No module named foo".
nekketsuuu
@DavidWolever It seems that
imp.new_module is deprecated since Python 3.4 (ref). Use importlib.util.module_from_spec instead.Here's something I bookmarked a while back that covers something similar:
It's a bit beyond what you want, but the basic idea is there.
Comments
I recently encountered trying to do this while trying to write unit tests for source code examples I put into a project's readme (I wanted to avoid just linking to small files or duplicating the text in a way that could get out of sync).
I came up with the following
import sys
import types
from importlib import import_module
def compile_and_install_module(module_name: str, source_code: str) -> types.ModuleType:
"""Compile source code and install it as a module.
End result is that `import <module_name>` and `from <module_name> import ...` should work.
"""
module = types.ModuleType(module_name, "Module created from source code")
# Execute source in context of empty/fake module
exec(source_code, module.__dict__)
# Insert fake module into sys.modules. It's now a real module
sys.modules[module_name] = module
# Imports should work now
return import_module(module_name)
And a quick example of how you can use it
$ cat hello.py
def foo():
print("Hello world")
bar = 42
$ python
Python 3.9.5 (tags/v3.9.5:0a7dcbd, May 3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from compile import compile_and_install_module
>>> compile_and_install_module("hello", open("hello.py").read())
<module 'hello'>
>>> import hello
>>> hello.foo()
Hello world
>>> from hello import bar
>>> bar
42
You can remove the return value and import_lib import if you
1 Comment
winwin
The
types.ModuleType seems to be the new way of creating modules dynamically, since imp is deprecated now. +1 to you.Python3 version
(attempted to edit other answer but the edit que is full)
import imp
my_dynamic_module = imp.new_module("my_dynamic_module")
exec("""
class Foo:
pass
""", my_dynamic_module.__dict__)
Foo = my_dynamic_module.Foo
foo_object = Foo()
# register it on sys
import sys
sys.modules[my_dynamic_module.__name__] = my_dynamic_module
