Modify Python/PIP to automatically install modules when failed to import

Question

Is there a way to modify python/pip to, whenever an import fails at runtime, it would try to install the module (by the same name) from pip and then import the module?

I'd say it would be a nicer default than to just throw an error. If after loading the module from pip any problems happen, then it would also throw an error, similar to when I notice I cannot import something, try pip install and then come to the same exact error message.

I know we can use requirements.txt for bundling a package, but I'm talking from a "client" (person running the script) rather than "provider" (person providing the script) perspective; that is, I, as a client, would like to be able to import any script and have dependencies be solved automatically.

I understand that this could potentially cause trouble, but whenever I see an ImportError I'd just try to pip install the module anyway. Only if the module wouldn't work after pip installation "would I ask further questions".

I thought of something like this snippet that would be "built in" to the python process:

def getDirectoryOfInterpreter(p):
    return "someway to return the directory"

try:
    import somemodule
except ImportError:
    os.system(getDirectoryOfInterpreter('THIS_INTERPRETER_BINARY') + ' pip install ' + "MODULE_NAME")
    import somemodule

Yes. That's what's done in Python Packages, when they are installed, they check the dependences and install rest of packages. — user4396006
– user4396006, Commented Mar 31, 2015 at 8:55
I'm talking about solving the dependencies at runtime of a script. — PascalVKooten
– PascalVKooten, Commented Mar 31, 2015 at 8:56
I may have a way to help you, but the dependency should be installed before — user4396006
– user4396006, Commented Mar 31, 2015 at 8:56

Community · Accepted Answer · 2020-06-20 09:12:55Z

15

+25

You can do this with pipimport, when using virtualenv. It probably works with the system python if you have appropriate privileges to write the necessary directories (at least site-packages, but your import might have some command that pip will try to put somewhere in the PATH). Since it is good practice to always use virtualenvs for you own development anyway, I never tried using pipimport with the system python.

You need to import pipimport into your virtualenv by hand:

virtualenv venv
source venv/bin/activate
pip install pipimport

Then create a file autopipimport.py that you import first in any module:

# import and install pipimport
import pipimport
pipimport.install()

Now in any other .py file you can do:

import autopipimport
import some.package.installable.by.pip

I once tried (auto of curiosity) to add the two pipimport related lines to venv/lib/python2.7/site.py to be automatically "loaded", but that was to early in the chain and did not work.

pipimport will only try to use pip to install a particular module once, storing information about what has been tried in a .pipimport-ignore file (should be under venv unless that is not writeable).

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Apr 6, 2015 at 5:53

Anthon

78.3k35 gold badges207 silver badges290 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Martijn Pieters Over a year ago

Note that this requires the package name to match the project name on PyPI; from bs4 import BeautifulSoup won't find the beautifulsoup4 project it is registered with.

Anthon Over a year ago

Although true in general, based on the constraint put in the question by the OP that match is a given.

PascalVKooten Over a year ago

@MartijnPieters I was also having exactly that one in mind, though it is rather an exception. Can you think of another? Either way, the goal is not to be able to count on not having any errors, but you'd just reduce loads of them.

PascalVKooten Over a year ago

@Anthon Perhaps there is even a way to somehow first run this script in any python session, without having to add this to every single script?

Anthon Over a year ago

@PascalvKooten As I indicated I tried to put it in site.py, and for that reason, but python got into some infinite import loop. I did not investigate further, maybe you can change the code. The idea is nice, but "explicit is better than implicit" and I have my install dependencies now in my setup.py that I need for uploading to PyPI anyway.

|

RayLuo · Accepted Answer · 2017-01-24 19:33:52Z

Here is something I have done for my own curiosity.

Yes this unsafely calls a shell in the background with arbitrary user input using a non-preferred function. Please don't use this code for production.

import os

Libraries_Installed = False

def Pip_Install(library):
    """Locate pip.exe on system and install library.
    Be sure to sanitize input to prevent RCE."""
    global Lib_Installed

    if os.path.isfile('C:\\Python27\\Scripts\\pip.exe'):
        print '[!] c:\Python27\Scripts\pip.exe => pip'
        stream = os.popen('C:\Python27\Scripts\pip.exe install %s' % library)
        Result = stream.read()

        if 'Successfully installed' in str(Result):
            print '\n\n[+] Successfully installed %s' % library
            Lib_Installed = True

        elif 'FAIL' in str(Result).upper() or 'FAILED' in str(Result).upper() \
        or 'FAILURE' in str(Result).upper():
            print '[!] Unable to install %d' % library
            print '[!] Please manually run the pip installer'
            print '[!] try: C:\Python27\Scripts\pip.exe install %s' % library

    elif os.path.isfile('D:\\Python27\\Scripts\\pip.exe'):
        stream = os.popen('D:\Python27\Scripts\pip.exe install %s' % library)
        Result = stream.read()

        if 'Successfully installed' in str(Result):
            print '\n\n[+] Successfully installed %s' % library
            Lib_Installed = True

        elif 'FAIL' in str(Result).upper() or 'FAILED' in str(Result).upper() \
        or 'FAILURE' in str(Result).upper():
            print '[!] Unable to install %d' % library
            print '[!] Please manually run the pip installer'
            print '[!] try: C:\Python27\Scripts\pip.exe install %s' % library

# Python library that may need to be installed
try:
    import Tkinter
    import tkFileDialog
except:
    print '[!] Error importing Tkinter ... Trying pip installer'
    Pip_Install('tkinter')

if Lib_Installed == True:
    print '\n[!] The following Python libraries were installed automatically:'
    for library in Libs:
        print str(library)
    Q = raw_input('''\n[!] Please exit the program and run a new instance.
[!] Enter [Y] to continue anyways''')
    if Q.upper()[0] == 'Y':
        pass
    else:
        exit()

Yes this unsafely calls a shell in the background with arbitrary user input using a non-preferred function. Please don't use this code for production.

Rafał Białas · Accepted Answer · 2025-05-08 06:33:13Z

0

import subprocess; # this must be installed

try:
    import entrypoints
except ModuleNotFoundError:
    proc = subprocess.run('pip install entrypoints', shell=True, capture_output=True, text=True)
    if proc.returncode == 0:
        print("%s %s, %s" % (str(datetime.datetime.now())+' ] ...READY ]', str(proc.returncode), "pip install 'entrypoints' COMPLETED"))

answered May 8 at 6:33

Rafał Białas

31

Collectives™ on Stack Overflow

Modify Python/PIP to automatically install modules when failed to import

3 Answers 3

6 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related