Why calling Python script from setup.py invokes Python shell instead?

Question

I try to call a Python script from setup.py during installation, using a custom install:

class CustomInstall(install):
    def run(self):
        install.run(self)

        ...

        p = subprocess.Popen(
            [sys.executable, 'demo_package/deploy_database.py'],
            shell=True,
            stdout=subprocess.PIPE,
            cwd=os.path.join(self.install_lib, 'demo_package'))

        out, err = p.communicate()

setup(..., cmdclass=dict(install=CustomInstall))

When deploying the package on a Ubuntu machine, the process, instead of executing deploy_database.py, shows nothing. When I stop it manually with Ctrl+C, the output seems to indicate that instead of actually running deploy_database.py, it simply starts Python:

^CDownloading/unpacking PypiPackagesMonitoring
  Downloading demo-1.0.64.zip  
  Running setup.py egg_info for package demo

Installing collected packages: demo
  Running setup.py install for demo

    Python 3.3.2+ (default, Oct  9 2013, 14:50:09)
    [GCC 4.8.1 on linux
    Type "help", "copyright", "credits" or "license" for more information.
Cleaning up...
Operation cancelled by the user
Storing complete log in /home/.../.pip/pip.log

What is wrong in the way I call a Python script? How should I do it instead?

shell=True ought to imply that you need a feature that cannot be expressed as a simple sequence of strings, i.e. when using shell=True you must use a single string that represents the whole shell command*(which is *not simply calling an external program, but includes some shell features such as pipes/redirection/calling a built-in etc.). Otherwise you should use shell=False and you can represent the program to execute and its arguments in the same format of sys.argv. Mixing the two methods results in this strange results. — Bakuriu
– Bakuriu, Commented Oct 30, 2013 at 20:29

Jonas Schäfer · Accepted Answer · 2013-10-30 19:20:05Z

1

shell=True is not needed with [sys.executable, 'demo_package/deploy_database.py'], and causes problems. The way you're doing it (if you omit shell=True) is the preferred way, because it bypasses the shell.

shell=True makes Popen hand over the passed command to a shell (e.g. /bin/bash, depending on the shell configured for the current user). Only the first command passed in the list is executed, the other elements of the list are passed as arguments to the shell itself (from the docs):

On Unix with shell=True […] If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself.

shell=True is a bit dangerous, but also useful in some cases. See the documentation for details.

edited Oct 30, 2013 at 19:20

answered Oct 30, 2013 at 19:13

Jonas Schäfer

20.9k5 gold badges59 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Why calling Python script from setup.py invokes Python shell instead?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related