3

I do have a bash script that needs to install some Python packages on the system instead of an virtual environment which may or may not be activated when the script is executed.

This script is called by people that may already have a python virtual environment activated and I do want to be sure that for few commands I do not use it.

I tried to use the deactivate command but it seems that is not available, even if bash detects the virtual environment (presence of the VIRTUAL_ENV variable).

As a side note, I don't want to permanently disable the virtual environment. I just want to run few commands outside it. How can I do this?

5
  • You should still be able to access the system Python by explicitly using /usr/bin/python. Commented May 13, 2016 at 18:10
  • @jonrsharpe That's not really a good solution because the system Python may not be at this location. For example on Mac OS X, it is a different location. Commented May 13, 2016 at 18:12
  • system python on OS X is accessible as /usr/bin/python Commented May 13, 2016 at 18:18
  • You sure? I'm on OS X and that's where mine is! Won't work on Windows, though... Commented May 13, 2016 at 18:24
  • @sorin, ...I've extended my answer to have a section directly on-point for your use case. Please let me know if there's anything further that should be added. Commented May 14, 2016 at 20:33

2 Answers 2

6

If activating before starting the script

If you did the activate step in a parent shell, not in the shell instance running the script itself, then non-exported variables and functions are unavailable during its runtime.

To be entirely clear about definitions:

source my-virtualenv/bin/activate # this runs in the parent shell

./my-shell-script # the shell script itself is run in a child process created by
                  # fork()+execve(); does not inherit shell variables / functions, so
                  # deactivate WILL NOT WORK here.

(source my-shell-script) # creates a subshell with fork(), then directly invokes
                         # my-shell-script inside that subshell; this DOES inherit shell
                         # variables / functions, and deactivate WILL WORK here.

You have three options:

  • Export the deactivate function and its dependencies from the parent shell, before starting the script.

    This is as given below, and looks something like:

    source my-virtualenv/bin/activate
    export VIRTUAL_ENV ${!_OLD_VIRTUAL_@}
    export -f deactivate
    ./my-script-that-needs-to-be-able-to-deactivate
    

    You could optionally define an activation function that does this for you, like so:

    # put this in your .bashrc
    activate() {
      source "$1"/bin/activate && {
        export VIRTUAL_ENV ${!_OLD_VIRTUAL_@}
        export -f deactivate
      }
    }
    
    # ...and then activate virtualenvs like so:
    activate my-virtualenv
    
  • Make some guesses, within the script, about what the prior Python environment looked like.

    This is less reliable, for obvious reasons; however, as virtualenv does not export the shell variables containing the original PYTHON_HOME, that information is simply unavailable to child-process shells; a guess is thus the best option available:

    best_guess_deactivate() {
      if [[ $VIRTUAL_ENV && $PATH =~ (^|:)"$VIRTUAL_ENV/bin"($|:) ]]; then
        PATH=${PATH%":$VIRTUAL_ENV/bin"}
        PATH=${PATH#"$VIRTUAL_ENV/bin:"}
        PATH=${PATH//":$VIRTUAL_ENV/bin:"/}
        unset PYTHONHOME VIRTUAL_ENV
      fi
    }
    

    ...used within a limited scope as:

    run_python_code_in_virtualenv_here
    (best_guess_deactivate; run_python_code_outside_virtualenv_here)
    run_python_code_in_virtualenv_here
    
  • Run the script in a forked child of the shell that first sourced activate with no intervening exec() call

    That is, instead of invoking your script as a regular subprocess, with:

    # New shell instance, does not inherit non-exported (aka regular shell) variables
    ./my-shell-script
    

    ...source it into a forked copy of the current shell, as

    # Forked copy of existing shell instance, does inherit variables
    (source ./my-shell-script)
    

    ...or, if you trust it to hand back control to your interactive shell after execution without messing up state too much (which I don't advise), simply:

    # Probably a bad idea
    source ./my-shell-script
    

    All of these approaches have some risk: Because they don't use an execve call, they don't honor any shebang line on the script, so if it's written specifically for ksh93, zsh, or another shell that differs from the one you're using interactively, they're likely to misbehave.

If activating within the script

The most likely scenario is that the shell where you're running deactivate isn't a direct fork()ed child (with no intervening exec-family call) of the one where activate was sourced, and thus has inherited neither functions or (non-exported) shell variables created by that script.

One means to avoid this is to export the deactivate function in the shell that sources the activate script, like so:

printf 'Pre-existing interpreter: '; type python

. venv-dir/bin/activate

printf 'Virtualenv interpreter: '; type python

# deactivate can be run in a subshell without issue, scoped to same
printf 'Deactivated-in-subshell interpreter: '
( deactivate && type python ) # this succeeds

# however, it CANNOT be run in a child shell not forked from the parent...
printf 'Deactivated-in-child-shell (w/o export): '
bash -c 'deactivate && type python' # this fails

# ...unless the function is exported with the variables it depends on!
export -f deactivate
export _OLD_VIRTUAL_PATH _OLD_VIRTUAL_PYTHONHOME _OLD_VIRTUAL_PS1 VIRTUAL_ENV

# ...after which it then succeeds in the child.
printf 'Deactivated-in-child-shell (w/ export): '
bash -c 'deactivate && type python'

My output from the above follows:

Pre-existing interpreter: python is /usr/bin/python
Virtualenv interpreter: python is /Users/chaduffy/test.venv/bin/python
Deactivated-in-subshell interpreter: python is /usr/bin/python
Deactivated-in-child-shell (w/o export): bash: deactivate: command not found
Deactivated-in-child-shell (w/ export): python is /usr/bin/python

Assuming you've fixed that, let's run once more through using a subshell to scope deactivation to make it temporary:

. venv-dir/activate

this-runs-in-venv

# minor performance optimization: exec the last item in the subshell to balance out
# ...the performance cost of creating that subshell in the first place.
(deactivate; exec this-runs-without-venv)

this-runs-in-venv
Sign up to request clarification or add additional context in comments.

5 Comments

Try it, you will get an error that deactivate command does not exists.
Not sure why, in my case the virtual environment is activate before I run the script and inside the bash script deactivate is not available.
@sorin, ...my guess is that you're running this in a child process that wasn't forked directly from the shell evaluating activate, and thus which didn't get local function definitions.
@sorin, ...I've expanded my answer to describe that scenario in detail, and a direct workaround.
@sorin, I just reread your prior comment: Activating before the script starts is clearly the problem here -- it creates the scenario where the functions and variables aren't present inside the script. Activate within the script instead, or run the necessary exports when doing your prior activation (from that parent process, since if the content isn't exported, there's no way for the script itself to see it).
0

You could always reference the global Python directly:

/usr/bin/python2.7 -E my_python_command

If you're concerned about such a path being unreliable, you could either:

  • Configure Python in a unique, safe, static location upon installation, and reference that
  • Invoke a subshell that is not inside the virtual environment
  • Use an alternate name for your virtualenv Python executable so that it doesn't conflict on the path. ie. the virtualenv Python would be python-virtual, and python would still lead to the global installation.

Then it would be something like:

python -E my_command # global  
python-virtual my_command # virtual

1 Comment

Explicitly calling /usr/bin/python doesn't undo the PYTHONHOME changes made by virtualenv. Consider making it /usr/bin/python -E my_command to ignore both PYTHONHOME and PYTHONPATH.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.