How to manage multiple Python versions in Windows 10 to use with Python tox?

Question

I'm trying to use tox to automate pytest testing of my project on Python 3.7 and 3.8 but am struggling with how to best set this up. What's the easiest way to get multiple Python versions installed on my Windows 10 machine in order to use them with tox?

If I just manually install them using the official installers how do I set up my environment variables? Because each installation contains python.exe, so if I install two versions and add both their paths C:\Program Files\Python37\ and C:\Program Files\Python38\ to my Path environment variable this won't differentiate them... python will just refer to the first one listed. So do I need to go and manually rename the python.exe files to names like python37.exe and python38.exe? This all seems very manual and clunky but I can't find any easier way or tutorial about this step.

The tox documentation just gives a solution using conda, but I'm not using conda and don't want to switch to it just to use tox.

If Python version paths and aliases are set up manually, then if someone clones my project to work on a PR and wants to run tests locally, tox would not work correctly if their Python install locations are different or they're on a different OS. Is there some standard way to define and set this all up so things are easy and consistent across machines?

If all this is unavoidable, what is an alternative to tox for consistently testing projects across multiple Python versions that can work on any machine or CI/CD pipeline?

You can always call C:\Program Files\Python37\python directly. Each Python knows the path to its own libraries. — Tim Roberts
– Tim Roberts, Commented Apr 28, 2021 at 18:42
I would suggest specifically calling the application by supplying the full path. Alternatively - you could make a shortcut for each version. — MF-
– MF-, Commented Apr 28, 2021 at 18:44
Then if someone clones my project to work on a PR and wants to run tests locally first, tox would not work correctly if their Python install locations are different or they're on a different OS. I was hoping there was some standard way to define and set this all up so things are easy and consistent across machines. How do big open source projects handle this? Do contributors all just manually edit their environment variables or install locations to match tox.ini? But then this would break if they contribute to two projects with different tox.ini files — pyjamas
– pyjamas, Commented Apr 28, 2021 at 18:47
Can you use the Python launcher for Windows that was created for the express purpose of managing multiple Python installations? py -3.7 -mtox or whatever (assumes tox installs as runnable module)? — ShadowRanger
– ShadowRanger, Commented Apr 28, 2021 at 18:59

Jürgen Gmach · Accepted Answer · 2021-04-28 21:47:34Z

tox has quite some logic implemented to find installed Python interpreters.

While the documentation lacks some details (maybe you want to create an issue?), we can still have a look a the source code:

@tox.hookimpl
def tox_get_python_executable(envconfig):
    spec, path = base_discover(envconfig)
    if path is not None:
        return path
    # second check if the py.exe has it (only for non path specs)
    if spec.path is None:
        py_exe = locate_via_pep514(spec)
        if py_exe is not None:
            return py_exe

    # third check if the literal base python is on PATH
    candidates = [envconfig.basepython]
    # fourth check if the name is on PATH
    if spec.name is not None and spec.name != envconfig.basepython:
        candidates.append(spec.name)
    # or check known locations
    if spec.major is not None and spec.minor is not None:
        if spec.name == "python":
            # The standard names are in predictable places.
            candidates.append(r"c:\python{}{}\python.exe".format(spec.major, spec.minor))
    return check_with_path(candidates, spec)

As you can see, there are five ways to determine the available Python interpreters on a Windows system.

Especially the second one looks promising -it is using the already mentioned Python launcher for Windows, also see https://www.python.org/dev/peps/pep-0514/

As far as I understand it, you just need to install your Python interpreters, and they will be automatically discoverable.

tox is definitely a very good way to test Python applications against multiple interpreters, both locally and in CI.

P.S.: Yes, it works!!

I just rdp'ed into a Windows box, installed Python 3.8 and Python 3.9 - just clicked through the default installer, and created the following tox.ini

[tox]
envlist = py38, py39

[testenv]
commands = python -c "print('hello')"
skip_install = true

Both interpreters were detected and both environments got executed.

(venv) C:\Users\jugmac00\Projects\stackoverflow>tox
py38 create: C:\Users\jugmac00\Projects\stackoverflow\.tox\py38
py38 run-test-pre: PYTHONHASHSEED='296'
py38 run-test: commands[0] | python -c 'print('"'"'hello'"'"')'
hello
py39 create: C:\Users\jugmac00\Projects\stackoverflow\.tox\py39
py39 run-test-pre: PYTHONHASHSEED='296'
py39 run-test: commands[0] | python -c 'print('"'"'hello'"'"')'
hello
_______________________________________________________ summary _______________________________________________________
  py38: commands succeeded
  py39: commands succeeded
  congratulations :)

(venv) C:\Users\jugmac00\Projects\stackoverflow>

Collectives™ on Stack Overflow

How to manage multiple Python versions in Windows 10 to use with Python tox?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related