We are using Python scripts to process satellite images using different toolboxes or libraries (like GDAL etc.). We now have run into the problem that some 3rd-party Python applications we use (where we have no influence) require different versions of these python packages (for example, the Sentinel2 SEN2COR atmospheric correction requires GDAL 1.x, where we also want to use GDAL 2.x to be able to modify JPEG2000 files).
What's the best way to get this thing set up? I would prefer a way where I can have multiple installations of the same python version (e.g. 2.7, but that does not REALLY matter), and install packages and versions separately for each of them. Like that I could make sure that the SEN2COR script runs in his own python installation, where I install the packages needed and never touch it again, and work in another python installation with my other scripts.
I think a thing like virtualenv would be perfect for that, but there is one important point: All our scripts are command line scripts and are started from a variety of sources, like MATLAB scripts, R scripts or even cron jobs sometimes. Is there a way in virtualenv where I can do something like /usr/bin/python-version-only-for-sen2cor process_data.py arguments and /location/of/other/python-version reload_table.py from the shell or from inside other programs? Whats the best way for our setup? I probably can just install python multiple times and always modify the environment vars to use a different python version while installing the packages, but I guess this is prone to errors. Any suggestions?
/home/me/.virtualenvs/sen2cor/bin/python), than that Python executable will automatically prioritize packages installed in/home/me/.virtualenvs/sen2cor/lib/pythonx.y/site-packages. There is no need to set a PYTHONPATH, just point to the correct executable. This is often how web projects that use Python are run through e.g. Apache (or at least, that's my main experience for this).