Loading python modules through a computing cluster

You could simply call your python program from the bash script with something like: PYTHONPATH=$HOME/lib/python /path/to/my/python my_python_script

I don't know how SunGrid works, but if it uses a different user than yours, you'll need global read access to your $HOME. Or at least to the python libraries.

First, whether or not this solution works for you depends heavily on how the cluster is set up. That said, the general solution to your problem is below. If the compute cluster has access to the same files as you do in your home directory, I see no reason why this would not work.

You need to be using a virtualenv. Install your software inside that virtualenv along with any additional python packages you need. Then in your batch bash script, provide the full path to the python interpreter within that virtualenv.

Note: to install python packages inside your virtualenv, you need to use the pip instance that is in your virtualenv, not the system pip.

Example:

$ virtualenv foo
$ cd foo
$ ./bin/pip install numpy

Then in your bash script:

/path/to/foo/bin/python /path/to/your/script.py

Have you tried to add these in your python code:

import sys
sys.path.append("..")
from myOtherPackage import myPythonFile

This works very well for my code when I run it on Cluster and I wanted to call my "myPythonFile" from other package "myOtherPackage"