Where to parallelize? Constructing a bash script for parallelization over multiple nodes

Handling from within the R script

When running an R script with srun, the way to have all instances do something different (other than using MPI, which is non trivial) is to refer to the SLURM_PROC_ID environment variable.

Insert a line such as

idx = as.numeric(Sys.getenv('SLURM_PROC_ID'))

and have all combinations of parameters in a list. Then choose the combination from the list depending on idx.

Handling from the Bash submission script

You can also manage the distribution in the submission script with a construct like the following (with https://www.gnu.org/software/parallel/parallel_tutorial.html)

parallel srun --exclusive -n 1 -c1 Rscript myscript.R ::: {1..10}

to run myscript.R 10 times, with one argument ranging from 1 to 10 respectively. You then get the value of the argument in the R script with commandArgs()

This seems like use case for MPI, a standard for writing distributed-memory applications. It is also available for use with R.

However, if you have an existing script that can take arguments specifying a subset of the problem, which you can submit to your cluster multiple times, your suggested approach is probably more feasible than rewriting your script for use with MPI.

In this case, I would recommend using your script as is, but writing a bash script (as you suggested yourself) to handle distribution over the nodes. This bash script should simply submit (srun) multiple R script executions with different parameter combinations to the cluster. Depending on how much work you want to put in, you could write code for automatically finding appropriate srun parameters depending on the total number of available cores and number of script runs to execute. You could also just manually figure out how many cores each execution should consume.

Using srun to launch an R-script causes all nodes and cores to execute the excat same code.

This, I do not fully understand. If you srun your R script with different parameter combinations, the different nodes will compute different parts of the problem. Of course, you will have to aggregate your results, either manually or automatically in your bash script.