4

I have a python program that I run it via command line (Mac OSX) as:

python -W ignore Experiment.py --iterations 10

The file Experiment.py should be run multiple times using different --iterations values. I do that manually one after another, so when one run is finished, I run the second one with different --iterations, and so on. However, I cannot always set near to my laptop to run all of them so I am wondering if there is a way using shell script where I can state all runs together and then the shell script executes them one after another (Not parallel, just sequentially as I would have done it by my self)? Something like:

python -W ignore Experiment.py --iterations 10
python -W ignore Experiment.py --iterations 100
python -W ignore Experiment.py --iterations 1000
python -W ignore Experiment.py --iterations 10000
python -W ignore Experiment.py --iterations 100000

Edit: What if I have multiple arguments --X --Y --Z?

Improve this question

2 Answers 2

Reset to default
5

You can use a for loop:

for iteration in 10 100 1000 10000 100000; do
    python -W ignore Experiment.py --iteration "${iteration}"
done

If you have multiple parameters, and you want all the various permutations of all parameters, as @Fox noted in a comment below, you can use nested loops. Suppose, for example, you had a --name parameter whose values could be n1, n2, and n3, then you could do:

for iteration in 10 100 1000 10000 100000; do
    for name in n1 n2 n3; do
         python -W -ignore Experiment.py --iteration "${iteration}" --name "${name}"
    done
done

You could put that in a file, for example runExperiment.sh and include this as the first line: #!/bin/bash. You could then run the script using either:

bash runExperimen.sh

Or, you could make the script executable, then run it:

chmod +x runExperiment.sh
./runExperiment.sh

If you're interested in some results before others, that'll guide how you structure the loops. In my example above, the script will run:

... --iteration 10 --name n1
... --iteration 10 --name n2
... --iteration 10 --name n3
... --iteration 100 --name n1

So it runs all experiments for iteration 10 before moving on to the next iteration. If instead you wanted all experiments for name n1 before moving on to the next, you could make the name loop the "outer" loop.

for name in ...; do
    for iteration in ...; do
Improve this answer
3
  • Oh Great! Thank you very very much. What if I have more than one parameters: --X --Y --Z, how do I make it ? Commented Jan 5, 2018 at 14:59
  • Yes, if you want all permutations of all parameters, the nested loop will do what you're after. Commented Jan 5, 2018 at 15:05
  • @AndyDalton Thank you Andy. Appreciate your answer Commented Jan 5, 2018 at 15:09
1

With GNU Parallel it looks like this:

parallel -j1 python -W ignore Experiment.py --iterations {} ::: 10 100 1000 10000 100000

Remove -j1 if you want to run one Experiment.py per CPU core in parallel.

To combine with multiple arguments, let us assume you have:

-s = S, M, L
--size = small, medium, large
--color =  red, green, blue

You want -s L to be linked to --size large but with all combinations of --color:

parallel echo s={1} size={2} color={3} ::: S M L :::+ small medium large ::: red green blue
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.