Sequentially run multiple bash scripts

Question

sorry if this question has already been asked. I simply couldn't figure out right vocabulary to search for it. I have multiple experiments that I need to run in a cluster with GPU. But for now I start one job and wait for it to finish before launching the other. What can I do to "give command" to run all the scripts sequentially? I have something like the following:

./experiment.sh configs/job1.sh --train <gpu-ID>
./experiment.sh configs/job2.sh --train <gpu-ID>
./experiment.sh configs/job2.sh --train <gpu-ID>

where for <gpu-ID> I put 0 or 1 for specific GPUs

Right now I start a job say:

./experiment.sh configs/job1.sh --train <gpu-ID>

And periodically check whether it finished so that I can start job2.

What's a more efficient way to automate this?

try && like cmd1 && cmd2 , this means, cmd2 will run after cmd1 fineshed sucessfully — P....
– P...., Commented Oct 23, 2019 at 15:41
This will run the jobs sequentially; when job1.sh finishes, job2.sh will start. — chepner
– chepner, Commented Oct 23, 2019 at 16:23
Can you please confirm that you have tried running the code you posted, and that you have found that it fails to do what you ask for? — that other guy
– that other guy, Commented Oct 23, 2019 at 16:42

Alice · Accepted Answer · 2019-10-23 15:42:30Z

5

Yes, you can use double amperstands : script1 && script2 && script3

In this example, script2 will be executed when script1 ends, only if script1 returns 0, ie doesn't fail

answered Oct 23, 2019 at 15:42

Alice

1,5923 gold badges10 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

chepner · Accepted Answer · 2019-10-23 16:26:43Z

1

What you have would run the jobs sequentially, if they appeared in a script:

$ cat all_jobs
./experiment.sh configs/job1.sh --train <gpu-ID>
./experiment.sh configs/job2.sh --train <gpu-ID>
./experiment.sh configs/job2.sh --train <gpu-ID>
$ sh all_jobs  # Runs job1, then run job2, then run job 3

If you want to run them from a single command line, separate them with ;:

$ ./experiment.sh configs/job1.sh --train <gpu-ID>; ./experiment.sh configs/job1.sh --train <gpu-ID>; ./experiment.sh configs/job1.sh --train <gpu-ID>;

answered Oct 23, 2019 at 16:26

chepner

538k77 gold badges594 silver badges746 bronze badges

Collectives™ on Stack Overflow

Sequentially run multiple bash scripts

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related