Execute shell script w/ arguments stored in another file

Question

I have a Python script that requires me to set many flags. I am trying to write some shell scripts to simplify the process of executing this Python script.

I would like to have two shell scripts, params.sh and run.sh. In params.sh, I set the shell variables to the desired values like so:

VAR1=val1
VAR2=val2
...

In run.sh, I want to take the variables that I've set in params.sh, create a unique name from the values of each variable, and then use those variables as the flags to execute the Python script. So in pseudocode, what I'd like to do is

PARAMS=get_params()
UNIQUE_NAME=make_unique_name(PARAMS)
python main.py --name=$UNIQUE_NAME --VAR1=${VAR1} --VAR2=${VAR2} ...

I'm stuck on how to create a shell function that takes an arbitrary number of arguments, and how I might process the list of params to execute the Python script as it appears in the last line of pseudocode. Any suggestions would be greatly appreciated.

Would it be feasible to instead have a text file containing the parameters (params.txt) and pass the path to params.txt as the sole argument to the Python script? The Python script will then be responsible for parsing params.txt and creating the unique name. — ndmeiri
– ndmeiri, Commented Mar 15, 2018 at 21:12
Does params.sh have to actually be a shell script? Why can't it be basically the same format, but treated as a .cfg/.ini/whatever file? Then you can write a simple script (trivial in Python, for example) that reads a params.cfg file and prints a list of args, and all run.sh has to do is python main.py --name=$UNIQUE_NAME $(python parse_cfg.py params.cfg) or the like. — abarnert
– abarnert, Commented Mar 15, 2018 at 21:12
Also, why is this tagged Python? You're asking how to do something complicated with bash. You'd do the same thing to run a compiled program, or a perl script, or anything else. So why do you want Python experts rather than Bash experts to help you? — abarnert
– abarnert, Commented Mar 15, 2018 at 21:13
The argparse module supports using files to store long lists of arguments, see this example. — import random
– import random, Commented Mar 15, 2018 at 21:14
I am unfamiliar with bash, so forgive me. The file containing the param values need not be a shell script. And the suggested tags included Python. I'll remove the tag. Thank you kindly for your suggestions :) — Roflcakzorz
– Roflcakzorz, Commented Mar 15, 2018 at 21:32

Charles Duffy · Accepted Answer · 2018-03-15 21:51:35Z

1

Consider:

#!/usr/bin/env bash

# read lines from file "vars", prefixing each with "--"
while IFS= read -r line; do
  args+=( "--$line" )
done <vars

# hash the contents of that variable
read -r _ unique_name _ < <(printf '%s\0' "${args[@]}" | openssl dgst -md5)

# use the hash as a name; then pass through the arguments
python main.py --name="$unique_name" "${args[@]}"

edited Mar 15, 2018 at 21:51

answered Mar 15, 2018 at 21:46

Charles Duffy

299k43 gold badges441 silver badges497 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Roflcakzorz Over a year ago

When I try this, the unique name appears to be empty.

Roflcakzorz Over a year ago

Not sure why, but deleting the underscores around unique_name seemed to fix the issue. Thanks!

Charles Duffy Over a year ago

@Roflcakzorz, the underscores consume the first and third-and-later words of the input, leaving only the second. On my system, openssl dgst -md5 omits output that looks like (stdin)= b1946ac92..., so the leading underscore is necessary to make sure that only the checksum, and not the (stdin)= at the front of the line, goes into the unique_name variable.

Charles Duffy Over a year ago

@Roflcakzorz, ...that said, I wonder if maybe your script sets a non-default value for IFS? That would explain why the splitting into words is happening on an undesired boundary. Or if you replaced openssl dgst -md5 with md5sum, then we'd need to remove second-and-later, not first.

Collectives™ on Stack Overflow

Execute shell script w/ arguments stored in another file

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related