Dynamic arguments for Python's argparse

Question

I have a need in Python to create a list of arguments dynamically. I've created a script to demonstrate this, named args.py, shown below:

#!/usr/bin/python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-args_file', default = 'args.txt')

with open(parser.parse_args().args_file, 'r') as f:
    args = f.readlines()

for arg in args:
    parser.add_argument('-' + arg.strip())

dynamic_args = parser.parse_args()

print dynamic_args

I also have a text file in the same folder, named args.txt, also shown below:

arg1
arg2
arg3

As expected, running args.py with no arguments results in:

Namespace(arg1=None, arg2=None, arg3=None, args_file='args.txt')

However, what I'm having trouble with is running my script with the -h argument. I would like the help to display the arguments found in the args_file, as seen in the example below:

usage: args.py [-h] [-args_file ARGS_FILE] [-arg1 ARG1] [-arg2 ARG2]
               [-arg3 ARG3]

What I'm seeing instead is:

usage: args.py [-h] [-args_file ARGS_FILE]

Moreover, if I run the script interactively (i.e. python -i arg.py), and at the interactive prompt type the command "parser.print_usage()", I get the wanted response (showing the -argN arguments). Also, typing "arg.py -arg1 1" or "arg.py arg1 1" result in "unrecognized arguments".

I've tried everything I can think of, but I've been unsuccessful thus far. Do any of the Python aficionados have any suggestions?

The help message you are seeing is printed at the first parse_args(), where the additional options have not yet been set. — hlt
– hlt, Commented Aug 14, 2014 at 21:23
You'd need to suppress passing -h during the first parse_args() call.. — Martijn Pieters
– Martijn Pieters, Commented Aug 14, 2014 at 21:23
As a side note, you don't need to do args = f.readlines() then for arg in args:. That f is already an iterable of lines; just do for arg in f: and you'll get the same thing (but simpler, briefer, and more efficient). — abarnert
– abarnert, Commented Aug 14, 2014 at 21:24
I found the solution below. Thanks for your help and also thanks for the side note. I've implemented that as well. — Mark Atkin
– Mark Atkin, Commented Aug 15, 2014 at 18:48

John Hazen · Accepted Answer · 2014-08-15 02:43:26Z

16

As Martjin pointed out, you can omit the help from the parser the first time. The other thing to do is to use parse_known_args the first time, so you only parse the args_file.

In cases like this, I like to keep things clear by using a throwaway parser for the first parse, and a full parser for the final parse:

import argparse
argfile_parser = argparse.ArgumentParser(add_help=False)
full_parser = argparse.ArgumentParser()
argfile_parser.add_argument('-args_file', default = 'args.txt')
full_parser.add_argument('-args_file', default = 'args.txt')
with open(argfile_parser.parse_known_args()[0].args_file, 'r') as f:
    for arg in f:
        full_parser.add_argument('-' + arg.strip())

dynamic_args = full_parser.parse_args()

print dynamic_args

For testing, I added a file args2.txt:

argA
argB
argC

And I think the result is what you're looking for:

lap:~$ python tmp.py -h
usage: tmp.py [-h] [-args_file ARGS_FILE] [-arg1 ARG1] [-arg2 ARG2]
              [-arg3 ARG3]

optional arguments:
  -h, --help            show this help message and exit
  -args_file ARGS_FILE
  -arg1 ARG1
  -arg2 ARG2
  -arg3 ARG3
lap:~$ python tmp.py -args_file args2.txt
Namespace(argA=None, argB=None, argC=None, args_file='args2.txt')
lap:~$ python tmp.py -h -args_file args2.txt
usage: tmp.py [-h] [-args_file ARGS_FILE] [-argA ARGA] [-argB ARGB]
              [-argC ARGC]

optional arguments:
  -h, --help            show this help message and exit
  -args_file ARGS_FILE
  -argA ARGA
  -argB ARGB
  -argC ARGC
lap:~$ python tmp.py -arg1 foo
Namespace(arg1='foo', arg2=None, arg3=None, args_file='args.txt')
lap:~$ python tmp.py -args_file args2.txt -argA bar
Namespace(argA='bar', argB=None, argC=None, args_file='args2.txt')

answered Aug 15, 2014 at 2:43

John Hazen

1,3811 gold badge9 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Mark Atkin Over a year ago

This is exactly what I needed! I had tried different combinations of add_help=False, multiple ArgumentParser objects, and parse_known_args, but I couldn't seem to connect all the dots. Thanks!

wesm Over a year ago

How would you "access" dynamic args, say if I only have a list of strings, how do I "lookup" an arg from the argparse output?

wesm Over a year ago

nvm getattr(script_args, 'arg_name')

gecko655 · Accepted Answer · 2021-05-06 03:00:23Z

2

From Hazen's answer, using parents option makes it easier.

https://docs.python.org/3/library/argparse.html#parents

import argparse
argfile_parser = argparse.ArgumentParser(add_help=False)
argfile_parser.add_argument('-args_file', default = 'args.txt')
full_parser = argparse.ArgumentParser(parents=[argfile_parser])
with open(argfile_parser.parse_known_args()[0].args_file, 'r') as f:
    for arg in f:
        full_parser.add_argument('-' + arg.strip())

dynamic_args = full_parser.parse_args()

print dynamic_args

answered May 6, 2021 at 3:00

gecko655

3523 silver badges11 bronze badges

Collectives™ on Stack Overflow

Dynamic arguments for Python's argparse

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related