Python argparse AssertionError

Question

I just started using argparse module. I wrote the following reduced snippet to demonstrate an issue I'm having.

from argparse import ArgumentParser

if __name__ == '__main__':
    parser = ArgumentParser('Test argparse. This string needs to be relatively long to trigger the issue.')
    parser.add_argument('-f', '--fin', help='a', required = True)
    parser.add_argument('-o', '--out ', help='b', required = True)
    parser.add_argument('-t', '--trans', help='c', required = True)

    args = parser.parse_args()
    print(repr(vars(args)))

AssertionError will be produced when script is run with argument -h

Traceback (most recent call last):
  File "arg.py", line 10, in <module>
    args = parser.parse_args()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 1707, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 1739, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 1945, in _parse_known_args
    start_index = consume_optional(start_index)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 1885, in consume_optional
    take_action(action, args, option_string)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 1813, in take_action
    action(self, namespace, argument_values, option_string)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 1017, in __call__
    parser.print_help()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 2341, in print_help
    self._print_message(self.format_help(), file)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 2325, in format_help
    return formatter.format_help()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 278, in format_help
    help = self._root_section.format_help()
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 208, in format_help
    func(*args)
  File "C:\Users\user\AppData\Local\Continuum\Anaconda\envs\py3k\lib\argparse.py", line 329, in _format_usage
    assert ' '.join(opt_parts) == opt_usage
AssertionError

Reducing the length of the description string passed to ArgumentParser makes it work correctly. ~~Removing one of the arguments will also help.~~

Am I doing something wrong here? My environment is:

Python 3.3.5 |Anaconda 1.9.2 (64-bit)| (default, Mar 10 2014, 11:25:04) [MSC v.1600 64 bit (AMD64)] on win32

Keep in mind that ArgumentParser.__init__ doesn't really take positional arguments. Given the order in which the keyword arguments are defined, you're passing an extremely long string to initialize the prog attribute, which is intended to store the name of the program as it appears in the help message. Which attribute are you trying to initialize with that string? — chepner
– chepner, Commented Apr 18, 2014 at 18:40
Part of the problem is that you're not specifying the description string correctly. You need to use a keyword argument: parser = ArgumentParser(description='Test argparse. This string needs to be relatively long to trigger the issue.') — khagler
– khagler, Commented Apr 18, 2014 at 18:46
Then you should use ArgumentParser(description="long string here"). — chepner
– chepner, Commented Apr 18, 2014 at 19:13

esorton · Accepted Answer · 2014-04-18 19:08:50Z

15

There is an extra space after --out in the code. Change:

parser.add_argument('-o', '--out ', help='b', required = True)

to:

parser.add_argument('-o', '--out', help='b', required = True)

The underlying cause of the problem is an assert check within the Python code that only occurs when Python attempts to break the help text into multiple lines because it is too long. After breaking the text into a list, the Python code joins it back together and compares it to the original to ensure that it is correct. However, the code that breaks the text apart drops the adjoining spaces resulting in a miscompare.

I added prints to the code (argparse.py, Python 2.7):

# wrap the usage parts if it's too long
text_width = self._width - self._current_indent
if len(prefix) + len(usage) > text_width:
    # break usage into wrappable parts
    part_regexp = r'\(.*?\)+|\[.*?\]+|\S+'
    opt_usage = format(optionals, groups)
    pos_usage = format(positionals, groups)
    opt_parts = _re.findall(part_regexp, opt_usage)
    pos_parts = _re.findall(part_regexp, pos_usage)
    print ' '.join(opt_parts)
    print opt_usage
    assert ' '.join(opt_parts) == opt_usage

And the results:

[-h] -f FIN -o OUT -t TRANS
[-h] -f FIN -o OUT  -t TRANS
Traceback (most recent call last):
  File "blah.py", line 9, in <module>
    args = parser.parse_args()

Note the extra space after OUT.

This explains all of the observed behavior:

Must be long enough to trigger the wrapping behavior.
Deleting the --trans argument moved --out to the end negating the behavior.
Deleting the --out argument negateted the behvaior.

edited Apr 18, 2014 at 19:08

answered Apr 18, 2014 at 18:27

esorton

1,58210 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

esorton Over a year ago

I just looked into the argparse.py code. The issue has to do with line wrapping. So, a long line caused wrapping and you hit the issue. A short line didn't need wrapping and you didn't hit the issue. I'll add some text to the answer a bit later to help clarify.

ohcamel Over a year ago

Oops. Sorry for deleting the comment above. I initially wrote "It's interesting that the length of description text affects the occurrence". But as @chepner pointed out in the topic comments, I wasn't really setting the description but instead the prog attribute.

hpaulj Over a year ago

That part of the help formatting code needs to be cleaned up. It's too complex and error prone. bugs.python.org/issue11874. But that '--out ' could still have caused problems when accessing the namespace. args.out would not work. getattr(args, 'out ') would.

esorton Over a year ago

@hpaulj Thanks for adding the information to the bug report.

Forrest Bice Over a year ago

I had trouble parsing the solution from this page when I bumped into this issue and ended up finding the solution in the comments of the bug report linked above. To hopefully help others until a patch is made here's my TL;DR fix TL;DR: All parameter of your constructed ArgumentParser (ex: usage, description, etc.) in combination must fit on one line and not wrap when displayed. In my case I have a large number of arguments which caused my usage to wrap lines. Thus my fix was overriding usage to make it shorter; however, this also made it less useful.

igr · Accepted Answer · 2018-03-12 11:48:49Z

9

I came here with the exact same problem/error, but without having any extra spaces after --out. My problem was that metavar was set to an empty string (metavar=''). Changing that resolved the issue.

answered Mar 12, 2018 at 11:48

igr

5,0662 gold badges18 silver badges20 bronze badges

3 Comments

joadha Over a year ago

Thanks for this. My problem was a set of square braces in one of my metavars.

akash Over a year ago

The metavar=' ' definition added an extra space for the particular argument and caused the error while parsing.

skwidbreth Over a year ago

I had the same issue with metavar, and removing it fixed the problem. However, I still really want to get rid of the metavar! Any thoughts on how to achieve that?

PJ_Finnegan · Accepted Answer · 2018-05-17 15:50:19Z

Python 3.5.2

It drove me crazy for a while but I've found the problem at last. It's definitely a usage line length issue, if it exceeds the COLUMNS environment variable set for the console/terminal, you get that error. I try from command line:

$ COLUMNS=80 python <myprog.py> -h

and I get this exception:

...
  File "/usr/lib/python3.5/argparse.py", line 1735, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.5/argparse.py", line 1767, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.5/argparse.py", line 1973, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/lib/python3.5/argparse.py", line 1913, in consume_optional
    take_action(action, args, option_string)
  File "/usr/lib/python3.5/argparse.py", line 1841, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/lib/python3.5/argparse.py", line 1025, in __call__
    parser.print_help()
  File "/usr/lib/python3.5/argparse.py", line 2367, in print_help
    self._print_message(self.format_help(), file)
  File "/usr/lib/python3.5/argparse.py", line 2351, in format_help
    return formatter.format_help()
  File "/usr/lib/python3.5/argparse.py", line 287, in format_help
    help = self._root_section.format_help()
  File "/usr/lib/python3.5/argparse.py", line 217, in format_help
    func(*args)
  File "/usr/lib/python3.5/argparse.py", line 338, in _format_usage
    assert ' '.join(opt_parts) == opt_usage
AssertionError

But with:

$ COLUMNS=<XX> python <myprog.py> -h

where XX > of the resulting usage line, everything is ok, it prints the usage + help and exits. So either you shorten your usage line or you increment the COLUMNS value.

EDIT:

I've found the error in my case: I was using the square brackets [] in my program/arguments description.

As others have correctly pointed out, looking at the python code where the exception occurred, you can see that argparse has a provision to automatically fold the usage/help to $COLUMNS columns. But to split the long rows, it uses the following RE:

(File "/usr/lib/python3.5/argparse.py", line 333:)

`part_regexp = r'\(.*?\)+|\[.*?\]+|\S+'`

When it rejoins the line to check, if the user has introduced square brackets, the assert fails, being them special characters used by argparse to mark optional values.

In short, I removed the rogue square brackets from my text and everything works fine, with the usage/help properly folded and formatted according to the $COLUMNS value.

Alexander Ejbekov · Accepted Answer · 2014-04-18 18:35:39Z

4

The problem is not the extra -h that you've added. Look at the error and the -o argument:

assert ' '.join(opt_parts) == opt_usage

it's joining the whitespace in '--out '. If you remove it, everything should work fine.

answered Apr 18, 2014 at 18:35

Alexander Ejbekov

6,0501 gold badge28 silver badges27 bronze badges

Comments

abruin · Accepted Answer · 2019-01-19 20:55:35Z

4

For me, it was setting both required=True and metavar=''. Removing one and keeping the other solved it.

answered Jan 19, 2019 at 20:55

abruin

712 bronze badges

1 Comment

aysljc Over a year ago

Thank you. It works. It would be interesting if someone could post why this works.

Collectives™ on Stack Overflow

Python argparse AssertionError

5 Answers 5

5 Comments

3 Comments

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

5 Comments

3 Comments

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related