In argparse, allow REMAINDER(...) arguments in a mutually exclusive group

[shaniarmon]

BPO	40509
Nosy	@rhettinger, @taleinat, @shaniarmon
PRs	bpo-40509: Enable REMAINDER argparse arguments to be used in mutually exclusive … #19919

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-05-05.10:30:44.470>
labels = ['type-bug', 'library', '3.9']
title = 'In argparse, allow REMAINDER(...) arguments in a mutually exclusive group'
updated_at = <Date 2020-10-18.01:48:44.194>
user = 'https://github.com/shaniarmon'

bugs.python.org fields:

activity = <Date 2020-10-18.01:48:44.194>
actor = 'paul.j3'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-05-05.10:30:44.470>
creator = 'Shani Armon'
dependencies = []
files = []
hgrepos = []
issue_num = 40509
keywords = ['patch']
message_count = 9.0
messages = ['368126', '368213', '368227', '368282', '368284', '368323', '369333', '378833', '378845']
nosy_count = 4.0
nosy_names = ['rhettinger', 'taleinat', 'paul.j3', 'Shani Armon']
pr_nums = ['19919']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40509'
versions = ['Python 3.9']

[shaniarmon]

Options specified with the REMAINDER nargs should be treated as optional.
And an empty list should be treated as default.

[paulj3]

Handling the positional with '?' and '*' in a mutually_exclusive_group is tricky enough!

I believe your user can use the '--' to get the same effect.

[shaniarmon]

Why is REMAINDER any more complicated than ZERO_OR_MORE? I need to make a subcommand system (think jupyter style) and -- isn't an intuitive interface. Jupyter just doesn't use argparse in the subcommand case. I prever to in order to support options given before the subcommand, outside the mutually exclusive group (like -C for git)

[paulj3]

A flagged argument with REMAINDER works just fine in a mutually exclusive group.

group.add_argument('-g', nargs='...')

positionals in such a group can only have ? or *. If you check the code, and past issues you'll see that those require some special handling. Normally that kind of positional is always 'seen', because an empty list (not strings) satisfies those nargs. One positional can work in a mutually exclusive group because it's been made to work, not because the fit is natural.

Search past issues for REMAINDER to check whether your concerns have been raised before. If I recall correctly, even without a group, handling a flagged argument with REMAINDER is more robust than a positional.

[shaniarmon]

Yes. The pull request contains the special handling. For the purpose of REMAINDER positionals, the default is changed to the empty list that was returned if no arguments were passed.

For the purpose of positional argument, that is equivalent to nothing being passed

[shaniarmon]

Also, this is quite different from previous issues with REMAINDER. This fits in line with how the example suggests remainder should be used. I wand some flags (version, configuration) to be separate and exclusive to subcommands. And the usage is not ambiguous.

[shaniarmon]

Bumping, since no reply was made to my previous comment. If this is not relevant at all can this be closed or marked as such, so that it may be raised in python-dev?

[taleinat]

I suggest bringing this up for discussion on Python-Ideas.

[paulj3]

A more recent issue shows that the use of a '*' positional in a multually_exclusive_group is poorly understood and documented.

https://bugs.python.org/issue41854

That's part of why I am not enthusiastic about extending this to include REMAINDER.

And on rereading Shani's posts, I realize I don't understand what this has to do with subcommands. Is this the 'add_subparsers' mechanism, or just the idea of using a trailing part of sys.argv as input to another parser or script?

I think discussion of this topic can only proceed with some concrete examples, and may be even a proposed code change (not necessarily a formal patch or pull request).

[serhiy-storchaka]

Options specified with the REMAINDER nargs should be treated as optional.

Done in #124307.

And an empty list should be treated as default.

An empty list is now treated as default for REMAINDER. Done in #124306.

Is it what you wanted, @shaniarmon? There are no examples or tests, so I am not sure.