allow_abbrev=False does not work for single-dash long options

[SunCrazy]

Bug report

The following codes get different results between python3.6.9 and python3.8.10

import argparse


if __name__ == "__main__":
    parser = argparse.ArgumentParser(
        allow_abbrev=False
    )

    parser.add_argument('-on', "--output-name", type=str, help="output")

    args, _ = parser.parse_known_args(["-on", "name", "-o", "dir"])
    print(args)

python3.6.9

[chenf@b12e0231:/mnt/ssd/chenf/projects/test_hhb_2.4/debug/cmd_config]$ python3 --version
Python 3.6.9
[chenf@b12e0231:/mnt/ssd/chenf/projects/test_hhb_2.4/debug/cmd_config]$ python3 ~/test.py
Namespace(output_name='name')

python3.8.10

[chenf@b11b0623:/mnt/ssd/chenf/project/hhb_test_2.4/cmd_config]$ python3 --version
Python 3.8.10
[chenf@b11b0623:/mnt/ssd/chenf/project/hhb_test_2.4/cmd_config]$ python3 ~/test.py
Namespace(output_name='dir')

Debug

I have tried to debug the codes and get some clues.

I have set the allow_abbrev=False to disable prefix match, but it works in python3.6.9 and fails in python3.8.10.
When I div into argparse.py and find that it considers the allow_abbrev in python3.6.9 but not in python3.8.10 as follow:

in python3.6.9 argparse.py

def _parse_optional(self, arg_string):
    ....

    if self.allow_abbrev:
            # search through all possible prefixes of the option string
            # and all actions in the parser for possible interpretations
            option_tuples = self._get_option_tuples(arg_string)

            # if multiple actions match, the option string was ambiguous
            if len(option_tuples) > 1:
                options = ', '.join([option_string
                    for action, option_string, explicit_arg in option_tuples])
                args = {'option': arg_string, 'matches': options}
                msg = _('ambiguous option: %(option)s could match %(matches)s')
                self.error(msg % args)

            # if exactly one action matched, this segmentation is good,
            # so return the parsed action
            elif len(option_tuples) == 1:
                option_tuple, = option_tuples
                return option_tuple
    ....

in python3.8.10 argparse.py

def _parse_optional(self, arg_string):
    ....

    # search through all possible prefixes of the option string
        # and all actions in the parser for possible interpretations
        option_tuples = self._get_option_tuples(arg_string)

        # if multiple actions match, the option string was ambiguous
        if len(option_tuples) > 1:
            options = ', '.join([option_string
                for action, option_string, explicit_arg in option_tuples])
            args = {'option': arg_string, 'matches': options}
            msg = _('ambiguous option: %(option)s could match %(matches)s')
            self.error(msg % args)

        # if exactly one action matched, this segmentation is good,
        # so return the parsed action
        elif len(option_tuples) == 1:
            option_tuple, = option_tuples
            return option_tuple
    
    ....

Linked PRs

• gh-104860: Fix allow_abbrev=False for single-dash long options #124340
• [3.13] gh-104860: Fix allow_abbrev=False for single-dash long options (GH-124340) #124749
• [3.12] gh-104860: Fix allow_abbrev=False for single-dash long options (GH-124340) #124750

[JelleZijlstra]

3.6 is unsupported and 3.8 is only receiving security fixes, so neither version is going to change here.

However, 3.12 behaves like 3.8 (Namespace(output_name='dir')). Do you think that behavior is buggy? It's not obvious to me which behavior is correct.

[SunCrazy]

3.6 is unsupported and 3.8 is only receiving security fixes, so neither version is going to change here.

However, 3.12 behaves like 3.8 (Namespace(output_name='dir')). Do you think that behavior is buggy? It's not obvious to me which behavior is correct.

Yes, I think that it is a bug. As described in the python document as follows:

Fuzzy matching should be prohibited while allow_abbrev=False is set. As above codes shows, I set allow_abbrev=False when create argparse obj, ['-o', 'dir'] is still parsed to '-on' and then rewrite the value that set by ['-on', 'name']. That's why the current result is Namespace(output_name='dir').

[hmc-cs-mdrissi]

I think --output-name is long option, while -on is a short option (1 vs 2 hyphens). And I'm confused how two letter short option is intended to be interpreted. It's not being treated as 2 short options of -o + -n because ['-n', 'dir'] does get ignored. While it feels like weird behavior, the bigger question to me is there any documentation for how one hyphen multiple character flag is supposed to behave? Is -on meant to be one 2 letter option or 2 one character options? If it's a 2 letter option what's difference between -on vs --on?

[sunmy2019]

Do you think that behavior is buggy?

I think so. The prefix matching code should be reverted to 3.6 version if we are following what the description is telling us.

[ronaldoussoren]

The code in. _parse_optional does not look at allow_abbrev in the current version of argparse, but _get_option_tuples does. That method explicitly only allows for abbreviations for long options, that is those whose name starts with two dashes.

The same method (_get_option_tuples) also explicitly only looks at the first character for multi-character short option (single dash at the start).

Current behaviour looks intentional ("-o dir" uses the short option, not an abbrev of the long option), but I'm far from an expert on argparse.

Regardless of this, the current behaviour is present in 3.8 - 3.12 and changing this can break other users.

[hpaulj]

Originally abbreviations were allowed. Then we added this parameter to turn that off. But that proved to be buggy, since it broke the chained short option case. That was patched, probably between 3.6 and 3.8.. Feel free to search for the relevant issues and patches.

So '-o' is allowed as an abbreviation of '-on', hence the 3.8 behavior.

While '-o' is the "ideal" short option, the distinction between long and short is handled piecemeal. So '-on' qualifies as short (in the abbreviation context), even though it wouldn't be handled as such in the chaining context.

I suppose we could add a test in add_argument to complain about hybrid shorts like '-on'. But is that change worth it? I say we've done enough fiddling with this abbreviation business. We don't have to close all edge cases. If programmers want to define flags that might confuse their own end users, let them.

Paul

[AA-Turner]

It seems that this is intended behaviour due to the changes mentioned above.

A

[sunmy2019]

It seems that this is intended behaviour due to the changes mentioned above.

No. This change is unexpected and problematic.

It's just that nobody is maintaining this library, and no one is willing to fix this bug for compatibility reasons.

[AA-Turner]

I read that from the below, sorry for the confusion.

So '-o' is allowed as an abbreviation of '-on'

[sunmy2019]

If the behavior is hard to change, at least we can document the current behavior.

The current explanation is ambiguous. Improving the documentation is achievable here.

[hpaulj]

What wording would clarify things for you? Developers tend to overlook ambiguities and unfamiliar interpretations. The `allow_abbrev` docs clearly says it applies to `long options`, but the distinction between long and short is not clearly defined elsewhere in the docs. I scaned it a few days ago, and my memory is that: In the section about setting the `dest`, '--long' is clearly given priority, with '-long' being a fall back choice. In another place '--long=va' and '-long=val' are treated the same. But for chaining shorts, a single character short is clearly assumed. Toward then end, the `Argument abbreviations (prefix matching)` section has examples with strings like '-badger' and '-bacon'. I haven't thought through that carefully, or tried to test them, but that may be buggy, and misleading. This documentation ambiguity relects, in part, the decentralized way in which the code itself handles single/double dash, and single/multicharacter options. So more systematically we have true long - anything begining with '--' (2 prefix) true short - one dash, one character pseudo long - one dash, more than one character equals - anyting with '='

[serhiy-storchaka]

This is not intentional behavior, but a bug that was introduced in process of fixing flag grouping with allow_abbrev=False, presumably in dffca9e (bpo-26967).

-on is not a short option, it is a single-dash long option. It can be abbreviated to -o if allow_abbrev=True (default) and cannot be grouped with short options. allow_abbrev=False should disallow abbreviation, but currently it does not work for single-dash long options.