smtpd-as-a-script feature should be documented and should use argparse

[masklinn]

BPO	11260
Nosy	@warsaw, @rhettinger, @ncoghlan, @merwok, @bitdancer, @masklinn, @berkerpeksag
Files	smtpd-to-argparse.diff: Port smtpd to argparse smtpd-as-script-doc.diff: Documentation of smtpd-as-a-script

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-01-10.00:54:36.213>
created_at = <Date 2011-02-20.20:03:50.521>
labels = ['type-feature', 'library', 'docs']
title = 'smtpd-as-a-script feature should be documented and should use argparse'
updated_at = <Date 2018-01-10.00:54:36.212>
user = 'https://github.com/masklinn'

bugs.python.org fields:

activity = <Date 2018-01-10.00:54:36.212>
actor = 'barry'
assignee = 'none'
closed = True
closed_date = <Date 2018-01-10.00:54:36.213>
closer = 'barry'
components = ['Documentation', 'Library (Lib)']
creation = <Date 2011-02-20.20:03:50.521>
creator = 'xmorel'
dependencies = []
files = ['20812', '20813']
hgrepos = []
issue_num = 11260
keywords = ['patch']
message_count = 17.0
messages = ['128917', '128918', '128998', '129042', '129044', '129047', '129048', '129078', '129117', '129123', '129132', '129157', '129467', '140278', '161834', '223270', '309750']
nosy_count = 9.0
nosy_names = ['barry', 'rhettinger', 'ncoghlan', 'eric.araujo', 'r.david.murray', 'xmorel', 'docs@python', 'BreamoreBoy', 'berker.peksag']
pr_nums = []
priority = 'low'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue11260'
versions = ['Python 3.5']

[masklinn]

argparse has been merged to the standard library in 3.2, and (tell me if I'm wrong) would therefore be the "best-practices" way to parse command-line arguments.

Numerous stdlib modules can be used as scripts, but they tend to have ad-hoc documentation (if they are at all documented) and arguments parsing (using any of the 4 available methods in Python: straight from sys.argv, getopt, optparse and argparse).

I picked smtpd as a first shot since it does something useful (SMTP proxy) and has a pretty good (if ad-hoc) command-line documentation.

smtpd is currently using getopt for its options parsing and the argument parsing is very cleanly factored: the port only had to replace the implementation of the parseargs function (and add and remove some helpers).

• The port keeps the existing arguments semantics (including the mandatory host:port syntax for the local and remote specs if overridden)

• The port tries to maintain the old error messages, but due to the way argparse works (or the way I used it for the specs) the parity is not perfect when providing incorrect specs

• The CLI help uses argparse's formatting, and the documentation for the local and remote specs was set on these arguments rather than as an epilog. The version string was also removed from the help screen

• Because they are set by argparse's arguments validation, the status codes on incorrect arguments are slightly different:

  • running smtpd.py as a regular user without the --nosetuid flag still exits with status 1
  • providing incorrect spec formats (missing or non-int port) or providing too many positional arguments (3 or more) now exits with status 2 (formerly 1)

[masklinn]

Second patch: documenting smtpd-as-a-script in the module's rst

[bitdancer]

Thanks for taking an interest in this and working up a patch. Unfortunately I do have some concerns.

As far as I know the only module that currently uses argparse is compileall, which was only changed recently.

Do tests currently exist for smtpd run as a script? If not, our experience with converting compileall to argparse indicates a thorough test suite is needed (and even so we missed some things we hadn't thought to test).

We converted compileall because its help output was broken, but in retrospect it might have been better to fix the help output in the existing code. In other words, if the current code works, is "updating" it a sufficient reason to change it, considering the chances of introducing new bugs? The answer might be yes, but I don't think it is obviously yes.

[masklinn]

Do tests currently exist for smtpd run as a script?

I have to confess I didn't think to check.

If not, our experience with converting compileall to argparse indicates a thorough test suite is needed (and even so we missed some things we hadn't thought to test).

OK, so if there is no test suite currently I should create one, and if there is one I should ensure it's complete? I guess I should use compileall as an example of how to test modules-as-scripts if the former? Overall, smtpd-as-a-script is really pretty simple, it totals 28 lines apart from the argument parsing itself (which is a bit under 60 lines ignoring the help text/pattern and gets a bit under 50 including it post-patch), and as I mentioned the only part which actually needed changing is the arguments parsing, which was very well factored out.

In other words, if the current code works, is "updating" it a sufficient reason to change it, considering the chances of introducing new bugs?

I'm not sure, but one of the ways I see standard libraries is not just as "ready to use" code, it's also as a repository of how to do things. And to that end, if argparse is now considered the canonical way to parse command-line arguments (which I would expect), there are very few examples of how to do things in the stdlib itself (though there are examples outside of it, due to the life argparse had pre-stdlib).

It also rose to the occasion as I was wondering about the numerous standard library modules-as-scripts which are either undocumented or under-documented: because it had a good command-line documentation and a clean separation between the configuration (options parsing) and the actual execution, but no module documentation (in Doc/) it seemed like a good starting point: if it's not feasible to correctly convert "best cases" (and smtpd is probably one, short of modules using optparse probably) then the whole idea is stillborn: I do not see how it would be possible to fare better on some of the fully undocumented modules using manual options parsing, yet it would have to be expected.

[rhettinger]

Xavier, I think these efforts are misguided in several ways:

• Many of the undocumented command-line interfaces are
  intentionally undocumented -- they were there for the
  convenience of the developer for exercising the module
  as it was being developed and are not part of the official API.
  Most are not production quality and would have been done
  much differently if that had been the intent.

• The standard library's primary purpose is to be a library
  for use in user programs, not to serve as a suite of applications.
  We do have a few applications such as pydoc that assist
  development, but that is not what we want to do for most
  modules. Real apps need more features than we usually
  offer and they need much faster development cycles than
  supported by our 18-24 month release cycle.

• To the extent the standard library serves as a set of
  examples, that is just side benefit. There are other
  ways to make code specifically for examples (we include
  tons of examples in the documentation; we had a Demos
  directory; there is the wiki; and there is the ASPN
  Cookbook site; etc)

• David Murray is correct is pointing-out that converting
  to argparse risks changing the semantics and possibly
  breaking someone's existing uses of the current interface.

• Though argparse is the shiny new kid on the block and
  it should be used for new code, there is no imperative
  to go change all existing command-line scripts. We haven't
  even done that same exercise for the new string formatting.

• All that being said, there are some exceptions and it
  make may sense to document the interface in some where
  we really do want a command-line app. I'll look at
  any patches you want to submit, but try to not go wild
  turning the library into a suite of applications. For
  the most part, that is not what the standard library
  is about.

[masklinn]

any of the undocumented command-line interfaces are intentionally undocumented -- they were there for the convenience of the developer for exercising the module as it was being developed and are not part of the official API.

I can understand that, but it's not clear from the source code which is which, and for several of them third-parties have taken up the convenience. Maybe a more formal process of defining which tools are to be considered "official" and which are easter-eggs is needed? In any case, even for "easter eggs" surely a good command-line documentation is a good idea (even if the module documentation side is not added due to the "easter egg" status) isn't it?

The standard library's primary purpose is to be a library for use in user programs, not to serve as a suite of applications.

Sure, and I don't think most of the module-as-scripts have what it takes to be seen as applications, but many are useful and/or interesting helpers/shortcuts for day to day tasks. http.server and smtpd are good examples of useful modules-as-scripts (http.server being the exact opposite of smtpd in that it has a module-as-script documentation but no documentation whatsoever on the command-line)

Real apps need more features than we usually offer and they need much faster development cycles than supported by our 18-24 month release cycle.

I think my suggestion has been misunderstood: I don't want to turn these into real apps, they're fine as basic scripts/helpers to spend 5mn on a task instead of half an hour. I think they deserve better than to be documented and known through StackOverflow or Reddit "what are the -m stdlib features" lists.

To the extent the standard library serves as a set of examples, that is just side benefit. There are other ways to make code specifically for examples (we include tons of examples in the documentation; we had a Demos directory; there is the wiki; and there is the ASPN Cookbook site; etc)

Sure, but I think it's still an important driver of how things "are done" in that they're *concrete* examples, not abstract (of course the Cookbook is generally concrete as well). I'm not discounting the importance or quality of the rest of the documentation at all, or at least that was not my intention.

All that being said, there are some exceptions and it make may sense to document the interface in some where we really do want a command-line app. I'll look at any patches you want to submit, but try to not go wild turning the library into a suite of applications. For the most part, that is not what the standard library is about.

As I said, my only intention here is be to document (and argparsify/formalize) what is already there. I considered doing more (e.g. for the specific case of smtpd-as-a-script making ports optional even when hosts are specified, that kind of things) but decided against this distraction, I really just want to make existing module-as-script features simpler to discover and use.

That said, do you have guidelines of which areas this idea would be most interestingly/efficiently applied? Maybe a list of modules-as-scripts you know are used regularly and could benefit from some improvements in their interface or documentation?

[masklinn]

One more note I forgot previously: the conversion of as much scripts as possible to argparse would be for three reasons:

• Make behavior consistent across the board (e.g. -h/--help)
• Make CLI documentation format consistent across the board either so users know what to expect and when
• Provide easy to reach (for users) examples of using argparse

[warsaw]

I'd support updating to argparse *if* you write a test suite for full coverage of the existing cli first. Once that passes, you'll have more confidence in your port. Modernizing the argument parsing in that case would be a useful addition. Not critical, but on the whole I think a good thing.

[masklinn]

Barry, do I correctly understand your comment to mean I should write end-to-end tests of the CLI (until reaching the already tested "meat" of smtpd), not just the CLI options parsing?