Skip to content

Argument parsing for functions #7

Open

Description

@emmatyping

To make writing Rust functions that take Python arguments ergonomic, we need an easy way to describe how to parse arguments, and ideally convert to Rust object abstractions. The C API for argument parsing is described here: https://docs.python.org/3/c-api/arg.html

C code in CPython uses a DSL to pre-generate C argument parsing code called argument clinic

We have a few different ways we could go here. PyO3 currently uses proc-macros to provide very ergonomic argument parsing. We could also modify argument clinic to generate Rust code and include!() the generated files, which might have better compilation times.

I am still undecided about which approach I prefer.

Activity

  1. youknowone commented on Dec 12, 2025

    @youknowone

    Maybe using clinic + proc-macro will is also possible.

    #[clinic]
    /// useful DSL
    fn xyz_impl(...) 
    ``
    
    Then #[clinic] can generate everything inside the macro. No `include!()` will be required.
    Cons: The generated code will not be searched 
    
    If saving generated code in the repository is important, generate the code and including it using `#[clinic]` will be also possible though.
  2. davidhewitt commented on Dec 12, 2025

    @davidhewitt

    I have no strong opinion about the build system integration, anything is possible. I think what matters most is going to be what the Rust code we write looks like. Does it use normal Rust attributes, or something more like argument clinic special syntax?

  3. Eclips4 commented on Dec 12, 2025

    @Eclips4
    Member

    I’m pretty much in favor of any approach except updating the Tools/clinic one. The basic problem is that CPython's clinic isn't guaranteed to work with third-party C modules. I think we can use a proc-macro instead, so we could make it usable for third-party modules that way.

  4. emmatyping commented on Dec 12, 2025

    @emmatyping
    Author

    I think we can use a proc-macro instead, so we could make it usable for third-party modules that way.

    Yeah I think this is a good idea. The APIs we design now should eventually be stablized and made usable by the public, so proc-macros seem like a natural use case for something we'll want to make ergonomic for users. It also makes a lot of sense as we will likely be iterating on function signatures initially a lot.

    If compile times are an issue, we can probably re-use the proc-macro code to generate argument parsing code that gets checked in and used via a build script hook (similar to how bindgen can generate bindings ahead of time).

  5. cmaloney commented on Mar 9, 2026

    @cmaloney

    Second not updating Tools/clinic. Recently I have been adding a new decorator and what I need to change and why isn't really obvious in updating AC internals. The code is effective but, to me, hard to extend.

    To me there are a couple core parts to the problems AC solves today:

    1. Specifying what the accepted Python function signature(s) are.
    2. Performing argument type / value checking and normalization
    3. Mapping the interpreter call representation (tp_call, vectorcall) to the implementation function ABI (currently C ABI) ideally in a way the compiler can see and potentially optimize.

    To me if 1 can be exactly Python syntax that's actually better than the current DSL. This is very close to type stubs but hand-annotations don't always match actual implementation. For example I/O .write() stub says it returns an int but it also returns None for some layers of the I/O stack. Doesn't come up in most cases but does exist.

    For 2 and 3 need something which exactly matches behavior to start at least which is harder than making something new. Ideally that can be seen/worked with by the JIT to eliminate interpreter overhead for more cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions