[jameslamb]

Contributes to #7470

Proposes adding numpydoc-validation and pydoclint, 2 linters that enforce standardization and correctness of Python docstrings. These caught a few issues (like eval_{X,y} missing from the Dask docstrings.

Both are fast enough to not be noticeable.

I think they'll help a lot with consistency and standardization, and relieve a small bit of review burden.

Notes for Reviewers

I tried to choose configurations for these tools that as closely as possible match the project's current conventions. See inline comments below.

I've added this branch (temporarily) to readthedocs configuration.

• example build: https://app.readthedocs.org/projects/lightgbm/builds/34868017/

[jameslamb]

numpydoc-validation would really prefer that functions that return a single object not name the returned object. See https://numpydoc.readthedocs.io/en/latest/format.html#returns

[jameslamb]

pydoclint directly compares the text of the type hint in the docstring to the text of the type hint in the code. This produces lots of false positives for lightgbm, for example where Python code has Path and the docstring has pathlib.Path.

I've requested a way to globally ignore checks, similar to ruff's "ignore" (jsh9/pydoclint#228 (comment)). But until then, I think it's not too bad to have these noqa comments everywhere. We don't add new public interfaces with new docstrings that often.

[jameslamb]

These had been in the docstrings inherited from the sklearn estimators but not actually written out in the code.

They were previously passed through **kwargs, and I'm just proposing promoting them to the signature:

1. to help with tab completion and discoverability via help()
2. to satisfy linter warnings about the "Parameters" section not matching the signature

[borchero]

Have you checked out pydocstyle? It's neat because you can integrate it directly with ruff; wouldn't need additional pre-commit hooks.

[jameslamb]

This project already uses the pydocstyle ruff rules:

LightGBM/python-package/pyproject.toml

Lines 164 to 165 in 439136e

	# pydocstyle
	"D",

That didn't catch any of the issues caught by numpydoc-validation or pydoclint.

These are complementary tools.

• pydocstyle = generic docstring conventions, but only looks at the docstrings (https://docs.astral.sh/ruff/rules/?linter=pydocstyle-d#legend)
• numpydoc-validation = compliance with numpydoc style specifically (https://numpydoc.readthedocs.io/en/latest/format.html), and also looks at the Python code to catch things like "undocumented argument"
• pydoclint = mix of generic things and numpydoc-specific things, and also looks at the Python code

There's some overlap between numpydoc-validation and pydoclint but they do also catch some different things.

I think we should have all 3 of these things enabled here. It would have avoided bugs like missing some docs updates when eval_y / eval_X were added in #6857

[borchero]

All right, thanks for the explanation @jameslamb!

[jameslamb]

Thanks very much for considering it and carefully reviewing!