Skip to content

Turn warnings into errors under pytest - #619

Merged
mdickinson merged 11 commits into
mainfrom
fix/615-pytest-filterwarnings
Aug 5, 2026
Merged

mdickinson merged 11 commits into
mainfrom
fix/615-pytest-filterwarnings

Conversation

@mdickinson

Copy link
Copy Markdown
Member

Towards #615. Stacked on #618 — base is fix/615-test-noise-warnings, so the diff here is just the one commit. GitHub will retarget this to main automatically once #618 merges.

It has to be stacked: filterwarnings = ['error'] only passes once #618's two fixes are in, since both the ResourceWarning and the PytestCollectionWarning would otherwise fail the suite.

Why

pyproject.toml had no [tool.pytest.ini_options] section, so pytest ran with Python's default warning filters — which ignore ResourceWarning and ImportWarning outright. That's precisely why the ResourceWarning fixed in #618 was invisible in the GitHub Actions jobs while showing up in the EDM runs, which go through etstool.py and use python -W default -m unittest discover. Without this, the next warning we introduce is equally invisible.

The one escape

TraitsUI emits an ImportWarning at import time when numpy is missing, and numpy isn't in Envisage's dependency closure — the test-with-pypi jobs install only pytest, pyside6 and .. So it fires there:

environment pytest with filterwarnings = ['error'] and no escape
with numpy 178 passed
without numpy 3 collection errors, all ImportWarning: ArrayEditor is not available due to missing numpy

The filter matches on the message rather than ignoring ImportWarning as a category, so we keep the protection for any other import warning:

'ignore:ArrayEditor is not available:ImportWarning',

Installing numpy in the test jobs would also have worked, and would exercise more of TraitsUI — but it adds a test-only dependency to the matrix to suppress a warning that isn't telling us anything about Envisage.

Scope

This governs pytest only. The EDM jobs stay at -W default: unittest has no config-file equivalent, and making that path strict too would mean putting -W error plus escapes into etstool.py, which is more coupling than the benefit justifies. The pytest jobs cover 3.8 and 3.10–3.14, which is enough to catch a new warning.

Verification

  • Full suite passes on Python 3.8, 3.10, 3.12 and 3.14, with and without numpy: 178 passed each.
  • Canary check: a test doing warnings.warn("canary", UserWarning) now fails, confirming the config is in effect rather than silently ignored.
  • black --check, isort --check, flake8 clean.

🤖 Generated with Claude Code

mdickinson and others added 9 commits August 5, 2026 13:03
Three independent sources of noise, all visible when the suite is run the
way etstool.py runs it (`python -W default -m unittest discover`), and all
hidden by the bare `python -m pytest` used in the GitHub Actions jobs.

* `HTTPResourceProtocol.file` caught the `HTTPError` raised by `urlopen`
  and discarded it without closing. `urllib.response.addbase` subclasses
  `tempfile._TemporaryFileWrapper`, so an unclosed `HTTPError` emits a
  `ResourceWarning` when collected -- and in real use leaves the
  connection underlying the error response to the garbage collector.
  Close it, guarding against Python 3.8 and 3.9, where an `HTTPError`
  built with `fp=None` doesn't initialize its file-like base classes and
  `close()` raises `KeyError`. (3.8 is in the test matrix.)

* `envisage.tests.test_plugin.TestPlugin` matched pytest's `Test*`
  collection pattern but has an `__init__`, so collecting it raised a
  `PytestCollectionWarning`. Renamed to `SamplePlugin`.

* Eleven log records at `WARNING` and above reached `stderr`, from two
  call sites. Both now have tests that assert them, which is the first
  coverage either has had; the tests that provoked them only incidentally
  suppress them instead.

The four `envisage.ui.tasks.tasks` warnings from the `TasksApplication`
tests are left alone -- those come from building a `TasksApplication`
without the `TasksPlugin` that registers the extension point, which looks
like a genuine wart rather than test noise, and is worth deciding on
separately.

Part of #615.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review feedback: the comment only needs to explain why fp is checked
rather than calling close() unconditionally. The rationale for closing
at all is in the changelog entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review feedback:

* The NoSuchResourceError raised by HTTPResourceProtocol.file reported
  the address with a "http:://" prefix. Sole occurrence, and no test
  asserts on the message.

* Drop the comment above the assertLogs block in
  test_exception_in_trait_contribution; the assertions say the same thing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reverts the comment to its state on main, byte for byte, so it no longer
appears in the diff at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Envisage's pytest runs used the default warning filters, which hide
ResourceWarning and ImportWarning entirely. That's why the ResourceWarning
fixed in the previous PR was invisible in the GitHub Actions jobs while
showing up in the EDM runs, which go through etstool.py and use
`python -W default -m unittest discover`.

Adding `filterwarnings = ['error']` means a newly-introduced warning fails
the suite rather than passing unnoticed.

One escape is needed: TraitsUI emits an ImportWarning at import time when
numpy isn't installed, and numpy isn't in Envisage's dependency closure, so
it fires in the test-with-pypi jobs (3 collection errors without it). The
filter is matched on the message rather than ignoring ImportWarning
wholesale.

This only governs pytest; the EDM jobs stay at `-W default`, since unittest
has no equivalent config file.

Verified on Python 3.8, 3.10, 3.12 and 3.14, with and without numpy
installed, plus a canary test confirming a stray warning now fails.

Part of #615.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Base automatically changed from fix/615-test-noise-warnings to main August 5, 2026 12:39
mdickinson and others added 2 commits August 5, 2026 13:40
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes pytest treat all warnings as errors to prevent new warnings from passing unnoticed in CI, while allowing a single known TraitsUI ImportWarning (missing numpy) to be ignored.

Changes:

  • Add [tool.pytest.ini_options] to pyproject.toml with filterwarnings = ['error', ...] and one targeted ignore for the TraitsUI ArrayEditor numpy warning.
  • Document the new “warnings as errors under pytest” behavior in the changelog.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
pyproject.toml Configure pytest to error on warnings, with one message-specific ignore to keep the non-numpy test env collecting.
CHANGES.rst Note the new pytest warning strictness in the “Tests” section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mdickinson
mdickinson merged commit 29a035c into main Aug 5, 2026
20 checks passed
@mdickinson
mdickinson deleted the fix/615-pytest-filterwarnings branch August 5, 2026 12:47
mdickinson added a commit that referenced this pull request Aug 5, 2026
A review of the 8.0.0 changelog against `7.0.4..main`. 7.0.4 was tagged
directly on main — `0c02d0b` is both the tag commit and the merge-base —
so
that range is exact. It holds 30 non-merge commits: 11 touching the
library or
packaging, 19 CI and infrastructure.

## The library side was already complete

All eight substantive changes had entries, and both deprecations 7.0.0
promised
to remove in 8.0 (#540, #544) are removed and listed. No
`DeprecationWarning`
remains anywhere in the library, so nothing is left dangling for 9.0.

## References now point at PRs, not issues

The two `Fixes` entries for the `HTTPError` leak cited #615, whose title
is
"Clean up test noise from warnings and WARNING-level log messages". The
leak was
found *via* that cleanup, but a reader following the reference lands on
a
test-hygiene issue rather than a resource-leak fix.

Rather than fix those two in isolation, every reference in the section
now
points at the PR that made the change: #574→#613, #616→#622, #548→#599,
#547→#612, #614→#617, and #615→#618/#619/#621 as appropriate. This also
supplies
the reference the `pkg_resources` entry was missing — #600, the PR,
rather than
#515, the issue.

Two notes:

- The "no longer writes `WARNING`-level log messages" entry cites
**both** #618
and #621. #618 deliberately deferred the four `envisage.ui.tasks.tasks`
records from the `TasksApplication` tests to a follow-up, so the claim
only
  became true once #621 landed.
- Sections for 7.0.4 and earlier are untouched; they already cite PRs.

## CI and build work added

Nineteen PRs had no entry at all, in contrast with 7.0.4, which listed
its
action bumps and copyright-header updates under `Build`. Five new
entries cover
them, aggregated where the work was one theme:

| Entry | PRs |
| --- | --- |
| Trusted publishing (OIDC) and SHA-pinned actions | #604, #608 |
| Copyright end years, and the year-rollover fix to the style check |
#596, #597, #607 |
| GitHub Action bumps | #591, #592, #593, #594, #595, #602, #603, #605,
#606, #609, #610 |
| Pre-release Pythons in the CI matrix | #601, #624 |
| `AGENTS.md` | #598, #626 |

Trusted publishing changes how releases are authenticated, so it seemed
worth
naming rather than folding into the general CI churn.

## Judgment calls worth a second opinion

**The CI matrix entry names no versions.** It reads "now includes
pre-release
Python versions" rather than "3.14, and 3.15 pre-releases". Naming them
is more
informative, but in a section whose `Removals` entry deliberately gives
no
upper bound on supported Pythons, a `Build` line ending in "3.15"
invites
exactly the reading that bound was left out to avoid. Easy to change if
you'd
rather have the numbers.

**No upper bound on Python versions.** Considered and rejected: it means
churn
on every Python release, and forwards compatibility is the reasonable
default
expectation for a package that runs warning-free on the current version.

**Deliberate omissions.** #626's other cleanups — an `IServiceRegistry`
docstring, the legacy examples README, dead Traits<6.0 test code —
change
nothing for users. There's no `Documentation` section because the doc
edits in
#599 are just the egg-manager material going away with the classes,
already
covered by the `Removals` entry. And `CorePlugin` gaining its own
`stop()` to
unregister service offers is an implementation move; the #617 entry
already
notes that `PluginActivator` no longer calls
`register_services`/`unregister_services`, which is the part a
downstream
`Plugin` subclass would notice.

## Left to the release PR

The `Released:` date line, an introductory paragraph, a "Thanks to:"
list, and
the `version = '7.0.4'` still in `pyproject.toml`. The date isn't
settled yet.

## Verification

`CHANGES.rst` parses under docutils with no warnings at report level 2
or above,
and all lines stay within 79 characters. Changelog only — no code
touched.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
mdickinson added a commit that referenced this pull request Aug 6, 2026
…ck (#628)

Towards #630, covering all of its checklist but the last item:

- [x] Remove `etstool.py` in favour of direct command invocations
- [x] Use `uv` as the primary driver for development and CI tasks
- [x] Switch from `setuptools` to `uv_build` for the build backend;
remove
      MANIFEST.in
- [x] Add a tracked `uv.lock`
- [ ] Consider switching to src/ layout — left to a follow-up, since
moving
      every file would swamp the rest of this PR

## What changes

- **`etstool.py` goes.** It was undocumented for human use, and CI used
2 of its
9 commands — always through a bootstrap Python whose only purpose was to
run
  it.
- **uv drives everything except EDM.** The documentation, style and test
  requirements become dependency groups in `pyproject.toml`, replacing
`docs/requirements.txt` and `style-requirements.txt`. `test-with-pypi`
drops
  `actions/setup-python` entirely.
- **`uv.lock` is tracked** and installed with `--locked`, so an upstream
release
can no longer break CI, and dependabot's `uv` ecosystem keeps it
current.
- **`uv_build` replaces `setuptools`.** Package data is picked up
automatically,
so `MANIFEST.in` and the five `[tool.setuptools.package-data]` entries
go.
- **Platform coverage shifts.** EDM drops macOS — no macOS installer is
published for 4.1.4 or 4.1.5 — and moves to 4.1.5, while
`test-with-pypi`
  gains macOS and Windows, so it now runs 3 OSes × 6 Pythons.
- **`ets-from-source` moves off EDM** to `uv add --branch main`, which
re-locks
  only the four ETS packages and leaves pytest at its pinned version.
- **A toolkit-free smoke test** (`test-without-toolkit`, Ubuntu / 3.14)
checks
that Envisage doesn't accidentally require a GUI toolkit and that the
tests
needing one skip rather than fail: 171 passed, 7 skipped. The EDM
workflow
covers this too, but shouldn't be the only thing that does. `pytest` and
`pyside6` move into `test` and `gui` groups so the job can ask for one
without
the other; `dev` includes both, so `uv run` is unchanged everywhere
else.

## Worth a look

**Artifact contents.** The wheel was diffed entry-by-entry against the
released
8.0.0 wheel on PyPI: identical bar two files, gaining
`envisage/ui/tasks/tests/data/README.md` (excluded before by a
`data/*.pkl`
glob) and losing setuptools' `top_level.txt`, which is in no packaging
standard.
`METADATA` is unchanged modulo field order. The sdist loses
`CHANGES.rst` —
nothing consumes it, since no `docs/` directory has ever shipped — and
gains a
`pyproject.toml.orig`, which is deliberate uv_build behaviour
([astral-sh/uv#18741](astral-sh/uv#18741)).

**The EDM jobs now run pytest rather than unittest**, so `filterwarnings
= error`
(#619) applies to them for the first time.

**Read the Docs was already broken, and this should fix it.**
`.readthedocs.yaml`
pinned Python 3.11 while #601 raised Sphinx to 9.1.0, which needs 3.12+,
so the
install step failed — see #631. Deliberately no closing keyword: better
to
confirm a green RTD build after merge.

**`publish-on-pypi` builds with `uv build` but still uploads with
`pypa/gh-action-pypi-publish`.** `uv publish` uploads PEP 740
attestations but
does not generate them, so using it would have published 8.1.0 without
the
provenance 8.0.0 has (PyPI's provenance endpoint: 200 for 8.0.0, 404 for
7.0.4).
The upload step is therefore byte-identical to `main`'s. `twine check
--strict`
is deliberately not restored — a weak check we're happy to lose.

**Four things PR CI cannot exercise**, since GitHub reads them from the
default
branch: `ets-from-source` and `publish-on-pypi` (schedule/dispatch
only), the
dependabot config, and the Read the Docs config. I dispatched
`ets-from-source`
against this branch — 3/3 green, installing from the ETS main branches.

*This PR was created with the assistance of an AI coding agent.*

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants