Skip to content

Release 9.2.0: Python 3.15 support, iterable_match_keys - #669

Merged
seperman merged 43 commits into
masterfrom
dev
Nov 18, 2026
Merged

seperman merged 43 commits into
masterfrom
dev

Conversation

@seperman

Copy link
Copy Markdown
Member

Release 9.2.0: merges dev into master.

New features

  • iterable_match_keys: match list items by key per path, e.g. {"root['orders']": "order_id", "root['orders'][*]['lines']": ("sku", "warehouse")}. Paths use the exclude_paths glob syntax; a key spec is a key name, a tuple of key names, or a callable. Matching is O(n + m), inserted/removed/reordered records are reported per record at real list indexes, and Delta round-trips (Declarative key-based matching of list items per path (iterable_match_keys) #664, Add iterable_match_keys for key-based list matching per path (#664), fix #665 #666).
    • Duplicate keys pair by occurrence; items without a key pair in order.
    • 1 and 1.0 are the same key; booleans and strings never match numbers.
    • With ignore_order, keyed lists are still paired by key, without reporting moves.
    • Subtree multiprocessing falls back to serial when iterable_match_keys is used.
    • New docs page: docs/iterable_match_keys.rst.

Python support

  • Added Python 3.15 support (Add Python 3.15 support and drop Python 3.10 #661).
  • Dropped Python 3.10, which reached end-of-life in October 2026. Supported versions are now 3.11–3.15.
    • cachebox>=6.1.2,<7 (first release with cp315 wheels); numpy ~=2.5.4 on 3.12+, ~=2.4.6 on 3.11; pandas ~=3.1.1.
    • Build backend allows flit_core<5.
    • Dropped the tomli dependency; stdlib tomllib is always used.
    • CI matrix and nox sessions updated to 3.11–3.15.

Bug fixes

Dependencies

Tooling and docs

SAY-5 and others added 30 commits May 23, 2026 22:17
number_to_string rebuilt complex numbers via "{real}+{imag}j".format(...).
When the imaginary part is negative this produced an invalid string like
"1.0+-1.0j", and the complex() round-trip raised ValueError -- so
number_to_string(1-1j) and, by extension, DeepHash(1-1j, significant_digits=3)
crashed. Join using the imaginary part's own sign instead.
- model.py: "comparision" -> "comparison" (DiffLevel.copy docstring)
- diff.py: "revisted" -> "revisited" (TODO comment)
- docstrings/view.rst: "representaion" -> "representation" (x2)
- docstrings/deephash_doc.rst: "agains" -> "against"
DeepDiff([2], [2.0]) returned {} even though the scalar equivalent
DeepDiff(2, 2.0) and the dict equivalent DeepDiff({'a': 2}, {'a': 2.0})
both correctly report an int -> float type_change.

Root cause: for ordered iterables whose items are all basic-hashable,
DeepDiff diffs the two sequences with difflib.SequenceMatcher. Because
2 == 2.0 (and hash(2) == hash(2.0)), SequenceMatcher classifies the pair
as an 'equal' opcode, so the elements are never handed to _diff() -- the
only place the type-change check (get_type(t1) != get_type(t2)) lives.
The whole difference silently disappeared. The same happened for other
equal-but-differently-typed pairs such as [1] vs [Decimal(1)] and
[True] vs [1], and for changes buried among unchanged items, e.g.
[1, 2, 3] vs [1, 2.0, 3].

Fix: in the 'equal' opcode branch of _diff_ordered_iterable_by_difflib,
recurse into _diff() for any aligned pair whose types differ. Routing
through _diff() reuses the existing type-change logic, so all options are
honored automatically -- ignore_numeric_type_changes=True and
ignore_type_in_groups still suppress the report, and genuinely identical
items still produce nothing.

Added regression tests (fail before this change, pass after):
- [2] vs [2.0] reports the int -> float type_change and matches the
  scalar result
- the change is detected among surrounding unchanged items
- ignore_numeric_type_changes=True still yields {}

Full test suite: 1251 passed, unchanged from baseline.

Fixes #605
The `deep` console script is always installed, but its dependencies
(`click`, and `pyyaml` for YAML files) live in the optional `cli` extra.
On a default install, running `deep` imported `deepdiff.commands`, which
imports `click` at module load, so the command died with a bare
`ModuleNotFoundError: No module named 'click'`.

Point the entry point at a thin wrapper that checks for `click` first and
exits with an actionable message (`pip install deepdiff[cli]`) instead of a
traceback, keeping the CLI dependencies optional as intended.

Fixes #594
`_dict_from_slots` unmangled each slot name using the concrete instance
type (`type(object).__name__`). A mangled slot such as `__id` is stored under
the name of the class that *declared* it (e.g. `_Parent__id`), so for a
subclass instance the lookup used the wrong name (`_Child__id`), `hasattr`
returned False, and the parent-defined slot was silently dropped.

Pair each slot with the class in the MRO that declared it and unmangle using
that class's name.

Closes #506
Bumps [soupsieve](https://github.com/facelessuser/soupsieve) from 2.8.3 to 2.8.4.
- [Release notes](https://github.com/facelessuser/soupsieve/releases)
- [Commits](facelessuser/soupsieve@2.8.3...2.8.4)

---
updated-dependencies:
- dependency-name: soupsieve
  dependency-version: 2.8.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Delta serializes each iterable Opcode NamedTuple, then rebuilds it in
the JSON deserializer with Opcode(**op_code), which assumes a mapping.
That only holds under orjson: its default callback runs JSON_CONVERTOR
and emits Opcodes as dicts. The stdlib json fallback (used on any
install without the optional orjson extra) encodes NamedTuples natively
as positional arrays, so Opcode(**op_code) raises TypeError and the
documented round-trip Delta(dump, deserializer=json_loads) crashes for
any list diff that goes through difflib opcodes (reorders / duplicates).

Accept both shapes on deserialization: mapping -> Opcode(**op_code),
array -> Opcode(*op_code) (fields are in NamedTuple order). Added a
regression test that forces the builtin-json path via
force_use_builtin_json=True so it guards the array shape even where
orjson is installed.
fix: skip time truncation for date objects in truncate_datetime
…ive-imaginary

Handle negative imaginary parts in number_to_string (#551)
Fix missing type changes for equal-comparing list items (#605)
fix(cli): give actionable error when 'deep' runs without cli extra (#594)
Fix mangled __slots__ attribute lost for subclass instances (#506)
Adds the two cases from the original fix that are not covered on dev:
- 'e' notation with negative imaginary part (1.0e+0-1.0e+0j)
- negative real with negative imaginary part (-1.0-1.0j)

The code fix itself is already on dev via #598; this keeps only the
extra test coverage. Targets dev, no CHANGELOG edit.
Replace the has_time check with a positive isinstance check so
truncate_datetime only applies to datetime and time objects. This fixes
the AttributeError when diffing timedelta values (timedelta has no
.replace()) and avoids the same crash for np.datetime64 in DeepHash.

Adds regression tests for timedelta and np.datetime64. Targets dev, no
CHANGELOG edit.
Bumps [certifi](https://github.com/certifi/python-certifi) from 2026.4.22 to 2026.10.18.
- [Commits](certifi/python-certifi@2026.04.22...2026.10.18)

---
updated-dependencies:
- dependency-name: certifi
  dependency-version: 2026.10.18
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Python 3.10 reached end-of-life in October 2026. Supported versions are
now 3.11 through 3.15.

- Bump cachebox to >=6.1.2,<7 (first release with cp315 wheels)
- numpy ~=2.5.4 on 3.12+, ~=2.4.6 on 3.11; pandas ~=3.1.1 everywhere
- Allow flit_core <5 as build backend
- Upgrade dev/test/docs/static dependencies (polars 2.1, dateutil 4.0,
  Sphinx 9.0, pytest 9.2, coverage 7.17, pydantic 2.14, etc.)
- Drop tomli dev dependency; always use stdlib tomllib
- Remove pre-3.11 enum hash branch from tests
- Update CI matrix and nox sessions to 3.11-3.15, run lint/coverage on 3.15
Add Python 3.15 support and drop Python 3.10
Replace hardcoded venv activation commands with guidance for detecting the
contributor's active environment (VIRTUAL_ENV, direnv, .venv, uv). Remove
stray references from another project, and gitignore CLAUDE.local.md for
personal, machine-specific instructions.
- scripts/pr_sandbox/sandbox-pr.sh: fetch a PR without checking it out, show
  changed/risky files with control characters stripped, then run only its
  deepdiff/ and tests/ in the sandbox (deps and pytest config from the
  trusted checkout).
- scripts/pr_sandbox/run-sandboxed.sh: bubblewrap sandbox (no network,
  read-only system and venv, throwaway $HOME with canary credentials and
  canary env vars) under strace, plus a PEP 578 audit hook.
- scripts/pr_sandbox/analyze.py: turns strace/audit logs and a $HOME
  before/after diff into ranked findings; exits 2 on suspicious behaviour.
  A clean run of the current suite yields no MEDIUM+ findings.
- CI: pr-sandbox-monitor runs the same on every PR; pr-risk-label
  (pull_request_target, no PR checkout) labels PRs touching build config,
  conftest, pickles, CI, agent config or the sandbox tooling.
- CLAUDE.md: treat PR content as untrusted data when reviewing.
The work dir from mktemp is 0700, so chgrp on report/ alone is not enough
for another user to read it. When the group exists and the user is in it,
open up the work dir (traverse/list) plus report/ and audit/; src/ and the
fake home stay private.
…r $HOME

With a uv-managed Python under the user's home, CPython's startup search
(~/lib/python315.zip etc.) was reported as MEDIUM home-read. Ignore such
probes only under directories that contain the interpreter, so reads like
~/.local/lib/python3.15 are still reported.
…tes it

The PR that introduces the monitor has no scripts/pr_sandbox on its base
commit, so the job failed with 'No such file'. Use the PR's copy in that
case, with a warning that the verdict is not trustworthy.
- make pr-sandbox PR=N: sandbox-pr.sh with a per-PR work dir under /tmp.
- make pr-review PR=N: start the user's zsh claude command in plan mode
  with review-settings.json (auto/bypass disabled, risky commands ask,
  PreToolUse guard that blocks tool use in auto/bypass mode) and a
  review prompt pointing at the latest sandbox report.
- make pr-clean: delete fetched PR refs and sandbox work dirs.
Plan mode prompted for every read of the report in /tmp. Use dontAsk with
the sandbox work dir added via --add-dir: reads, read-only commands and
gh pr view/diff/checks run without prompts, anything else is refused
automatically. Deny code execution, network, edits and PR-mutating gh
commands outright. Tell the reviewer to rely on analyze.py's findings
instead of re-deriving them from strace.log.
seperman and others added 13 commits November 16, 2026 11:54
Sandboxed, monitored test runs for untrusted PRs
DeepDiff(t1, t2, iterable_match_keys={path: key_spec}) pairs the items
of the selected lists by key instead of by position. Paths use the
exclude_paths glob syntax; a key spec is a key name, a tuple of key
names, or a callable. Matching runs inside _get_matching_pairs in
O(n + m), so results keep real list indexes and Delta round-trips.

- Duplicate keys pair by occurrence; items without a key pair in order.
- 1 and 1.0 are the same key; booleans and strings never match numbers.
- Records with different keys at the same index stay added + removed
  instead of being merged into values_changed.
- With ignore_order, keyed lists are still paired by key, without
  reporting moves.
- Lists inside a moved record skip difflib, and Delta no longer swaps
  paths for items of opcode-diffed lists, so mixed diffs round-trip.
- Subtree multiprocessing falls back to serial, since workers can't
  see absolute paths.

Also fixes iterable_compare_func dropping an added item whose content
equals an already matched item: matched t2 items are now tracked by
index, not content hash.

Closes #664
Closes #665
Add iterable_match_keys for key-based list matching per path (#664), fix #665
…trip

Fix Delta JSON round-trip crash when orjson is not installed.
Thanks.
…ry-number-to-string

test: handle negative imaginary parts in number_to_string
…imedelta

  datetime_normalize now truncates only datetime.datetime and datetime.time
  objects. It used to try every type except a plain date, so
  truncate_datetime raised AttributeError for timedelta in DeepDiff and for
  np.datetime64 in DeepHash. Neither type has a .replace() that takes time
  fields.

  Behaviour change: truncate_datetime no longer touches timedelta or
  np.datetime64 values, which are compared exactly. This matches existing
  behaviour elsewhere: DeepHash already hashed timedelta untruncated, and
  DeepDiff never truncated np.datetime64.

  Adds regression tests for timedelta (DeepDiff) and np.datetime64 (DeepHash).
truncate_datetime is intended to apply only to datetime and time objects.
The previous np.datetime64 test used date-only values, where truncation is
a no-op, and compared DeepHash objects, which always differ for different
keys. Use values with a time component that differ only below the
truncation unit and compare per-object hashes. Add tests for sub-second
timedelta differences and for combining truncate_datetime with
ignore_order=True.
@seperman seperman changed the title Release 9.2.0 Release 9.2.0: Python 3.15 support, iterable_match_keys Nov 18, 2026
@codecov

codecov Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.32402% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.21%. Comparing base (f4e58b5) to head (3ec2b9a).
⚠️ Report is 48 commits behind head on master.

Files with missing lines Patch % Lines
deepdiff/cli.py 75.00% 2 Missing ⚠️
deepdiff/diff.py 97.77% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #669      +/-   ##
==========================================
+ Coverage   95.10%   95.21%   +0.11%     
==========================================
  Files          17       19       +2     
  Lines        4900     5055     +155     
==========================================
+ Hits         4660     4813     +153     
- Misses        240      242       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@seperman
seperman merged commit a444192 into master Nov 18, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants