Skip to content

BUG: Reuse a single Selection during table-format HDFStore.select reads#64430

Closed
Copilot wants to merge 2 commits intomainfrom
copilot/investigate-issue-22082
Closed

BUG: Reuse a single Selection during table-format HDFStore.select reads#64430
Copilot wants to merge 2 commits intomainfrom
copilot/investigate-issue-22082

Conversation

Copy link

Copilot AI commented Mar 6, 2026

Repeated reads from table-format HDF keys were constructing selection state twice per select call in the read path. This added unnecessary allocation/retention pressure and is consistent with the long-standing memory-growth behavior reported for repeated table reads.

  • Read-path change

    • AppendableFrameTable.read now creates one Selection and reuses it for both:
      • axis extraction (_read_axes)
      • post-read filtering/reindexing (process_axes)
    • _read_axes was updated to accept an existing Selection instead of rebuilding from where/start/stop.
  • API/internal contract cleanup

    • Table._read_axes signature changed from (where, start, stop) to (selection), making ownership of selection state explicit and avoiding duplicate construction.
  • Regression coverage

    • Added a focused test in pandas/tests/io/pytables/test_select.py that monkeypatches Selection.__init__ and asserts only one initialization per temp_hdfstore.select("df"), while verifying result equality.
# before
result = self._read_axes(where=where, start=start, stop=stop)
selection = Selection(self, where=where, start=start, stop=stop)
df = self.process_axes(df, selection=selection, columns=columns)

# after
selection = Selection(self, where=where, start=start, stop=stop)
result = self._read_axes(selection=selection)
df = self.process_axes(df, selection=selection, columns=columns)

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: jbrockmendel <8078968+jbrockmendel@users.noreply.github.com>
Copilot AI changed the title [WIP] Investigate issue #22082 BUG: Reuse a single Selection in HDFStore.select table reads Mar 6, 2026
Copilot AI changed the title BUG: Reuse a single Selection in HDFStore.select table reads BUG: Reuse a single Selection during table-format HDFStore.select reads Mar 6, 2026
@jbrockmendel jbrockmendel deleted the copilot/investigate-issue-22082 branch March 6, 2026 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants