Add Python object reference tracking #752
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pablogsal
force-pushed
the
pyrefs2
branch
7 times, most recently
from
32d2317 to
d748e4d
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #752 +/- ##
==========================================
- Coverage 92.01% 90.66% -1.36%
==========================================
Files 95 98 +3
Lines 11940 12383 +443
Branches 413 420 +7
==========================================
+ Hits 10987 11227 +240
- Misses 953 1156 +203
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Tests require 3.13.3 at least because before there is a bug that causes some objects to be identified wrongly. cibuildwheel only has this when pypa/cibuildwheel#2359 passes |
pablogsal
force-pushed
the
pyrefs2
branch
5 times, most recently
from
6b081eb to
a03e9de
Compare
|
I just released cibuildwheel with 3.13.3 support! |
godlygeek
force-pushed
the
pyrefs2
branch
2 times, most recently
from
227cf37 to
9de0d3b
Compare
godlygeek
requested changes
Aug 19, 2025
|
|
||
| # THEN | ||
| # No surviving objects should be recorded when reference_tracking is disabled | ||
| assert len(tracker.get_surviving_objects()) == 0 |
There was a problem hiding this comment.
I think we should probably raise an exception if this is called when reference tracking was never enabled. It seems like user error...
Also, this test doesn't prove what it says it does. It would pass even if reference tracking were enabled, because no objects leak (thanks to the del and gc.collect() calls)
pablogsal
force-pushed
the
pyrefs2
branch
3 times, most recently
from
9b3c024 to
263bedf
Compare
This feature adds the capability to track Python objects throughout their lifecycle in memray. The implementation introduces a reference tracking system that records PyObject creation and destruction events, storing them as TrackedObject records in the memray capture file with associated stack traces. The Tracker class now accepts a reference_tracking flag to enable this functionality and provides a get_surviving_objects() method to retrieve live objects at the end of tracking. Object tracking works across threads and supports native, Python, and hybrid stack traces, providing insights into Python object lifetimes. This enhancement extends memray beyond memory allocation tracking to give developers a powerful tool for understanding object lifecycles and identifying potential reference leaks. With this addition, memray can now correlate memory allocations with the Python objects that trigger them, enabling more precise debugging of memory usage patterns. The feature is particularly useful for identifying objects that persist longer than expected or detecting unexpected object creation patterns. By tracking both memory allocations and object references, memray provides a more complete picture of a program's memory behavior, helping developers optimize memory usage and resolve leaks more effectively. This implementation leverages the new Python reference tracing API that was introduced in CPython 3.13. Signed-off-by: Pablo Galindo <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This feature adds the capability to track Python objects throughout
their lifecycle in memray. The implementation introduces a reference
tracking system that records PyObject creation and destruction events,
storing them as TrackedObject records in the memray capture file with
associated stack traces. The Tracker class now accepts a
reference_tracking flag to enable this functionality and provides a
get_surviving_objects() method to retrieve live objects at the end of
tracking.
Object tracking works across threads and supports native, Python, and
hybrid stack traces, providing insights into Python object
lifetimes. This enhancement extends memray beyond memory allocation
tracking to give developers a powerful tool for understanding object
lifecycles and identifying potential reference leaks.
With this addition, memray can now correlate memory allocations with the
Python objects that trigger them, enabling more precise debugging of
memory usage patterns. The feature is particularly useful for
identifying objects that persist longer than expected or detecting
unexpected object creation patterns. By tracking both memory allocations
and object references, memray provides a more complete picture of a
program's memory behavior, helping developers optimize memory usage and
resolve leaks more effectively.
This implementation leverages the new Python reference tracing API that
was introduced in CPython 3.13.
Signed-off-by: Pablo Galindo [email protected]
Issue number of the reported bug or feature request: #
Describe your changes
A clear and concise description of the changes you have made.
Testing performed
Describe the testing you have performed to ensure that the bug has been addressed, or that the new feature works as planned.
Additional context
Add any other context about your contribution here.