-
Notifications
You must be signed in to change notification settings - Fork 22
Update accessors to store weak reference to data #894
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
thehomebrewnerd
commented
May 5, 2021
- Update accessors to store weak reference to data
- Closes Use weak references to refer to the original data in the accessor #880
Codecov Report
@@ Coverage Diff @@
## main #894 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 44 44
Lines 6878 6887 +9
=========================================
+ Hits 6878 6887 +9
Continue to review full report at Codecov.
|
@thehomebrewnerd I think this implementation solves the original issue! I visualized the object graph with this branch compared to the latest release and the accessor no longer has a reference to the original dataframe: This branch0.3.0 releaseThe memory footprint looks a lot better in evalml. Here is the memory consumption of automl search on the NYC taxi dataset (1.5 million rows) This branch0.3.0 releaseI've also run our full set of benchmark datasets and we see similar improvements in memory without sacrificing accuracy/performance. So thank you! 👏 |
Co-authored-by: Gaurav Sheni <[email protected]>
Co-authored-by: Gaurav Sheni <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! One comment about something that I was a little confused about, but feel free to ignore if you think adding a second example will be redundant!