Skip to content

[ADD] Post-Hoc ensemble fitting #260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from

Conversation

ravinkohli
Copy link
Contributor

Adds a fit ensemble method enabling users to post hoc build an ensemble after search is done. This feature allows the user to run a search and sequentially build an ensemble later. This PR also adds tests for the functionality as well as updating traditional_run_history.json.

@ravinkohli ravinkohli changed the title Add tests and fit ensemble method [ADD] Post-Hoc ensemble fitting Jun 18, 2021
@codecov
Copy link

codecov bot commented Jun 18, 2021

Codecov Report

Merging #260 (4d55a7d) into development (8237f2c) will increase coverage by 0.15%.
The diff coverage is 78.43%.

Impacted file tree graph

@@               Coverage Diff               @@
##           development     #260      +/-   ##
===============================================
+ Coverage        81.76%   81.91%   +0.15%     
===============================================
  Files              151      151              
  Lines             8641     8677      +36     
  Branches          1326     1333       +7     
===============================================
+ Hits              7065     7108      +43     
+ Misses            1104     1096       -8     
- Partials           472      473       +1     
Impacted Files Coverage Δ
autoPyTorch/api/base_task.py 84.02% <78.00%> (-0.63%) ⬇️
autoPyTorch/datasets/base_dataset.py 80.48% <100.00%> (+0.48%) ⬆️
...ipeline/components/setup/network_backbone/utils.py 87.21% <0.00%> (-1.51%) ⬇️
autoPyTorch/ensemble/ensemble_builder.py 73.58% <0.00%> (-0.63%) ⬇️
autoPyTorch/utils/stopwatch.py 70.10% <0.00%> (+1.03%) ⬆️
autoPyTorch/ensemble/ensemble_selection.py 96.84% <0.00%> (+1.05%) ⬆️
...Torch/pipeline/components/training/metrics/base.py 68.65% <0.00%> (+2.98%) ⬆️
autoPyTorch/optimizer/smbo.py 83.19% <0.00%> (+3.36%) ⬆️
...reprocessing/feature_preprocessing/TruncatedSVD.py 93.10% <0.00%> (+3.44%) ⬆️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8237f2c...4d55a7d. Read the comment docs.

Copy link
Collaborator

@nabenabe0928 nabenabe0928 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.
I left some comments.

Copy link
Contributor

@franchuterivera franchuterivera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the changes. They look good but I am requesting mainly two things to make it even better:

Add an example where we fit the API sequentially. First search is called with 1 core, then fit the ensemble

Then add some extra checks as mentioned to gain more confidence on the chance.

Copy link
Collaborator

@nabenabe0928 nabenabe0928 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find large issues except what Francisco already mentioned.
I will wait for another major change.

@ravinkohli ravinkohli marked this pull request as draft September 30, 2021 16:37
@ravinkohli ravinkohli closed this Oct 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants