Skip to content

Cocktail hotfixes #245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Jun 3, 2021
Merged

Cocktail hotfixes #245

merged 6 commits into from
Jun 3, 2021

Conversation

ArlindKadra
Copy link

  1. Updated the dropout implementation, since the dropout rates were not calculated correctly.
  2. Fixed the dropout implementation.
  3. Removed the flatten layer in NoHead.
  4. Updated the normalization procedure in the backbones since the addition of conditional skip connection broke things.

ravinkohli
ravinkohli suggested changes Jun 3, 2021
View reviewed changes
Copy link
Contributor

@ravinkohli ravinkohli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for the PR. After merging this PR, we should see a considerable improvement over the current results. I have requested a few small changes. One of them is, maybe you can fix the skip connection bug in the ResBlock module which we discussed on mattermost.

ravinkohli
ravinkohli reviewed Jun 3, 2021
View reviewed changes
autoPyTorch/pipeline/components/setup/network_backbone/ResNetBackbone.py Outdated
# we will never apply the start_norm for the first block,
# which is why we should account for this case.
if not self.config['use_skip_connection']:
layers.append(nn.BatchNorm1d(in_features))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this not have a condition to check if we should use batch norm?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ups, I missed it. Very good catch.

@ArlindKadra ArlindKadra requested a review from ravinkohli June 3, 2021 13:03
ArlindKadra
ArlindKadra commented Jun 3, 2021
View reviewed changes
autoPyTorch/pipeline/components/setup/network_backbone/ShapedResNetBackbone.py Outdated
)[:-1]

dropout_shape = [
dropout / 1000 * self.config["max_dropout"] for dropout in dropout_shape
dropout for dropout in dropout_shape
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I see it, this is actually the same list.

@ravinkohli ravinkohli merged commit ee685a9 into refactor_development_regularization_cocktails Jun 3, 2021
github-actions bot pushed a commit that referenced this pull request Jun 3, 2021
Arlind Kadra: Cocktail hotfixes (#245)
fdbdaf1
@ArlindKadra ArlindKadra mentioned this pull request Jun 7, 2021
Merged
@ravinkohli ravinkohli deleted the cocktail_hotfixes branch October 22, 2021 09:37
ravinkohli pushed a commit that referenced this pull request Dec 8, 2021
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
fc5f2d6 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments
ravinkohli pushed a commit that referenced this pull request Dec 8, 2021
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
f0f2bf6 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
ravinkohli pushed a commit that referenced this pull request Dec 21, 2021
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
107750a 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
ravinkohli pushed a commit that referenced this pull request Jan 24, 2022
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
23a3b6c 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
ravinkohli pushed a commit that referenced this pull request Jan 28, 2022
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
09ad0d7 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
ravinkohli pushed a commit that referenced this pull request Feb 28, 2022
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
be411a2 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
ravinkohli pushed a commit that referenced this pull request Feb 28, 2022
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
298fc7c 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
ravinkohli pushed a commit that referenced this pull request Mar 9, 2022
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
88b21b7 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
ravinkohli pushed a commit to ravinkohli/Auto-PyTorch that referenced this pull request Apr 12, 2022
@ArlindKadra @ravinkohli
Cocktail hotfixes (automl#245)
a6eab14 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
ravinkohli pushed a commit that referenced this pull request Jul 26, 2022
@ArlindKadra @ravinkohli
Cocktail hotfixes (#245)
0c8d2ff 
* Fixes for the development branch and regularization cocktails

* Update implementation

* Fix unit tests temporarily

* Implementation update and bug fixes

* Removing unecessary code

* Addressing Ravin's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments

[refactor] Address Shuhei's comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ravinkohli ravinkohli ravinkohli approved these changes

@franchuterivera franchuterivera Awaiting requested review from franchuterivera

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ArlindKadra @ravinkohli