Skip to content

[FIX] Passing checks #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 27 commits into from
Dec 7, 2021
Merged

[FIX] Passing checks #298

merged 27 commits into from
Dec 7, 2021

Conversation

ravinkohli
Copy link
Contributor

@ravinkohli ravinkohli commented Oct 22, 2021 •
edited
Loading

This PR fixes the following

  • tests which were failing due to the updated implementation in the branch.
  • In addition as certain components (preprocessors and network embedding) require a fix before tests can be run on them, I have currently skipped those tests.
  • mypy and flake checks

The goal of this PR is to pass all checks for the reg_cocktails branch.

@ravinkohli ravinkohli changed the title Initial fix for all tests passing locally py=3.8 [FIX] Passing checks Oct 22, 2021
nabenabe0928
nabenabe0928 requested changes Nov 4, 2021
View reviewed changes
nabenabe0928
nabenabe0928 reviewed Nov 8, 2021
View reviewed changes
Copy link
Collaborator

@nabenabe0928 nabenabe0928 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing my comments.
I noticed that my comments was not completed properly, so I added comments.

ArlindKadra
ArlindKadra reviewed Dec 6, 2021
View reviewed changes
Copy link

@ArlindKadra ArlindKadra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, although I have a few uncertainties :)

ArlindKadra
ArlindKadra reviewed Dec 6, 2021
View reviewed changes
test/test_data/test_feature_validator.py
assert np.shape(input_data_featuretest) == np.shape(transformed_X)
categories_ = validator.column_transformer.\
named_transformers_['categorical_pipeline'].named_steps['onehotencoder'].categories_
assert any(('0' in categories) or (0 in categories) or ('missing_value' in categories) for categories in
Copy link

@ArlindKadra ArlindKadra Dec 6, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This are the unique category values for the one hot encoder right, what would be the case where they are 0?
Is it not 'missing_value' for categorical columns?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will be 0 when the column is categorical but the dtype of the column is int

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to this

Auto-PyTorch/autoPyTorch/data/tabular_feature_validator.py

Lines 380 to 381 in 10aea66

if column_dtype in ['category', 'bool']:
categorical_columns.append(column)

It will be a categorical column only when it has a string or bool dtype.
And if between test and train there are differences, it will either throw an error or convert a null column to object for test if not empty in train.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it fail if you run it without the 0 checks?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be a categorical column only when it has a string or bool dtype

not string, but category. And category can be string or int.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works fine without '0' check but removing 0 gives an error

@ArlindKadra ArlindKadra merged commit ef1057a into refactor_development_regularization_cocktails Dec 7, 2021
@ArlindKadra ArlindKadra deleted the fix_tests branch December 7, 2021 20:06
github-actions bot pushed a commit that referenced this pull request Dec 7, 2021
Ravin Kohli: [FIX] Passing checks (#298)
827bca1
github-actions bot pushed a commit that referenced this pull request Dec 8, 2021
Ravin Kohli: [FIX] Passing checks (#298)
49fe363
ravinkohli added a commit that referenced this pull request Dec 8, 2021
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
f45dbf2 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit that referenced this pull request Dec 8, 2021
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
87ee242 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit that referenced this pull request Dec 21, 2021
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
6e48d7c 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit that referenced this pull request Jan 24, 2022
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
ff6e8c4 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit that referenced this pull request Jan 28, 2022
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
c310ef6 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit that referenced this pull request Feb 28, 2022
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
98c93c4 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit that referenced this pull request Feb 28, 2022
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
0d475fc 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit that referenced this pull request Mar 9, 2022
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
3e50b27 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit to ravinkohli/Auto-PyTorch that referenced this pull request Apr 12, 2022
@ravinkohli @nabenabe0928
[FIX] Passing checks (automl#298)
ce74450 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
ravinkohli added a commit that referenced this pull request Jul 26, 2022
@ravinkohli @nabenabe0928
[FIX] Passing checks (#298)
392f07a 
* Initial fix for all tests passing locally py=3.8

* fix bug in tests

* fix bug in test for data

* debugging error in dummy forward pass

* debug try -2

* catch runtime error in ci

* catch runtime error in ci

* add better debug test setup

* debug some more

* run this test only

* remove sum backward

* remove inplace in inception block

* undo silly change

* Enable all tests

* fix flake

* fix bug in test setup

* remove anamoly detection

* minor changes to comments

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <[email protected]>

* Address comments from Shuhei

* revert change leading to bug

* fix flake

* change comment position in feature validator

* Add documentation for _is_datasets_consistent

* address comments from arlind

* case when all nans in test

Co-authored-by: nabenabe0928 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ArlindKadra ArlindKadra ArlindKadra left review comments

@nabenabe0928 nabenabe0928 nabenabe0928 requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ravinkohli @ArlindKadra @nabenabe0928