Skip to content

Commit 74c9625

Browse files
author
Github Actions
committed
Ravin Kohli: Additional metrics during train (#194)
1 parent 82e59c7 commit 74c9625

19 files changed

+177
-327
lines changed

‎refactor_development_regularization_cocktails/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_custom_configuration_space.rst.txt

Lines changed: 26 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ the search. Currently, there are two changes that can be made to the space:-
4646

4747
.. code-block:: none
4848
49-
<smac.runhistory.runhistory.RunHistory object at 0x7fa5fa61bac0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
49+
<smac.runhistory.runhistory.RunHistory object at 0x7f952f61b4f0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
5050
data_loader:batch_size, Value: 32
5151
encoder:__choice__, Value: 'OneHotEncoder'
5252
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -87,7 +87,7 @@ the search. Currently, there are two changes that can be made to the space:-
8787
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
8888
trainer:StandardTrainer:weighted_loss, Constant: 1
8989
trainer:__choice__, Value: 'StandardTrainer'
90-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.00165557861328125, budget=0), TrajEntry(train_perf=0.17543859649122806, incumbent_id=1, incumbent=Configuration:
90+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0018584728240966797, budget=0), TrajEntry(train_perf=0.1578947368421053, incumbent_id=1, incumbent=Configuration:
9191
data_loader:batch_size, Value: 32
9292
encoder:__choice__, Value: 'OneHotEncoder'
9393
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -128,17 +128,18 @@ the search. Currently, there are two changes that can be made to the space:-
128128
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
129129
trainer:StandardTrainer:weighted_loss, Constant: 1
130130
trainer:__choice__, Value: 'StandardTrainer'
131-
, ta_runs=1, ta_time_used=12.98829174041748, wallclock_time=14.392967700958252, budget=5.555555555555555)]
132-
{'accuracy': 0.8554913294797688}
131+
, ta_runs=1, ta_time_used=13.231909990310669, wallclock_time=14.629006624221802, budget=5.555555555555555)]
132+
{'accuracy': 0.8497109826589595}
133133
| | Preprocessing | Estimator | Weight |
134134
|---:|:------------------------------------------------------------------|:----------------------------------------------------------|---------:|
135-
| 0 | None | CatBoostClassifier | 0.58 |
136-
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.2 |
137-
| 2 | None | KNNClassifier | 0.14 |
138-
| 3 | None | RFClassifier | 0.04 |
139-
| 4 | None | LGBMClassifier | 0.02 |
140-
| 5 | None | ExtraTreesClassifier | 0.02 |
141-
<smac.runhistory.runhistory.RunHistory object at 0x7fa55a4b5130> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
135+
| 0 | None | RFClassifier | 0.42 |
136+
| 1 | None | CatBoostClassifier | 0.2 |
137+
| 2 | None | ExtraTreesClassifier | 0.14 |
138+
| 3 | None | KNNClassifier | 0.08 |
139+
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.08 |
140+
| 5 | None | SVC | 0.04 |
141+
| 6 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
142+
<smac.runhistory.runhistory.RunHistory object at 0x7f952c18f760> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
142143
data_loader:batch_size, Value: 32
143144
encoder:__choice__, Value: 'NoEncoder'
144145
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -177,7 +178,7 @@ the search. Currently, there are two changes that can be made to the space:-
177178
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
178179
trainer:StandardTrainer:weighted_loss, Constant: 1
179180
trainer:__choice__, Value: 'StandardTrainer'
180-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0017666816711425781, budget=0), TrajEntry(train_perf=0.19883040935672514, incumbent_id=1, incumbent=Configuration:
181+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0017552375793457031, budget=0), TrajEntry(train_perf=0.22807017543859653, incumbent_id=1, incumbent=Configuration:
181182
data_loader:batch_size, Value: 32
182183
encoder:__choice__, Value: 'NoEncoder'
183184
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -216,19 +217,18 @@ the search. Currently, there are two changes that can be made to the space:-
216217
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
217218
trainer:StandardTrainer:weighted_loss, Constant: 1
218219
trainer:__choice__, Value: 'StandardTrainer'
219-
, ta_runs=1, ta_time_used=8.5232412815094, wallclock_time=9.917478799819946, budget=5.555555555555555)]
220-
{'accuracy': 0.8728323699421965}
221-
| | Preprocessing | Estimator | Weight |
222-
|---:|:--------------------------------------------------------------|:-------------------------------------------------------------------|---------:|
223-
| 0 | SimpleImputer,NoEncoder,MinMaxScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.28 |
224-
| 1 | None | RFClassifier | 0.22 |
225-
| 2 | None | ExtraTreesClassifier | 0.22 |
226-
| 3 | None | KNNClassifier | 0.12 |
227-
| 4 | None | CatBoostClassifier | 0.06 |
228-
| 5 | None | LGBMClassifier | 0.04 |
229-
| 6 | SimpleImputer,NoEncoder,Normalizer,PolynomialFeatures | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
230-
| 7 | None | SVC | 0.02 |
231-
| 8 | SimpleImputer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
220+
, ta_runs=1, ta_time_used=9.11508822441101, wallclock_time=10.526266098022461, budget=5.555555555555555)]
221+
{'accuracy': 0.8670520231213873}
222+
| | Preprocessing | Estimator | Weight |
223+
|---:|:--------------------------------------------------------------|:----------------------------------------------------------------|---------:|
224+
| 0 | None | CatBoostClassifier | 0.32 |
225+
| 1 | None | ExtraTreesClassifier | 0.3 |
226+
| 2 | None | RFClassifier | 0.14 |
227+
| 3 | None | KNNClassifier | 0.1 |
228+
| 4 | SimpleImputer,NoEncoder,Normalizer,PowerTransformer | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.08 |
229+
| 5 | None | LGBMClassifier | 0.02 |
230+
| 6 | SimpleImputer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
231+
| 7 | SimpleImputer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
232232
233233
234234
@@ -357,7 +357,7 @@ the search. Currently, there are two changes that can be made to the space:-
357357
358358
.. rst-class:: sphx-glr-timing
359359

360-
**Total running time of the script:** ( 11 minutes 24.852 seconds)
360+
**Total running time of the script:** ( 11 minutes 31.636 seconds)
361361

362362

363363
.. _sphx_glr_download_advanced_tabular_example_custom_configuration_space.py:

0 commit comments

Comments
 (0)