Optuna Mode doesn't work when tuning LightGBM

[AkshayNovacene]

Great package, I love that it supports a wide range of functionalities.

However, when I try to create an AutoML instance in Optuna mode for Light GBM, it fails and gives the following error message, it works when I use other ML models though:

[W 2023-12-11 18:14:57,553] Trial 0 failed with parameters: {'learning_rate': 0.1, 'num_leaves': 1598, 'lambda_l1': 2.840098794801191e-06, 'lambda_l2': 3.0773599420974e-06, 'feature_fraction': 0.8613105322932351, 'bagging_fraction': 0.970697557159987, 'bagging_freq': 7, 'min_data_in_leaf': 36, 'extra_trees': False} because of the following error: The value None could not be cast to float..
[W 2023-12-11 18:14:57,554] Trial 0 failed with value None.

These are the parameter settings for AutoML:

automl = AutoML(
    mode="Optuna",
    eval_metric="f1",
    golden_features=False,
    ml_task='binary_classification',
    kmeans_features=False,
    start_random_models=1,
    stack_models=False,
    train_ensemble=False,
    optuna_time_budget=100,
    optuna_verbose=True,
    features_selection=False,
    algorithms=["LightGBM"],
    validation_strategy={
        "validation_type": "kfold",
        "k_folds": 3,
        "shuffle": True,
        "stratify": True,
    }

It would be great if you could help me with this.

[pplonski]

Hi @AkshayNovacene,

Thank you for reporting the issue. Looks like a bug, maybe LightGBM model wasnt trained, that's why None in score value.

[AkshayNovacene]

Thank you for the swift response @pplonski I realized that the error might be related to #645 and I got it fixed. Apologies for raising a new issue.

[pplonski]

no problem, I think LightGBM should be updated in the requirements.txt.

[lijm1358]

Hi! I've encountered exactly same error as above, which seems to be caused by lgb.train in tuner/optuna/lightgbm.py.

gbm = lgb.train(
    param,
    self.dtrain,
    valid_sets=[self.dvalid],
    valid_names=["validation"],
    verbose_eval=False,
    callbacks=[pruning_callback],
    num_boost_round=self.rounds,
    early_stopping_rounds=self.early_stopping_rounds,
    feval=self.custom_eval_metric,
)

argument verbose_eval and early_stopping_rounds has removed, so removing verbose_eval, early_stopping_rounds and add early stopping callback fixed this issue for me.

early_stopping_callback = lgb.early_stopping(self.early_stopping_rounds)

gbm = lgb.train(
    param,
    self.dtrain,
    valid_sets=[self.dvalid],
    valid_names=["validation"],
    callbacks=[pruning_callback, early_stopping_callback],
    num_boost_round=self.rounds,
    feval=self.custom_eval_metric,
)

I'm new to this package, is it ok to make pull request like above?

[pplonski]

Hi @lijm1358,

Sure, PR is welcome!

[pplonski]

Thanks to @lijm1358 for PR, we released a new version with fix.