Calculate logLikeLihoodsAbove from samplesBelow by c-bata · Pull Request #151 · c-bata/goptuna

c-bata · 2020-08-06T10:43:08Z

See https://github.com/optuna/optuna/blob/e28e9febbde9ba4ba2c16468401bbb49116353b0/optuna/samplers/_tpe/sampler.py#L326

github-actions · 2020-08-06T10:44:58Z

Benchmark result of Ackley function

Report ID: b3139c02dca21d5fe7d20a97e3bd2c17632b51280bab9425780a6631d9116f4d
Kurobako Version: 0.2.0
Number of Solvers: 5
Number of Problems: 1
Metrics Precedence: best value -> AUC

Please refer to "A Strategy for Ranking Optimizers using Multiple Criteria" for the ranking strategy used in this report.

Please expand here for more details.

Overall Results

Solver	Borda	Firsts
Goptuna (CMA-ES)	3	1
Goptuna (TPE)	1	0
Optuna-CMAES	3	1
Optuna-TPE	1	0
Random	0	0

Individual Results

(1) Problem: sigopt/evalset/Ackley(dim=2)

Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Goptuna (CMA-ES) (study)	0.720523 +- 0.584595	448.157 +- 182.932	0.036 +- 0.015
1	Optuna-CMAES (study)	1.164011 +- 0.905332	482.710 +- 149.598	0.526 +- 0.073
3	Optuna-TPE (study)	3.074214 +- 1.116972	585.226 +- 126.096	2.011 +- 0.472
3	Goptuna (TPE) (study)	3.467162 +- 0.626148	628.950 +- 121.013	0.158 +- 0.032
5	Random (study)	7.067197 +- 1.265246	871.279 +- 125.875	0.000 +- 0.000

Solvers

ID: bcb361930b088ad765b33edfe444986095c910402687ed162e8f6c11a5351b43

recipe:

{
  "command": {
    "path": "/home/runner/work/goptuna/goptuna/bin/goptuna_solver",
    "args": [
      "cmaes"
    ]
  }
}

specification:

{
  "name": "Goptuna (CMA-ES)",
  "attrs": {
    "github": "https://github.com/c-bata/goptuna"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: 5c2f3ce0f48edaa415f646290c199434d68ef4ad4638bf963c13f9c1a5d1bd2b

recipe:

{
  "command": {
    "path": "/home/runner/work/goptuna/goptuna/bin/goptuna_solver",
    "args": [
      "tpe"
    ]
  }
}

specification:

{
  "name": "Goptuna (TPE)",
  "attrs": {
    "github": "https://github.com/c-bata/goptuna"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: fd78b69a4424cea6bdf87dc5e79a74a8c6d0aff8371badd7c1ba11c5389b799a

recipe:

{
  "name": "Optuna-CMAES",
  "optuna": {
    "loglevel": "error",
    "sampler": "CmaEsSampler"
  }
}

specification:

{
  "name": "Optuna-CMAES",
  "attrs": {
    "github": "https://github.com/optuna/optuna",
    "paper": "Akiba, Takuya, et al. \"Optuna: A next-generation hyperparameter optimization framework.\" Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.",
    "version": "optuna=2.0.0, kurobako-py=0.1.8"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: 3071ba4560e75197b0d9e85d0117b988af460904c42b15fb6ccf579d166a4e9a

recipe:

{
  "name": "Optuna-TPE",
  "optuna": {
    "loglevel": "error",
    "sampler": "TPESampler"
  }
}

specification:

{
  "name": "Optuna-TPE",
  "attrs": {
    "github": "https://github.com/optuna/optuna",
    "paper": "Akiba, Takuya, et al. \"Optuna: A next-generation hyperparameter optimization framework.\" Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.",
    "version": "optuna=2.0.0, kurobako-py=0.1.8"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: adba9490e43fe66d939f067852b174c90815eb5f7d736b45282d07bd324579d4

recipe:

{
  "random": {}
}

specification:

{
  "name": "Random",
  "attrs": {
    "version": "kurobako_solvers=0.2.0"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "LOG_UNIFORM_DISCRETE",
    "CATEGORICAL",
    "CONDITIONAL",
    "MULTI_OBJECTIVE",
    "CONCURRENT"
  ]
}

Problems

ID: 17966ef7eccd1a5b5e29f23659e90202e5d228948069153cce3b1f1a03751803

recipe:

{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 2
  }
}

specification:

{
  "name": "sigopt/evalset/Ackley(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.8"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

Studies

ID: 1976426ff91e4113bb730f4fba67724f4bd21fdaac5cc2166fb9d3e057458b7d

problem: sigopt/evalset/Ackley(dim=2)
solver: Goptuna (CMA-ES)
budget: 100
repeats: 10
concurrency: 1

ID: 23c993067b5293e699061e27627e38dcabe267f63caf9773842c9562e4e88b8f

problem: sigopt/evalset/Ackley(dim=2)
solver: Goptuna (TPE)
budget: 100
repeats: 10
concurrency: 1

ID: d162d78007748fa9bda3dce403be5c63ed9110740c59be258ea73c0e9ebf708b

problem: sigopt/evalset/Ackley(dim=2)
solver: Optuna-CMAES
budget: 100
repeats: 10
concurrency: 1

ID: 7bcc41ef32dc0cca32d7c01b844ec32abd6fece5f863ab7fc7ec330ba63913d6

problem: sigopt/evalset/Ackley(dim=2)
solver: Optuna-TPE
budget: 100
repeats: 10
concurrency: 1

ID: fe994927342992c810fcb1f862b16d7d8759ece4e09b05b0c6e742969acbfd14

problem: sigopt/evalset/Ackley(dim=2)
solver: Random
budget: 100
repeats: 10
concurrency: 1

github-actions · 2020-08-06T10:46:02Z

Benchmark result of Rastrigin problem

Report ID: d356f012b6d8d34591a7fd2771672f33bbbc3d0da8f8dcaf4194442ebd8eaad6
Kurobako Version: 0.2.0
Number of Solvers: 4
Number of Problems: 1
Metrics Precedence: best value -> AUC

Please refer to "A Strategy for Ranking Optimizers using Multiple Criteria" for the ranking strategy used in this report.

Please expand here for more details.

Overall Results

Solver	Borda	Firsts
Goptuna (BIPOP-CMA-ES)	0	1
Goptuna (CMA-ES)	0	1
Goptuna (IPOP-CMA-ES)	0	1
Random	0	1

Individual Results

(1) Problem: Rastrigin function (dim=2)

Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Goptuna (BIPOP-CMA-ES) (study)	0.628570 +- 0.621310	3004.628 +- 1954.645	9.849 +- 0.958
1	Random (study)	1.392878 +- 0.512078	5576.501 +- 1551.925	0.005 +- 0.002
1	Goptuna (IPOP-CMA-ES) (study)	0.583980 +- 0.569993	3036.752 +- 1822.481	9.909 +- 0.829
1	Goptuna (CMA-ES) (study)	1.341307 +- 1.118099	3757.611 +- 2586.556	10.084 +- 0.570

Solvers

ID: 9b2ad76978c9cab636e881f48d36cb398e7812c07cf0cf044ad74b88ba37f902

recipe:

{
  "command": {
    "path": "/home/runner/work/goptuna/goptuna/bin/goptuna_solver",
    "args": [
      "bipop-cmaes"
    ]
  }
}

specification:

{
  "name": "Goptuna (BIPOP-CMA-ES)",
  "attrs": {
    "github": "https://github.com/c-bata/goptuna"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: bcb361930b088ad765b33edfe444986095c910402687ed162e8f6c11a5351b43

recipe:

{
  "command": {
    "path": "/home/runner/work/goptuna/goptuna/bin/goptuna_solver",
    "args": [
      "cmaes"
    ]
  }
}

specification:

{
  "name": "Goptuna (CMA-ES)",
  "attrs": {
    "github": "https://github.com/c-bata/goptuna"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: b40e4010fb9c8506d051f50c41db99f67e5d52d585d04ba4ef88e2d6490b6e15

recipe:

{
  "command": {
    "path": "/home/runner/work/goptuna/goptuna/bin/goptuna_solver",
    "args": [
      "ipop-cmaes"
    ]
  }
}

specification:

{
  "name": "Goptuna (IPOP-CMA-ES)",
  "attrs": {
    "github": "https://github.com/c-bata/goptuna"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: adba9490e43fe66d939f067852b174c90815eb5f7d736b45282d07bd324579d4

recipe:

{
  "random": {}
}

specification:

{
  "name": "Random",
  "attrs": {
    "version": "kurobako_solvers=0.2.0"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "LOG_UNIFORM_DISCRETE",
    "CATEGORICAL",
    "CONDITIONAL",
    "MULTI_OBJECTIVE",
    "CONCURRENT"
  ]
}

Problems

ID: 0091bc29d1a812f56db93aa64502974e93cc18283ec26b6c5c99b085b81529b8

recipe:

{
  "command": {
    "path": "/home/runner/work/goptuna/goptuna/bin/rastrigin_problem",
    "args": [
      "2"
    ]
  }
}

specification:

{
  "name": "Rastrigin function (dim=2)",
  "attrs": {},
  "params_domain": [
    {
      "name": "x1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 5.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "x2",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 5.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Rastrigin",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

Studies

ID: 8c50e86759933a0dbaf04c154ae3cebfbbf6b397a49a1db1b47ac5dae7365a89

problem: Rastrigin function (dim=2)
solver: Goptuna (BIPOP-CMA-ES)
budget: 2500
repeats: 10
concurrency: 1

ID: 5d4812251fac7d96cd6bcb683c134e553741f955e2379673731a49deb201616d

problem: Rastrigin function (dim=2)
solver: Goptuna (CMA-ES)
budget: 2500
repeats: 10
concurrency: 1

ID: af7aa97997c9e5ca6c80b9a7adc66f4979f841b5abba604f9329d12e88fd05fe

problem: Rastrigin function (dim=2)
solver: Goptuna (IPOP-CMA-ES)
budget: 2500
repeats: 10
concurrency: 1

ID: aeab515b58a5d09dd615e3f47cad686f03c0f24f168a34ff5c3a667d9b408078

problem: Rastrigin function (dim=2)
solver: Random
budget: 2500
repeats: 10
concurrency: 1

github-actions · 2020-08-06T10:49:11Z

Benchmark result of HPOBench-Naval problem

Report ID: 6a239ba15e4dc4175337926a7db941b28344c9e843031940cb2410826522908a
Kurobako Version: 0.2.0
Number of Solvers: 5
Number of Problems: 1
Metrics Precedence: best value -> AUC

Please refer to "A Strategy for Ranking Optimizers using Multiple Criteria" for the ranking strategy used in this report.

Please expand here for more details.

Overall Results

Solver	Borda	Firsts
Optuna-RANDOM-ASHA	0	1
Optuna-RANDOM-MEDIAN	0	1
Optuna-TPE-ASHA	0	1
Optuna-TPE-MEDIAN	0	1
Random	0	1

Individual Results

(1) Problem: HPO-Bench-Naval

Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Optuna-TPE-ASHA (study)	0.000483 +- 0.000580	0.069 +- 0.070	29.515 +- 10.524
1	Optuna-RANDOM-MEDIAN (study)	0.000035 +- 0.000004	0.027 +- 0.020	24.923 +- 2.506
1	Random (study)	0.000152 +- 0.000045	0.044 +- 0.037	0.000 +- 0.000
1	Optuna-TPE-MEDIAN (study)	0.000068 +- 0.000024	0.029 +- 0.030	25.916 +- 6.364
1	Optuna-RANDOM-ASHA (study)	0.000053 +- 0.000011	0.010 +- 0.006	6.950 +- 0.946

Solvers

ID: d7f35c18c3d80bc06651efa2be60cd39448525dac4938682f161f8721dc6eda3

recipe:

{
  "name": "Optuna-RANDOM-ASHA",
  "optuna": {
    "loglevel": "error",
    "sampler": "RandomSampler",
    "pruner": "SuccessiveHalvingPruner"
  }
}

specification:

{
  "name": "Optuna-RANDOM-ASHA",
  "attrs": {
    "github": "https://github.com/optuna/optuna",
    "paper": "Akiba, Takuya, et al. \"Optuna: A next-generation hyperparameter optimization framework.\" Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.",
    "version": "optuna=2.0.0, kurobako-py=0.1.8"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: 581be156c7d4cd5286f9f0ae7f9cf4555d39923045b738cb2fcf233491871f0b

recipe:

{
  "name": "Optuna-RANDOM-MEDIAN",
  "optuna": {
    "loglevel": "error",
    "sampler": "RandomSampler",
    "pruner": "MedianPruner"
  }
}

specification:

{
  "name": "Optuna-RANDOM-MEDIAN",
  "attrs": {
    "github": "https://github.com/optuna/optuna",
    "paper": "Akiba, Takuya, et al. \"Optuna: A next-generation hyperparameter optimization framework.\" Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.",
    "version": "optuna=2.0.0, kurobako-py=0.1.8"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: 00b6d16a8f2934ddb046c283a5520fed93476954acf55a12814a9bdac7e278f2

recipe:

{
  "name": "Optuna-TPE-ASHA",
  "optuna": {
    "loglevel": "error",
    "sampler": "TPESampler",
    "pruner": "SuccessiveHalvingPruner"
  }
}

specification:

{
  "name": "Optuna-TPE-ASHA",
  "attrs": {
    "github": "https://github.com/optuna/optuna",
    "paper": "Akiba, Takuya, et al. \"Optuna: A next-generation hyperparameter optimization framework.\" Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.",
    "version": "optuna=2.0.0, kurobako-py=0.1.8"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: d70996cc6ff2f96bf615d9623f0325d60a1012c914f2a7383d6f9059106411cb

recipe:

{
  "name": "Optuna-TPE-MEDIAN",
  "optuna": {
    "loglevel": "error",
    "sampler": "TPESampler",
    "pruner": "MedianPruner"
  }
}

specification:

{
  "name": "Optuna-TPE-MEDIAN",
  "attrs": {
    "github": "https://github.com/optuna/optuna",
    "paper": "Akiba, Takuya, et al. \"Optuna: A next-generation hyperparameter optimization framework.\" Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.",
    "version": "optuna=2.0.0, kurobako-py=0.1.8"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: adba9490e43fe66d939f067852b174c90815eb5f7d736b45282d07bd324579d4

recipe:

{
  "random": {}
}

specification:

{
  "name": "Random",
  "attrs": {
    "version": "kurobako_solvers=0.2.0"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "LOG_UNIFORM_DISCRETE",
    "CATEGORICAL",
    "CONDITIONAL",
    "MULTI_OBJECTIVE",
    "CONCURRENT"
  ]
}

Problems

ID: 5c382dedcdf8170f494a51e00713979e4f1754e3520a61df2ab570c3a115f032

recipe:

{
  "hpobench": {
    "dataset": "/home/runner/work/goptuna/goptuna/tmp/fcnet_tabular_benchmarks/fcnet_naval_propulsion_data.hdf5"
  }
}

specification:

{
  "name": "HPO-Bench-Naval",
  "attrs": {
    "github": "https://github.com/automl/nas_benchmarks",
    "paper": "Klein, Aaron, and Frank Hutter. \"Tabular Benchmarks for Joint Architecture and Hyperparameter Optimization.\" arXiv preprint arXiv:1905.04970 (2019).",
    "version": "kurobako_problems=0.1.8"
  },
  "params_domain": [
    {
      "name": "activation_fn_1",
      "range": {
        "type": "CATEGORICAL",
        "choices": [
          "tanh",
          "relu"
        ]
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "activation_fn_2",
      "range": {
        "type": "CATEGORICAL",
        "choices": [
          "tanh",
          "relu"
        ]
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "batch_size",
      "range": {
        "type": "DISCRETE",
        "low": 0,
        "high": 4
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "dropout_1",
      "range": {
        "type": "DISCRETE",
        "low": 0,
        "high": 3
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "dropout_2",
      "range": {
        "type": "DISCRETE",
        "low": 0,
        "high": 3
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "init_lr",
      "range": {
        "type": "DISCRETE",
        "low": 0,
        "high": 6
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "lr_schedule",
      "range": {
        "type": "CATEGORICAL",
        "choices": [
          "cosine",
          "const"
        ]
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "n_units_1",
      "range": {
        "type": "DISCRETE",
        "low": 0,
        "high": 6
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "n_units_2",
      "range": {
        "type": "DISCRETE",
        "low": 0,
        "high": 6
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Validation MSE",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 100
}

Studies

ID: b98e3a6ce1e7943d7212740ed82ea19bff92a33cf48802b8e02d6fea0e1b3a19

problem: HPO-Bench-Naval
solver: Optuna-RANDOM-ASHA
budget: 100
repeats: 5
concurrency: 1

ID: a128278035279dea09fa9c9be248ab43cf039fe1dde948657e1794e03ff89a58

problem: HPO-Bench-Naval
solver: Optuna-RANDOM-MEDIAN
budget: 100
repeats: 5
concurrency: 1

ID: c911c8bc0385b87b358dcd8f3a8243be61d5fd18c82fe74015bf550ad1277f19

problem: HPO-Bench-Naval
solver: Optuna-TPE-ASHA
budget: 100
repeats: 5
concurrency: 1

ID: 8bca403316c1c0aa23af7c097c7b13436b20ad4a69c7c5deac166fd577cbcda0

problem: HPO-Bench-Naval
solver: Optuna-TPE-MEDIAN
budget: 100
repeats: 5
concurrency: 1

ID: 5aae795e88e4a0125a6e1b625f2ce3c64cccc045eb33dd7ab6f7e2c445207879

problem: HPO-Bench-Naval
solver: Random
budget: 100
repeats: 5
concurrency: 1

Calculate logLikeLihoodsAbove from samplesBelow

90be6b6

c-bata added the bug Something isn't working label Aug 6, 2020

c-bata merged commit 146588a into master Aug 6, 2020

c-bata deleted the fix-log-likelihoods branch August 6, 2020 10:49

Conversation

c-bata commented Aug 6, 2020

Uh oh!

github-actions Bot commented Aug 6, 2020

Benchmark result of Ackley function

Table of Contents

Overall Results

Individual Results

(1) Problem: sigopt/evalset/Ackley(dim=2)

Solvers

ID: bcb361930b088ad765b33edfe444986095c910402687ed162e8f6c11a5351b43

ID: 5c2f3ce0f48edaa415f646290c199434d68ef4ad4638bf963c13f9c1a5d1bd2b

ID: fd78b69a4424cea6bdf87dc5e79a74a8c6d0aff8371badd7c1ba11c5389b799a

ID: 3071ba4560e75197b0d9e85d0117b988af460904c42b15fb6ccf579d166a4e9a

ID: adba9490e43fe66d939f067852b174c90815eb5f7d736b45282d07bd324579d4

Problems

ID: 17966ef7eccd1a5b5e29f23659e90202e5d228948069153cce3b1f1a03751803

Studies

ID: 1976426ff91e4113bb730f4fba67724f4bd21fdaac5cc2166fb9d3e057458b7d

ID: 23c993067b5293e699061e27627e38dcabe267f63caf9773842c9562e4e88b8f

ID: d162d78007748fa9bda3dce403be5c63ed9110740c59be258ea73c0e9ebf708b

ID: 7bcc41ef32dc0cca32d7c01b844ec32abd6fece5f863ab7fc7ec330ba63913d6

ID: fe994927342992c810fcb1f862b16d7d8759ece4e09b05b0c6e742969acbfd14

Uh oh!

github-actions Bot commented Aug 6, 2020

Benchmark result of Rastrigin problem

Table of Contents

Overall Results

Individual Results

(1) Problem: Rastrigin function (dim=2)

Solvers

ID: 9b2ad76978c9cab636e881f48d36cb398e7812c07cf0cf044ad74b88ba37f902

ID: bcb361930b088ad765b33edfe444986095c910402687ed162e8f6c11a5351b43

ID: b40e4010fb9c8506d051f50c41db99f67e5d52d585d04ba4ef88e2d6490b6e15

ID: adba9490e43fe66d939f067852b174c90815eb5f7d736b45282d07bd324579d4

Problems

ID: 0091bc29d1a812f56db93aa64502974e93cc18283ec26b6c5c99b085b81529b8

Studies

ID: 8c50e86759933a0dbaf04c154ae3cebfbbf6b397a49a1db1b47ac5dae7365a89

ID: 5d4812251fac7d96cd6bcb683c134e553741f955e2379673731a49deb201616d

ID: af7aa97997c9e5ca6c80b9a7adc66f4979f841b5abba604f9329d12e88fd05fe

ID: aeab515b58a5d09dd615e3f47cad686f03c0f24f168a34ff5c3a667d9b408078

Uh oh!

github-actions Bot commented Aug 6, 2020

Benchmark result of HPOBench-Naval problem

Table of Contents

Overall Results

Individual Results

(1) Problem: HPO-Bench-Naval

Solvers

ID: d7f35c18c3d80bc06651efa2be60cd39448525dac4938682f161f8721dc6eda3

ID: 581be156c7d4cd5286f9f0ae7f9cf4555d39923045b738cb2fcf233491871f0b

ID: 00b6d16a8f2934ddb046c283a5520fed93476954acf55a12814a9bdac7e278f2

ID: d70996cc6ff2f96bf615d9623f0325d60a1012c914f2a7383d6f9059106411cb

ID: adba9490e43fe66d939f067852b174c90815eb5f7d736b45282d07bd324579d4

Problems

ID: 5c382dedcdf8170f494a51e00713979e4f1754e3520a61df2ab570c3a115f032

Studies

ID: b98e3a6ce1e7943d7212740ed82ea19bff92a33cf48802b8e02d6fea0e1b3a19

ID: a128278035279dea09fa9c9be248ab43cf039fe1dde948657e1794e03ff89a58

ID: c911c8bc0385b87b358dcd8f3a8243be61d5fd18c82fe74015bf550ad1277f19

ID: 8bca403316c1c0aa23af7c097c7b13436b20ad4a69c7c5deac166fd577cbcda0

ID: 5aae795e88e4a0125a6e1b625f2ce3c64cccc045eb33dd7ab6f7e2c445207879

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant