The goal of our thesis was to analyze Bayesian optimization on the problem of hyperparameter optimization.
The basis for the analysis is the commonly used tools for hyperparameter optimization: random search and grid search.
We introduce Bayesian optimization with an emphasis on Gaussian processes and acquisition functions: EI, PI, LCB.
We perform ten experiments in which we optimize the hyperparameters of five different models.
In experiments, we analyze two really important metrics: speed of optimization and results that the optimized model achieved.
Models optimized with Bayesian optimization in comparable time achieved better results on average than those that were optimized with random search and grid search.
|
