Hi all, Is anyone using a library for optimizing hyperparameters such as

hyperopt

in combination with

MLForecast

? I have tried doing some smaller experiments using

hyperopt

but it fails as the library does not have access to the loss function.

I have used optuna and followed this pattern using cross validation and one of the loss functions from utilsforecast (using random ints for the window params here):

cv_results = fcst.cross_validation(

train_df,

n_windows=6,

h=12,

step_size=2,

)

cv_results['id_cutoff'] = cv_results['unique_id'] + '_' + cv_results['cutoff'].astype(str)

score = loss(cv_results, models=['LGBMRegressor'], id_col='id_cutoff')['LGBMRegressor'].mean()

We also just released a built-in hyperparameter optimization: https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/hyperparameter_optimization.html, that should be easier to use. It's on v0.12.1

Perfect timing! I will try and give the new implementation a look! (Y)