Matej10/07/2023, 2:38 PM
• yet I do not see more CPUs being utilized and the fitting takes very long time. • num_threads in mlforecast works amazingly well, so I wonder maybe this is perhaps due to the nature of statsforecast algos, that they cant be parallelized as well as e.g. the lgbm ? • (I do run the analysis in jupyter notebook but I doubt that is the culprit) Thanks and have a great weekend.
models_stats = [ AutoCES(season_length=96), AutoETS(season_length=96) ] # Instantiate StatsForecast class with the models sf = StatsForecast( models=models_stats, freq='15T', n_jobs = 7 # NOTE: n_jobs instead of num_threads ) sf.fit(df_train)
Kevin Kho10/08/2023, 6:06 AM
. Statsforecast parallelization is done across models. I think what you can do is break these up into multiple
jobs and that should parallelize.
José Morales10/09/2023, 4:05 PM
chunks and process each one in parallel. If you have less than 7 series you won't see a linear speedup and if you have one nothing will change.
Kevin Kho10/09/2023, 8:53 PM