Folks who are experienced with NBEATS, which param...

# neural-forecastf

Farzad E

02/10/2023, 5:21 PMFolks who are experienced with NBEATS, which parameters are the most important ones to tune? I have been playing around with n_harmonics, n_polynomials and learning_rate but it is clear that the model is not learning anything. R2 barely reaches 50% and MAPE is at 100%! I'm using 4 years of daily data to predict 2022. My data doesn't look that difficult to predict. I must be missing something simple. Here is the fit (magenta is the actual and cyan is the NBEATS:

Farzad E

02/10/2023, 5:22 PMThis is my code:

Farzad E

02/10/2023, 5:22 PMAnd the full data from 2018 onward:

Farzad E

02/10/2023, 5:26 PMThe interesting thing is that it never learns the amplitude changes at the daily level. It tries to follow the changes of the amplitude over the year (although poorly) but it's not even close when it comes to the amplitude changes at the daily level. Completely ignores the signal amplitude there.

c

Cristian (Nixtla)

02/10/2023, 5:47 PMHi **@Farzad E**. This is actually a relatively hard problem given the horizon size of 365. We have a specific paper and model for long horizon forecasting (https://arxiv.org/abs/2201.12886) in our library, the NHITS. Our experiments indeed show that the NBEATS interpretable (the default configuration, what you are running) does not perform well with long horizon.

Cristian (Nixtla)

02/10/2023, 5:49 PMI recommend you to use the

`Auto`

versions of the models (`AutoNHITS`

,`AutoNBEATS`

) which already have a predefine search space for hyperparameter tuning. You can also specify a custom search space, see: https://nixtla.github.io/neuralforecast/examples/automatic_hyperparameter_tuning.htmlCristian (Nixtla)

02/10/2023, 5:52 PMFor the specific case of the NBEATS, we don' usually change the n_harmonics and n_polynomials. To make the model more flexible you can increase the number of blocks (

`n_blocks`

) for each stack type, for example to `[3,3,3]`

and the `mlp_units`

.Cristian (Nixtla)

02/10/2023, 5:53 PMFinally, adding exogenous calendar variables might help, such as month dummies. Here we have the documentation for adding variables: https://nixtla.github.io/neuralforecast/examples/exogenous_variables.html

Cristian (Nixtla)

02/10/2023, 5:59 PMLooking at the complete data, 2022 is also hard to forecast because it has larger values than previous years, thats probably why the forecasts are lower. I strongly recommend you to try the NHITS model, we designed it specifically for this setting. Hope it helps!

f

Farzad E

02/10/2023, 6:15 PMFarzad E

02/10/2023, 6:24 PMm

Martin Bel

02/21/2023, 7:39 PMweekly data is hard with any method, at least in my experience.

👍 1

f

Farzad E

02/21/2023, 7:45 PMm

Martin Bel

02/21/2023, 7:52 PMI'd start with the statsforecast methods first, AutoArima for example is generally hard to beat. Then you can try the mlforecast package, this should be simpler to use than NHiTS.

👍 1

f

Farzad E

02/21/2023, 7:55 PMArima won't work for period of 365 and even for 52 it takes forever to compute. It's more practical for monthly or quarterly forecasts at least in my experience. I haven't tried mlforecast so thanks for mentioning that. I'll look into it.

m

Martin Bel

02/21/2023, 7:57 PMIt works ok for 365 or 52 periods. But I've seen this being slow also. Compared to R much slower in these frequencies.
This doesn't make it a "bad" model for this problems. It's generally the main model.

👍 1

f

Farzad E

02/21/2023, 8:00 PMYeah it does work like it gives you results. What I meant was that it becomes so slow specially if you have a few years of data and hundreds of time series, that in practice it won't work for your project. I haven't coded in R in years but I hear it is a bit faster in forecasting but not sure if that would help much.

m

Martin Bel

02/21/2023, 8:01 PMif you have hundreds of time series, perhaps you can use hierarchicalforecast.
It always depends on the data. But it's a hard to beat benchmark.

👍 1

f

Farzad E

02/21/2023, 8:04 PMThanks. I'll check it out.

2 Views