Folks who are experienced with NBEATS which parameters are t

Question

Folks who are experienced with NBEATS which parameters are the most important ones to tune I have been playing around with n harmonics n polynomials and learning rate but it is clear that the model is not learning anything R2 barely reaches 50 and MAPE is at 100 I m using 4 years of daily data to predict 2022 My data doesn t look that difficult to predict I must be missing something simple Here is the fit magenta is the actual and cyan is the NBEATS

Farzad E · Answer

This is my code

Farzad E · Answer

And the full data from 2018 onward

Farzad E · Answer

The interesting thing is that it never learns the amplitude changes at the daily level It tries to follow the changes of the amplitude over the year although poorly but it s not even close when it comes to the amplitude changes at the daily level Completely ignores the signal amplitude there

Cristian (Nixtla) · Answer

Hi < Farzad E> This is actually a relatively hard problem given the horizon size of 365 We have a specific paper and model for long horizon forecasting in our library the NHITS Our experiments indeed show that the NBEATS interpretable the default configuration what you are running does not perform well with long horizon

Cristian (Nixtla) · Answer

I recommend you to use the `Auto` versions of the models `AutoNHITS` `AutoNBEATS` which already have a predefine search space for hyperparameter tuning You can also specify a custom search space see <https nixtla github io neuralforecast examples automatic hyperparameter tuning html>

Cristian (Nixtla) · Answer

For the specific case of the NBEATS we don usually change the n harmonics and n polynomials To make the model more flexible you can increase the number of blocks `n blocks` for each stack type for example to ` 3 3 3 ` and the `mlp units`

Cristian (Nixtla) · Answer

Finally adding exogenous calendar variables might help such as month dummies Here we have the documentation for adding variables <https nixtla github io neuralforecast examples exogenous variables html>

Cristian (Nixtla) · Answer

Looking at the complete data 2022 is also hard to forecast because it has larger values than previous years thats probably why the forecasts are lower I strongly recommend you to try the NHITS model we designed it specifically for this setting Hope it helps

Farzad E · Answer

< Cristian Nixtla > This is incredible help Thanks a lot I didn t know about limitations of NBEATS I ll look into NHITS and will try the Auto versions as you mentioned

Farzad E · Answer

< Cristian Nixtla > On a related note I know classical methods like ARIMA have difficulty forecasting series with long periods e g 365 or even 52 and that s why I started looking into deep learning methods What are your thoughts on aggregating data and forecasting at weekly level Do most deep learning forecasters have a hard time with weekly data as well like ARIMA has or will they perform much better at weekly compared to daily

Martin Bel · Answer

weekly data is hard with any method at least in my experience

Farzad E · Answer

< Martin Bel> what is your go to method for weekly daily And what type of data do you work with of you don t mind me asking My current project is forecasting weekly and daily for the housing market data

Martin Bel · Answer

I d start with the statsforecast methods first AutoArima for example is generally hard to beat Then you can try the mlforecast package this should be simpler to use than NHiTS

Farzad E · Answer

Arima won t work for period of 365 and even for 52 it takes forever to compute It s more practical for monthly or quarterly forecasts at least in my experience I haven t tried mlforecast so thanks for mentioning that I ll look into it

Martin Bel · Answer

It works ok for 365 or 52 periods But I ve seen this being slow also Compared to R much slower in these frequencies This doesn t make it a bad model for this problems It s generally the main model

Farzad E · Answer

Yeah it does work like it gives you results What I meant was that it becomes so slow specially if you have a few years of data and hundreds of time series that in practice it won t work for your project I haven t coded in R in years but I hear it is a bit faster in forecasting but not sure if that would help much

Martin Bel · Answer

if you have hundreds of time series perhaps you can use hierarchicalforecast It always depends on the data But it s a hard to beat benchmark

Farzad E · Answer

Thanks I ll check it out