Hey Team, I'm using the AutoNBEATS model and have been struggling to get good performance out of it. I understand it's quite an open-ended question, but I'd like to know what are some common pitfalls for this model that could result in poor performance? Feel free to ask me any questions. This is my code:

horizon = 8
samples = 40

models=[
        AutoNBEATS(h=horizon, num_samples=samples , gpus=1, backend='optuna', alias="Auto-NBEATS"),
        NBEATS(h=horizon, input_size=horizon * 5),
        ]



nf_Nbeats = NeuralForecast(models=models, freq='H')


crossvalidation_df = nf_Nbeats.cross_validation(df=df, step_size=horizon, n_windows = (split - horizon + 1 ) // horizon)

      Model  Steps         MAE        RMSE        R2
0  A-NBEATS      8  343.775928  474.768585  0.311836
1    NBEATS      8  366.314055  502.145682  0.230183

Hi Blauzo! It is difficult to provide concrete feedback without more info on your case. Can you share more details? What would be a "good" performance? Can you run some simple baselines from statsforecast as well? It will give an idea on the forecasteablitiy of the problem.

Hey, so sorry for the delay. I will give you a quick rundown. My data is wind power generation from a wind farm, it has a few exogenous variables such as generator temp and wind direction (no wind speed due to a reading error). I have trained and cross-validated a few baseline models like ARIMA, SARIMA and Holt-Winter. The image has all the metrics I calculated with the forecasts. As you can see the N-BEATS model underperforms in relation to the baseline models. Right now I have managed to improve these results a bit, but they are still below the ARIMA and SARIMA baselines. I think it's a bit of an skill issue on me haha. I did test my code on another dataset (univariate wind power generation) and got it working well with little effort.

Thanks for sharing more details @Blauzo. Can you try using the

AutoNBEATS

? It will automatically select the best hyperparameters for your case. Also, if you have exogenous covariates you can include them in the

AutoNBEATSx

and

NBEATSx

models.

These results are with the auto-nbeats and auto-nbeatsx models. Just to be sure, what is the usual range for the

n_samples

you recommend? I got my best results in the 20-30 range. Also every now and then the auto model goes very bad, like -1000 MAE. Any ideas why?