Hi, I just want to make sure I fully understand everything. I have the following config dictionary

mlp_config = {
        "input_size": tune.choice([2, 4, 12, 20]),
        "hidden_size": tune.choice([256, 512, 1024]),
        "num_layers": tune.randint(2, 6),
        "learning_rate": tune.loguniform(1e-4, 1e-1),
        "batch_size": tune.choice([32, 64, 128, 256]),
        "windows_batch_size": tune.choice([128, 256, 512, 1024]),
        "random_seed": tune.randint(1, 20),
        "hist_exog_list": tune.choice([pcc_list]),
        "futr_exog_list": tune.choice([fcc_list]),
        "max_steps": tune.choice([500, 1000]),
        "scaler_type": tune.choice(["robust"]),
    }

The

hist_exog_list

and

futr_exog_list

are lists of historical and future exogenous variables. When I fit the model I use

nf.fit(df=df, val_size=20)

where the

df

is my dataframe that contains the target variable as well as the exogenous variables. Then when I predict I use

nf.predict(futr_df=futr_df)

where

futr_df

contains only the observations of the future exogenous variables that extend beyond the time period of the point of prediction. Does this seem correct? Or am I doing something wrong in the specification of the future exogenous variables. In my case there is only one period beyond the cut-off for the target where future exogenous variables are available, so the

future_df

dataframe only has one row and many columns (for the different features).

Hey @Dawie van Lill, • The config seems correct, • The "input_size" controls for the lags/autorregresive features of the method, it is convenient to make it as multiples of horizon, (example for 24 hours ahead you may want input_size=2*24 or input_size=7*24). • Regarding the model variance between horizons, ◦ you can try an NBEATSx architecture with

dropout_prob_theta

regularization. ◦ a robust loss like MAE/HuberLoss (https://nixtla.github.io/neuralforecast/losses.pytorch.html#huber-loss). ◦ increase the valid_size, to improve the validation signal h=1, h=2, is a very small window and the hyperparameter optimization will be noisy (https://nixtla.github.io/neuralforecast/common.base_auto.html).

Thanks for the feedback. I am working with quarterly data and I am mostly interested in the one period ahead forecast, thus the

h=1

. I realise after playing around with the code that the

valid_size

is dependent on the horizon, so I fixed that to be about 10% of the entire sample size (at around 25). I am also interested in the longer term forecast

h=4

, which gives me all the quarterly forecasts up to one year ahead. However, this is where I notice a significant difference between my specification with

h=1

and

h=4

. Nothing is different between the models and I use the MAE loss function, like you specified. I might be missing something here. Then finally, I tried the regularisation with the NBEATSx architecture and NHITS, but I am getting some errors with regard to dropout. The code runs fine without it, but once I include the dropout component it breaks for some reason. I have

"dropout_prob_theta": tune.choice([0.1, 0.3])

specified in the config.

You might need to update NeuralForecast from main: !pip install git+https://github.com/Nixtla/neuralforecast.git