https://github.com/nixtla logo
#neural-forecast
Title
# neural-forecast
d

Dawie van Lill

05/30/2023, 9:52 AM
Hi, I just want to make sure I fully understand everything. I have the following config dictionary
Copy code
mlp_config = {
        "input_size": tune.choice([2, 4, 12, 20]),
        "hidden_size": tune.choice([256, 512, 1024]),
        "num_layers": tune.randint(2, 6),
        "learning_rate": tune.loguniform(1e-4, 1e-1),
        "batch_size": tune.choice([32, 64, 128, 256]),
        "windows_batch_size": tune.choice([128, 256, 512, 1024]),
        "random_seed": tune.randint(1, 20),
        "hist_exog_list": tune.choice([pcc_list]),
        "futr_exog_list": tune.choice([fcc_list]),
        "max_steps": tune.choice([500, 1000]),
        "scaler_type": tune.choice(["robust"]),
    }
The
hist_exog_list
and
futr_exog_list
are lists of historical and future exogenous variables. When I fit the model I use
nf.fit(df=df, val_size=20)
where the
df
is my dataframe that contains the target variable as well as the exogenous variables. Then when I predict I use
nf.predict(futr_df=futr_df)
where
futr_df
contains only the observations of the future exogenous variables that extend beyond the time period of the point of prediction. Does this seem correct? Or am I doing something wrong in the specification of the future exogenous variables. In my case there is only one period beyond the cut-off for the target where future exogenous variables are available, so the
future_df
dataframe only has one row and many columns (for the different features).
Oh, as an aside. Is there a way to include lagged features in the neuralforecast models?
Last question, I promise! When I forecast over different horizons, say
h=1
vs
h=2
, I get very different results with respect to the one-period ahead forecast. Why would this be the case? I am determining the size of the validation set in the
fit
method. Are there other places where the horizon features in tuning of hyperparameters? I have fixed the
input_size
as well, so that the
input_size
is not dependent on the horizon.
k

Kin Gtz. Olivares

05/30/2023, 1:10 PM
Hey @Dawie van Lill, • The config seems correct, • The "input_size" controls for the lags/autorregresive features of the method, it is convenient to make it as multiples of horizon, (example for 24 hours ahead you may want input_size=2*24 or input_size=7*24). • Regarding the model variance between horizons, ◦ you can try an NBEATSx architecture with
dropout_prob_theta
regularization. ◦ a robust loss like MAE/HuberLoss (https://nixtla.github.io/neuralforecast/losses.pytorch.html#huber-loss). ◦ increase the valid_size, to improve the validation signal h=1, h=2, is a very small window and the hyperparameter optimization will be noisy (https://nixtla.github.io/neuralforecast/common.base_auto.html).
d

Dawie van Lill

05/30/2023, 1:22 PM
Thanks for the feedback. I am working with quarterly data and I am mostly interested in the one period ahead forecast, thus the
h=1
. I realise after playing around with the code that the
valid_size
is dependent on the horizon, so I fixed that to be about 10% of the entire sample size (at around 25). I am also interested in the longer term forecast
h=4
, which gives me all the quarterly forecasts up to one year ahead. However, this is where I notice a significant difference between my specification with
h=1
and
h=4
. Nothing is different between the models and I use the MAE loss function, like you specified. I might be missing something here. Then finally, I tried the regularisation with the NBEATSx architecture and NHITS, but I am getting some errors with regard to dropout. The code runs fine without it, but once I include the dropout component it breaks for some reason. I have
"dropout_prob_theta": tune.choice([0.1, 0.3])
specified in the config.
k

Kin Gtz. Olivares

05/30/2023, 2:25 PM
You might need to update NeuralForecast from main: !pip install git+https://github.com/Nixtla/neuralforecast.git