Nixtla's Open Source Time Series Ecosystem.

Nixtla Community

Hi <@U055ZP43Z4K>, can you provide more details? What model and you running and which hyperparameters?  how long is your data?

the model is Automlp  the size  of my data is 2522 hours the horizan  is 630  and the huperparameters is
"input_size_multiplier": [1, 2, 3, 4, 5],
    "h": None,
    "hidden_size": tune.choice([256, 512, 1024]),
    "num_layers": tune.randint(2, 6),
    "learning_rate": tune.loguniform(1e-4, 1e-1),
    "scaler_type": tune.choice([None, "robust", "standard"]),
    "max_steps": tune.choice([500, 1000]),
    "batch_size": tune.choice([16, 32, 64, 128]),
    "windows_batch_size": tune.choice([64, 128, 256, 512]),
    "loss": None,
    "check_val_every_n_epoch": tune.choice([100]),
    "random_seed": tune.randint(1, 20),
    **TRAINER_CONF,
}

try reducing the input_size_multiplier, it is larger than your data. Keep only 1 and 2

because we multiple the input size by the horizon right ?

the size of my data doesnt fixed how can i put condition to avoid this problem for futur because i am trying to make the code automatic

If you are specifying your own config, you also need to drop the `input_size_multiplier` and change it to `input_size` directly. You could have if conditions before defining the config dictionary

so you change the input size based on the length of your data

but it should smaller than my data there are no other condition ?

try to have input size a lot smaller than your data. because models operate based on windows of size input_size+h, if it is too large then there will be very few windows, which is not good

having short data is a challenging setting for deep learning methods

the model does it internally, it is part of the `BaseWindows` and `BaseRecurrent` base classes, in the `create_windows` method

or how can i undersatnd what actually happend in fit and predict for auto model is there any documentation for this ?

this is general for all models, not for the `auto` part to select hyperparameters. Windows-based models are functions from past values (`input_size`) to future values (`horizon`). During training, models sample a batch of windows (of size `input_size+h` ) starting at random timestamps. They produce the forecasts for all windows of the batch, compute training loss (based on the last `h` points of each window), and update parameters.

The unique number of windows is ts_length-input_size-h+1, so if input_size and h are too large, the model is trained on only a small set of windows, potentially overfitting to them.

`auto` models automatize the pipeline of trying different hyperparameters, observing their performance in a validation set, and selecting the best. They train `num_samples` different models with different hyperparameters, evaluate them on the validation set, and store the best based on the validation loss.

does the fit function choose the validation set size and test set size to choose the best collection from hyperparams or it doesnt have test set , is my question clear ??

models are evaluated on the validation set. You can specify it will the `val_size` parameter of the `fit` and `cross_validation` methods. If you dont specify `val_size`, auto models will use a validation set equal to 1 horizon of length. The test set is specified by the user, and it is only used for the `cross_validation` function. The models is NOT automatically evaluated in the test set, instead, the function will return the forecasts for the test set so you can visualize/evaluate them.

and also i have this in auto model every time i run the auto model i get different result why i didn't get the same predictions for my variable , do i should get same prediction or not ,and this difference, is it  because of random seed  and i set the random seed to fixed value  but still get different prediction every time i run the auto model