Is there a way to set up a validation set in neural forecast in which instead of testing on a portion of time, we instead test on different set of timeseries? I'm in a situation where I have many short timeseries. I'd like to use a validation set to tune my model but can't really afford using temporal cross validation. If instead I could train on a subset of timeseries and validation on another, I think I could maximize the value of my data better.

Hi @Phil! This is a great idea. The

core

methods are using the

TimeSeriesDataModule

to create the loaders, and it currently assumes that the dataset (collection of time series) is the same for both training and validation sets. I will modify the module to allow for different datasets, and then you will be able to pass a

valid_df

in

core

methods.

Awesome! thank you @Cristian (Nixtla)! Let me just say how great the community you've all created is. I'm excited by your efforts