https://github.com/nixtla logo
Powered by
c

Chris Gervais

05/31/2022, 2:33 PM
quick question about AutoNF - does NeuralForecast create new data loaders every time 
hyperopt
suggests a new model config? and if so, are each of those loaders guaranteed to contain the same data splits?
k

Kin Gtz. Olivares

05/31/2022, 2:35 PM
Hi @Chris Gervais , The most important data split is the time series cross validation one:
c

Chris Gervais

05/31/2022, 2:37 PM
sorry just to clarify the question, when you're running hyperopt, does it create new data loaders on every trial?
k

Kin Gtz. Olivares

05/31/2022, 2:37 PM
It does
c

Chris Gervais

05/31/2022, 2:37 PM
and you're probably anticipating the follow up question 🙂 are the splits identical for every train / valid / test loader?
k

Kin Gtz. Olivares

05/31/2022, 2:37 PM
Some experiment settings depend on things like batch size or the sampling frequency
Yes the train/validation/test splits are the same as the one in the picture
Some preprocessing of the windows might be different depending on hyperparameters As long as you maintain the ‘ds_in_test’ or the ‘mask_df’ parameters constant the split is respected
c

Chris Gervais

05/31/2022, 2:40 PM
gotchya, so you fix the index range for each split (i think it's called 
get_mask_dfs
)?
got it, perfect thanks!
k

Kin Gtz. Olivares

05/31/2022, 2:41 PM
Cool, if you find anything weird in the splits tell us. It is super important
👍 1
c

Chris Gervais

05/31/2022, 3:09 PM
seems to work as intended, but i did notice in 
model_fit_predict
, line 893 on tag 
0.0.9
that 
f_cols
doesn't get passed down to 
fit
. looks like it's on main as well. i could be wrong but that might be blocking exogenous variables from being used in auto-tuned models
k

Kin Gtz. Olivares

05/31/2022, 3:09 PM
The ‘f_cols’ is a placeholder, it is intended to distinguish past exogenous variables from future exogenous (available at prediction time) If you are using exogenous have in mind that they are used as future exogenous by default
c

Chris Gervais

05/31/2022, 3:11 PM
ah, it's not used for anything at the moment?
k

Kin Gtz. Olivares

05/31/2022, 3:12 PM
‘f_cols’ is not used for the moment
👍 1
Powered by