Hi everyone, I’m encountering the following error message: ValueError: There are missing combinations of ids and times in futr_df. You can run the make_future_dataframe() method to get the expected combinations or the get_missing_future(futr_df) method to get the missing combinations. This happens when I set refit=True in cross-validation. Can anyone explain what this error means and possibly help narrow down the potential causes? Thanks!

That's unfortunate! Do you have a piece of code that I can run that reproduces the error? On face value this error shouldn't happen in cross-validation, but hard to comment if I don't know the specifics of what you're doing.

@Olivier # ============== Define Models ============== H = 7*7 INPUT_SIZE = 7*6 models = [ NBEATSx( h=H, input_size=INPUT_SIZE, futr_exog_list=FEATURE_COLUMNS, n_harmonics=1, n_polynomials=2, stack_types=['trend', 'trend', 'trend'], activation='ReLU', n_blocks=[3, 2, 3], max_steps=100000, val_check_steps=10, early_stop_patience_steps=15, random_seed=RANDOM_STATE, batch_size=64, loss=MAE(), valid_loss=MAE(), ), LSTM( h=H, input_size=INPUT_SIZE, futr_exog_list=FEATURE_COLUMNS, learning_rate=0.0001, encoder_n_layers=50, encoder_hidden_size=256, context_size=INPUT_SIZE, decoder_hidden_size=128, max_steps=100000, val_check_steps=10, early_stop_patience_steps=25, random_seed=RANDOM_STATE, batch_size=64, loss=MAE(), valid_loss=MAE(), ), KAN( h=H, input_size=INPUT_SIZE, futr_exog_list=FEATURE_COLUMNS, grid_size=15, spline_order=4, n_hidden_layers=6, hidden_size=256, max_steps=100000, val_check_steps=10, early_stop_patience_steps=15, random_seed=RANDOM_STATE, batch_size=64, loss=MAE(), valid_loss=MAE(), ), TSMixerx( h=H, input_size=INPUT_SIZE, n_series=192, futr_exog_list=FEATURE_COLUMNS, ff_dim=258, max_steps=100000, val_check_steps=192, early_stop_patience_steps=5, random_seed=RANDOM_STATE, batch_size=64, loss=MAE(), valid_loss=MAE(), ), ] --------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[12], line 2 1 nf = NeuralForecast(models=models,freq='d') ----> 2 Y_hat_df = nf.cross_validation(df=data_nf,val_size=H,refit=True) 3 Y_hat_df = Y_hat_df.reset_index() File c\ProgramData\anaconda3\envs\py312\Lib\site packages\neuralforecast\core.py1292, in NeuralForecast.cross_validation(self, df, static_df, n_windows, step_size, val_size, test_size, sort_df, use_init_models, verbose, refit, id_col, time_col, target_col, prediction_intervals, level, **data_kwargs) 1290 else: 1291 futr_df = None -> 1292 preds = self.predict( 1293 df=predict_df, 1294 static_df=static_df, 1295 futr_df=futr_df, 1296 sort_df=sort_df, 1297 verbose=verbose, 1298 level=level, 1299 **data_kwargs, 1300 ) 1301 preds = ufp.join(preds, cutoffs, on=id_col, how="left") 1302 fold_result = ufp.join( 1303 preds, test[[id_col, time_col, target_col]], on=[id_col, time_col] 1304 ) File c\ProgramData\anaconda3\envs\py312\Lib\site packages\neuralforecast\core.py933, in NeuralForecast.predict(self, df, static_df, futr_df, sort_df, verbose, engine, level, **data_kwargs) 931 expected_cmd = "make_future_dataframe(df)" 932 missing_cmd = "get_missing_future(futr_df, df)" --> 933 raise ValueError( 934 "There are missing combinations of ids and times in

futr_df

.\n" 935 f"You can run the

{expected_cmd}

method to get the expected combinations or " 936 f"the

{missing_cmd}

method to get the missing combinations." 937 ) 938 if futr_orig_rows > futr_df.shape[0]: 939 dropped_rows = futr_orig_rows - futr_df.shape[0] ValueError: There are missing combinations of ids and times in

futr_df

. You can run the

make_future_dataframe()

method to get the expected combinations or the

get_missing_future(futr_df)

method to get the missing combinations.

Thanks! I'm thinking this could be due to insufficient data - can you check by reducing H and INPUT_SIZE (e.g. to like H=7 and INPUT_SIZE=7, for example)?