Hello friends, I am getting the following error in a very interesting way. I found the reason for the error but I could not fully understand the reason. I am using M5 data and I want to do hierarchical reconciliation. I am training LightGBM and XGBoost models to estimate the total, cat_id, dept_id and item_id levels in M5 data.

fcst = MLForecast(
        models=[LGBMRegressor(verbosity = -1), XGBRegressor(random_state=0, n_estimators=100)],
        freq="MS",
        lags=[1, 2, 3, 4, 5, 6, 12],
        lag_transforms = {
            1:  [RollingMean(3), RollingMean(6), RollingMin(6), RollingMax(6), RollingStd(6),Combine(
                RollingMean(window_size=3),
                RollingMean(window_size=12),
                operator.truediv,
            )],
            6:  [RollingMean(3), RollingMean(6), RollingMin(6), RollingMax(6), RollingStd(6),Combine(
                RollingMean(window_size=3),
                RollingMean(window_size=12),
                operator.truediv,
            )],
            12: [RollingMean(3), RollingMean(6), RollingMin(6), RollingMax(6), RollingStd(6)]
        },
        date_features = ["year", "month", "quarter", month_sin_transform, month_cos_transform, quarter_sin_transform, 
                        quarter_cos_transform, is_summer, is_winter, is_fall, is_spring, is_first_month, is_last_month],
        target_transforms=[GlobalSklearnTransformer(sk_log1p)],
        num_threads=8
    )

When I create the MLForecast object as above, I get the following error. However, when I remove the Combine features, I do not get any error.

fcst = MLForecast(
        models=[LGBMRegressor(verbosity = -1), XGBRegressor(random_state=0, n_estimators=100)],
        freq="MS",
        lags=[1, 2, 3, 4, 5, 6, 12],
        lag_transforms = {
            1:  [RollingMean(3), RollingMean(6), RollingMin(6), RollingMax(6), RollingStd(6)],
            6:  [RollingMean(3), RollingMean(6), RollingMin(6), RollingMax(6), RollingStd(6)],
            12: [RollingMean(3), RollingMean(6), RollingMin(6), RollingMax(6), RollingStd(6)]
        },
        date_features = ["year", "month", "quarter", month_sin_transform, month_cos_transform, quarter_sin_transform, 
                        quarter_cos_transform, is_summer, is_winter, is_fall, is_spring, is_first_month, is_last_month],
        target_transforms=[GlobalSklearnTransformer(sk_log1p)],
        num_threads=8
    )

How can the feature I created with Combine cause this error? Can you help me? I can share the notebook if you want.

Hey. This error happens with the forecasts, right? Do you see anything wrong with them? Like nans or very large values?

Yes, This error happens when I run the reconcile method with forecasts, but it only happens when I use the forecasts I get when I add the feature I created with Combine. When I add the Combine feature, the y_hat and y_hat_fitted forecasts are as follows. There is no NaN value or very large value in sight. Everything looks normal.

Can you run

describe()

on the predictions or plot some histograms to see if their distributions are very different?

Hmm, there doesn't seem to be anything wrong with the predictions, so it's probably related to the reconciliation @Olivier do you have any ideas?

This is my notebook

Very late response (I missed this) but the error message is clear: MinT-shrink apparently doesn't produce a valid answer when you use the combine features. So, use another reconciliation method or try a different set of hyperparameters or algorithms. Methods like MinT-shrink can produce ill-conditioned answers, depending on the inputs. Apparently that's the case here. I'd guess that the Combine features perhaps don't meaningfully add to the prediction? Maybe check the feature importance of the model.