A longer question on MLForecast, probably for <@U0...
# support
t
A longer question on MLForecast, probably for @José Morales. If it's more than we want to do via email, I'll redirect him to our Slack (I already pointed him in that direction, but he still replied via email). Thank you!
1
Ricardo Inácio <up202302742@edu.fe.up.pt> wrote:
Good afternoon, I hope thod email finds you well.
I am having an issue regarding the AutoMlForecast class. I am confused as in the github repository, and in the documentation (https://nixtlaverse.nixtla.io/mlforecast/auto.html), the method "forecast_fitted_values()", allows for the computation of prediction intervals. However, when I call the method, it says that such method doesnt exist. Also, the "fitted" attribute from the "fit()" method is not defined. Is there any other way to obtain the intervals, as I am using an LGBMRegressor with AutoModel?
I asked for the code he's using
I am using the AutoModel and AutoMLForecast classes to optimize and tune a lightbm regressor:
Copy code
I am using  the AutoModel and AutoMLForecast classes to optimize and tune a lightbm regressor:



def train(self):

        """

        Trains the LightGBM model and performs hyperparameter tuning.

        """

 

        # fixed parameters

        lgbm_params = {

            "random_seed": 42,

            "boosting_type": "gbdt",

            "verbosity": -1,

        }

 

        # tunable parameters

        def my_lgb_config(trial: optuna.Trial):

            return {

                "learning_rate": trial.suggest_categorical(

                    "learning_rate", [0.02, 0.03, 0.04, 0.05]

                ),

                "num_leaves": trial.suggest_categorical(

                    "num_leaves", [4, 8, 16, 32, 64]

                ),

                "max_depth": trial.suggest_categorical("max_depth", [5, 10, 15]),

                "n_estimators": trial.suggest_categorical(

                    "n_estimators", [50, 100, 150]

                ),

            }

 

        def init_config(trial: optuna.Trial):

            return {

                "lags": [i for i in range(1, 12)],

                "lag_transforms": self.create_lag_transforms([12], 12),

                "target_transforms": [Differences([12])],

            }

 

        tuned_lgb = AutoModel(

            # scikit-learn compatible regressor

            model=lgb.LGBMRegressor(**lgbm_params),

            config=my_lgb_config,

        )

 

        optuna.logging.set_verbosity(optuna.logging.ERROR)

        init = time()

        self.lgbm = AutoMLForecast(

            # Hyperparameter optimization helper

            models={"LGBM": tuned_lgb},

            freq=self.frequency,

            season_length=12,

            init_config=init_config,

        ).fit(self.train_set, h=self.horizon, num_samples=10, n_windows=2)

        end = time()

        self.execution_time = (end - init) / 60  # Time in minutes

 

 

I then realized that the method I am referring to, only computes the predictions for the train set:

<https://nixtlaverse.nixtla.io/mlforecast/auto.html#automlforecast-forecast-fitted-values>

 

In this case, I was looking for making predictions with, let's say, a 95% prediction interval, when calling the AutoMLForecast.predict() method:

def forecast(self):

        """

        Generates forecast using the trained model.

        """

        self.prediction = self.lgbm.predict(self.horizon)


I was looking for making predictions on the test set (forecasting) alongside uncertainty.
One exemple could be the implementation form using the MLForecast directly:


mlf = MLForecast(

    models=regressor,

    freq="M",

    lags=[i for i in range(1, 12)],

    lag_transforms={ 12: [RollingMean(12)]},

    target_transforms=[Differences([12])],

) 

 

mlf.fit(

    train,

    prediction_intervals=PredictionIntervals(h=horizon)

)

 

mlf.predict(h=horizon, level=[95])

 

Is there a way to make this directly on the AutoModel / AutoMLForecst? I am using the M3 competition Monthly frequency dataset.
I then realized that the method I am referring to, only computes the predictions for the train set:
https://nixtlaverse.nixtla.io/mlforecast/auto.html#automlforecast-forecast-fitted-values
In this case, I was looking for making predictions with, let’s say, a 95% prediction interval, when calling the AutoMLForecast.predict() method:
Copy code
def forecast(self):

        """

        Generates forecast using the trained model.

        """

        self.prediction = self.lgbm.predict(self.horizon)
I was looking for making predictions on the test set (forecasting) alongside uncertainty. One exemple could be the implementation form using the MLForecast directly:
Copy code
mlf = MLForecast(

    models=regressor,

    freq="M",

    lags=[i for i in range(1, 12)],

    lag_transforms={ 12: [RollingMean(12)]},

    target_transforms=[Differences([12])],

) 

 

mlf.fit(

    train,

    prediction_intervals=PredictionIntervals(h=horizon)

)

 

mlf.predict(h=horizon, level=[95])
Is there a way to make this directly on the AutoModel / AutoMLForecst? I am using the M3 competition Monthly frequency dataset. Thanks in advance, Ricardo
j
The forecast fitted values method was included in yesterday's release v0.13.1, so upgrading should be enough for that. For the prediction intervals the auto model doesn't support that yet, but we can add it
t
Excellent, thank you! I'll let him know.
j
I just merged the prediction intervals in auto. They can be used by installing from github and will be in the next release
t
Awesome, thanks! I shared that with him.