Nixtla's Open Source Time Series Ecosystem.

Nixtla Community

image.png

Hi, I was observing that for the same cutoff, the same model will produce different predictions if used in cross-validation vs fitted and calling predict after.
Then I was reading the API description and it says it splits the data based on *n_windows* and refits the model at the start (assuming *refit*=False).
How can I keep the fitted model used in crossval to call *predict* later? I tried setting *fitted*=True but does not seem to work.
Is the train/test data division as simple as test_data = [ds_final-n_windows*horizon : ds_final], train_data=df - test_data?

Hey.
• The trained models are available in the `cv_models_` attribute
• Yes, that's the way the splits are done, unless you set a different step size (the default is the horizon). This can be inspected in the `cutoff` column from the CV result

Thank you! And does the *fitted* parameter make a difference?

That computes the in sample predictions in each window, so that you can then use `MLForecast.cross_validation_fitted_values()` to extract them

Ain't those the same as the output dataframe?

And is there any API to get the train/test data used in crossvalidation after passing the n_windows param?

I tried using the fitted model from `cv_models_` but couldn't get it to predict for new data. If I pass it to a new MLForecast it gives the same error:
```forecast_ml_cv_model = forecast_ml.cv_models_[0]['LinearRegression']
forecast_ml_cv = MLForecast(models=[forecast_ml_cv_model], 
                         lags=range(1, horizon+1),
                         lag_transforms={
                            1: [ExpandingMean()],
                            horizon: [RollingMean(window_size=horizon)],
                         },
                         freq='B')```
I am not sure on how to pass the data for the sklearn model without encapsulating it in the MLForecast()

• The output dataframe has the out of sample predictions (the forecasts on the validation sets), the fitted values are the in sample
• No but you can use the `utilsforecast.processing.backtest_splits` function to get them. It returns cutoffs, train, valid
To predict with the cv models you can do something like this:
```orig_models = forecast_ml_cv.models
forecast_ml_cv.models = forecast_ml_cv.cv_models_[0]
forecast_ml_cv.predict(...)
forecast_ml_cv.models = orig_models```

On a side note, I was able to use PatchTST from neural-forecast straight after crossval and it gave the same prediction. But for every prediction it shows this. Is it possible to omit the verbose output?

```import logging

logging.getLogger('pytorch_lightning').setLevel(logging.ERROR)```
for the progress bar you can disable it on the model constructor by setting `enable_progress_bar=False`

Ok, thanks again! The logging one didn't do much but the progress bar trick worked.