Hi, I'm using NHITS with a configuration that performs well (tuned via Optuna). When I run cross-validation without prediction intervals, the results look good. However, as soon as I add prediction intervals, I have to set refit=True. At that point the model performance drops drastically, like the forecasts become flat lines even though I'm using the exact same config. Why is this and what can I do to mitigate this?

Are you using conformal intervals? Maybe training with a distribution loss would give better results.

Hi, thanks for your response. What distribution loss would you recommend looking into for conformal intervals?

With distribution losses, you don't need to use conformal intervals. Something like MQLoss could be a good starting point

It is for a study in which I am comparing prediction intervals, therefore I am using conformal intervals