Setting and tuning `conformal_distribution`. How h...
# mlforecast
a
Setting and tuning
conformal_distribution
. How high is a good amount to use for tuning the prediction intervals? I have data back to 2019 and I am forecasting pr. hour. Currently I am setting
n_windows=7
and
h=168
. So as far as I understand it will use the last
7*168 = 1176
observations for each time serie
168
for the first window,
168
for the second etc. - is this sufficient or is it waaay to little to tweak the interval?
👍 1
j
Hey. It's kind of like using cross validation to assess the score, the more data you use the better estimate you'll get, however the improvement decreases. Usually 4 or 5 windows is enough and you can even use 2, so I'd say 7 should be fine, but you could test this by moving the number of windows and measuring the change in the interval width.
👍 1
a
Sounds good 👍 But when I run with 7 windows I see some very negative values for some unique ids (at times where actual is at some of the lower values, but not 0). Is there any way I can avoid this? I am forecasting electricity demand using a LGBM, so I know that it will always have some sort of demand (and never negative). I am using a
max_horizon != None
so I am training one model pr. horizon.
Also I can see that I can set method to
conformal_distribution
or
conformal_error
- any where I can see documentation which explaines the difference? 😁
j
Are the negative values in the lower part of the interval? Or in the average? If they are in the average you can try using a different objective that ensures positive values like poisson or using a boxcox target transformation for example. If they are in the interval there's not a lot to do since the interval is symmetric and if the mean is zero the lower bound of the interval will always be negative. We don't have documentation on that, you can find the implementations here: • distributionerror The difference is that the distribution takes the quantiles from the means +/- scores and the errors takes the quantiles directly from the scores
🔥 2
v
We need this in neuralforecast.