https://www.nixtla.io logo
Join Slack
Powered by
This message was deleted.
# neural-forecast
s
This message was deleted.
c
Hi @Devin Gaffney! You can generate confidence intervals by changing the training 
loss
parameter of the model. For example, use the 
MQLoss
for multi-quantile, or a 
DistributionLoss
with the level parameter. See https://nixtlaverse.nixtla.io/neuralforecast/examples/longhorizon_probabilistic.html and https://nixtlaverse.nixtla.io/neuralforecast/examples/uncertaintyintervals.html
d
@Cristian (Nixtla) that seems to do it but.... I think whatever my parameters are, I'm not quite doing it right, lol
Copy code
>>> DistributionLoss(distribution='StudentT', level=[80, 90])
DistributionLoss()
>>> REPORTED_CONFIDENCE_INTERVALS
[75, 95, 99]
>>> models = [NBEATS(loss=DistributionLoss(distribution='StudentT', level=REPORTED_CONFIDENCE_INTERVALS), input_size=90, h=365, max_steps=50)]
Seed set to 1
>>> nf = NeuralForecast(models=models, freq='D')

>>> nf.fit(df=df)
Epoch 49: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  0.83it/s, v_num=6, train_loss_step=13.40, train_loss_epoch=13.40]
>>> forecast = nf.predict()                                                                                                                                                                                                                                                                
Predicting DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  6.45it/s]
>>> forecast
                  ds         NBEATS  NBEATS-median  NBEATS-lo-99  NBEATS-lo-95   NBEATS-lo-75  NBEATS-hi-75  NBEATS-hi-95  NBEATS-hi-99
unique_id                                                                                                                              
value     2024-01-15  880775.312500  831069.625000 -3.607529e+06 -1.258905e+06    4012.242188  1.769661e+06  3.098526e+06  5.329811e+06
value     2024-01-16  419336.093750  420239.812500 -3.836427e+06 -2.122078e+06 -533110.250000  1.403346e+06  3.009732e+06  5.239796e+06
value     2024-01-17  130431.367188  129012.929688 -3.735450e+05 -2.702184e+05  -87161.585938  3.545828e+05  5.109500e+05  6.274552e+05
value     2024-01-18  232599.687500  248514.937500 -2.859646e+06 -1.180909e+06 -210152.687500  6.923285e+05  1.576491e+06  2.881696e+06
value     2024-01-19   92505.289062   88933.375000 -1.869064e+05 -1.223944e+05  -36367.312500  2.211216e+05  3.258989e+05  3.916269e+05
...              ...            ...            ...           ...           ...            ...           ...           ...           ...
value     2025-01-09   45985.933594   39398.875000 -3.478920e+06 -1.647986e+06 -654498.625000  7.804569e+05  1.961868e+06  4.630426e+06
value     2025-01-10   73194.757812   78829.296875 -3.975971e+06 -1.964086e+06 -561564.687500  7.029534e+05  1.760353e+06  3.714584e+06
value     2025-01-11  108481.429688  120182.046875 -4.464982e+06 -2.120763e+06 -782349.687500  9.235725e+05  2.024399e+06  4.179062e+06
value     2025-01-12   30367.712891   32235.263672 -2.866290e+06 -1.771848e+06 -606043.312500  6.718169e+05  1.742667e+06  3.884185e+06
value     2025-01-13    9406.344727   48160.300781 -6.880750e+06 -1.874436e+06 -606047.187500  7.107528e+05  1.950270e+06  6.936456e+06

[365 rows x 9 columns]
>>> models = [NBEATS(input_size=90, h=365, max_steps=50)]
Seed set to 1
>>> nf = NeuralForecast(models=models, freq='D')
>>> nf.fit(df=df)
Epoch 49: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.16it/s, v_num=8, train_loss_step=7.27e+3, train_loss_epoch=7.27e+3]
>>> forecast = nf.predict()                                                                                                                                                                                                                                                                
Predicting DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 31.67it/s]
>>> 
>>> forecast
                  ds         NBEATS
unique_id                          
value     2024-01-15  110708.234375
value     2024-01-16  110636.796875
value     2024-01-17  110543.484375
value     2024-01-18  110596.789062
value     2024-01-19  110740.296875
...              ...            ...
value     2025-01-09  116703.382812
value     2025-01-10  116712.656250
value     2025-01-11  116823.351562
value     2025-01-12  115551.054688
value     2025-01-13  116534.046875

[365 rows x 2 columns]
when I run this nbeats without the confidence intervals, I get sensible results (e.g. the forecast is 110 going up to 116 within the year, totally within band). When I run with confidence intervals it falls apart and is offering forecasts orders of magnitude outside the observed data
Something wrong with my param selection or not training enough etc? Any sense on ways to not have it do this to me lol
c
Can you try scaling the data: https://nixtlaverse.nixtla.io/neuralforecast/examples/time_series_scaling.html. In particular with the 
scaler_type
parameter on the model. The DistributionLoss is very susceptible to the scale
d
awesome, will try that - thank you @Cristian (Nixtla)!
2 Views
Open in Slack
PreviousNext
Powered by