https://github.com/nixtla logo
#neural-forecast
Title
# neural-forecast
m

Manuel

04/18/2023, 7:05 AM
I'm trying to use
neuralforecast
with
hierarchicalforecast
for hierarchical reconciliation (I've also seen
HINT
, but the provided reconciliation methods,
MinTrace
and
BottomUp
, are not suitable for my case). Now the problem is this: since time series in the dataset have a hierarchical relationship and time series at the higher levels tend to have higher values (because they are the sum of the hierarchically lower time series), loss functions such as plain
RMSE
and
MAE
seem to be unsuitable because the errors at the lower levels of the hierarchy, being smaller in absolute value, tend to be penalized too little. A workaround might be to use scale invariant loss functions such as
SMAPE
, which being a percentage error might mitigate the problem. Another solution might be to give more weight in the loss function (e.g., RMSE) to hierarchically lower time series. Do you know if with neuralforecast (and in particular with
NHITS
) there is a way to give a higher weight to some unique_id during model training? Do you have any other ideas about this? I also tried using losses such as PMM, GMM and NBMM but they did not give good results. Thanks
k

Kin Gtz. Olivares

04/18/2023, 10:21 PM
From what I understood: • You have stark scale differences between bottom and top levels. • Due to the scale difference, absolute metrics like RMSE/MAE prioritize the top levels. • You used SMAPE to help with the top-bottom scale difference. • You wonder if there are weighted losses in NF Answering the questions: • Like SMAPE, probabilistic losses (DistributionLoss, PMM, GMM...) are scale independent by definition. • Regarding weighted losses, check scale-independent errors and weighted MQLoss here is the NeuralForecast losses catalog. • Before ditching the probabilistic approach I recommend you to carefully explore: ◦ Use HierarchicalForecast tutorial to set an accuracy expectation. ◦ Explore your forecasting features, control for seasonalities if any. ◦ Model hyperparameters (learning rate and random seed in particular). Hope this helps.
m

Manuel

04/24/2023, 9:17 AM
@Kin Gtz. Olivares Thanks! I have yearly seasonality, what do you mean by "control for seasonalities"?
k

Kin Gtz. Olivares

04/24/2023, 10:06 PM
You can add month dummies, like the ones in this example: https://nixtla.github.io/neuralforecast/examples/hierarchicalnetworks.html
7 Views