https://github.com/nixtla logo
#neural-forecast
Title
# neural-forecast
f

Francisco Trejo

05/31/2023, 6:23 PM
Hello I have a question with regards to how the unique_id impacts NeuralForecast. I have been using the NHITS model with hierarchical data like in the first image and when I use the aggregate function from the HierarchicalForecast to create the data frame the data looks like the second image. I noticed when using the unique_ids from the HF aggregate function I get different forecast results despite everything else being 100% the same. This leads to RMSE values that are substantially different. The current work around I have is to just rename the unique_ids back to the aggregated ones given by the aggregate function from HF once I get the forecasts but just wondering if anyone might have any insight as to why this is happening and if its supposed to be behaving this way. Thank you!
k

Kin Gtz. Olivares

05/31/2023, 6:36 PM
Hey @Francisco Trejo, A common problem I have faced while working with hierarchical data, is that usually you want to preserve the hierarchical order of the unique_ids. If the unique_id is a string NeuralForecast sorts it lexicographically. To maintain the hierarchical order of the unique_id I like to change it from string into a categorical variable. Here is an example with a
sort_df_hier
function : https://nixtla.github.io/neuralforecast/examples/hierarchicalnetworks.html Let me know if this helps.
e

Eric Braun

05/31/2023, 6:51 PM
@Kin Gtz. Olivares Thank you for the insight - I'm confused, however, as to why the order of the unique_id's would change the results prior to reconciliation? Does it have something to do with how the time series are batched during training?
k

Kin Gtz. Olivares

06/01/2023, 1:44 PM
The bootstrap reconciliation is performing a matrix operation that requires the series to be hierarchically ordered: y_tilde = S P y_hat https://github.com/Nixtla/neuralforecast/blob/main/neuralforecast/models/hint.py#L259
f

Francisco Trejo

06/01/2023, 5:52 PM
@Kin Gtz. Olivares I should clarify that I am not using HINT or any reconciliation method yet. I am merely using NHITS with NeuralForecast. The biggest difference is that the way the unique_ids are formatted and the order of them seem to matter. When I use unique_ids with the "/" in them I get different results than the unique_ids that have "_" in them. Even if the order is completely the same.
k

Kin Gtz. Olivares

06/01/2023, 7:50 PM
Sorry @Francisco Trejo, I would need more information to know what is breaking in your application, still my intuition is that taking care of the unique_id ordering should do the trick.
3 Views