Good morning. I have a quick question regarding degrees of freedom calculation for timegpt. I would like to calculate the error metrics CV(RMSE) and NMBE for the baseline model using historic data being used to train the model. However, in order to calculate the error metrics I need to find the degrees of freedom of the model, i.e., n - p, where n is the number of observations and p is the number of model parameters. Is there a way to calculate these number of parameters or degrees of freedom directly in timegpt or using other nixtla package like utils? Thanks!

The number of parameters is definitely very large, and it's something we cannot disclose. Otherwise, you can use the utilsforecast library and calculate of these metrics.

Thanks @Marco. I tried using the utilsforecast.evaluation library to calculate MAE, MSE and RMSE for 4 different dataframes: 1. Future predictions w/o finetuning 2. Future predictions w/ 10 finetuning steps and RMSE as the loss func 3. Historical predictions w/o finetuning 4. Historical predictions w/ 10 finetuning steps and RMSE as the loss function And the error metrics for all of them were very close to each other (I would have expected the finetuned predictions to be significantly better especially RMSE) Also the actual value of the RMSE seems to be much higher than what I would have expected (I was hoping the utilsforecast.evaluation RMSE method would incorporate the number of model parameters p in the metric calc even if it cannot be disclosed. But it looks like they're maybe using n-1 in the denominator instead of n-p?) The reason I'm asking about this is so that I can do an apples to apples comparison with an older "advanced" model we were using for predicting energy use data. Timegpt predictions look way better than this older model, but the error metrics for timegpt look worse which doesn't make any sense (we are primarily concerned with CVRMSE and NMBE). Here are a few questions I had: 1. Is the model still finetuning on previous data (e.g., the first 6 weeks of hourly data) even when calculating historical predictions? Or are historical predictions just straight up zero shot even if finetuning steps/loss func were passed? 2. Why do you think the error metrics came out to be the same for historic/future predictions w/ and w/o finetuning? The only common thing for all 4 cases was the train_df. I'll appreciate any help/advice you could provide for this. Thanks a lot!