Saeed Asadi at University of Ottowa is conducting some research on forecasting performance of pre-trained models. and shared results with us. His results and some questions in thread. Maybe @azul (she/her) (nixtla) or @Cristian Challu have some thoughts?

sad

thank you!

Giving my opinion even though nobody asked for it! 😄 Predicting the stock market is not really possible, and their results pretty much align with what we did on predicting bitcoin prices where a simple naive model is best. Pretty sure a naive model would give even better results than GARCH, but that depends on the horizon.

hey @Marco! thank you so much for this. i also share that intuition! could you try to write a response to their questions considering that? for the last one i think we can say that we dont have an experiment yet, but we have seen that exogenous variables usually help (maybe It might be a good a idea to point they out to our bitcoin tutorial)

Draft answer (let me know what you think): Thank you for sharing the results with us and for using TimeGPT! We notice that you are trying to forecast stock prices, which is commonly known to be unpredictable using only historical closing prices. It has been demonstrated that even the simplest model, like a naive model predicting the last known value, performs better than more sophisticated approaches, especially on short horizons. This also aligns with our conclusions from our experiment in predicting Bitcoin prices (see here). The main challenge when forecasting financial data is in discovering exogenous features that are predictive of the target. Including them is likely going to generate better results. Of course, finding these predictive exogenous features is a colossal challenge.

thank you, @Marco! pretty cool! i would add just a couple of suggestions: Thank you for sharing the results with us and for using TimeGPT! We notice that you are trying to forecast stock prices, which is commonly known to be unpredictable using only historical closing prices. It has been demonstrated that even the simplest model, like a naive model predicting the last known value, performs better than more sophisticated approaches, especially on short horizons. This also aligns with our conclusions from our experiment in predicting Bitcoin prices (see here). The main challenge when forecasting financial data is in discovering exogenous features that are predictive of the target. Including them is likely going to generate better results. Of course, finding these predictive exogenous features is a colossal challenge. One thing that we have seen that usually works well with TimeGPT for financial applications is ensemble it (using median or mean) with other classical models. Also, we suggest using

model="timegpt-1-long-horizon"

for long-horizon tasks. Given said that, we are working on improved versions of TimeGPT, and we expect to have better results in next iterations of the model for financial applications. i think we can answer that @Tracy Teal. thank you!

Excellent, thank you @Marco and @azul (she/her) (nixtla)!