It was my monograph subject.
"Abstract
Since its proposal in 2019, Transformers models have been revolutionizing applications in various tasks, from Large Language Models to Generative Artificial Intelligence. In the context of time series it was no different, since the beginning variations of the original model have been proposed to assess whether such technology could also benefit prediction tasks. On the other hand, recent studies questions the real applicability of these models, due to their high cost but inferior prediction quality compared to simpler and more efficient models. This study aims to evaluate the main known Transformers models adapted for the prediction of time series, comparing them with widely used classical models for this same task, specifically in the domain of financial observations. The results demonstrate that even for a dataset with few observations, Transformers can obtain results similar to those of models traditionally used for this task, but at a higher computational cost. The use of benchmark datasets and the understanding of the behavior of well-established models will allow the construction of a relevant reference link for comparison with other models and studies."
Unfortunately it's written on Portuguese but these 3 Transformer models got similar MAE, RMSE and POCID scores to ARIMA, MLP, DLinear and NLinear. They got even better results than LSTM. I used the M6 dataset. The "only" drawback was their resources consumption, 20 to 30 times more than ARIMA and MLP, but since they got similar results than the most know algorithms, my conclusion is that they're indeed effective (not efficient) for time series forecasting.
Anyway that was just an academic monograph, It's finally concluded and i'm now studying for a more practical approach for financial time series forecasting. I'm putting all neuralforecasting automodels on Nixtla library to run now and compare their performance and resource consumption.
I'm really interested on better models, your comment made me curious, which ones do you suggest for that task?
Best regards!