Hi there, I've got some promising good results usi...
# timegpt
j
Hi there, I've got some promising good results using TimeGPT. So good, I feel I should be skeptical 🤨 😂 It appears the main caveats are (and I hope there are inaccuracies/workarounds/solutions here ;): 1. Fine-tuning can't be carried forward between runs, thus if we have a data stream every second or minute (for many series), we'd have to run the entire pipeline every time. 2. This inherently introduces latency, and our primary optimization path would be to reduce size/complexity of data 3. Does using AzureAI and/or Spark assist? I suppose each series can be run in parallel, but does Azure enable project-specific fine-tuning/training? Impressive product, though indeed. Hoping I can find a way to make it work
j
Hey. We have a working implementation to save and reuse finetuned models, it should be available soon. At the moment the overhead can't be avoided.
j
ok that's great, thx. Seems somewhat synonymous with the Save and Load Models capabilities in NeuralForecast. would that rely on a service like Azure? Preference would be to be able to manage infra ourselves for security and such. One more Q. Our infra is currently in AWS. Any plans to support AWS in a similar fashion as Azure? Leveraging managed spark alongside the managed ML services could open up a lot of scalability potential. If you have projected timelines, that'd be great to understand. thx again
j
The way it'll work is you call a finetuning endpoint with your data and optionally provide an id and that model is saved and the id is returned to you to use it in other endpoints. For Azure the models would be saved in the container itself, we'd have to coordinate with microsoft on how to handle permissions for them to be saved elsewhere and that could take a very long time. I don't think we have plans on supporting other clouds but @Max (Nixtla) would know for sure.
j
Today in Azure, the models get saved in the client containers?
j
today the models don't get saved at all, they're finetuned and deleted when the function ends
j
ok thx. Also noticed the self-hosted option. Would that offer any more options for optimization? Again, it's a very impressive product and I'd love to find a way to make it work... Perhaps our use case is helpful in quantifying the scale of optimization potential... • 10-20 _hist_exog_ • freq=5min • Timestamps per Serie = 51,840 • forecast every 5 min during business hrs (96 per day) • up to 200 series At 200 series with fine-tuning, your calculator puts it at ~$70 per run, where there'd actually be < $0.0006 worth of new tokens being evaluated. Perhaps more than the $, I'd have a hard time grappling with the moral quandary of wasting so much electricity, (esp w/Microsoft as a beneficiary) 😂
I'll be doing some optimization/testing to determine which features can be removed, and how much history can be dropped before affecting performance
I was just reading how small the team is. Can hardly believe what you've accomplished with a lean team! hats off🤘
@José Morales I've upgraded to a paid API acct to facilitate more testing/optimization. I came across an article, where someone tested how much historical data is used by the model. he found the max number of rows used (anything > [number]) didn't appear to change predictions. He had a pessimistic view of it, but I find it so much more impressive if that's true. Few related Qs: 1. Can you share any insight into how much historical data would be leveraged for various horizons? a. Would want to pass in the minimum amount to reduce latency b. no sense increasing costs for you with unnecessary data while on API plan c. Assuming we end up with self-hosted or azure deploy, this has significant impact on cost estimation 2. For the training data set you used, up to what dates did that include? before forward testing, would like to confirm that the data we're testing wasn't included in your training set. 3. For enterprise self-hosted, would we have any additional controls over saving/continuously fine-tuned models? 4. Do you find the long horizon model can work better for horizons < 24 hrs? I've mainly tested forecasts < 2 hrs (freq * 24 ), but would like to see how it does with longer. Not sure where the line is for long model. Any insight here is appreciated. 5. Somewhat related: the docs have an interesting clue "Currently, the historical forecasts are not affected by
h
, and have a fix horizon depending on the frequency of the data." - Is that predefined? We'll be forward testing, but if we knew what the auto-h was for our freq=5min, we'd have a good idea of where what our benchmark is. This would also be very informative in looking at history to benchmark performance over different n training rows