https://github.com/nixtla logo
#neural-forecast
Title
# neural-forecast
p

Phil

10/23/2023, 9:17 PM
I'm trying to log the metrics from AUTONHITS to our MLfLow tracking. How can I set this up?
j

José Morales

10/23/2023, 9:26 PM
Hey. Do you mean the results of each configuration?
p

Phil

10/23/2023, 9:27 PM
Yes that's right. For each configuration, I'd like to have some kind of callback configured which could somewhow know the model id for that configuration and log the training and validation loss along with the chosen hyper-parameters
I know there is a tensor board setup but It's difficult to access from our cluster... If possible, I'd love to set it up so that it can log it in real time ( or atleast at every epoch) or something like that
j

José Morales

10/23/2023, 9:30 PM
Are you using ray or optuna?
p

Phil

10/23/2023, 9:30 PM
I'm using ray
j

José Morales

10/23/2023, 10:20 PM
Are you able to install from github?
p

Phil

10/23/2023, 10:21 PM
Depends on the package?
j

José Morales

10/23/2023, 10:21 PM
It'd just be neuralforecast
I just pushed some initial work to the callbacks branch, it'd be great if you could try it out. It adds a
callbacks
argument to AutoNHITS where you can provide a list of callbacks like the one in the example here
So if you can run:
pip install git+<https://github.com/nixtla/neuralforecast.git@callbacks>
you should be able to use it
p

Phil

10/23/2023, 10:24 PM
I won't be able to use it on our cluster since it automatically installs packages from our artifactory
but once this code is released, I can add the latest release version to our package artifactory and try it then
Can I see your PR?
j

José Morales

10/23/2023, 10:27 PM
Sure, it's this one. It needs more work because all the auto models need this argument, I only added it to nhits
p

Phil

10/23/2023, 10:28 PM
Very cool. I guess LinkedIn is using it's own callback method called:
Copy code
AIMLoggerCallback
But I could add it to the callbacks arguments
j

José Morales

10/23/2023, 10:30 PM
is it this one?
p

Phil

10/23/2023, 10:31 PM
looks like it
I guess they tweaked it slighly since It is an internal repo but it looks the same
j

José Morales

10/23/2023, 10:51 PM
ok. this will be available in the next release
❤️ 1
p

Phil

10/23/2023, 10:51 PM
Thank you very much @José Morales!