How can I update the

storage_path

for the

AutoNHITS

or

HyperOptSearch

? I'm trying to get the

ray

or

tune

working with

neuralforecast

and I keep getting this error below.

(_train_tune pid=21000) 2023-09-13 08:02:48,610 ERROR syncer.py:466 -- Caught sync error: Sync process failed: GetFileInfo() yielded path 'C:/Users/bubu/ray_results/_train_tune_2023-09-13_08-02-34/_train_tune_7a885f1f_1_h=12,input_size=24,learning_rate=0.0032,loss=ref_ph_de895953,max_steps=10,n_freq_downsample=24_12_1,n_pool__2023-09-13_08-02-39/error.pkl', which is outside base dir 'C:\Users\bubu\ray_results\_train_tune_2023-09-13_08-02-34\_train_tune_7a885f1f_1_h=12,input_size=24,learning_rate=0.0032,loss=ref_ph_de895953,max_steps=10,n_freq_downsample=24_12_1,n_pool__2023-09-13_08-02-39\'. Retrying after sleeping for 1.0 seconds...
(_train_tune pid=21000) 2023-09-13 08:02:49,617 ERROR syncer.py:466 -- Caught sync error: Sync process failed: GetFileInfo() yielded path 'C:/Users/bubu/ray_results/_train_tune_2023-09-13_08-02-34/_train_tune_7a885f1f_1_h=12,input_size=24,learning_rate=0.0032,loss=ref_ph_de895953,max_steps=10,n_freq_downsample=24_12_1,n_pool__2023-09-13_08-02-39/error.pkl', which is outside base dir 'C:\Users\bubu\ray_results\_train_tune_2023-09-13_08-02-34\_train_tune_7a885f1f_1_h=12,input_size=24,learning_rate=0.0032,loss=ref_ph_de895953,max_steps=10,n_freq_downsample=24_12_1,n_pool__2023-09-13_08-02-39\'. Retrying after sleeping for 1.0 seconds...

It seems this is a known bug and a suggested workaround is to predefine the storage path as below https://github.com/ray-project/ray/issues/36448#issuecomment-1673274452

from ray import air, tune

tuner = tune.Tuner(
    train_fn,
    run_config=air.RunConfig(storage_path="./")
)
tuner.fit()

However, I'm not sure where to define

run_config=air.RunConfig(storage_path="./")

with

AutoNHITS

Really appreciate any tips on this issue, thanks!

Hello @Afiq Johari! You can find the tune.Tuner method in the neuralforecast/common/_base_auto.py file, in line 180.

I think this PR is trying to address that: https://github.com/Nixtla/neuralforecast/pull/735

@Cristian (Nixtla) thanks for this tips, I'll check this out

while Ray fixes the issue I recommend you to try the NHITS directly. We know that Tune has a lot of issues, we are considering implementing alternatives when Tune is not an option.

@Cristian (Nixtla) yup, we are currently using NHITS, quite successfully, but just that it feels we're missing out for not leveraging the AutoNHITS given the automation

This certainly looks promising! Let us know how it works.

@Cristian (Nixtla) I tested

optuna

with

NHITS

, and able to run the hyperparameter tuning, and more importantly I can make use of the GPU on my windows server