https://github.com/nixtla logo
#neural-forecast
Title
# neural-forecast
c

Chris Gervais

01/06/2023, 2:19 PM
I'm trying to compare our hand-rolled cross-validation methods (which uses
sktime
) with the
neuralforecast
native cross-validation but we're running into a merge issue. The culprit seems to be in the core
NeuralForecast
class here:
Copy code
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[56], line 1
----> 1 nf.cross_validation(df=df, static_df=static_df, val_size=val_size)

File ~/Desktop/rtobots/.venv/lib/python3.9/site-packages/neuralforecast/core.py:386, in NeuralForecast.cross_validation(self, df, static_df, n_windows, step_size, val_size, test_size, sort_df, verbose, **data_kwargs)
    383 fcsts_df = pd.concat([fcsts_df, fcsts], axis=1)
    385 # Add original input df's y to forecasts DataFrame
--> 386 fcsts_df = fcsts_df.merge(df, how="left", on=["unique_id", "ds"])
    387 return fcsts_df
and the actual error appears to be related to index types that don't match:
Copy code
ValueError: You are trying to merge on datetime64[ns] and object columns. If you wish to proceed you should use pd.concat
Replication: • using the exogenous tutorial here https://nixtla.github.io/neuralforecast/examples/exogenous_variables.html • replacing the
.fit()
code chunk with
.cross_validation(df=df, static_df=static_df, val_size=int(len(df) * .2))
k

Kin Gtz. Olivares

01/06/2023, 2:25 PM
Hi @Chris Gervais I think this is a common error where the
ds
columns do not match in their types. You would need to convert one of them using
<http://pd.to|pd.to>_datetime
.
c

Chris Gervais

01/06/2023, 2:39 PM
ahh yep, sorry about that 👍 i dropped the conversion to troubleshoot some tz stuff and forgot to add it back in. thank you!
circling back on this, it was actually a tz-aware error:
Copy code
ValueError: You are trying to merge on datetime64[ns] and datetime64[ns, tzfile('/usr/share/zoneinfo/US/Eastern')] columns. If you wish to proceed you should use pd.concat
@Kin Gtz. Olivares do you know if tz-aware timestamps have been tested with
.cross_validation()
?
k

Kin Gtz. Olivares

01/06/2023, 10:10 PM
@Chris Gervais, I am unfamiliar of tz-aware timestamps it might depend on the pandas versions that we currently have, in particular it would depend on wether pandas has updated the methods we are using within the library for tz.
I would try them and see if the result seems reasonable
c

Chris Gervais

01/08/2023, 7:57 PM
i gave it a try, the normal
.fit()
method works but
.cross_validation()
fails, i think it's when merging the cv forecast dataframes. i can toss up an issue with replication steps if that's helpful?
k

Kin Gtz. Olivares

01/08/2023, 7:57 PM
Sounds good
c

Chris Gervais

01/08/2023, 8:00 PM
on a separate note, are there any examples of passing
pl.Trainer
kwargs down from either the core
NeuralForecast
class or the individual models, ie
NHITS
?
trying to get some more control over the trainer kwargs directly:
pl.Trainer(log_dir=..., auto_lr_find=...)
circling back on the trainer stuff, it's a bit hacky but we found the easiest way to get a good starting LR was to: • instantiate the model class (NHiTS, NBEATsx etc) with
max_steps=1
• init the
NeuralForecast
class and run
.fit()
• deepcopy
nf.models[0].trainer.datamodule
• init your own trainer class and train with
pl.Trainer(...).fit(nf.models[0], datamodule=datamodule)
2 Views