Hi guys, I'm trying to train AutoARIMA model with exogenous variables using spark in databricks. I have added exogenous variables after the y variable as suggested. train_df_monthly is in format of Spark dataframe. What options do I have, if I want to use spark dataframe and not pandas, as my dataset is really big.
Kind regards,
Daniel
I am executing this code:
# generate forecasts
sf = StatsForecast(
models = [AutoARIMA(season_length=season_length)],
freq = 'MS',
n_jobs=-1,
fallback_model=Naive()
)
y_pred = sf.forecast(
df = train_df_monthly,
h = h
)
And receiving this error:
PythonException: 'KeyError: "['EX_VAR1', 'EX_VAR2'] not in index"'. Full traceback below: