How long does this usually take to run? Generatin...
# generalg
Ginger Holt
09/26/2022, 11:55 PMHow long does this usually take to run? Generating the fitted values seems to take a long time, and I ended up canceling.
Copy code
fcst = StatsForecast(df=Y_df_train,
models=[ETS(season_length=4, model='ZZA')],
freq='QS', n_jobs=-1)
Y_hat_df = fcst.forecast(h=8, fitted=True)
Y_fitted_df = fcst.forecast_fitted_values()m
Mariana Menchero
09/27/2022, 12:02 AMHi Ginger. Executing that cell took a minute in Colab. I just did it, so maybe re-start again your execution and try again.
k
Kevin Kho
09/27/2022, 12:38 AMDo you have anything else running? That n_jobs=-1 might be competing for resources with something else?
👍 1
g
Ginger Holt
09/27/2022, 3:22 AMThank you @Mariana Menchero and @Kevin Kho for the help. I took out n_jobs=-1, and it ran in 59 seconds. 😃.
f
fede (nixtla) (they/them)
09/27/2022, 7:42 PMGreat, @Ginger Holt! 🙌
fede (nixtla) (they/them)
09/27/2022, 7:55 PMHey @Ginger Holt! is it possible to share the specs of your environment (os, python version, num cores, ram)? Just to see if there is a bug in the library related to
n_jobs=-1g
Ginger Holt
09/27/2022, 8:55 PM@fede (nixtla) (they/them): I am using Databricks: https://learn.microsoft.com/en-us/azure/databricks/release-notes/runtime/10.4 which has these System environment parameters:
• Operating System: Ubuntu 20.04.4 LTS
• Java: Zulu 8.56.0.21-CA-linux64
• Scala: 2.12.14
• Python: 3.8.10
• R: 4.1.2
• Delta Lake: 1.1.0
Ginger Holt
09/27/2022, 8:57 PMAlso, here is my cluster info: Summary
2-8 Workers
128-512 GB Memory32-128 Cores1 Driver
64 GB Memory, 16 CoresRuntime
10.4.x-scala2.12
Unity Catalog
Standard_D16s_v3
h
Han Wang
09/27/2022, 10:23 PM@Ginger Holt do you need to train multiple models on different partitions of the dataframe?
Han Wang
09/27/2022, 10:23 PMi am curious why you use databricks with a number of workers
3 Views