https://github.com/nixtla logo
#general
Title
# general
g

Ginger Holt

09/26/2022, 11:55 PM
How long does this usually take to run? Generating the fitted values seems to take a long time, and I ended up canceling.
Copy code
fcst = StatsForecast(df=Y_df_train, 
                     models=[ETS(season_length=4, model='ZZA')], 
                     freq='QS', n_jobs=-1)
Y_hat_df = fcst.forecast(h=8, fitted=True)
Y_fitted_df = fcst.forecast_fitted_values()
https://github.com/Nixtla/hierarchicalforecast/blob/main/nbs/examples/AustralianDomesticTourism.ipynb
m

Mariana Menchero

09/27/2022, 12:02 AM
Hi Ginger. Executing that cell took a minute in Colab. I just did it, so maybe re-start again your execution and try again.
k

Kevin Kho

09/27/2022, 12:38 AM
Do you have anything else running? That n_jobs=-1 might be competing for resources with something else?
👍 1
g

Ginger Holt

09/27/2022, 3:22 AM
Thank you @Mariana Menchero and @Kevin Kho for the help. I took out n_jobs=-1, and it ran in 59 seconds. 😃.
f

fede (nixtla) (they/them)

09/27/2022, 7:42 PM
Great, @Ginger Holt! 🙌
Hey @Ginger Holt! is it possible to share the specs of your environment (os, python version, num cores, ram)? Just to see if there is a bug in the library related to
n_jobs=-1
g

Ginger Holt

09/27/2022, 8:55 PM
@fede (nixtla) (they/them): I am using Databricks: https://learn.microsoft.com/en-us/azure/databricks/release-notes/runtime/10.4 which has these System environment parameters: • Operating System: Ubuntu 20.04.4 LTS • Java: Zulu 8.56.0.21-CA-linux64 • Scala: 2.12.14 • Python: 3.8.10 • R: 4.1.2 • Delta Lake: 1.1.0
Also, here is my cluster info: Summary 2-8 Workers 128-512 GB Memory32-128 Cores1 Driver 64 GB Memory, 16 CoresRuntime 10.4.x-scala2.12 Unity Catalog Standard_D16s_v3
h

Han Wang

09/27/2022, 10:23 PM
@Ginger Holt do you need to train multiple models on different partitions of the dataframe?
i am curious why you use databricks with a number of workers
3 Views