https://github.com/nixtla logo
Powered by
n

Naren Castellon

07/10/2023, 8:23 PM
I am training an LSTM model and I find that when I want to train it with the nf.fit(Y_train_df) method it gives me the following error, can you help me with the problem. Attached model and error. Help me, please!!!
k

Kin Gtz. Olivares

07/10/2023, 8:28 PM
Hey @Naren Castellon, Here is the github issue associated with your problem: https://github.com/Nixtla/neuralforecast/issues/620
n

Naren Castellon

07/10/2023, 8:49 PM
Look, I do it as it says, but the problem is not solved
k

Kin Gtz. Olivares

07/10/2023, 8:51 PM
You need to declare the system variable before importing pytorch, pytorch-lightning or neuralforecast
Copy code
import os
os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
n

Naren Castellon

07/10/2023, 8:56 PM
Is it necessary to use this dataset, *_ = TimeSeriesDataset.from_df(df = Y_train_df) , or just use the normal training and test data as well Y_train_df = df[:len(df)-720] Y_test_df = df[len(df)-720:]
Another question, I was looking at neuralforecast does not have the n_job=- parameter, is there something similar that I can use in my model so that I can use all the kernels?
k

Kin Gtz. Olivares

07/10/2023, 8:59 PM
It is better to rely directly on NeuralForecast core class. To make the calls to the lower level models. Here is a tutorial: https://nixtla.github.io/neuralforecast/examples/getting_started.html Here are the core class methods' documentation: https://nixtla.github.io/neuralforecast/core.html#neuralforecast.fit You can make the separation of train and test datasets using fit/cross_validation val_size/test_size parameters
n

Naren Castellon

07/10/2023, 9:11 PM
Now I have the following problem
k

Kin Gtz. Olivares

07/10/2023, 9:12 PM
You might be hitting the limits of your RAM
You can try reducing valid_batch_size
But, if you are running this in your local machine you might need to cut the number of series in your dataset
10s is super high frequency
n

Naren Castellon

07/10/2023, 9:14 PM
And in that case what can I do? The data that I am using is 80,832, it is very small, but the frequency of the model is freq='6Min10S', I don't know if that has anything to do with it.
k

Kin Gtz. Olivares

07/10/2023, 9:14 PM
If you are running this locally I would close Chrome and everything else and then rerun
to have free RAM
But, overall I would really recommend running this in a Google Colab GPU
you can read your data directly from dropbox or drive
with pandas
n

Naren Castellon

07/10/2023, 9:18 PM
This is the model I'm running, look
c

Cristian (Nixtla)

07/10/2023, 9:25 PM
@Naren Castellon can you provide more details on your hardware/OS and how you installed the library (conda/pip/clone)?
The frequency has to be supported by pandas, I dont think '6Min10S' will work
n

Naren Castellon

07/10/2023, 9:28 PM
clone neuralforecast
I have the data with that frequency, look @Cristian (Nixtla), It is the closest with that frequency
c

Cristian (Nixtla)

07/10/2023, 9:33 PM
It seems that the delta changes right? from the second row to the third is not 6min10s. Still, the 
predict
method needs to create the future values with a frequency supported by pandas, like this.
but I think the errors you have are not caused by this, it seems to be an issue with the installation/compatibility as Kin was mentioning earlier. I will come back with potential solutions!
n

Naren Castellon

07/10/2023, 9:37 PM
I know that, but right there on the pandas page there is an example that they use (freq="2h20min"), so it occurred to me to try it that way... now suppose that you can't use that frequency "6Min10s", What can I do in this case?
c

Cristian (Nixtla)

07/10/2023, 9:41 PM
thanks for showing that example, I didn't know that it was a valid frequency! The frequency is only used to generate the 
ds
column of the future values, and nothing else
n

Naren Castellon

07/10/2023, 9:46 PM
@Cristian (Nixtla) It keeps giving me the same memory error, what can I do? Can you give me any suggestions?
f

fede (nixtla) (they/them)

07/10/2023, 9:55 PM
hey @Naren Castellon! Can you use a different loss? For example, 
RMSE
? To see if there is a problem with 
DistributionLoss
🙌
n

Naren Castellon

07/10/2023, 9:58 PM
Hello @fede (nixtla) (they/them), ok, i would do the test
f

fede (nixtla) (they/them)

07/10/2023, 9:58 PM
Also, exporting 
KMP_DUPLICATE_LIB_OK=TRUE
before launching the notebook usually solves a lot of issues with M1 🙌
n

Naren Castellon

07/10/2023, 10:00 PM
And where do I use that parameter, in the construction of the model or outside?
k

Kin Gtz. Olivares

07/10/2023, 10:02 PM
loss=MQLoss(level=your_levels) or loss=RMSE()
In the initialization of the model
n

Naren Castellon

07/10/2023, 10:03 PM
and the "KMP_DUPLICATE_LIB_OK=TRUE"
f

fede (nixtla) (they/them)

07/10/2023, 10:06 PM
Copy code
import os
os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
Powered by