https://github.com/nixtla logo
#neural-forecast
Title
# neural-forecast
j

James Wei

05/01/2023, 10:58 PM
Hi everyone! Firstly, thanks for providing this excellent package – I hope to also contribute once I get a better grasp of the underlying code. My question is regarding the memory usage:
model = [RNN(h = test_length,
input_size = test_length*3,
scaler_type = 'robust')]
nf = NeuralForecast(models=model, freq='H')
nf.fit(df=df)
where test_length=1000. The training data consist of a single length 9000 time series. Using the default hidden layer numbers and sizes, a very rough estimate of the memory usage gives me: 1,280,000 parameters for 5.12 MB. Of course this would be multiplied by a few times for forward and backward pass memory and storing optimizer state in order to obtain total memory usage. What I don’t understand is that when I run:
nf.predict()
I receive the error message “OutOfMemoryError: CUDA out of memory. Tried to allocate 6.71 GiB (GPU 0; 11.75 GiB total capacity; 7.10 GiB already allocated; 3.88 GiB free; 7.13 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.” I have checked to make sure there is negligble GPU memory usage before executing nf.predict(). What do you think might be causing this issue? I am happy to provide the dataframe df I used if needed. I should also mention that this error does not occur when I restrict the RNN to single encoder/decoder hidden layers of size 100, rather than the default double layers of size 200.
k

Kin Gtz. Olivares

05/02/2023, 1:44 PM
Hey @James Wei, Thanks for using NeuralForecast 🙂 This is currently among one of our priority fixes in NeuralForecast inference. A partial fix was already implemented allowing you to use the
inference_input_size
, parameter that effectively trims the length of the time series to avoid using them entirely. I would recommend setting it to
inference_input_size=test_length*3
. Let me know if this helps. Would you be able to report this in a github issue if your problem persists?
j

James Wei

05/02/2023, 2:08 PM
This worked, thank you very much!
k

Kin Gtz. Olivares

05/02/2023, 3:24 PM
Good to know @James Wei Would you be able to post this as an issue on github? That way if somebody has similar issues they find the solution :)
j

James Wei

05/02/2023, 3:53 PM
Of course, will do it now!
k

Kin Gtz. Olivares

05/02/2023, 6:21 PM
Thanks
j

James Wei

05/04/2023, 5:25 PM
It looks like for fitting RNN, memory consumption allocated is still orders of magnitude larger than what is implied by the number of parameters. For example, using a dataset of length 10000 with 14 endogenous time series using default network & batch sizes uses much more than my 12GB capacity. Have you tried running nf.fit() on dataframes of similar size, and if so, what was the peak memory usage?
k

Kin Gtz. Olivares

05/05/2023, 3:05 PM
Hey @James Wei, We might have a curse of dimensionality problem showing up. Our RNN-based methods use a forking sequences optimization technique that creates the windows of size [series, time, channels, horizon]. Possible ways to alleviate the memory usage: • reduce the series batch size. • filter the length of the series of your data directly ir with input_size and inference_input_size • change to WindowsBased models like MLP, NBEATS, NHITS From our side we need to batch at the time level the creation of the forecasts.
j

James Wei

05/05/2023, 8:29 PM
Thanks for your info, I will take a closer look at the source code to get a better idea of how this technique works in practice!