Hi everyone! Firstly, thanks for providing this excellent package – I hope to also contribute once I get a better grasp of the underlying code. My question is regarding the memory usage:

model = [RNN(h = test_length,

input_size = test_length*3,

scaler_type = 'robust')]

nf = NeuralForecast(models=model, freq='H')

nf.fit(df=df)

where test_length=1000. The training data consist of a single length 9000 time series. Using the default hidden layer numbers and sizes, a very rough estimate of the memory usage gives me: 1,280,000 parameters for 5.12 MB. Of course this would be multiplied by a few times for forward and backward pass memory and storing optimizer state in order to obtain total memory usage. What I don’t understand is that when I run:

nf.predict()

I receive the error message “OutOfMemoryError: CUDA out of memory. Tried to allocate 6.71 GiB (GPU 0; 11.75 GiB total capacity; 7.10 GiB already allocated; 3.88 GiB free; 7.13 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.” I have checked to make sure there is negligble GPU memory usage before executing nf.predict(). What do you think might be causing this issue? I am happy to provide the dataframe df I used if needed. I should also mention that this error does not occur when I restrict the RNN to single encoder/decoder hidden layers of size 100, rather than the default double layers of size 200.

Hey @James Wei, Thanks for using NeuralForecast 🙂 This is currently among one of our priority fixes in NeuralForecast inference. A partial fix was already implemented allowing you to use the

inference_input_size

, parameter that effectively trims the length of the time series to avoid using them entirely. I would recommend setting it to

inference_input_size=test_length*3

. Let me know if this helps. Would you be able to report this in a github issue if your problem persists?

This worked, thank you very much!

Good to know @James Wei Would you be able to post this as an issue on github? That way if somebody has similar issues they find the solution :)

Of course, will do it now!

Thanks

It looks like for fitting RNN, memory consumption allocated is still orders of magnitude larger than what is implied by the number of parameters. For example, using a dataset of length 10000 with 14 endogenous time series using default network & batch sizes uses much more than my 12GB capacity. Have you tried running nf.fit() on dataframes of similar size, and if so, what was the peak memory usage?

Hey @James Wei, We might have a curse of dimensionality problem showing up. Our RNN-based methods use a forking sequences optimization technique that creates the windows of size [series, time, channels, horizon]. Possible ways to alleviate the memory usage: • reduce the series batch size. • filter the length of the series of your data directly ir with input_size and inference_input_size • change to WindowsBased models like MLP, NBEATS, NHITS From our side we need to batch at the time level the creation of the forecasts.

Thanks for your info, I will take a closer look at the source code to get a better idea of how this technique works in practice!