Performance question: I am using AutoNHITS and passing cpus=32 but when I look at the Ray's dashboard I only see one core maxed out at 100% while 31 cores are sitting idle. Has anyone seen issues with parallel processing? My versions are neuralforecast==1.3.0 & ray==2.2.0 and this is running on AWS EC2 with Amazon Linux.
02/10/2023, 8:00 PM
Can you try with cpus=1? In tune its the opposite, this is how much to reserve for each process (I believe that pl.lightning then limits the usage to 1 per process).
We recommend to use GPUs. On this data training 1 nbeats/nhits model should take less than a minute with a single GPU. You can therefore try many configurations (
) in a short period of time. We usually use small AWS instances with 1 GPU such as the g3.4xlarge
02/10/2023, 8:08 PM
Thanks. I did cpus=1 and I see it's running in parallel now. It's still not using all the cores but that's probably due to other settings and how much the algorithm decides it needs. I'll have to see if we have access to GPU instances but that is a good advice.