https://github.com/nixtla logo
v

Valeriy

06/25/2022, 9:25 AM
c

Chris Gervais

07/01/2022, 11:55 AM
NeuralProphet
ftw
c

Chris Gervais

08/18/2022, 11:53 PM
Definitely interesting! We've had a lot of success with
NeuralProphet
but always happy to try out other libraries 🙂 do you have a tutorial recommendation in
statsforecast
for multi-step prediction with exogenous lagged + future regressors?
would be super interesting to see the results between
statsforecast
and
NeuralProphet
when more extensive hyperparam optimization is applied, this looks like a single architecture was used throughout: https://github.com/Nixtla/statsforecast/blob/main/experiments/neuralprophet/src/neuralprophet.py ... FWIW, we've found we need around ~70 NN architecture searches followed by about the same number of LR tuning runs, definitely not ideal but not unlike other NN approaches. fortunately the auto LR finder works reasonably well which makes the arch searches a lot more reasonable. we'll try to bring in the example from https://github.com/Nixtla/statsforecast/blob/main/experiments/arima_xreg/src/statsforecast.py this weekend and report back on any interesting findings we come across!
v

Valeriy

08/21/2022, 3:42 PM
@Chris Gervais please share will be interesting to see the results
c

Chris Gervais

08/29/2022, 3:40 PM
okay, we still need to do some more testing but so far we're finding that a properly tuned model from
NeuralProphet
consistently outperforms both
AutoARIMA
and
ETS
from
statsforecast
on a wide range of power system forecasting tasks. i can't comment on whether this extends to other time series baseline datasets but i suspect the ERCOT baseline is very similar to our datasets. obviously the tuning time to get decent results is a disadvantage relative to
AutoARIMA
and
ETS
but it seems the same tradeoff exists between
NeuralProphet
and
Nbeats / N-HiTS
in that it's relatively quick to spin up decent models in
NeuralProphet
whereas
Nbeats / N-HiTS
requires a much more extensive hyperparmeter sweep to get comparable results. we're still pretty new to the
nixtla
ecosystem of open source forecasting tools though, so i'm sure there are many things we're doing wrong, happy to chat about opportunities for improving our integrations with either
statsforecast
or
neuralforecast
FWIW, we may replace
RandomForestRegressor
from
sklearn
as our baseline in favour of
AutoARIMA
from
statsforecast
- works great and super efficient