https://github.com/nixtla logo
#neural-forecast
Title
# neural-forecast
o

Oleksii Kaplenko

07/11/2023, 6:09 AM
Two main concerns: 1) train vs valid losses range has absolutely different scale 2) valid loss seems to grow during training, is it overfitting?
k

Kin Gtz. Olivares

07/11/2023, 9:52 AM
By default the model trains with MAE, and you are evaluating with MSE, which explains the losses’ scale difference. And early stopping is one of the strongest regularizers with regressions and neural networks.
o

Oleksii Kaplenko

07/11/2023, 10:10 AM
I've hardcoded
loss=MSE(),
valid_loss=MSE()
Is your previous explanation still valid? @Kin Gtz. Olivares
c

Cristian (Nixtla)

07/11/2023, 3:46 PM
Hi @Oleksii Kaplenko! The validation loss is scaled back to the original scale of the data
while the train loss is normalized
o

Oleksii Kaplenko

07/11/2023, 3:49 PM
Got it, thanks
r

Rafael Correia Da Silva

07/11/2023, 6:53 PM
hi @Oleksii Kaplenko just curious, how are you pulling your tensorboard plots?
o

Oleksii Kaplenko

07/12/2023, 8:51 AM
@Rafael Correia Da Silva using fit() method creates lightning_logs folder, inside this folder lies tensorboard files for all runs, to see these charts I use command:
tensorboard --logdir ./lightning_logs
🔥 2
amazing 1
r

Rafael Correia Da Silva

07/12/2023, 12:39 PM
wow I was doing this stuff by hand 🤡 this is soo helpful, thanks!