On the topic of TFT, any suggestions on speeding up the training (besides GPUs obviously)? There seems to be some bottlenecks, maybe on the dataset loader side?

Hi Chris! Yes, TFT is an intrinsically slow model. We analyzed the cost of each component and the model itself accounts for almost all the computation (not the loader). For each batch it has to unroll the LSTMs and then it has the attention layer, with quadratic cost on the size of the window). There are many hyperparameters you can tweak to reduce the cost/time: • Reduce

windows_batch_size

(less windows per batch) • Reduce

input_size

(shorter input window) • Reduce

hidden_size

(smaller model)

Hey Cristian, thanks for running this to ground. We'll try these suggestions and let you know how it goes. lol TFT looks like a snail next to TCN 😄

It is, TFT is a WindowsBased approach TCN uses the forking sequences optimization