Nixtla's Open Source Time Series Ecosystem.

Nixtla Community

hey <@U03DDFKMT0X>! As Kin mentioned, you’re right. The new syntax allows fitting multiple models without defining a dataset/loader for each of them. Also, the main idea behind `NeuralForecast` is that input and output are dataframes.  You can find an example here: <https://nixtla.github.io/neuralforecast/examples/getting_started_with_nbeats_and_nhits.html>.

Please let us know your thoughts about the new interface and if you have further questions.

Hi <@U03DDFKMT0X>, as you noticed, we have been working on a new version of NeuralForecast to make usage easier. We would love for you to play around and tell us if you want or need something.
New Exciting Features:
* Updated Getting Started: <https://nixtla.github.io/neuralforecast/examples/getting_started_with_nbeats_and_nhits.html>
* Unified interface for training and predicting
* Classes for automatic hyperparameter optimization for different models: <https://nixtla.github.io/neuralforecast/models.html>
* Multi Quantile Forecast
* Out of the Box GPU Support
* Unified Datasets/Dataloader class for all models
* Support for different hyperopt backends
* Pytorch lighting support

awesome, yah so far so good! we're tuning on our end but passing a single model to the wrapper seems to be working well. i did notice a change to the license though that appears to prohibit any commercial use of the library, do you happen to have any more details on the implications of <https://github.com/Nixtla/neuralforecast/pull/285/files>?

You can still use the library for basically whatever purpose you want. It’s the same Apache license + one specific restriction.
```May I create, distribute, offer as SaaS, and/or "sell" my products using Commons Clause licensed components?
Yes!

Commons Clause only forbids you from "selling" the Commons Clause software itself. You may develop on top of Commons Clause licensed software (adding applications, tools, utilities or plug-ins) and you may embed and redistribute Commons Clause software in a larger product, and you may distribute and even "sell" (which includes offering as a commercial SaaS service) your product. You may even provide consulting services (see clarifying discussion here). You just can't sell a product that consists in substance of the Commons Clause software and does not add value.

This is not a new concept. It's similar to "value-add" requirements in many licenses. For example let's say you use a library containing numerical algorithms from Rogue Wave Software. Can you create an application with the library and sell the application? Yes. Can you offer that application as SaaS and charge for it? Yes. Can you change the name of the library and change some function names and sell the library or offer it as SaaS? No.

Let's apply the example to Commons Clause licensed software. Commons Clause-licensed Redis Graph is a graph database module for BSD-licensed Redis. Can you create applications with Redis Graph and distribute and/or sell them? Yes. Can you redistribute Redis Graph along with your application? Yes. Can you offer that application as SaaS and charge for it? Yes. Can you take Redis Graph itself, call it ElastiGraph and offer it as SaaS and charge for it. No.```

Here is a complete FAQ: <https://commonsclause.com/>

Gotchya, yep that makes perfect sense. I may follow up in a few days with clarifying questions but I think that clears things up

Hi <!channel>: We have been working on a new version of NeuralForecast to make usage easier. We would love for you to play around and tell us if you want or need something.
*New Exciting Features:*
* Updated Getting Started: <https://nixtla.github.io/neuralforecast/examples/getting_started_with_nbeats_and_nhits.html>
* Unified interface for training and predicting
* Classes for automatic hyperparameter optimization for different models: <https://nixtla.github.io/neuralforecast/models.html>
* Multi Quantile Forecast
* Out of the Box GPU Support
* Unified Datasets/Dataloader class for all models
* Support for different hyperopt backends
* Pytorch lighting support

Hi <@U0449S695P0>, currently these models assume the data is sampled at regular intervals

Hello <@U03619XRWKD>, Thanks for answering the query. What would happen if we try to fit multiple irregular time series in these Global models. whether the model will fit the data or it will fail with an error?

During training, the ds column is only used for sorting the time-series, so it shouldn't fail with an error. However, the performance will be seriously affected as the lags will correspond to different times. Some methods and their parameters also assume regular time-series. For example, the val_size and test_size are in terms of number of time-stamps, so the splits wont be correct if series are irregular.

In short, we dont recommend using it for irregular time series.

One alternative is to make them regular by discretizing time (if its continuous), and fill the gaps with zeros for example

Thanks, <@U03619XRWKD> for the clarification and suggestion.

Hello <@U046XT7QW58>! Thanks for the interest in our library. These models are univariate, meaning that they produce forecasts for each time-series independently from the rest. However, these are "global models", so the same model is trained and used to forecast all the time-series of your dataset. Our research (and many other papers) show that these global univariate models forecast more accurately than the most recent multivariate models. Here is a link to the most recent documentation on how to use them: <https://nixtla.github.io/neuralforecast/examples/getting_started_with_nbeats_and_nhits.html>

Hello <@U03619XRWKD>. thank you for the response. The link says "File not found".

Hi! Here is the link of the updated version: <https://nixtla.github.io/neuralforecast/examples/getting_started.html>

thank you for this link. now I know that this models are momentarily for univariate forecasting. please let me know if the system have added for multivariate models. thanks.

Yes, the data is in the form of time series.

we can definitely attest to a difference in ease of tuning, but the lift against xgboost is significant (at least in our domain) and persists across ~95% of all of our ~1k forecasting tasks

XGBoost is all about creating features. Could this be the issue? 

we ran both with and without tsfresh features, same finding held up.

Nice there is also new one for features could be interesting for team <https://github.com/predict-idlab/tsflex|https://github.com/predict-idlab/tsflex>

Did DL win on global models (multi series) or univariate as well?

yah for clarity this is all for multioutput single series using exogenous variables

Too much work recently <@U03DDFKMT0X> so not much time but will be soon starting forecasting project so will pickup again. What DL model is shining?

<@U03DDFKMT0X> Did you also test out TFT out of curiosity?

We did a little while back but only locally, nothing in production. We struggled to auto tune TFT ourselves but part of that was probably bc we didn’t study the search space well and we were training on CPUs. I recall training being quite slow. We’ve been focused on infrastructure and orchestration the past couple weeks but expect to shift back to ML on the next two sprints. Will give it a try and circle back! 

Hi <@U04816WA7N3>,

1. I am currently working on the latter, to include probabilistic outputs on all the neuralforecast models, in this <https://github.com/Nixtla/neuralforecast/tree/DeepAR|branch>. I might have something working by the end of this week. I will let you know. If you have links to PyTorch implementation of tweedie and zero inflated losses we would appreciate it.
2. Regarding the categorical embeddings, depending on the model some have and others don't. We tried to keep as parsimonious as possible the baseline models (LSTM, RNN, MLP, TCN, ...). I am considering returning categorical encoders to NBEATSx, and NHITS.
3. If you need a model with categorical embeddings take a look to our temporal fusion transformer <https://nixtla.github.io/neuralforecast/models.tft.html|TFT>.

<@U038ZT9UDGA> Awesome! RE: tweedie - the third place solution to the M5 competition used deepar with a custom tweedie loss: <https://github.com/devmofl/M5_Accuracy_3rd>

Cool, thanks for the pointer <@U04816WA7N3>

Hey <@U04816WA7N3>,

If you want to check, we added to main a first iteration of `DistributionLoss`, here:
• <https://github.com/Nixtla/neuralforecast/commit/f6b7102299970c6a2810692557096b4a6975f1fa>
• <https://github.com/Nixtla/neuralforecast/blob/main/nbs/losses.pytorch.ipynb>
We are still polishing the interaction of the data's scale and the DistributionLoss scale parameters.
For count data, the `DistributionLoss(distribution='Poisson')` in combination with 'ReLU' encoders networks   (`MLP`, `NBEATS`, `NHITS`, `TCN`) was giving me reasonable results already.

<@U038ZT9UDGA> Thanks! I'm looking forward to digging in.

We are still working out some details around the parameters as outputs of the network.
It might take some weeks.

Early results with the Normal and StudentT seem promising

Understood. Does that mean you've seen an issue with the Poisson distribution?

I am exploring an optimization technique that decouples scale/location from the parameters estimation.
The parameters need to be restricted to reasonable domains (Poisson rates need to be positive for example).

If you combine a scaler with the DistributionLoss the normalization and the domain constraints interact poorly.

I'll be greatly interested to see how that work pans out! Are you aware of how other major DL forecasting libraries, like gluonts, deal with the issue?

GluonTS also has a decoupled optimization strategy that helps its networks with the distribution parameters estimation.
We are taking some inspiration from it, but I intend to improve some details.

Screen Shot 2022-11-23 at 9.27.49 AM.png

Hi <@U045VS8T481>,

The way that one can accomodate seasonalities in the <https://nixtla.github.io/neuralforecast/models.nhits.html|NHITS> model is through the `n_freq_downsample` parameter.
• In your case with monthly data you can try for example `n_freq_downsample=[6,3,1]` . 
• It seems that the scale of your data might be challenging too, you might want to try the `scaler_type='norm'` to help the optimization of the network (even ReLU nonlinearities might struggle with very large signals).
• You might want to train the network for a bit longer epochs.

<@U038ZT9UDGA>, thanks for advice. i have increase the epochs = 100, and added downsamples as you suggested.
thing did changed afterwards. but still not close. may try scaler next. but seasonality is not included it the model, so far.

i check a post and it seems their nbeats not forecasting seasonality ups an downs either: <https://pytorch-forecasting.readthedocs.io/en/stable/tutorials/ar.html>

also, how to import AutoNHITS?
from neuralforecast.models import AutoNHITS , AutoNBEATS does not work.

```from neuralforecast.auto import (
    AutoMLP, AutoNBEATS, 
    AutoRNN, AutoTCN, AutoDilatedRNN,
)```

I think that the scale might be challenging to the models.
Can you try the Box-Cox transform, taking logarithms of your data and then training the network?

TFT comes in mind if you want to do forecasting. But first try with a simple lin reg and then move on to random forest/lightgbm. Get those as baselines. Then move over to more deep learning stuff imo.

I already trained LR and tree based architectures and I am considering those as baseline models and I read TFT paper but TFT comes for time series problem where they are taking the timestamp information separately, I don't have timestamp info but I can use index as timestamp but it will not make any sense but it is worth giving it a try.

Have you tried a simple NN? Like a 2-3 layered one?

fine tuned it ? did you do neural architecture search ?

Yes, I fine tuned it but no luck with these approaches. I didn't do the NAS till now but will have a look for sure.

try fione tuning nr of layers and nr of neurons

Hey <@U0443KG42EQ>,

If you have a classic regression problem, you might want to treat it as such.
P(Y_i | X_i)

The methods that we focus on Nixtla are forecasting methods, solving:
P(Y_[t+1:t+h] | Y_[ :t ])

Key <https://stats.stackexchange.com/questions/65287/difference-between-forecast-and-prediction#:~:text=In%20time%20series%2C%20forecasting%20seems,respect%20to%20the%20given%20data.|regression/forecasting differences>:
• forecasting aims to predict several steps ahead, making it a multivariate regression problem.
• forecasting usually exploits time dependencies in the features, for example through lags (past series values). TFT, along with most Nixtla's methods are actively exploiting the time dependency structures.
• regression often times does not consider if the prediction will be future, current or past.
As <@U04BX22QMHT>, suggested you might want to try RF/LGBM and linear regression first.
If your problem does not have time dependencies, forecasting methods might not be suited.

If you are interested in trying TFT here is a usage example:
<https://nixtla.github.io/neuralforecast/models.tft.html>

it's not a perfect comparison bc most of the hparams don't line up quite right between the two libraries but it is definitely noticeable

wondering if it has to do with your handling of lagged regressors, the `ptfc` version seems to limit you to lagged regressors from the target variable only

Hi <@U03DDFKMT0X>

That is cool, and as you noticed: NeuralForecast's NHITS allows for future, lagged and static exogenous variables.
We made these exogenous variables available for all the models that we host.

In the NBEATSx paper, exogenous variables gave us an advantage of 25% over the original NBEATS model.

image.png

Hi <@U04G03SQC4T>, thanks for using our library!

We are currently working on a tutorial on how to use exogenous variables, it will be posted later this week. In the meantime, there is a simple example in the usage example section of each model, such as <https://nixtla.github.io/neuralforecast/models.nhits.html>.

In short, you need to define the temporal variables (future and lagged) as columns in the `df` input dataframe of the `fit`  method, and the static variables with the `static_df` input. Also, you need to specify the names of the variables in the corresponding model inputs, such as `futr_exog_list` .

Thanks, it looks promising. I get down to implementing this :)

Hi Eric. Yes, all temporal features are scaled based on the scaler_type

In the following tutorial we show all the details on how to add exogenous features: <https://nixtla.github.io/neuralforecast/examples/exogenous_variables.html>

Hey <@U04G6E1TVHB> 

Here is a cross validation example with TFT:
<https://nixtla.github.io/neuralforecast/examples/forecasting_tft.html|https://nixtla.github.io/neuralforecast/examples/forecasting_tft.html>

Here is a cross validation example with StatsForecast:
<https://nixtla.github.io/statsforecast/examples/crossvalidation.html|https://nixtla.github.io/statsforecast/examples/crossvalidation.html>

Hope it helps

Oh, thank you so kind. I believe that even that is more than enough. Thanks again ! :+1:

Hi <@U04GUDQLHR8>,

Sends for using the library hope it gives you service.
The `n_harmonic`  parameter is particular to the NBEATS model as it controls the sines/cosines space of the seasonality projections. We explain it in page 7 of the NBEATSx paper: <https://arxiv.org/pdf/2104.05522.pdf>

The NHITS model specializes over frequencies based on the `n_freq_downsample` .

You can find here an example of the NHITS' decompositon through frequencies:
<https://github.com/Nixtla/neuralforecast/blob/main/nbs/models.nhits.ipynb>

Great thanks and for the swift reply. I will try it.

Hi <@U04BX22QMHT>,

You are right, using the `NeuralForecast.predict` method without additional arguments is analogous to forecast.

The Automatic Forecasting methods are low level classes that operate purely with PyTorch and Numpy.
For comparison of the `AutoTFT`, `AutoNBEATS`, and `AutoNHITS` we recommend you to use directly the `NeuralForecast`
high-level wrapper class and its methods.

Here is a `NeuralForecast` class documentation and usage example:
• <https://nixtla.github.io/neuralforecast/core.html>
• <https://nixtla.github.io/neuralforecast/examples/longhorizon_with_nhits.html>
I opened this issue:  <https://github.com/Nixtla/neuralforecast/issues/394>

Great! Thanks a lot!

Another question, when do you think you would be able to also have an AutoNBeatsx ?

Because exogeneous variables seem very important to my data

Hi <@U04BX22QMHT>,

For the moment we don't have `AutoNBEATSx`, thanks for noticing it. We opened an issue for it.

You can still recover an `AutoNBEATSx`-like behavior with the `AutoNHITS` model by setting `n_freq_downsample`
to a list of ones.

The `AutoNHITS` model already accepts, static, past and future exogenous data.
Here is an example:
<https://github.com/Nixtla/neuralforecast/blob/main/nbs/examples/Exogenous_Variables.ipynb>

Hey <@U04GUDQLHR8>,

The particular documentation of the models is a bit hidden, here is the documentation for `NBEATS` model:
<https://nixtla.github.io/neuralforecast/models.nbeats.html>

You can control the number of stacks through the `stack_types`, `mlp_units` , and `n_blocks` parameters. Beware that you need them to be synchronized.

If you want to try a simpler model increasing the stacks, there is also the `MLP`:
<https://nixtla.github.io/neuralforecast/models.mlp.html>

okay and thanks for quick reply; glad I didn’t miss the obvious this time :slightly_smiling_face:.

Hi <@U03DDFKMT0X>

I think this is a common error where the `ds` columns do not match in their types.
You would need to convert one of them using `<http://pd.to|pd.to>_datetime`.

ahh yep, sorry about that :+1: i dropped the conversion to troubleshoot some tz stuff and forgot to add it back in. thank you!

circling back on this, it was actually a tz-aware error:
```ValueError: You are trying to merge on datetime64[ns] and datetime64[ns, tzfile('/usr/share/zoneinfo/US/Eastern')] columns. If you wish to proceed you should use pd.concat```

<@U038ZT9UDGA> do you know if tz-aware timestamps have been tested with `.cross_validation()`?

<@U03DDFKMT0X>, I am unfamiliar of tz-aware timestamps it might depend on the pandas versions that we currently have, in particular it would depend on wether pandas has updated the methods we are using within the library for tz.

I would try them and see if the result seems reasonable

i gave it a try, the normal `.fit()` method works but `.cross_validation()` fails, i think it's when merging the cv forecast dataframes. i can toss up an issue with replication steps if that's helpful?

on a separate note, are there any examples of passing `pl.Trainer` kwargs down from either the core `NeuralForecast` class or the individual models, ie `NHITS`?

trying to get some more control over the trainer kwargs directly: `pl.Trainer(log_dir=..., auto_lr_find=...)`

circling back on the trainer stuff, it's a bit hacky but we found the easiest way to get a good starting LR was to:
• instantiate the model class (NHiTS, NBEATsx etc) with `max_steps=1`
• init the `NeuralForecast` class and run `.fit()`
• deepcopy `nf.models[0].trainer.datamodule`
• init your own trainer class and train with `pl.Trainer(...).fit(nf.models[0], datamodule=datamodule)`

Hi <@U04G6E1TVHB>,

Would you be able to elaborate a bit more on the compiling problems?

Sorry I had lost the connection. By the way I have a model that works well with NHITS but not with TFT, while in the documentation it follows the same prediction logic as NHITS. For cross-validation, I dare not even touch it!

<https://colab.research.google.com/drive/1CTsfcYR-pQ2oZfI_bRy8mLANzTV-aK-1?usp=sharing>

Hello <@U046XT7QW58>. The "unique_id" is used to identify different time-series on your dataset. This is user generated because it depends on your data.

There is no constrains on the values, and the actual values will not affect the forecast. Simply use different identifiers (eg. "product_a", "product_b") for all the observations of each time-series. We have some examples here: <https://nixtla.github.io/neuralforecast/examples/getting_started.html>

complementing on <@U03619XRWKD>, here’s a description of the data format <@U046XT7QW58>: <https://nixtla.github.io/neuralforecast/examples/data_format.html>

Does this help? <https://nixtla.github.io/neuralforecast/examples/getting_started.html>

Hi <@U04KPJ0SD4K>

Complementing on <@U035DR8HD6D>. You can use the `.predict,`  method from the core class.
It is similar to StatsForecast's forecast method.
Here is the core class documentation: <https://nixtla.github.io/statsforecast/core.html>

Thank you <@U035DR8HD6D> &amp; <@U038ZT9UDGA>, It was a small over look from my side. It works with simple `.predict` function.

Hi <@U04LP14L7P0>. The library does not support mixing temporal variables with different resolutions. Both the target variable and exogenous variables should be included as columns in the same input dataframe with the same datestamps. We have a tutorial on adding exogenous variables: <https://nixtla.github.io/neuralforecast/examples/exogenous_variables.html>

The easiest solution is to repeat the monthly value for every day of the month. Current models assume all temporal variables have information at every timestamp.

Hi <@U04KPJ0SD4K>! You are correct, `predict()` is designed to forecast future observations after the dataset.

The closest method we have to predict past observations is the `cross_validation` method, designed to forecast multiple past windows. However, these windows are meant to be used as a validation set/test_set, so the model wont be trained on them.

We are currently working on a `predict_in_sample` method to forecast historical data, after the model is trained. We will let you know when we finish it.

Thanks for the reply, <@U03619XRWKD>. Is there any ETA on the `predict_in_sample` method?

We should have it in our main branch by Friday

Hi <@U03619XRWKD>, can you let me know once this function is pushed to the main branch?

We are pushing a new function `predict_rolled`. With this function you can predict many past windows, and the function will not fit models.

Hi <@U0449S695P0>! We have these additional models: MLP, RNNs (including GRU, LSTM, DilatedRNN), TCN, and Transformer-based (TFT). The full list of the `auto` models is in <https://nixtla.github.io/neuralforecast/models.html>

It is possible the DistributionLoss is not yet in PyPi, we will update the version in the following days. In the meantime, you can install it directly from the main branch with _`pip install git+<https://github.com/Nixtla/neuralforecast.git@main>`_

Hi <@U03619XRWKD>: The installation from the main branch is failing as well on my system. I am using clean virtual env to install it. please find the error below:
```pip install git+<https://github.com/Nixtla/neuralforecast.git@main> 
Collecting git+<https://github.com/Nixtla/neuralforecast.git@main>
  Cloning <https://github.com/Nixtla/neuralforecast.git> (to revision main) to c:\users\lenovo\appdata\local\temp\pip-req-build-anqkfyqq
  Running command git clone --filter=blob:none --quiet <https://github.com/Nixtla/neuralforecast.git> 'C:\Users\lenovo\AppData\Local\Temp\pip-req-build-anqkfyqq'
  Resolved <https://github.com/Nixtla/neuralforecast.git> to commit 6fe9b933fc658a0f96bf968945c9ba59f2bfded8
  Preparing metadata (setup.py) ... done
Collecting numpy&gt;=1.21.6
  Using cached numpy-1.24.2-cp310-cp310-win_amd64.whl (14.8 MB)
Collecting pandas&gt;=1.3.5
  Using cached pandas-1.5.3-cp310-cp310-win_amd64.whl (10.4 MB)
Collecting torch&gt;=1.12.1
  Using cached torch-1.13.1-cp310-cp310-win_amd64.whl (162.6 MB)
Collecting pytorch-lightning==1.6.5
  Using cached pytorch_lightning-1.6.5-py3-none-any.whl (585 kB)
ERROR: Could not find a version that satisfies the requirement ray[tune]==2.0.1 (from neuralforecast) (from versions: 2.2.0)
ERROR: No matching distribution found for ray[tune]==2.0.1```

Hi <@U03DDFKMT0X>! Yes, this is by design. All features should be pre-processed before passing them to the models. Can you provide more details on what issues do you have with the encoded features?

You can set scaler_type to None to remove scaling, to identify if this is the issue.

sure yah, so for example we're taking a column of data representing hour of day (1-24) and converting those to dummy variables via hot encoding. but in the example re: exogenous the variable `weekday` doesn't get hot encoded before passing it to the model

In this example we didn't one hot encoded the variable, but it should work (run) in both cases

circling back on this - it wasn't the encoding, we had gaps in our y values. but that brings me to another question - is it possible to create a future df that doesn't rely on "current" y values?

example: fitting to 2022 hourly data and using it to forecast Feb 9th 2023, *without* having access to y values from Jan 1 - Feb 8th 2023

moving this to general chat to avoid overthreading this haha sorry

Hey <@U04KSHMA6MU>,

We currently added Gaussian/Poisson Multivariate Mixture distributions that can be used as the output layer of any neural forecast model.

If you are certain on the existence of strong correlations of the target variables you can fit such multivariate models.

My recommendation when fitting multivariate models is to have a strong univariate baseline to which you can compare to make sure there are benefits of more complex models.

Hi <@U03DDFKMT0X> ,

That is the “cold-start” problem.
Although we have not explored that functionality. In principle neural forecast methods should be able of making such predictions.

The most important input for the “cold start” problem are the static features.

As long as other series with similar static features are available in your dataset a prediction with no temporal information should be feasible.

i think i may have misrepresented the problem :slightly_smiling_face: we have lots of y values historically, say from jan - dec 2022, as well as plenty of future exogenous variables. but we have no `y` values from jan - feb 2023. is it possible to use only exogenous variables and have no target lags?

Would you be able to create predictions and feed the predictions to the model in rolling window fashion?

Or for such series with missing values fit and predict a long horizon and drop the predictions with the missing lags?

good question, it might be possible. would it help if we provided some sample data from fitting + the input data frames? 