Nixtla's Open Source Time Series Ecosystem.

Nixtla Community

image (1).png

hey <@U04CVB9M7L1>! What version of python are you using? I guess that may be the reason the minimum supported version is python3.7.

hey <@U04CVB9M7L1>! We have just released a version of HierarchicalForecast that does not depend on `mycolorpy` (the library that threw the error you showed). You can install it using: `pip install "hierarchicalforecast&gt;=0.2.1"`

Please let me know if any errors appear :raised_hands:

Which implementation of LightGBM are you using?

I mean are you using something like <http://Github.com/nixtla/mlforecast|Github.com/nixtla/mlforecast> or skforecast

`model = lgb.LGBMRegressor`
`models_ml = model(**params)`
`ml_fcst = Forecast(`
 `models=models_ml,`
`freq="W-MON",`
   `lags=[1, 2, 3, 4, 5, 6, 7, 8], )`
`ml_fcst.fit(`
`df_train,`
`id_col="unique_id",`
`time_col=column_name_for_date,`
`target_col=column_name_for_qty,)`
`pred_ml_df = ml_fcst.predict(weeks_to_forecast)`

Does this happen with statsforecast ets as well?

I use only your frameworks/libs. so yes, ets from statsforecast

I'd say it's probably related to your data. Are you able to share a small sample?

the dataset is quite small, around 300 rows, with not huge variance

<@U03UX9KR75W>
`model = ETS`
`models_stats = [model(**params)]`
`stats_fcst = StatsForecast(`
`df=df_train, models=models_stats, freq="W-MON", n_jobs=-1`
`)`

`stats_fcst.fit(df=df_train)`
`pred_stats_df = stats_fcst.predict(weeks_to_forecast)`

hey <@U04BX22QMHT>! Which `params`  are you using? The default `season_length` is 1.

ETS finds the best model with and without trend. If the data does not exhibit a clear trend, it will opt for the trendless model, so the forecast will be a flat line. Although visually not very attractive, this is the expected behavior as ETS is a weighted average of past observations in its trendless and seasonality-free version.

If you want to explore the seasonality of the weekly data, I recommend setting it using MSTL. For that model you can use `season_length=52`. (ETS currently does not allow season lengths greater than 24).

As for LightGBM, we have seen that it is quite good at capturing seasonalities, but be sure to include the relevant features. One way to model the seasonality in the `MLForecast`  library is by using the `differences` argument (<https://nixtla.github.io/mlforecast/forecast.html>).

Hello all! Do you happen to know if there is a problem for ETS if you have for 5 rows for y 0 as a value?

My dataset contains 260 inputs, which 4 or 5 of them contains 0 as y.

And when I've tried training ETS model with the season_lengths=24 (as <@U0316L4HWQ7>) mentioned above I get the following error: ValueError: Multiplicative seasonality is not appropriate for zero and negative values

However, if the seasonal_lengths is disable I don't receive this error and the training runs end2end (with almost the same predictions as <@U04BX22QMHT> posted above (and by almost the same predictions I mean same values for all the predictions))

Also <@U0316L4HWQ7> I don't quite get it what this parameter differences is? Can you please provide more details about how I should set it? Thank you in advance :slightly_smiling_face:

<@U04CLTA2EJD> As far as I reckon with differences you get the ts to be stationary like, but <@U0316L4HWQ7> can correct me

hey <@U04CLTA2EJD>! ETS had a problem handling data with zeros. It is fixed in the latest version of StatsForecast. You can install it using `pip install -U statsforecast`  or `pip install statsforecast&gt;=1.3.2` . Please let me know if the problem persists after changing the version. :raised_hands:

Yes, precisely <@U04BX22QMHT>! With the `differences` argument, you can make the time series stationary. It is similar to the integrated component of an arima model <@U04CLTA2EJD>. In the context of mlforecast, it allows considering trend and seasonality

Thanks <@U0316L4HWQ7>! Indeed the problem for ETS was because I've used statsforecast 1.3.1, after upgrading to 1.3.2 the problem didn't persist anymore!

Could you provide best practices for setting differences? On the dataset <@U04BX22QMHT> sent here for example? Thank you in advance once again, <@U0316L4HWQ7>!

And also, do you happen to know what am I missing in implementing the LGBM Classifier model for a classification task? I want to train the LGBM Classifier to predict binary classes (0 or 1) and the predictions I get are always the same (meaning all the predictions are 1s or all the predictions are 0s).

model = lgb.LGBMClassifier
models_ml = model(**best_hp)
ml_binary_fcst = Forecast(
    models=models_ml,
    freq="W-MON",
    # differences = [1],
)
ml_binary_fcst.fit(
    train_binary_df, id_col="unique_id", time_col="ds", target_col="y"
)
pred_ml_binary_df = ml_binary_fcst.predict(weeks_to_forecast)

image.png

I tried setting differences param like this : `differences = [1]` and here it is the output:

```model = lgb.LGBMRegressor
models_ml = model(**best_hp)
ml_fcst = Forecast(
    models=models_ml,
    freq="W-MON",
    differences = [1],
    lags=[i for i in range(1, weeks_to_forecast+1)],
)
ml_fcst.fit(train_df, id_col="unique_id", time_col="ds", target_col="y")
pred_ml_df = ml_fcst.predict(weeks_to_forecast)```

where best_hp are the best params the Optuna Search found using Bayesian Search (TPESampler) after a certain nr of trials. Any idea why the results are this bad? Am I missing something?

hey <@U04CLTA2EJD>! Maybe your data don’t exhibit a clear trend. In that case, omitting `differences` might be a good option. Also, since you’re working with weekly data, you could pass `differences=[52]`  to capture yearly trends.

Dear community, please share some love:

<https://twitter.com/MergenthalerMax/status/1596217596248047616?s=20&amp;t=YBg78-kLy3AFiWT-ei85fA>

Hello and thank you. I like the MSTL model for my timeseries. Do you recommend adjusting any hyperparameters for the MSTL or AutoARIMA models? As noted in the example notebook, I am using MSTL and AutoARIMA as trend_forecaster and am hoping to enhance prediction accuracy. I tried AutoTheta, but it takes much longer than AutoARIMA and the predictions are nearly same. Thank you very much.

hey <@U04DZF9R90Q>! Thanks for using our library. If your data exhibits a clear trend, you could use `RandomWalkWithDrift`  as a trend forecaster. Trying `AutoETS`   and `DynamicOptimizedTheta`  would also be a good idea.

<@U0316L4HWQ7>, Thank you for the suggestions. I tried to use AutoETS but I couldn't able to get it to work. Please see the error below

models = [MSTL(
    season_length=[24*7, 24*31],
    #trend_forecaster=AutoARIMA()
    trend_forecaster=AutoETS()
)]

```Exception                                 Traceback (most recent call last)
Cell In [474], line 1
----&gt; 1 models = [MSTL(
      2     season_length=[24*7, 24*31],  #8
      3     #trend_forecaster=AutoARIMA(truncate=True)
      4     trend_forecaster=AutoETS()
      5 )]

File ~\miniconda3\envs\neuralforecast\lib\site-packages\statsforecast\models.py:2975, in MSTL.__init__(self, season_length, trend_forecaster)
   2973 if repr(trend_forecaster) == "AutoETS":
   2974     if trend_forecaster.model[2] != "N":
-&gt; 2975         raise Exception(
   2976             "Trend forecaster should not adjust " "seasonal models."
   2977         )
   2978 if hasattr(trend_forecaster, "season_length"):
   2979     if trend_forecaster.season_length != 1:

Exception: Trend forecaster should not adjust seasonal models.```


hey <@U04DZF9R90Q>! Passing `AutoETS(model='ZZN')`  should work. In the default setting `AutoETS`  also adjust seasonal models, but in this case, we don’t want to model those patterns (they are already modeled by MSTL). Passing `model='ZZN'`  tells `AutoETS` to omit the seasonal models.

Thank you for responding. I used model='ZZN' and it worked. Sorry if this has previously been covered in the docs; I'm new to NBEATS and this package. How do I make levels work with AutoETS? When I pass levels through sf.forecast, I get
```ValueError: Columns must be same length as key```

Yesterday we released a new version of StatsForecast, including prediction intervals for `AutoETS` . You can install the new version using `pip install statsforecast&gt;=1.4.0` .

No problem, <@U04DZF9R90Q>!  let us know any questions you have

Thank you very much. I will update. Sorry for disturbing you; I'd also like to know if NBEATS may be used as a trend forecaster. I use NBEATS for my other timeseries and was wondering if the same could be done with MSTL

The timeseries I'm using with MSTL has a frequency of one hour. I have data going back five years. Initially, I tried NBEATS, but it was unable to identify the multiple seasonalities. I have seasonalities of 8 hours, weekly, and monthly. Then I tried your MSTL model, and it accurately reflected the seasonality.

Currently, only StatsForecast models are supported as trend forecasters for MSTL. An interesting approach to improve performance might be to ensemble MSTL and NBEATS (or our NTHIS model). Here’s an example of how to use NHITS using the NeuralForecast library: <https://nixtla.github.io/neuralforecast/examples/getting_started.html>

Thank you very much. I'll give NHITS a shot. I appreciate everything you've done. Have a wonderful day.

Dear community, here is our newest release:
<https://twitter.com/MergenthalerMax/status/1598397618572259328?s=20&amp;t=G__jeLNo12N2j7uCbwQxBA>

WhatsApp Image 2022-11-29 at 11.16.52.jpeg

WhatsApp Image 2022-11-29 at 11.19.43.jpeg

WhatsApp Image 2022-11-29 at 11.24.23.jpeg

We created some memes that we did end up not using: feel free to take them lol

Ha, I for one love these memes <@U035DR8HD6D>

Yay <https://medium.com/@valeman/-86655805a676>

<@U035DR8HD6D> what is this DOT thing, is there an article link please somewhere?

The theta family of models has been shown to perform well in various datasets such as M3. `StatsForecast` includes four models of the family: `Theta`, `OptimizedTheta`, `DynamicTheta`, `DynamicOptimizedTheta`. The implementation is based on the <https://www.sciencedirect.com/science/article/pii/S0169207016300243|work of Jose A. Fioruccia, Tiago R. Pellegrini, Francisco Louzada, Fotios Petropoulos, and Anne B.Koehlerf>, and optimized using numba.

Yes, you’re right! Dynamic Optimized Theta is an improvement of the original Theta method :raised_hands:

We saw it, we are preparing something for transfer learning too.

<http://app.nixtla.io/transfer-learning/>

Yeah I saw the transfer learning repo - I am working my way through all of your tools : ).

We have a framework for building real-time, lightweight AI applications that just uses Python: <https://wave.h2o.ai/>. I've built out a few applications to use our own time series forecasting tools but want to extend it to statsforecast, neuralforecast, etc. Maybe we can jointly host a webinar on all things time series in 2023 if you are up for that.

Sounds exciting, <@U042MMJG9JM>! Let’s do it!

Great, it's unlikely that I can get it going in December with the impending holidays, let me chat with our marketing team and I'll be in touch. If you guys want to DM me the emails of the best folks to work with, that would be a great next step

But you can see the referecne to that same paper on the posts

I wasn’t impressed by the acronym as well at the time. 

Hi <@U04DB0W0T0D>! Thanks for joining. <@U03619XRWKD> did some work on anomaly detection at Amazon; he might be an interesting talking partner.
Congrats on Third Eye, looks very exciting.

I would like to say the same about the matrix profile, but I think that thing is above my intellectual capabilities.

We are happy to help in way possible.

<@U04DB0W0T0D> Matrix Profile is great but have you checked out Conformal Prediction for anomaly detection?

thanks <@U035DR8HD6D>. I find what you’re doing at Nixtla more complex that matrix profile :upside_down_face:
<@U03L39VLJBA> nope, do you have a good reference to start with?

<@U04DB0W0T0D> I do, here <https://valeman.medium.com/how-to-detect-anomalies-state-of-the-art-methods-using-conformal-prediction-e02691659412|https://valeman.medium.com/how-to-detect-anomalies-state-of-the-art-methods-using-conformal-prediction-e02691659412>

nice! I spend a lot of time in my company explaining that forecasting-based methods are not good for anomaly detection
will dig into KNN-CAD

<@U04DB0W0T0D> spot on forecasting is not great for anomaly detection, Conformal Prediciton and MP are much better. Your might also check more recent papers by Vovk on <http://alrw.net|alrw.net> <http://alrw.net/articles/32.pdf> <http://alrw.net/articles/31.pdf> <http://alrw.net/articles/29.pdf> this paper from Fedorova and Vovk powers microsoft azure anomaly detection <http://alrw.net/articles/04.pdf> <https://www.youtube.com/watch?v=lg-vkdIPK3A>

Let me know if you find something interesting please I am not expert in this subject yet but would like to deep dive

Just saw preview from another open source forecasting library that used conformal prediction to boost probabilistic prediction intervals capabilities. The results are amazing, stay tuned will be announced soon. <@U035DR8HD6D> <@U031KHZH06B> 

Screenshot 2022-12-06 at 15.28.26.png

I found this comment from their paper especially true of our times:

```Classifying forecasting methods as being either of a "machine learning" or "statistical" nature has become commonplace in parts of the forecasting literature and community, as exemplified by the M4 competition and the conclusion drawn by the organizers. We argue that this distinction does not stem from fundamental differences in the methods assigned to either class. Instead, this distinction is probably of a tribal nature, which limits the insights into the appropriateness and effectiveness of different forecasting methods.```
... if there were ever just one word to describe the "DL vs ML vs Stats" debate in forecasting, *tribal* has got to be it lol

maybe i'm missing something but afaict this only really matters to academics. in practice the recipe seems actually very straight forward:
• try a bunch of fast + cheap shit, see what works
• try a bunch of slightly less fast + cheap shit, see what works
• repeat until you're happy or out of time / money
... i'd also love to meet the sucker who spent 11k training a DL forecaster :rolling_on_the_floor_laughing: we're down to ~10-15 cents a pop!

Yes, intermittent demand is hard and sometimes and sometimes an average is better. 
You could also try croston. Thank two guides might be helpful. 

<https://www.databricks.com/blog/2022/12/06/intermittent-demand-forecasting-nixtla-databricks.html|https://www.databricks.com/blog/2022/12/06/intermittent-demand-forecasting-nixtla-databricks.html>

<@U03L39VLJBA> could you eli5 me conformal prediction and it s advantages?

<@U04BX22QMHT> wrong thread for posting  I was planning to post in general. Here is explanation for 2 year old <https://www.tiktok.com/@rajistics/video/7145960334270975275>

You can find much more here <https://github.com/valeman/awesome-conformal-prediction>

So it`s basically getting quantiles for the predictions?

<@U04BX22QMHT> no, conformal prediction is a general framework it does not produce quantiles as such what it produces are Prediction Intervals where true values are guarantees to lie in with user specified probability. Classical quantile regression does not produce true quantiles - just because it says 75% quantile there are no guarantees there will be 75 percent of observations below it. You can see an example of what it does and what is the difference here. NeuralProphet trained with quantile regression shown to provide miscalibrated intervals but Conformal Prediction has corrected it. So it can both produce prediction intervals that are well calibrated or alternatively correct prediction intervals from other models. <https://valeman.medium.com/probabilistic-forecasting-with-conformal-prediction-and-neuralprophet-af9c87901d94|https://valeman.medium.com/probabilistic-forecasting-with-conformal-prediction-and-neuralprophet-af9c87901d94>

New paper from the team that published the first ever paper applying Conformal Prediction to forecasting Sequential Predictive Conformal Inference for Time Series <https://arxiv.org/pdf/2212.03463.pdf>

Hi <@U03L39VLJBA> fugue is an abstraction on top of spark dask as ray, Nixtla uses Fugue to interact with these backends. We achieved amazing result on Spark, please read this article <https://towardsdatascience.com/distributed-forecast-of-1m-time-series-in-under-15-minutes-with-spark-nixtla-and-fugue-e9892da6fd5c>

and more importantly, it's easy to use. With Statsforecast interface, it is trivial to scale out on Spark, here is a code example <https://github.com/Nixtla/statsforecast/blob/8ea8c9e0d72b8e2b2edb4fb18a81ef5f713ec8f0/nbs/examples/Prophet_spark_m5.ipynb#L112>

Hi <@U03QQBDG3EX> that’s awesome thanks for sharing these links.

Yes, spark is great. We use fugue as a backend that connects with spark, ray and dask. Is a great tool. <@U03QQBDG3EX> and <@U03SA41952R> have been great allies in our mission. :) 

:tada::tada: Announcing Conformal Prediction in NeuralProphet :tada::tada:

The team at NeuralProphet has been busy building exciting innovations in NeuralProphet and recently released day v0.5.0 that now included Conformal Prediction for time series forecasting :fire::fire::fire::fire::fire::rocket::rocket::rocket::rocket::rocket:

My Medium article ‘Probabilistic Forecasting with Conformal Prediction and NeuralProphet’ showcases using NeuralProphet to predict energy consumption. <https://valeman.medium.com/probabilistic-forecasting-with-conformal-prediction-and-neuralprophet-af9c87901d94>

Course on Conformal Prediction! <https://mindfulmodeler.substack.com/p/e-mail-course-on-conformal-prediction>

Use the `cutoff column`.  Here is an example: <https://nixtla.github.io/statsforecast/examples/crossvalidation.html>

Hi <@U04BKDZFQ2K>! To optimize speed, the cross-validation method does not have a fit predict architecture.

However, if you are interested in the in-sample predictions, you can use the `StatsForecast.cross_validation_fitted_values` method.

```After executing StatsForecast.cross_validation, you can access the insample prediction values for each model and window. To get them, you need to pass fitted=True to the StatsForecast.cross_validation method and then use the StatsForecast.cross_validation_fitted_values method.```
Ref: <https://nixtla.github.io/statsforecast/core.html#statsforecast.cross_validation>

Screenshot 2022-12-14 at 11.25.47 AM.png

Hi <@U04F78WGXEE>,

In the NeuralForecast library we are prioritizing modularity.

During instantiation our models allow you to customize the loss functions that you can choose from a catalog, or in its absence you can use your own.

In particular for the under and overshooting loss (inbalanced losses), we have `QuantileLos` and `MQLoss` that are based on the NewsVendor problem.

Here is the NeuralForecast losses catalog: <https://nixtla.github.io/neuralforecast/losses.pytorch.html>
Here is an MQLoss example: <https://nixtla.github.io/neuralforecast/examples/uncertaintyintervals.html>

Your paper looks really interesting, I will try to read it carefully.
It looks also that you are beginning to look into reinforcement learning. :)

We are also fans of quantile regression and its links to the News Vendor problem.

Hi <@U04F78WGXEE>, it would be an honor to have you as a contributor :slightly_smiling_face:

Got it. I'm currently a student and am doing finals but after the semester ends, I will try to find some time to contribute.

I also love how you use nbdev! I use it for my open source projects too.