Nixtla's Open Source Time Series Ecosystem.

Nixtla Community

Hi dear members of the community: we just released some features. Please share some love:
<https://www.linkedin.com/posts/mergenthaler_machinelearning-timeseries-forecasting-activity-7033898837886816256-X4jI?utm_source=share&amp;utm_medium=member_desktop|https://www.linkedin.com/posts/mergenthaler_machinelearning-timeseries-forecasting-ac[…]837886816256-X4jI?utm_source=share&amp;utm_medium=member_desktop>

Special mention to <@U03L39VLJBA>, that has been a promoter of conformal prediction.

Christmas :santa:  comes early, amazing release team 

I am not part of Nixtla team. But for any open source project you should do it. What's the worst that can happen? They reject your PR? You can then fork the repo and build your own extension. There are thousands of examples out there. Either you succeed or fail. Either way it will be a great experience.

Well, I just want to ask for permission, haha. I needed to use RollingMovAvg and ExpMovAvgs and that`s why I thought of this. We got a 2x speed increase of our pipeline by making these changes

Screenshot 2023-02-22 at 12.45.31.png

Nixtla tweet on fire 30K :fire::fire::fire::fire::fire:

<https://threadreaderapp.com/thread/1627714706830942209.html>

had this problem too. ended up hacking in a solution:

```fitted_models = sf.fit().fitted_```
You can then pickle these fitted models. Note there may be more than one model.

When you want to load them again:
```new_sf = StatsForecast(df=df, models=placeholder_models)  # the models you put in here are irrelevant
sf.fitted_ = fitted_models
sf.predict(horizon)```


<@U038TNSC4TG> maybe I should have put up a PR for this, apologies for laziness on my end!

Hey Sam, thanks for the response, from what I gathered `.fit` is callable of `StatsForecast` class. That assumption is based on the fact that when calling `.forecast` method it autofits the models.
First question: Would it be better to call `.fit` after initiating the class with all the params?
Second question: would `fit` create ?incapsulated list? with all the fitted models? hence this code:
```[model.fitted_ for model in sf.fit()]```

1. Indeed forecast() fits the models. Only call .fit() then .predict() imo if you want to split these steps for some other reason.
2. Ye it does create a list of fitted models.

okay, so utilising `fit` and `predict` in the case of manual fitting.
Would be great to have a PR about creating a `save` method.

Yeah I chatted with them about it. The reason is that the overhead can be a bit big. If you have 1000 timeseries, the weights for each timeseries would be saved so it’s more common to just call the forecast straight. The PR makes sense though

<@U04PWQLN9BQ> I dabbled a bit around with fitted models and it is a bit confusing on what's happening under the hood, when looking at `sf.fit().fitted_` I can see replicas of the a single model that was issues, are they same model but fitted with different params?

AutoARIMA example:
```[array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=objec
t), array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=ob
ject), array([AutoARIMA], dtype=object), array([AutoARIMA], dtype=object)]```

<@U03SA41952R> I just attempted a test run with 400 data rows (not much, just a test) and 10 above ARIMA models, the size of the pickle is 3.9kb

there may be a separate model for each unique_id in your data

:man-facepalming:omg that makes 100% sense hahaha

dw it confused me too when i first saw it

is there a way to associate the model with its unique id? incremental increase according to alphabetical order?

i haven't figured that one out yet, but you can probably hack it

Yes the best model for each fitted model for each unique id in your data. So if you try ETS and AutoARIMA, you will see double the models. The unique_id should resolve when you call predict so you shouldn’t need to handle it yourself?

check out this thread <@U04R9CEKAF5>. I think it answers what you were asking.

<@U04QFEWJZT4> this is for statsforecast, I'm using mlforecast, which does not have .fitted_ attribute, unfortunately

Hi! Does anybody have any idea why AutoARIMA seems to suddenly forecast large quantities where it should not ? Like it`s weird, because a month ago, it seemed to be working fine and predicting at a high enough rate and we did not change anything.

I don't think season_length is something you should change unless your series has multiple seasonalities in which case you should probably use MSTL. If your series has only one seasonality then that's what you pass to season_length. Also, for longer seasonalities it always takes longer to fit specially with ARIMA. I don't see anything weird about that.

<https://robjhyndman.com/hyndsight/longseasonality/|https://robjhyndman.com/hyndsight/longseasonality/>

Well yes, we only want to use yearly seasonality. But is we give seasonality = 52, it`s up to 4x slower than when seasonality =1 . that`s what`s weiiiiird

sf.fitted_   attribute should give you what the model is  built of.

Thanks for reply,But iam looking for how to check thier co-efficients?

`sf.fitted_[model_index][unqiue_id].model_`

edit: i tried try catching or some workarounds and it still nott working


I have decent data as an input ... so I do not think it`s from the data.

See this:
<https://nixtlacommunity.slack.com/archives/C031EJJMH46/p1673558640014259?thread_ts=1673547105.504069&amp;cid=C031EJJMH46|https://nixtlacommunity.slack.com/archives/C031EJJMH46/p1673558640014259?thread_ts=1673547105.504069&amp;cid=C031EJJMH46>

Not sure what environment.yml is for but I don't use Conda and I have Prophet installed. Also, what does prophet have to do with Statsforecast or Neuralforecast?!

hi <@U04QSAXB727>! Yes, installing prophet from pypi usually throws errors :disappointed: we use conda instead

<@U0316L4HWQ7> wouldn't creating a docker image be better?

<@U040AMG6N69> ... you clearly did not read <https://github.com/Nixtla/statsforecast/blob/main/CONTRIBUTING.md>

<@U04QSAXB727>  why should I have read that?! I don't have a PR. My suggestion is that you provide more details in your questions if you want to get help from the community. While the main developers are very nice and frequently come here and answer our questions, it's not their job to do so. The idea of having a Slack channel is to build a community and hope that users are kind enough to help each other and give back to the community. In that sense, the burden is on the person who asks a question to provide enough information for others to understand the issue. Giving details into what you are trying to do helps everyone to be in a better position to help you.

hey man, can you chill and move on? clearly the thread isn't targeting you

So you post a question to the channel (not to any specific person) and someone comes and asks you to provide more information, you ignore their question which is fine but then you proceed to specifically ping them by name and make an assumption that they should have read something that is related to your issue and when they suggest you provide more context your response is chill and move on?!! Wow! Maybe next time don't ping people by name if you don't want to engage in a civil conversation (i.e., follow your own advice and move on if you're done with what you are looking for!)

Hi guys, we are really thankful for your contributions and comments. Both of you are valued and active members of the community. However, this thread is getting unnecessary heated. Clearly there are areas of opportunity in how we name things. It can be definitely confusing to have the same name for a dev environment that is necessary for contributing (and includes Prophet) and a regular environment that does not include prophet. I think this discussion makes this clear. We will think about how to make this clearer.
We are only as strong as our community, and both of you are very active and valued. Please keep up the good work.

hey <@U04R9CEKAF5>! Thanks for using mlforecast. The tutorial splits the time series into train and test sets to show the performance of the models. But if you want to actually predict the future, you can pass the entire dataset (without splitting it) and you will be able to get forecasts for the future.

but shouldn't then the prediction array be longer than the test set you pass? If you wanted to predict 48 horizon, you should get an entire test set + 48 additional rows, correct?

hey <@U04RDGR25DY>! The `input_size`  parameter controls the length of the in-sample time series: <https://nixtla.github.io/statsforecast/core.html#statsforecast.cross_validation>. You can use it to perform cross-validation with sliding windows.

Hey Fede, thanks a lot for getting back to me so quickly.  Sorry I missed that!

Hi <@U0316L4HWQ7>, I am now using MLForecast and I was wondering whether there is a plan for the `input_size` argument to be implemented in `MLForecast.cross_validation`  to enable a sliding window?  Happy to raise an issue and contribute if possible.

Hey <@U04RDGR25DY>! `MLForecast`  has the `keep_last_n`  argument instead of `input_size`  to perform sliding windows. We are working on standardizing argument names across the nixtlaverse. :slightly_smiling_face: Here’s the reference to the cross-validation method: <https://nixtla.github.io/mlforecast/forecast.html#mlforecast.cross_validation>

Ah, sorry Fede.  I thought that was dropping part of the forecast horizon as is done in some trading markets.  Thanks for the clarification.

Hi Again <@U0316L4HWQ7>.  Hope you are well.  I have been making good progress evaluating MLForecast for our production solution, however, I am having an issue using `keep_last_n` and `cross_validation`.  In the <https://github.com/Nixtla/mlforecast/blob/main/nbs/docs/electricity_peak_forecasting.ipynb|electricity_peak_forecasting notebook> when using the `keep_last_n` argument the code it fails if `keep_last_n &lt; (Y_df.shape[0] - window_size)`.  In the example the minimum window that works is `keep_last_n = 6528` :

I have carried different tests and the problem remains with and without differencing/exogenous features.

I have been continuing my investigation and have looked at whether this problem is just present when using `cross_validation.`  I have just tried the `end_to_end_walkthrough.ipynb` notebook Training and Forecasting sections and the same error occurs when `differences =[24]` and `keep_last_n` is &lt; 1008.  No error is raised if the `differences` argument is not used.  This therefore looks like a problem when slicing the data when there are exogenous/differences present and is an issue when both when using `fit` / `predict` or `cross_validation` .

Good afternoon <@U0316L4HWQ7>, I have done some digging and I think there might be two bugs when using `keep_last_n` in MLForecast.

First, <https://github.com/Nixtla/mlforecast/blob/e4ae1c7a8eee68f7582ad7fd11ad4334aa73da1a/mlforecast/core.py#L387|here> the `self.last_dates` is not correct when using `keep_last_n` .   This results in null values for the exogenous features when there is a merge in `_get_features_for_next_step` <https://github.com/Nixtla/mlforecast/blob/e4ae1c7a8eee68f7582ad7fd11ad4334aa73da1a/mlforecast/core.py#L603|here>.  I corrected this using a bit of a hack:

`self.last_dates = pd.DatetimeIndex([sorted_df.index.get_level_values(self.time_col)[-1]])`

This appears to work for my use case but I don't know the design of MLForecast well so this might not be correct for other cases such as multiple `unique_id`'s,

Secondly, once this was fixed I noticed that the `X` and `y` used in `fit_model`  <https://github.com/Nixtla/mlforecast/blob/e4ae1c7a8eee68f7582ad7fd11ad4334aa73da1a/mlforecast/forecast.py#L363|here> had all the data and not just the last n samples.  I implemented the following hack before `return self.fit_models(X, y)`:
```if keep_last_n is not None:
    X, y  = X[-keep_last_n:], y[-keep_last_n:]```
 This is not the right place to fix this as it should be done in `core.py` I think but I just did this quickly to fix and get some results.

Do you have any thoughts?  Happy to keep digging into the code if that helps, raise an issue on Github or share this with someone else if you don't have time?

Thanks again

hey <@U04RDGR25DY>! Thank you for taking the time to dig into the problem. Please help us to raise an issue on GitHub to track the problem. :) <@U03UX9KR75W>, do you have any thoughts about the issue?

No problem at all.  I’ll raise an issue tomorrow morning.  Let me know if I can help. 

Hi. The `keep_last_n` argument is used only for predicting, it is meant to be an efficiency parameter for cases when you have very long series and your updates don't require all the history.
For example if your series are of length 10,000 and your features only require the last 50 days, then setting `keep_last_n=50` makes it so that only the last 50 values of each serie are kept and used to compute the updates, this is because in the updates the whole transformation is computed but only the last value is kept.

I think it'd be better to add the `input_size` argument to do exactly the same as in statsforecast, I'll work on that and let you know when it's done.

Although those errors you're getting seem a bit odd, the keep_last_n argument should only impact the predict step, I'm not sure why you get errors on the transform. I'd really appreciate if you can open an issue with a minimal reproducible example.

Thanks a lot for this <@U03UX9KR75W>,  that makes sense.  Regarding the `input_size` argument, thanks a lot for working on this as it is really important for my current use case.  Once the code is ready (even in a separate branch/fork) if you could let me know I will start using it to test on my data.

I will create a minimal reproducible example and raise an issue later today - I'll need to have a think about this error based on on my new understanding of what `keep_last_n` is doing so it might be later this evening.  I think it might only be occurring in the predict step.   I'll tag you in the issue later.

Thank you <@U03UX9KR75W> for clarifying the actual behavior of `keep_last_n`  and for including the new feature :slightly_smiling_face:
Sorry for the misunderstanding <@U04RDGR25DY> :raised_hands:

Absolutely no problem <@U0316L4HWQ7>.  Looking forward to Jose's work going live.  :smile:

hey <@U04RDGR25DY>, we just merged the PR adding the input_size argument, so if you install from the main branch you should be able to use it. please let us know how it goes. keep in mind that the number you set there won't necessarily be the number of training samples per serie because they can be shorter in the window and also some rows will be dropped unless you set `dropna=False`

Amazing José, thanks a lot for the very quick turnaround.  I’ll install today and get back to you early next week.  

Image from: <https://openforecast.org/adam/rollingOrigin.html>

This would be useful in energy demand/price forecasting where capturing short-term dynamics is key.

Hi dear community:
Today we are comparing Nixtla and Fugue with Google.
Please show some love.

<https://www.linkedin.com/feed/update/urn:li:activity:7036392086559973376?utm_source=share&amp;utm_medium=member_desktop>

Welcome to the community!
By features you mean exogenous features? Some models accept exogenous features and some do not. Which models are you using?

Welcome  <@U04RVB0EJRJ>!
Here you have a tutorial for exogenous variables in <https://nixtla.github.io/statsforecast/examples/exogenous.html|StatsForecast>.
Here one for<https://nixtla.github.io/mlforecast/docs/end_to_end_walkthrough.html#adding-features| ML Forecast.>
And here one for <https://nixtla.github.io/neuralforecast/examples/exogenous_variables.html|NeuralForecast>.

Hi <@U04RVB0EJRJ>. We just added a multivariate model based on graph neural networks (StemGNN) in NeuralForecast. We are preparing a tutorial on how to use it.

image.png

Wow, thank you so much for all of your replies. This will certainly help me a lot!
For a start, I am developing a baseline with univariate ts. Would this methodology be correct (I have a background in ML but no real ts knowledge)?
I have 6 years' worth of data (24 h freq), and a 7-day target. So I would split 5 y -&gt; training / 1y -&gt; test. For the windows sampling, I was thinking to go for a 3 weeks input 1-week target scenario since I do not have any trend.

Hi Merlin, If you are new to time series, I'd suggest you follow the examples in the documentation (that are great BTW).
I'd start with statsforecast and then move to other models available in mlforecast and neuralforecast.
Run then line by line and you will learn a lot. You will probably be able to adapt one of those examples to your data.

I would suggest you try this <https://nixtla.github.io/statsforecast/examples/getting_started_complete.html|tutorial>.

Please share if anything does not work :slightly_smiling_face:

sure, I will try it and let you guys know how it went :+1:

<@U035DR8HD6D> ~are you guys working on SARIMA implementation as well ?~
got it

Hi <@U04S3MT8RH8>! It seems you are using external regressors, is that correct? If so, future exogenous regressors must be provided to generate forecasts. Here’s a tutorial on the topic: <https://nixtla.github.io/statsforecast/examples/exogenous.html>

Hi <@U04RZ5FTPRC> ,

Thanks for using the library the MiddleOut and TopDown reconciliation methods are available for strictly hierarchical structures (not grouped series).

Here is an explanation of the difference between hierarchical and grouped series:
<https://otexts.com/fpp3/hierarchical.html|https://otexts.com/fpp3/hierarchical.html>

It looks like this is already possible with the Darts package: <https://unit8co.github.io/darts/generated_api/darts.models.forecasting.sf_auto_arima.html>

Hi <@U04SDV009D0>. I saw you already created an issue in github for this. Thanks for letting us know this is a useful feature that other libraries already have. We'll take it into consideration for further developments.

Hi <@U04R9CEKAF5>! <@U03UX9KR75W> can probably help you.

Turns out I also had to specify lags for it to work. Odd stuff, possible fix in some newer version?

Hi Adi. DM me, i think I can help  you with AutoArima :slightly_smiling_face:

There is one for Nixtla+Ray+AWS and you can find help on Ray+Azure but I haven't seen Nixtla on Azure tutorial. I haven't used Azure but once you know how to spin up your Ray cluster on Azure, using Nixtla should be similar to what we do on EC2.

Cool. Thanks, <@U040AMG6N69>. I found this about creating a Ray Cluster within Azure.

<https://github.com/microsoft/ray-on-aml>

So I should be able to give this a try soon-ish.

Hey <@U04RVB0EJRJ>! Both libraries include the `refit`  argument in `cross_validation` . If `refit=True` , then the models will be retrained for each window. Currently, in `statsforecast`  that argument only works with the `Auto`  models.