Hi <!channel> I just joined and hit the wall when trying to import ConformalIntervals

from statsforecast.utils import ConformalIntervals

gives the error:

ImportError: cannot import name 'ConformalIntervals' from 'statsforecast.utils'

Has anyone else experienced this? Python = 3.10.9 & statsforecast = 1.5.0 Thanks a bunch.

I'm still trying to figure out how to add sample weights to MLForecast. According to https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html#, sample weights can be added in two places; either in the parameters given when LGBMRegressor is called, or in the LGBMRegressor.fit method. However, the parameter only works "in case of loading data directly from text file" (https://lightgbm.readthedocs.io/en/latest/Parameters.html), and the fit call from MLForecast doesn't allow a weight parameter (https://nixtla.github.io/mlforecast/forecast.html#mlforecast.fit). Anyway, my thought was that I could make a work-around by making a MLForecast object, doing the preprocessing, then extracting the model and training it with lightgbm calls, plugging it back in to MLForecast, and then using the predict feature of MLForecast (hence allowing me to take advantage of Nixtla's rolling feature calculations for future features). Code (see below) did not work as I was hoping (ValueError: No fitted models found. You have to call fit or preprocess + fit_models.). Is there a way to make this work?

import pandas as pd
import numpy as np
from mlforecast import MLForecast
import lightgbm as lgb

cols = 4        # col 0 is target, col 1 is weights, cols 2 and 3 are features
rows = 20000

data=np.random.rand(rows,cols)
columns = ['y','weight','feat_1','feat_2']

data_df = pd.DataFrame(data[:,0:4], columns=columns)
data_df['unique_id'] = 1
data_df['ds']= [i for i in range(0, rows)]

model_1 = MLForecast(
    models={
        'LGBM': lgb.LGBMRegressor()
    },
    freq=1,
    lags = [1, 2],
)

prep = model_1.preprocess(data_df, static_features = None, dropna = False)
train_weight = prep['weight']
train_x = prep.iloc[:,[2,3,6,7]]
train_y = prep['y']
model_1.models['LGBM'].fit(train_x,train_y,train_weight)

data_future = pd.DataFrame({'unique_id': [1,1,1],'ds':[rows,rows+1,rows+2], 'feat_1':[1,2,3], 'feat_2':[1,2,3]})
model_1.predict(3,X_df=data_future)

Hi all, I'm new to the project, just got token and experimenting with timegpt, i'm constantly hitting this error , does anyone also encounter this ? I'm just using the sample code . I'm able to get the forecast results. but statsforecast spit some error. I've checked both df, there is no null value.. Appreciate any help .. I'm on python@3.11/3.11.5 , numba 0.57.1. I suspect it is the version issue?

Traceback (most recent call last):
  File "/Users/lilypad/Documents/Code/timegpt/test1.py", line 19, in <module>
    sf.plot(df, fcst_df, level=[80, 90], max_insample_length=24 * 5)
  File "/opt/homebrew/lib/python3.11/site-packages/statsforecast/core.py", line 1789, in plot
    axes[idx, idy].fill_between(
  File "/opt/homebrew/lib/python3.11/site-packages/matplotlib/__init__.py", line 1446, in inner
    return func(ax, *map(sanitize_sequence, args), **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/matplotlib/axes/_axes.py", line 5425, in fill_between
    return self._fill_between_x_or_y(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/matplotlib/axes/_axes.py", line 5330, in _fill_between_x_or_y
    ind, dep1, dep2 = map(
    ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/numpy/ma/core.py", line 2360, in masked_invalid
    return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)
                          ^^^^^^^^^^^^^^^^^^^^^^^
TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

python script is :

import pandas as pd
import os
from nixtlats import TimeGPT
from statsforecast import StatsForecast as sf

print(os.environ)


df = pd.read_csv('<https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short.csv>')

timegpt = TimeGPT(token=os.environ['TIMEGPT_TOKEN'])
fcst_df = timegpt.forecast(df, h=24, level=[80, 90])

print("NaN count in df:")
print(df.isna().sum())
print("NaN count in fcst_df:")
print(fcst_df.isna().sum())

sf.plot(df, fcst_df, level=[80, 90], max_insample_length=24 * 5)

Hello! Thanks for the awesome project! I have a question regarding cross_validation function. Is it possible to define an initial "training window" size and let the algorithm perform rolling window until end of timeseries is reached without explicitly defining "training_size" or "n_windows"? I have multiple timeseries of different length and it seems a bit weird to specify a fixed amount of n_windows because I want as many cross validations on each timeseries as possible. My current approach is:

# Find timeseries with least data
min_ts_count = big_df_for_stats.groupby(by="unique_id").count()["y"].min()
# This is the amount of minimum trained wanted for rolling! window
initial_training_size = 365
step_size = 4
h = 1

test_size_parameter = ((min_ts_count - initial_training_size)//step_size)
# This is the workaround to prevent the exception which happens when "(test_size-h)%step_size != 0"
test_size_parameter = test_size_parameter - (test_size_parameter%step_size) + h

res_df = sf.cross_validation(h=h, step_size=step_size, fitted=True, test_size=test_size_parameter, n_windows=None)

Another question about MLForecast: Is it possible to train different models on different subsets features, without having to make separate MLForecast objects? I'd like to be able to do the preprocessing only once, but then train multiple models on different sets of features. I can run the preprocessing and train the models easily enough, but when I go to predict them using the MLForecast.ts.predict() method I get an error like: ValueError: Number of features of the model must match the input. Model n_features_ is 31 and input n_features is 32 Is it possible to manipulate the data in the MLForecast object to only select the features used in training?

Also, I think I found a bug in the utilsforecast.preprocessing.fill_gaps() method. When using weekly frequency, data is lost unless specific weekly bucketing is used:

from utilsforecast import preprocessing as pr
import pandas as pd
import numpy as np

df = pd.DataFrame(
    {
        'unique_id': [0, 0, 0, 1, 1],
        'ds': pd.to_datetime(['2018-11-11', '2018-11-18', '2018-12-02', '2018-11-11', '2018-12-02']),
        'y': np.arange(5),
    }
)
df1 = pr.fill_gaps(
    df,
    freq='W',
)
print(df)
print(df1)

##### OUTPUT ######

   unique_id         ds  y
0          0 2018-11-11  0
1          0 2018-11-18  1
2          0 2018-12-02  2
3          1 2018-11-11  3
4          1 2018-12-02  4
   unique_id         ds   y
0          0 2018-11-08 NaN
1          0 2018-11-15 NaN
2          0 2018-11-22 NaN
3          0 2018-11-29 NaN
4          1 2018-11-08 NaN
5          1 2018-11-15 NaN
6          1 2018-11-22 NaN
7          1 2018-11-29 NaN

This is happening because the numpy definition for week is bucketed starting on Thursdays:

np.datetime64(pd.to_datetime("2018-11-11"),'W')

##### OUTPUT ######

numpy.datetime64('2018-11-08')

If I comment out the last two lines of _determine_bound in the fill_gaps method, the problem is resolved:

def fill_gaps(
    df: pd.DataFrame,
    freq: str = 'W',
    start: str = "per_serie",
    end: str = "global",
    id_col: str = "unique_id",
    time_col: str = "ds",
) -> pd.DataFrame:
    delta = np.timedelta64(1, freq) if isinstance(freq, str) else freq
    times_by_id = df.groupby(id_col)[time_col].agg(["min", "max"])
    starts = _determine_bound(start, freq, times_by_id, "min")
    ends = _determine_bound(end, freq, times_by_id, "max") + delta
    sizes = ((ends - starts) / delta).astype(np.int64)
    times = np.concatenate(
        [np.arange(start, end, delta) for start, end in zip(starts, ends)]
    )
    if isinstance(freq, str):
        times = times.astype("datetime64[ns]", copy=False)
    uids = np.repeat(times_by_id.index, sizes)
    idx = pd.MultiIndex.from_arrays([uids, times], names=[id_col, time_col])
    return df.set_index([id_col, time_col]).reindex(idx).reset_index()
    
def _determine_bound(bound, freq, times_by_id, agg) -> np.ndarray:
    if bound == "per_serie":
        out = times_by_id[agg].values
    else:
        # the following return a scalar
        if bound == "global":
            val = getattr(times_by_id[agg].values, agg)()
            if isinstance(freq, str):
                val = np.datetime64(val)
        else:
            if isinstance(freq, str):
                # this raises a nice error message if it isn't a valid datetime
                val = np.datetime64(bound)
            else:
                val = bound
        out = np.full(times_by_id.shape[0], val)
    #if isinstance(freq, str):
        #out = out.astype(f"datetime64[{freq}]")
    return out

df = pd.DataFrame(
    {
        'unique_id': [0, 0, 0, 1, 1],
        'ds': pd.to_datetime(['2018-11-11', '2018-11-18', '2018-12-02', '2018-11-11', '2018-12-02']),
        'y': np.arange(5),
    }
)
df1 = fill_gaps(
    df,
    freq='W',
)
print(df)
print(df1)

##### OUTPUT #####
   unique_id         ds  y
0          0 2018-11-11  0
1          0 2018-11-18  1
2          0 2018-12-02  2
3          1 2018-11-11  3
4          1 2018-12-02  4
   unique_id         ds    y
0          0 2018-11-11  0.0
1          0 2018-11-18  1.0
2          0 2018-11-25  NaN
3          0 2018-12-02  2.0
4          1 2018-11-11  3.0
5          1 2018-11-18  NaN
6          1 2018-11-25  NaN
7          1 2018-12-02  4.0

However, with this change problems occur when the weekly bucketing of the data is not precise (i.e. if I accidentally put '2018-12-01' in it would error)

Regarding my initial question (https://nixtlacommunity.slack.com/archives/C031EJJMH46/p1693989589210909) I would like to ask if my approach is the recommended way to do an extensive cross_validation in the special case that the forecasting horizon is only 1? I am asking because all examples in the documentation show the usage of 3<n_windows<20 which essentially means only "20 crossvalidated days " in the case of h=1. Also, I would like to ask if the Auto* Models perform hyperparameter tuning and model selection for each window inside the cross validation? Or, do they only perform one initial model selection on the whole time series, find the best model, and use that best model for each window inside the cross validation? Thank you in advance!

Using hierarchicalforecast, with MiddleOut (forecast proportions) reconciliation, it happened that for a certain point the second-to-last level has a forecasted value of 4 and the last (lowest) level, for the same timestep, has a forecasted value of over 7000 (I expect it to be equal or lower than 4). Could it be related to some division by a value close to zero that is happening somewhere?

Welcome!

BWBarber

joined #general.

Hi Everyone, I went through some of the previous messages and everyone seems doing well with the platform, so excuse me if my question seems quite rookie. I just found out about Nixtla a week ago and was trying to go through after getting a token from the platform, however, my requests were denied. I have already applied on the platform to gain access (https://www.nixtla.io/.) but didn't receive any feedback. can anyone please advise if I should do anything else ? or when to expect a reply

Hello everyone. First I would like to thank everyone on the team for implementing the fill_gaps function, my team and I are already using it a lot here! We are testing scale modeling for approximately 40k time series with mlforecast, the results are looking promising. But we would like to test with TimeGPT as well. However, when passing the tocken, it started to run, but soon after the following message appeared:

ApiError: status_code: 429, body: {'data': None, 'message': 'Too many requests', 'details': 'You have reached your request limit, email <mailto:ops@nixtla.io|ops@nixtla.io> to continue using the API', 'code': 'A20', 'requestID': 'CJA2TUG4YJ', 'support': 'If you have questions or need support, please email <mailto:ops@nixtla.io|ops@nixtla.io>'}

Hello, I have been working in R with Hyndman et al.'s fable family of packages. I'm trying to replicate my work using StatsForecast and MLforecast to see if I can improve performance (both accuracy and speed). Right now it appears what I've produced using Fable family of packages and Nixtla packages are getting somewhat similar results in terms of MASE on data trained using CV. However, in some cases Fable does better and others Nixtla. I believe there are two reasons: 1) I'm able to use Prophet with Fable (I know Nixtla's research suggestes ensembles using StatsForcast can produce on par or better, but I'm looking at ensembles and prophet is still getting better results); and 2) in Fable I'm able to use the "common exogenous" (https://fable.tidyverts.org/reference/common_xregs.html) variables trend and season. I've searchd for answers to these questions in previous posts, but didn't find answers that directly addressed them. My questions are: • Is there a way to replicate the common exogenous regressors using Nixtla packages? I'm thinking I might need to decompose the series and then merge that data into the dataframe. If that's right, how do I deal with needing those external regressors in the future frame for forecasts? Or is there something else I might have missed? • Is there a way to incorporate prophet into MLforecast? I think only sklearn, xgboost, and lightgbm work, but wanted to confirm. I have been trying to also forcast using the prophet API and them merging the data, but it'd be better to have it all in one set from the get-go. Appreciate any guidance.

Hi! Timegpt is a amazing work. Can you share what is the self-supervised pre-training task for TimeGPT? Contrastive learning or mask reconstruction

I'm interested in ensemble/combination modeling. The R Fable takes care of the math in the background for both the point forecast and the prediction intervals (both are important for my current problem). The link below is all I've been able to find. It references the

nixtla

package, which I can't find. I haven't had luck with google/stackoverflow searches. I did find one ticket/issue on the GitHub page about FFORMA (which would be great), but right now I'm just looking to do combo models of simple averages--with statsforecast and mlforecast models. Is this a feature I've just not been able to find or something perhaps in the works? https://medium.com/@gosshhh9/nixtla-using-ensembling-method-for-time-series-e1d23964ac30

Hi everyone! Just waiting for the API KEY activation and start my journey with TimeGPT with OpenBB!

morning folks, I'd like to know your tooling stack if you have built end to end ml ops pipeline for the nixtla libraries - esp ML Forecast. Our current approach is 1. to use cloud pickle to export the model binary 2. track experiments using our own custom built scripts. We are specifically looking into either mlflow or sagemaker.

If feeding the same data to TimeGPT, should I expect the same (stable) predictions each time or no? Does it have a temperature parameter?

Do the Auto* models from neuralforecast also include the exogenous covariates into the hyperparameter search? I am asking because there is no "futr_exog_list" parameter for the Auto* models.

Hey! Y'all think that any of the libraries present can be used for anomaly detection in audio files? If you've tried it

Welcome!

GoostValley

joined #general.

Hi everyone! Thanks to Max and all of you for admitting me as a TimeGPT Beta Test user! I will be cool If you can help me with some credits to start, my registered email is hernandogaitandiaz@gmail.com

Hello everybody !

Hey Nixtla, what a cool product you’re developed. I’ve watched the youtube speech from Azul, very impressive! I’m the team lead of the Machine Learning team at energyworx, a SaaS for grid operators and energy supplier. Forecasting timeseries, energy usage, energy price etc is essential for us. And we’d love to contribute to TimeGPT to make it even better. I applied for an API through the developer portal, however that key was just usable once, and after that I got 429 errors. Please let me know how can we start trying the product, thanks so much

My understanding is that forecasts can become erratic when too many exogenous variables are introduced. Does anyone have data on what the upper limit of exogenous variables is?

👋 Hi everyone!

Hi, is it possible to save a trained MLForecast model object and later load it for prediction ?

Like some others here, we’re also on the waitlist for TimeGPT. Would it be possible to share an outlook when more people gets access? Or better said: what’s the current limitation on admitting more people? Generally more users == more feedback too 🙂

Hi. I was admitted into the TimeGPT beta but am trying to figure out how to get my credits and get it up and running.

Hello, Team! Thank you for letting us use your great tools. High level question: for a case where we have lots of series (in our case it's monthly ARR for different customers) do you recommend that we try multiple series handling and or use hierarchical methods in some of your packages? To be more clear are there any pages that document the theory and more detailed use of forecasting multiple series? Thank you all in advance!

I'm following the steps in https://nixtla.github.io/statsforecast/docs/how-to-guides/exogenous.html with my own data and having a difficulty time debugging this warning

/local_disk0/.ephemeral_nfs/envs/pythonEnv-6ba80c76-407b-4675-a454-0051f1cc3503/lib/python3.10/site-packages/statsforecast/arima.py1557 UserWarning:

xreg not required by this model, ignoring the provided regressors

I've been plumbing the source code and I think

forecast_arima

might be dispatching with a model that I don't intend it to (no exog regressors). Does anybody have any advice on how I can debug further?