Nixtla Community #general

Phi Nguyen

09/08/2022, 4:42 PM

let me share a reproducible script

Phi Nguyen

09/08/2022, 4:43 PM

Copy code

from time import time
import ray
import pandas as pd
#from neuralforecast.data.datasets.m5 import M5, M5Evaluation
from statsforecast import StatsForecast
from statsforecast.models import ETS

ray.init(address="auto")

Y_df = pd.read_parquet('<s3://m5-benchmarks/data/train/target.parquet>')
Y_df = Y_df.rename(columns={
    'item_id': 'unique_id', 
    'timestamp': 'ds', 
    'demand': 'y'
})
Y_df['ds'] = pd.to_datetime(Y_df['ds'])

Y_df = Y_df[Y_df.unique_id == "FOODS_1_001_CA_1"]

constant = 10
Y_df['y'] += constant

fcst = StatsForecast(
    df=Y_df, 
    models=[ETS(season_length=7, model='ZNA')], 
    freq='D'
)

Y_hat = fcst.forecast(28)

Max (Nixtla)

09/08/2022, 4:53 PM

Y_df = Y_df[Y_df.unique_id == "FOODS_1_001_CA_1"]

# The problem was that the type was a category. # Add this line

Y_df['unique_id'] = Y_df['unique_id'].astype(str)

Max (Nixtla)

09/08/2022, 4:54 PM

Copy code

from time import time
import ray
import pandas as pd
#from neuralforecast.data.datasets.m5 import M5, M5Evaluation
from statsforecast import StatsForecast
from statsforecast.models import ETS

ray.init(address="auto")

Y_df = pd.read_parquet('<s3://m5-benchmarks/data/train/target.parquet>')
Y_df = Y_df.rename(columns={
    'item_id': 'unique_id', 
    'timestamp': 'ds', 
    'demand': 'y'
})
Y_df['ds'] = pd.to_datetime(Y_df['ds'])

Y_df = Y_df[Y_df.unique_id == "FOODS_1_001_CA_1"]
# Add this line
Y_df['unique_id'] = Y_df['unique_id'].astype(str)

constant = 10
Y_df['y'] += constant

fcst = StatsForecast(
    df=Y_df, 
    models=[ETS(season_length=7, model='ZNA')], 
    freq='D'
)

Y_hat = fcst.forecast(28)

Phi Nguyen

09/08/2022, 5:35 PM

Thanks! it worked!

🎉 1

Slackbot

09/12/2022, 4:09 PM

This message was deleted.

👀 1

Slackbot

09/13/2022, 9:47 PM

This message was deleted.

👀 1

Slackbot

09/14/2022, 4:50 PM

This message was deleted.

👀 1

Slackbot

09/19/2022, 8:32 AM

This message was deleted.

mlforecast_xgb_issue.ipynb

👀 1

Slackbot

09/26/2022, 11:55 PM

This message was deleted.

Ginger Holt

09/28/2022, 10:29 PM

I am running: reconcilers = [ BottomUp(), MinTrace(method='mint_shrink'), MinTrace(method='ols') ] hrec = HierarchicalReconciliation(reconcilers=reconcilers) Y_rec_df = hrec.reconcile(Y_hat_df, Y_fitted_merged_df, S, tags) and getting this error: KeyError: 'None of [RangeIndex(start=0, stop=11553, step=1)] are in the [index]' Any pointers?

Slackbot

09/28/2022, 10:42 PM

This message was deleted.

Ginger Holt

09/29/2022, 5:17 PM

I am running: Y_hat_df = Y_hat_df.reset_index().set_index('unique_id') Y_fitted_merged_df = Y_fitted_merged_df.reset_index().set_index('unique_id') reconcilers = [ BottomUp(), MinTrace(method='mint_shrink'), MinTrace(method='ols') ] hrec = HierarchicalReconciliation(reconcilers=reconcilers) Y_rec_df = hrec.reconcile(Y_hat_df, Y_fitted_merged_df, S, tags) and getting this error: LinAlgError: Array must not contain infs or NaNs There are no infs or NaNs in the data. I am getting "False" for all of these checks: np.isinf(Y_fitted_merged_df).any() np.isinf(Y_hat_df).any() np.isnan(Y_fitted_merged_df).any() np.isnan(Y_hat_df).any() Data size is ~40K rows. So it may be a matrix computation issue at that scale. Have others gotten this working at the same scale?

Slackbot

09/29/2022, 5:19 PM

This message was deleted.

Slackbot

10/06/2022, 3:43 PM

This message was deleted.

J T

10/06/2022, 9:14 PM

Hi Team Nixtla... got a question.. i was able to run the statsforecast. but the results came back funny - one month very high and the next month very low.. but the historical actual is not like that.. and my run on using pmdarima is very good. One thing i noticed is that the output DS was pointing to month end, instead of 1st day of the month.. not sure if that matters.

J T

10/06/2022, 9:15 PM

Copy code

Ran under AZ databrick.
here is the code: 
#Select SARIMA with seasonality 12
autoARIMA = AutoARIMA(season_length=12)

# Select ETS with seasonality 12 and multiplicative trend
model = StatsForecast(df=products2.set_index('unique_id'), 
                      models=[autoARIMA],
                      freq='m', n_jobs=-1)
Y_hat_df = model.forecast(horizon).reset_index()

J T

10/06/2022, 9:16 PM

here is the output results: unique_id ds AutoARIMA 0 0 2022-01-31 410.899323 1 0 2022-02-28 62994.761719 2 0 2022-03-31 493.694214 3 0 2022-04-30 65027.785156 4 0 2022-05-31 484.681763

Slackbot

10/06/2022, 9:20 PM

This message was deleted.

Valeriy

10/09/2022, 11:34 AM

https://twitter.com/ChristophMolnar/status/1579051843107184641?s=20&t=moJyU55p_TECAzmelJ_SkQ this is way to funny

🤣 1

Max (Nixtla)

10/10/2022, 2:56 PM

Complex Exponential Smoothing (CES) is now avaialable in StatsForecast! https://www.linkedin.com/posts/mergenthaler_timeseries-forecasting-python-activity-69[…]029659398144-rlS8?utm_source=share&utm_medium=member_desktop

👏 3

Slackbot

10/11/2022, 9:28 AM

This message was deleted.

👀 1

Valeriy

10/11/2022, 9:29 AM

The MIT group that produces the library and NeurIPS paper does a lot of interesting work in forecasting.

Valeriy

10/11/2022, 9:30 AM

https://abdullaho.me/publication/mssa/

J T

10/14/2022, 6:10 PM

trouble installing numba, statsmodels and statsforecst. it errors out today be never last week. something has changed? %pip install -U numba Python interpreter will be restarted. WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5fdd406fd0>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/numba/

👀 1

Slackbot

10/14/2022, 6:55 PM

This message was deleted.

Max (Nixtla)

10/14/2022, 6:58 PM

Something like:

Copy code

conda env create --name statsforecastenv --file=environment.yml

And then

Copy code

source activate statsforecastenv

Slackbot

10/17/2022, 12:58 PM

This message was deleted.

Slackbot

10/23/2022, 2:31 AM

This message was deleted.