This message was deleted.
# general
s
This message was deleted.
m
Thanks for the comment. Cc @José Morales
j
Hey @Arsa Nikzad. I think that if you have any null values in your target you get an error when trying to run the preprocess because all transformations propagate the null values, were you able to run it with missing values in the middle?
a
Hi @José Morales. thanks for the reply. the target does not have null values. the issue is with
rolling_std
when we have a set of consecutive zeros larger than window size in target. the root cause of issue seems to be
_rolling_std
in
window_ops.rolling
where it generates large negative numbers in above situation and these negative numbers are then converted to NAN. attached is an example.
Copy code
data = pd.DataFrame({
    'date': pd.date_range(start='2019-01-01', end='2020-12-31', freq='MS'),
    'sprid': 1.,
    'target': [1., 2., 0., 4., 0., 0., 0., 0., 9., 10., 11., 12] * 2
})

models = [lgb.LGBMRegressor(**{})]
fcst = MLForecast(
    models=models,
    freq='MS',
    lags=[1],
    lag_transforms={
        1: [(rolling_std, 3)]
    }
)

preprocessed_df = fcst.preprocess(data, id_col='sprid', time_col='date', target_col='target', dropna=False)
print(preprocessed_df)

## check _rolling_std
from window_ops.rolling import  _rolling_std
a = np.array([1, 2, 0, 4, 0, 0, 0, 0, 9, 10, 11, 12] * 2)
print(_rolling_std(a, 3))
j
Thanks for the example! I think a warning for these cases is definitely useful. Will add it soon
👍 1
a
additionally,
rolling_std
should generate zeros instead of NAN for these cases.
m
Hi @Arsa Nikzad! Happy to work on that. Would you mind opening an issue for that on GH. Here is the link.
a
Sure, will do!
j
Hey @Arsa Nikzad, we just pushed window-ops 0.0.14 which should fix this issue with the rolling_std
a
Thank you.