https://github.com/nixtla logo
#general
Title
# general
r

rariwa

04/09/2022, 11:17 PM
hi, I am trying to build model using NBEATSx. however, I only have one unique id. I got an error as follow :
Copy code
AssertionError                            Traceback (most recent call last)
Input In [32], in <cell line: 61>()
     57 mc['n_layers'] =  len(mc['stack_types']) * [ mc['constant_n_layers'] ]
     59 from neuralforecast.experiments.utils import create_datasets
---> 61 train_dataset, val_dataset, test_dataset, scaler_y = create_datasets(mc=mc,
     62                                                                      S_df=s_df, Y_df=y_df, X_df=x_df,
     63                                                                      f_cols=['Exogenous1', 'Exogenous2'],
     64                                                                      ds_in_val=180,
     65                                                                      ds_in_test=984)
     67 train_loader = TimeSeriesLoader(dataset=train_dataset,
     68                                 batch_size=int(mc['batch_size']),
     69                                 n_windows=mc['n_windows'],
     70                                 shuffle=True)
     72 val_loader = TimeSeriesLoader(dataset=val_dataset,
     73                               batch_size=int(mc['batch_size']),
     74                               shuffle=False)

File ~\Anaconda3\envs\SiT\lib\site-packages\neuralforecast\experiments\utils.py:249, in create_datasets(mc, S_df, Y_df, X_df, f_cols, ds_in_test, ds_in_val, verbose)
    244 train_mask_df, valid_mask_df, test_mask_df = get_mask_dfs(Y_df=Y_df,
    245                                                           ds_in_val=ds_in_val,
    246                                                           ds_in_test=ds_in_test)
    248 #---------------------------------------------- Scale Data ----------------------------------------------#
--> 249 Y_df, X_df, scaler_y = scale_data(Y_df=Y_df, X_df=X_df, mask_df=train_mask_df,
    250                                   normalizer_y=mc['normalizer_y'], normalizer_x=mc['normalizer_x'])
    252 #----------------------------------------- Declare Dataset and Loaders ----------------------------------#
    254 if mc['mode'] == 'simple':

File ~\Anaconda3\envs\SiT\lib\site-packages\neuralforecast\experiments\utils.py:202, in scale_data(Y_df, X_df, mask_df, normalizer_y, normalizer_x)
    200     for col in X_cols:
    201         scaler_x = Scaler(normalizer=normalizer_x)
--> 202         X_df[col] = scaler_x.scale(x=X_df[col].values, mask=mask)
    204 return Y_df, X_df, scaler_y

File ~\Anaconda3\envs\SiT\lib\site-packages\neuralforecast\data\scalers.py:43, in Scaler.scale(self, x, mask)
     40 elif self.normalizer == 'norm1':
     41     x_scaled, x_shift, x_scale = norm1_scaler(x, mask)
---> 43 assert len(x[mask==1] == np.sum(mask)), 'Something weird is happening, call Cristian'
     44 nan_before_scale = np.sum(np.isnan(x))
     45 nan_after_scale = np.sum(np.isnan(x_scaled))

AssertionError: Something weird is happening, call Cristian
here is my data
m

Max (Nixtla)

04/09/2022, 11:22 PM
Lol let's call @Cristian (Nixtla).
c

Cristian (Nixtla)

04/10/2022, 3:32 PM
Hi rariwa! That assert has a mistake, we already fixed in the git repository and pip. You can also fix it by removing the assert.
r

rariwa

04/11/2022, 9:40 AM
now it shows new error
Copy code
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Input In [8], in <cell line: 61>()
     57 mc['n_layers'] =  len(mc['stack_types']) * [ mc['constant_n_layers'] ]
     59 from neuralforecast.experiments.utils import create_datasets
---> 61 train_dataset, val_dataset, test_dataset, scaler_y = create_datasets(mc=mc,
     62                                                                      S_df=s_df, Y_df=y_df, X_df=x_df,
     63                                                                      f_cols=['Exogenous1', 'Exogenous2'],
     64                                                                      ds_in_val=180,
     65                                                                      ds_in_test=984)
     67 train_loader = TimeSeriesLoader(dataset=train_dataset,
     68                                 batch_size=int(mc['batch_size']),
     69                                 n_windows=mc['n_windows'],
     70                                 shuffle=True)
     72 val_loader = TimeSeriesLoader(dataset=val_dataset,
     73                               batch_size=int(mc['batch_size']),
     74                               shuffle=False)

File ~\Anaconda3\envs\SiT\lib\site-packages\neuralforecast\experiments\utils.py:252, in create_datasets(mc, S_df, Y_df, X_df, f_cols, ds_in_test, ds_in_val, verbose)
    247 train_mask_df, valid_mask_df, test_mask_df = get_mask_dfs(Y_df=Y_df,
    248                                                           ds_in_val=ds_in_val,
    249                                                           ds_in_test=ds_in_test)
    251 #---------------------------------------------- Scale Data ----------------------------------------------#
--> 252 Y_df, X_df, scaler_y = scale_data(Y_df=Y_df, X_df=X_df, mask_df=train_mask_df,
    253                                   normalizer_y=mc['normalizer_y'], normalizer_x=mc['normalizer_x'])
    255 #----------------------------------------- Declare Dataset and Loaders ----------------------------------#
    257 if mc['mode'] == 'simple':

File ~\Anaconda3\envs\SiT\lib\site-packages\neuralforecast\experiments\utils.py:207, in scale_data(Y_df, X_df, mask_df, normalizer_y, normalizer_x)
    205     for col in X_cols:
    206         scaler_x = Scaler(normalizer=normalizer_x)
--> 207         X_df[col] = scaler_x.scale(x=X_df[col].values, mask=mask)
    209 return Y_df, X_df, scaler_y

File ~\Anaconda3\envs\SiT\lib\site-packages\neuralforecast\data\scalers.py:35, in Scaler.scale(self, x, mask)
     33 nan_before_scale = np.sum(np.isnan(x))
     34 nan_after_scale = np.sum(np.isnan(x_scaled))
---> 35 assert nan_before_scale == nan_after_scale, 'Scaler induced nans'
     37 self.x_shift = x_shift
     38 self.x_scale = x_scale

AssertionError: Scaler induced nans
c

Cristian (Nixtla)

04/11/2022, 3:24 PM
does the data have missing values/NaNs before the normalization?
Hi @rariwa, where you able to run it?
r

rariwa

05/10/2022, 11:50 PM
still get the error
p

Pablo Andreone

08/20/2022, 12:31 PM
Hi, following this : https://nixtla.github.io/neuralforecast/models_nhits__nhits.html I get the same error
Copy code
neuralforecast\experiments\utils.py:263, in create_datasets(mc, S_df, Y_df, X_df, f_cols, ds_in_test, ds_in_val, verbose)
    257 train_mask_df, valid_mask_df, test_mask_df = get_mask_dfs(Y_df=Y_df,
    258                                                           ds_in_val=ds_in_val,
    259                                                           ds_in_test=ds_in_test)
    261 #---------------------------------------------- Scale Data ----------------------------------------------#
    262 Y_df, X_df, scaler_y = scale_data(Y_df=Y_df, X_df=X_df, mask_df=train_mask_df,
--> 263                                   normalizer_y=mc['normalizer_y'], normalizer_x=mc['normalizer_x'])
    265 #----------------------------------------- Declare Dataset and Loaders ----------------------------------#
    267 if mc['mode'] == 'simple':

KeyError: 'normalizer_y'
c

Cristian (Nixtla)

08/22/2022, 1:39 PM
Hi @Pablo Andreone. The "normalizer_y" and "normalizer_x" hyperparameters are deprecated, it is called "scaler" now
You can see the changes here: https://github.com/Nixtla/neuralforecast/blob/main/nbs/experiments__utils.ipynb, and also in https://nixtla.github.io/neuralforecast/models_nhits__nhits.html. You probably need to clone or download the latest version of utils.py
2 Views