This message was deleted.

Can you provide some sample code? Sounds like a scaler issue

Hi @José Morales, My sample code below. Data pre-processing to scale the features and Qty (target)

## Scaling by Item_ID

items = df_required['Item_ID'].unique()

item_scalers_x = {}

item_scalers_y = {}

# Select all numeric columns

ignore_qty = ['Qty']

numeric_cols = df_required_train.select_dtypes(include=['int8','int16','int32','int64','float16','float32','float64']).columns

numeric_cols = [col for col in numeric_cols if col not in ignore_qty]

for item in items:

item_filtered_fit = df_required_train[df_required_train['Item_ID'] == item].copy()

item_filtered_transform = df_required_full[df_required_full['Item_ID'] == item].copy()

scaler = MinMaxScaler()

scaler = scaler.fit(item_filtered_fit[numeric_cols])

item_filtered_transform[numeric_cols] = scaler.transform(item_filtered_transform[numeric_cols])

item_scalers_x[item] = scaler

scaler = MinMaxScaler()

scaler = scaler.fit(item_filtered_fit.loc[:, ['Qty']])

item_filtered_transform.loc[:, ['Qty']] = scaler.transform(item_filtered_transform.loc[:, ['Qty']])

item_scalers_y[item] = scaler

df_required_full.update(item_filtered_transform)

# Save the scalers as pkl files

scaler_feature_position = "Item__Scaler_feature_position.pkl"

scaler_x_file_name = "Item_Scaler_X.pkl"

scaler_y_file_name = "Item_Scaler_y.pkl"

if not os.path.exists(scalers_dir):

print("Path does not exist")

os.makedirs(scalers_dir)

else:

print("Path already exists")

# save scaler x

with open(scalers_dir + "/" + scaler_x_file_name, "wb") as file:

pkl.dump(item_scalers_x, file)

# save scaler y

with open(scalers_dir + "/" + scaler_y_file_name, "wb") as file:

pkl.dump(item_scalers_y, file)

# save the feature positions

with open(scalers_dir + "/" + scaler_feature_position, "wb") as file:

pkl.dump(numeric_cols, file)

Model Training and Predict with Unseen Data

# config for tunning etc. Most parameters taken of by the Auto models.

myconfig = {'input_size': tune.choice([horizon,horizon*2,horizon*3])

,'batch_size': tune.choice([32,64])

,'max_steps': tune.choice([500])

,'learning_rate': tune.loguniform(1e-4, 1e-2)

,'random_seed' : 666 # set a fixed seed for reproducibility but some randomness will exist

,'scaler_type': tune.choice([None]) # data is already scaled.

,'hist_exog_list' : myhist_exog_list

,'futr_exog_list' : myfutr_exog_list}

# model setup

model = [AutoNHITS(h=horizon, config=myconfig, loss=RMSE()

, search_alg=HyperOptSearch(), num_samples=3, cpus=1, verbose=False)]

# Fit the model

nf1 = NeuralForecast(models=model, freq='W-MON')

nf1.fit(df=historic_df)

# do predictions with model on unseen data

pred_df = nf1.predict(futr_df=future_df)

# Save the full data model

model_file_name = "NFC_AutoNHITS" + ".pkl"

if not os.path.exists(models_dir):

os.makedirs(models_dir)

with open(models_dir + "/" + model_file_name, "wb") as file:

pkl.dump(nf1, file)

Load the saved model and predict future rows

## Load saved model & scalers, predict and reverse scaled

# Open the pickle file and load the model

with open('./Models/NFC_AutoNHITS.pkl','rb') as f:

mymodel = pkl.load(f)

# Load scalers from pickle file for reverse scaling

with open('./Models/item_scalers_y.pkl', "rb") as file:

item_scalers_y = pkl.load(file)

# test predict mv

all_predictions = mymodel.predict(df=df_historic, futr_df=df_future)

columns = set(all_predictions.columns) - set(['Item_ID'])

for item in all_predictions['Item_ID'].unique():

# Filter the DataFrame

filtered_df = all_predictions[all_predictions['Item_ID'] == item].copy()

for column in columns:

# Reverse scale the Qty and all the predicitons

filtered_df.loc[filtered_df.index, column] = item_scalers_y[item].inverse_transform(filtered_df[[column]])

# Update the original DataFrame with the modified values

all_predictions.loc[filtered_df.index, column] = filtered_df[column]

Are the predictions inflated before you run your inverse_transform?

cant really tell if they are inflated because they are scaled. when I did the predict on unseen data, the inverse transformed data looks perfectly in the ball park of actual Qty

Are you scaling your exogenous features as well? In that case you should run transform on them before calling predict

Yes, I did for the dataframes in df=df_historic, futr_df=df_future before calling predict. I didnt show the sample code here, apologies.

Sorry, there's a lot going on in your code, if you're able to narrow it down to a small reproducible example I'm happy to help, otherwise I could just be guessing all day.

thanks. I will try to code up a reproducible sample and revert. It may take a few days.

One other thing you could check to verify is getting the scaler and verifying it's an identity scaler, i.e.

mymodel.models[0].scaler.scaler

should be the identity_scaler

what is an identity_scaler and what to expect when I run mymodel.models[0].scaler.scaler?

it just a dummy scaler that subtracts 0 and divides by 1

I just ran mymodel.models[0].scaler.scaler after loading the model and before predict, and its says 'AutoNHITS' object has no attribute scaler.

ah, the auto saves it inside the model, so it's probably

mymodel.models[0].model.scaler.scaler

it now says this

<function neuralforecast.common._scalers.identity_scaler(x, x_shift, x_scale)>

I found under Documentation - Time Series Scaling, there is a parameter local_scaler_type. Do the Auto models use this parameter? What is the default of this parameter? It seems that in addition to scaler type in config, NFC models also have a core scaler. Maybe this is causing my issue as my input data is already scaled and resulted in double scaling? _Fit the model by instantiating a

NeuralForecast

object and using the

fit

method. The

local_scaler_type

parameter is used to specify the type of scaling to be used. In this case, we will use

standard

, which scales the data to have zero mean and unit variance.Other supported scalers are

minmax

,

robust

,

robust-iqr

,

minmax

, and

boxcox

._

Hi @lobbie lobbie. No, by default the core object does not scale the data. I do recommend to switch from pre-processing and scaling the data yourself to use the scaling method of the core class, since it is already doing your idea of scaling the complete series. The core class will store the scaling method and will automatically scale any new

df

or

futr_df

to keep the domain space constant for the model. So you would remove all the code before "Model Training and Predict with Unseen Data", and simply specify

minmax

in the nf class.

@José Morales and @Cristian (Nixtla) The reason for scaling in preprocessing step is because we are running several nfc and non-nfc models in our training pipeline, and pre-scale streamline the data prep process. Will definitely try your suggestions during summer break in Aus and see the effect and hopefully identify root cause. We found that the inflated numbers only happen to 1 or 2 items and the rest looks 'normal' so the cause maybe something else. Will investigate on that front too. It is good to know the core scaler does not scale the data by default unless we explicitly specify. I also did a simulation using Excel and it looks like even double scaling and subsequently reverse scaling do not affect the numbers. Thanks for your help and suggestions.

@lobbie lobbie were you ever able to figure out why NHiTS was giving you inflated predictions? I have similar problem where I see the forecast amplitudes are larger especially at peaks and troughs. I am already using NHiTS instead of Auto version and I have tried different scalings but the issue remains. Did you have any luck?

@Farzad E, hi. i ended up reviewing my input data ie target and exogenous features. there were outliers so i removed them before model training. i also tried regression trees models to first establish baseline model and compare nfc model performance with. i still do normalisation and scaling in steps remove outlier, cube root, min max scale, train, score, reverse min max scale, cube. clip any negative forecast to 0. it worked for me. also when i do scaling and normalisation, i did the df as a whole instead of item by item. HTH.

@lobbie lobbie thanks a lot for getting back to me. A few questions: • What do you mean by cube root? I don't have any step like that. • Do you use the scaler in NHiTS or do you scale outside of NHiTS or both? I only set the NHiTS scaler param to min-max and don't do any other scaling besides that. • What do you mean by scaling df as a whole? You mean scaling the y column of df_train across all unique_ids as opposed to scaling each time series individually?

@Farzad E • Cube root transformation on the target and exogenous features. You can also try square root, log and etc transformation. Purpose is to normalise my data as they are quite skewed, and I am also using regression models (baseline models). • No, I did not use the core scaler within NFC. I normalise and scale in preparing data for modelling as I am also using other non NFC models. • Yes, I normalise and scale the target y, and all the exogenous features in df train and df test as a whole and not by each ts, because I build global models instead of local models. My time series are sales of items in the same product category. If your time series eg are sales of items in different product category, then suggest you normalise and scale them individually. You will also need to reverse transform and reverse scale when you do these outside the models as mentioned in my steps earlier. HTH

@lobbie lobbie thanks a lot! Really appreciate your response.

@Farzad E, no worries. One more note on scaling which I forgot to mention. If you are scaling outside of NFC, ensure you scale df train, save the scaler, and use the saved scaler to scale df test else you will be introducing data leakage. The saved scaler will also be used to reverse scale your predictions.