Hi Team, I am trying to do an evaluation of my forecasts with the

utilsforecast.evaluation.evaluate

function but I keep hitting the following error when adding the train_df parameter. I’ve had a look at the source code which indicates that train_df is optional, however if I omit the train_df I get the same error. call:

from utilsforecast.evaluation import evaluate
from utilsforecast.losses import (
    mse,   # mean square error
    mape,  # mean absolute percentage error
    mae,   # mean absolute error
    mase,  # mean absolute scaled error
    rmse,  # root mean square error
    mqloss, # multi-quantile loss
    scaled_crps, # scaled continues ranked probability score
    )
eval_df = evaluate(forecasts, metrics=[metrics], train_df=train)

> File “/Users/.pyenv/versions/datascience/lib/python3.11/site-packages/utilsforecast/evaluation.py”, line 72, in evaluate > metric_requires_y_train = { > ^ > File “/Users/.pyenv/versions/datascience/lib/python3.11/site-packages/utilsforecast/evaluation.py”, line 73, in dictcomp > _function_name(m): “train_df” in inspect.signature(m).parameters > ^^^^^^^^^^^^^^^^^ > File “/Users/.pyenv/versions/datascience/lib/python3.11/site-packages/utilsforecast/evaluation.py”, line 22, in _function_name > name = f.name > ^^^^^^^^^^ > AttributeError: ‘list’ object has no attribute ‘__name__‘. Did you mean: ‘__ne__‘? my forecasts dataframe has the following columns:

unique_id

ds

Naive

Naive-lo-95

Naive-hi-95

MSTL

MSTL-lo-95

MSTL-hi-95

AutoETS

AutoETS-lo-95

AutoETS-hi-95

combined-lo

combined-hi

combined-base

my train dataframe has the following columns:

ds

y

unique_id

and the metrics list is simply [mse, rmse, mape] Any assistance would be appreciated. Thanks in advance.

Hello! I think your forecasts dataframe must include the actual values. By default, it looks for a column called

y

. Otherwise, you can specify the column name using

target_col

. I created a minimal example to mimic your situation and it works on my end:

data = {
    'unique_id': [1, 1],  # Setting constant value to 1
    'ds': ['2024-01-01', '2024-01-02'],
    'y': [102, 103],
    'Naive': [100, 105],
    'Naive-lo-95': [95, 100],
    'Naive-hi-95': [105, 110],
    'MSTL': [102, 107],
    'MSTL-lo-95': [97, 102],
    'MSTL-hi-95': [107, 112],
    'AutoETS': [103, 108],
    'AutoETS-lo-95': [98, 103],
    'AutoETS-hi-95': [108, 113],
    'combined-lo': [96, 101],
    'combined-hi': [106, 111],
    'combined-base': [101, 106]
}

# Creating the DataFrame
df_sample = pd.DataFrame(data)

evaluation_df = evaluate(df=df_sample, metrics=[mse, rmse, mape])

I hope this helps!

Thank you for your rep[y I will try that.