Darius Marginean
12/16/2022, 8:29 AMcrossvalidation
method from the MLForecast
class it is done using Time Series cross-validation & in statsforecast, the crossvalidation
method from StatsForecast class it is done using Expanding Window?
2. I've walked over your tutorial for crossvalidation
in statsforecast
(https://nixtla.github.io/statsforecast/examples/crossvalidation.html) and when you want to evaluate the results using RMSE from datasetsforecast.losses
, I've seen that it is computed on the whole crossvalidation_df
not on each fold separately & then aggregate the results ( unlike in mlforecast: https://nixtla.github.io/mlforecast/docs/end_to_end_walkthrough.html, where you've put an example on how to compute the losses on each fold separately using evaluate_cf(df):
functionstep_size
, which is missing from mlforecast) Is there a method to do Expanding window cross_validation in mlforecast too?offset
variable from the backtest_splits()
function did what I wanted (Expanding window with step_size=1
)
Previous:
def backtest_splits(
data,
n_windows: int,
window_size: int,
freq: Union[pd.offsets.BaseOffset, int],
time_col: str = "ds",
):
for i in range(n_windows):
offset = (n_windows - i) * window_size
...
Now:
def backtest_splits(
data,
n_windows: int,
window_size: int,
freq: Union[pd.offsets.BaseOffset, int],
time_col: str = "ds",
expending_window = int,
):
if expending_window == 1:
for i in range(n_windows):
offset = window_size+n_windows-1-i
... (the rest of the code is left unchanged)
else:
Previous implementation
Also I added expending_window as an argument in the cross_validation method from the MLForecast class like so:
def cross_validation(
self,
data: pd.DataFrame,
n_windows: int,
window_size: int,
id_col: str,
time_col: str,
target_col: str,
expending_window: int = 0,
static_features: Optional[List[str]] = None,
dropna: bool = True,
keep_last_n: Optional[int] = None,
dynamic_dfs: Optional[List[pd.DataFrame]] = None,
predict_fn: Optional[Callable] = None,
**predict_fn_kwargs,
):
If it's not specified it will use the previous implementation, if it's 1 it will use Expanding window with size of 1. Most probably not the best aproach but I needed it fastfede (nixtla) (they/them)
12/16/2022, 5:29 PMstep_size
in mlforecast is window_size
. We are working on matching the methods, arguments, and functionality between mlforecast and statsforecast, we will prioritize the cross validation method 🙂Max (Nixtla)
12/16/2022, 5:53 PM