https://github.com/nixtla logo
n

Nitin Kandpal

03/17/2023, 6:34 AM
Hi everyone, Is there any way to get fitted values in statsforecast spark pipeline (FugueBackend), right now there are only two functions forecast and cross_validation_fitted_values
s

Sam Miller

03/17/2023, 8:41 AM
have you tried calling
cross_validate(fitted=True)
, then
cross_validation_fitted_values()
?
n

Nitin Kandpal

03/17/2023, 8:54 AM
Hi Sam, But there is no function cross_validate for spark FugueBackend, may you please share some snippet
s

Sam Miller

03/17/2023, 8:57 AM
Ah my apologies - wasn't aware of that
m

Max (Nixtla)

03/17/2023, 6:36 PM
Thanks @Nitin Kandpal, currently the dist engines don't support returning the fitted values. If that is something you need, we could definitely work on it. It that is the case, could you open a GitHub issue here.
And thanks @Sam Miller!
n

Nitin Kandpal

03/18/2023, 1:04 PM
Hi @Max (Nixtla), Thank you for your reply. There is already a ticket for this request : https://github.com/Nixtla/statsforecast/issues/382
m

Max (Nixtla)

03/18/2023, 2:13 PM
Thanks! You are right.
n

Nitin Kandpal

03/20/2023, 5:27 AM
Hi @Max (Nixtla), in the mean time is there any other way I can get the fitted value while I am using spark dataframes (distributed environment)
f

fede (nixtla) (they/them)

03/20/2023, 6:59 PM
hey @Nitin Kandpal! An alternative would be to modify by hand the
FugueBanckend
class to return the fitted values (https://github.com/Nixtla/statsforecast/blob/main/statsforecast/distributed/fugue.py#L47).
h

Han Wang

03/21/2023, 5:34 AM
i think maybe we can add that
and end users should not need to use FugueBackend, fugue should be implicit now