https://github.com/nixtla logo
v

Valeriy

12/09/2022, 3:39 PM
Hi @Max (Nixtla) @fede is Nixtla using Spark? From what I recollect Fugue was mentioned I presume Nixtla looked at Spark at some point and decided to use / not to use it. Was wondering if Spark is suitable for building forecasting system?
h

Han Wang

12/09/2022, 5:26 PM
Hi @Valeriy fugue is an abstraction on top of spark dask as ray, Nixtla uses Fugue to interact with these backends. We achieved amazing result on Spark, please read this article https://towardsdatascience.com/distributed-forecast-of-1m-time-series-in-under-15-minutes-with-spark-nixtla-and-fugue-e9892da6fd5c
👍 1
and more importantly, it's easy to use. With Statsforecast interface, it is trivial to scale out on Spark, here is a code example https://github.com/Nixtla/statsforecast/blob/8ea8c9e0d72b8e2b2edb4fb18a81ef5f713ec8f0/nbs/examples/Prophet_spark_m5.ipynb#L112
👍 1
v

Valeriy

12/10/2022, 11:05 AM
Hi @Han Wang that’s awesome thanks for sharing these links.