Hi <@U035DR8HD6D> <@U031KHZH06B> is Nixtla using S...
# generalv
Valeriy
12/09/2022, 3:39 PMHi @Max (Nixtla) @fede is Nixtla using Spark? From what I recollect Fugue was mentioned I presume Nixtla looked at Spark at some point and decided to use / not to use it. Was wondering if Spark is suitable for building forecasting system?
h
Han Wang
12/09/2022, 5:26 PMHi @Valeriy fugue is an abstraction on top of spark dask as ray, Nixtla uses Fugue to interact with these backends. We achieved amazing result on Spark, please read this article https://towardsdatascience.com/distributed-forecast-of-1m-time-series-in-under-15-minutes-with-spark-nixtla-and-fugue-e9892da6fd5c
👍 1
Han Wang
12/09/2022, 5:29 PMand more importantly, it's easy to use. With Statsforecast interface, it is trivial to scale out on Spark, here is a code example https://github.com/Nixtla/statsforecast/blob/8ea8c9e0d72b8e2b2edb4fb18a81ef5f713ec8f0/nbs/examples/Prophet_spark_m5.ipynb#L112
👍 1
v
Valeriy
12/10/2022, 11:05 AMHi @Han Wang that’s awesome thanks for sharing these links.