I'm trying to fix a rabbit hole issue in statsforecast which causes our tests to fail in this PR. Basically, Spark tests fail because a Spark session can't be created. It seems this is due to upgrading the manylinux image that is used for the tests. The upgrade is necessary because otherwise yet another error occurs because DuckDB did an upgrade which requires an upgrade to the linux image we run in our tests. And fugue uses DuckDB, which in turn otherwise fails the test - as you can see in this PR. I can't figure out how to make these Spark tests comply, if I google the issue it seems the error results from JAVA_HOME not being defined but I tried that but that doesn't work. Maybe @Han Wang knows about this? I'm tempted to just delete/ignore these tests because now all Python>3.10 tests that use Statsforecast in any of our libs fail because of another statsforecast dependency issue which needs to be resolved but this is blocking the fix (as our tests keep failing on this). @Han Wang should I otherwise just force a downgrade of fugue or of DuckDB? That's also an option but this whole rabbit hole started with trying to pin a scipy version to fix our tests and then I end up pinning even more packages....

This Friday just keeps bringing bad luck 😅

Jup....

I think, this is because of Java version, should be a very simple fix in the GitHub workflow config, I will take a look

Thanks! I found the Java version to be the issue indeed

Nice have you resolved the issue?

Yes!

Great thanks!