@Han Wang For future reference, this run completed successfully with xLSTM @8xH100. Weirdly, in console the non-zero rank processes still errored out after training. I could evaluate the run successfully though.