This message was deleted.

Hey. We currently support only pandas dataframes, so you'd need to load it into memory. You could build a lightgbm dataset after the preprocessing and train using that, not sure if it that'd help a lot though.

Makes sense. Thanks for the quick response!

We try to keep the types where possible, so if you define the id as categorical and the target as float32 you could reduce the memory usage