https://github.com/nixtla logo
Join Slack
Powered by
hi, what’s the best way to save the `MLForecast` m...
# general
w
hi, what’s the best way to save the 
MLForecast
model? I can save the lightbgm under the hood but would lose the other attributes. I want to save the model artifacts and write an inference pipeline for serving?
j
Hi. We currently rely on pickle (more info here), please let us know if this doesn't suit your needs
w
thanks! this looks fine. Do you have any experience deploying such models in sagemaer endpoint?
j
Sorry, I haven't done it in a long time. Maybe @fede (nixtla) (they/them) has a more recent experience
w
thanks! @fede (nixtla) (they/them) let me know if you have done that before.
wow just realize this pickle file is more than 5G
is it possible to just save the lightgbm (~10mb) and later initialize the MLForecast object with the lightgbm model that’s been fitted?
j
I think the features_ attribute may be too heavy. I'm planning on removing it soon but that may help, can you try deleting it before saving to see if it reduces the size? e.g. 
del fcst.features_
and then pickling
👀 1
w
@João Matias thanks! this is super helpful. With 
del(fcst.ts.features_)
, the model size reduces from ~5G to 250MB. I’ll check if loading still works fine. Any possibility its size can be further reduced?
j
Nice! There are a couple more attributes which sizes could be reduced. Which lag features are you using?
w
Copy code
lag_features=['lag1', 'lag2', 'lag3', 'lag4', 'lag5', 'lag6', 'lag7', 'lag8', 'lag9', 'lag10', 'lag11', 'lag12', 'lag13', 'lag14', 'lag15', 'lag16', 'lag17', 'lag18', 'lag19', 'lag20', 'lag21', 'lag22', 'lag23', 'lag24', 'lag25', 'lag26', 'lag27', 'lag28', 'lag29', 'lag30', 'lag31', 'lag32', 'lag33', 'lag34', 'lag35', 'lag36', 'lag37', 'lag38', 'lag39', 'lag40', 'lag41', 'lag42', 'lag43', 'lag44', 'lag45', 'lag46', 'lag47', 'lag48', 'lag49', 'lag50', 'lag51', 'lag52', 'lag53', 'lag54', 'lag55', 'lag56', 'lag57', 'lag58', 'lag59', 'lag60']
I also have a bunch of 
date_features
with custom functions
j
In that case you can keep only the last 60 values of each serie (right now we store all the history). I think something like: 
<http://fcst.ts.ga|fcst.ts.ga> = fcst.ts.ga.take_from_groups(slice(-60,None))
And the same for fcst.ts._ga would reduce the size further
w
cool, thanks! will try and update here 🙂
👍 1
Thanks it helps! It reduces to 170~mb but still not comparable to native lightgbm or xgboost model, which is only ~10mb.
I guess there’s still data in the forecast object being serialized, in addition to the native model artifact and featurization functions?
j
Hmm. The other things I can think about are the models themselves. Can you remove them just to check? 
del fcst.models_
Oh before you do that you can serialize them separately, e.g. 
fcst.models_['LGMBRegressor'].booster_.save_model('model.txt')
You'd then have to restore it at load time but that may help with the siE
Size*
w
yeah, 
fcst.models_['LGMBRegressor'].booster_.save_model('model.txt')
is very small.
if I go with this path, what else needs to be saved separately in order to restore the forecast object? I would like to have the benefit of getting the preprocessing capability from MLForecast
j
For the preprocess you only need the feature functions which are still there, so it should work. The part where you'd need the models is the predict
w
right, but 
predict
needs to be aware of the feature processing in order to make forecast.
j
Yes, that's all saved in attributes of the ts object. I can take a look later at the other possibly heavy attributes that we may not need
Please share the size without the models if possible
w
you mean the size with just the model by doing this 
fcst.models_['LGMBRegressor'].booster_.save_model('model.txt')
?
j
I mean saving it as you are doing, deleting the models_ attribute from the forecast object and then serializing it
In order for it to work when loading you'd have to load the booster object and save it back to the models attribute. Like
Copy code
bst = lgb.Booster(model_file='model.txt')
fcst.models_ = {'LGMBRegressor': bst}
w
175MB without model
I think model is pretty small. It’s the other attributes (probably data) that need to be deleted. Appreciate your help and spending time looking into this!
j
Damn haha. Thank you for raising this, it's important for the serialized forecast object to be as small as possible and we haven't focused on that
I think it's probably the static features (a dataframe), but we need that to predict. Can you check the shape? 
fcst.ts.static_features_
w
len(fcst.ts.features)
is 89
fcst.ts.static_features.shape
-> 119299
unique identifier is static_feature right? I think serializing this data is also pretty heavy but we kind of need it if forecasting for the same id, instead of new data.
j
How many columns does it have? The unique ID can be a static feature if you defined it as one, otherwise it's just there for merging the predictions. If that df is big you could try saving it as parquet, deleting the attribute and restoring it on load.
w
are you talking about the original columns, or after featurization?
the unique ID is just an identifier. In that case, it doesn’t make sense to make it a static feature, right?
j
I meant the 
fcst.ts.static_features_
attribute. If you have an older version it may be 
ts.ts.static_features
. That attribute holds the values for the static features for each serie (if you don't have any it stores only the unique ID). The unique id can be used as a static feature if the model can use it, for example LightGBM
Open in Slack
PreviousNext
Powered by