I am using the XGBoost Python API to train a model with the multi_strategy parameter set to multi_output_tree.
I am able to get the leaf indices for each tree using the predict() method with the pred_leaf parameter set to True. However, I am not sure how to convert these leaf indices to the output of each tree (weights) and add with the base score to get the output of the model manually.
Specifically, I am trying to do the following:
- Get the leaf indices for each tree using the
predict()method with thepred_leafparameter set toTrue. - For each tree and leaf index, get the output value vector of the tree.
- Add the output values from all of the trees to get the final output of the model.
Basically, I am looking for something like a booster.get_nodes_weights() method. This method would return a list of lists, where each sublist contains the output values of the nodes in a single tree.
Is there any way to do this?
Additionally, I am wondering if there is any way to set the leaf values of the trees manually. This would be useful for implementing a “fine tuning” update.
I know about refresh update in updater options, but what i’m looking for is for example to change leaf values with SGD with Momentum after the tree structure is formed.
Thanks in advance for your help!
Additional information:
- I am using the following version of XGBoost:
xgboost==2.0.0 - I am training my model with
multi_strategyparameter set tomulti_output_tree - booster.get_dump method is not clean and doesn’t work for in
multi_output_treesetting.