Custom Loss function



Is there a way to pass on additional parameters to an XGBoost custom loss function? Also can we track the current structure of the tree at every split?



Depends on how far you’re willing to go to reach this goal. In this case you’d have to edit C++ code.

Loss function in general is used to calculate gradients and hessians. that’s it.

However, you can modify the code that calculates loss change. I can point you where that is if you really want to.


Sure, please let me know. Thanks


If you use ‘hist’ option to fit trees, then this file is the one you need to look at


FindSplit is the routine that finds split. You’ll see a parralell call to EnumerateSplits that looks for the best split

In EnumerateSplit routine, look for calculations of loss_chg. This is where you can add your regularization terms. fid variable there is your column id.

After the best split is selected inside if statement
if (best.loss_chg > kRtEps) {

you can use the selected column id to store in whatever structure you need for your regularization. the selected column id is best.SplitIndex()

Something like that. good luck.