Custom Loss function

#1

Hi,

Is there a way to pass on additional parameters to an XGBoost custom loss function? Also can we track the current structure of the tree at every split?

Thanks
Kshitij

#2

Depends on how far you’re willing to go to reach this goal. In this case you’d have to edit C++ code.

Loss function in general is used to calculate gradients and hessians. that’s it.

However, you can modify the code that calculates loss change. I can point you where that is if you really want to.

#3

Sure, please let me know. Thanks

#4

If you use ‘hist’ option to fit trees, then this file is the one you need to look at

src/tree/updater_histmaker.cc

FindSplit is the routine that finds split. You’ll see a parralell call to EnumerateSplits that looks for the best split

In EnumerateSplit routine, look for calculations of loss_chg. This is where you can add your regularization terms. fid variable there is your column id.

After the best split is selected inside if statement
if (best.loss_chg > kRtEps) {

you can use the selected column id to store in whatever structure you need for your regularization. the selected column id is best.SplitIndex()

Something like that. good luck.