Does "refresh" option update split thresholds of tree nodes?

Hi,
I am using xgboost for transfer learning. I am expecting both the leaf node values and the split thresholds of the non-leaf nodes will be adapted to a new set of data.

Specifically, I set the ‘process_type’ parameter to be ‘update’ and ‘updater’ to be ‘refresh’. From the manual I know that the leaf node update is controlled by setting ‘refresh_leaf’. I am just wondering after setting ‘update’=‘refresh’, will the split thresholds of the non-leaf nodes be retrained/updated as well? The manual said that the “node stats” will be updated, but I guess that does not refer to the split thresholds but the statistics like cover or gain of features?

It would be helpful if someone could let me know whether my understanding is right. And if the split thresholds are not updated, is there any reason not doing so?

Thanks a lot!

No, split thresholds will not be updated. The reason is that XGBoost is a batch algorithm, requiring the entire training data to be present in order to determine best set of thresholds and features in non-leaf nodes. By the time the model receives a new batch of data, the old batch of data is no longer in memory, so it is not possible to find new split thresholds that would account of both old and new data.

1 Like