I refer to https://xgboost.readthedocs.io/en/latest/tutorials/param_tuning.html
The page says that “there are two ways to improve [the model]” which depends on what you are trying to improve:
Firstly:
If you care only about the overall performance metric (AUC) of your prediction
Secondly:
If you care about predicting the right probability
What is the difference in these cases? When would you prefer the one over the other?
If you correct data imbalance via assigning data weights, you will introduce a bias to the predicted probability, i.e. the predicted probability for the minority class will be over-estimated.
Right, so using scale_pos_weight may overestimate the minority class then.
Curious though, in what cases would you be interested in optimizing AUC more than getting the right probability? IE, why optimize for the first, rather than the second.