Using XGBoost with the following parameters:
{‘objective’: ‘rank:ndcg’, ‘eta’: 0.1, ‘gamma’: 1.0, ‘eval_metric’: ‘ndcg@3’, ‘min_child_weight’: 0.1, ‘max_depth’: 6}
After building a model I get NDCG@3 larger than 1:
eval-ndcg@3:72015.195312
Given what I know of NDCG (that is normalized by the ideal ranking DCG and has to be within the (0,1) range) this has to be a bug, right?
I’m not that concerned with this bug in the evaluation since I can compute NDCG on my own from the ranking and the labels. However, I am concerned about how the model was trained given that the objective may also reach wrong values.
Another thing to note: the labels were originally between 0 and 1. But I discovered that XGBoost rounds down the labels to the nearest integer, meaning that most labels were 0. To overcome this issue, I multiplied all the labels by a factor of 1000. (Is there another way around this?). That resulted in some very large labels.
Going back to the large NDCG values, I guess it might be caused by an overflow (raising 2 to the power of a large number in the computation of NDCG).
Any suggestions?
Thank you!