Negative labels in XGBRanker w/ listwise objective

#1

Do I understand correctly that XGBRanker w/ listwise objective does not support negative labels? Given that labels are unsigned integers: rank_metric.cc

#2

Yes, the labels should be non-negative integers: 0, 1, 2, 3, …

#3

@hcho3 May I ask could you clarify the input label requirements? Are non-negative integer labels required for all the LTR objectives (rank:pairwise, rank:ndcg, rank:map)? I have been experimenting with the Python Client using real-valued labels and the model seemed to train properly. Maybe there are some nuances I am not aware of.

#4

The learning-to-rank task is formulated so that the relevance of each document is indicated with a non-negative integer. You can look up the definition of NDCG metric.

#5

Thank you, Philip, for the prompt reply. For my application, some documents are known to be 10x or even 100x more relevant than others. Do you recommend consecutive integer labels 1,2,3, or scaled values 1,10,100?

Also, may I ask which formulation of DCG is used by xgboost LTR, the traditional formula with simple relevance numerator or the alternate formula with exponential relevance?
See https://en.wikipedia.org/wiki/Discounted_cumulative_gain#Discounted_Cumulative_Gain

#6

XGBoost uses the alternate formula with 2^relevance term. I don’t recommend integer labels that are too big, due to possible numeric instability.

#7

Thank you Philip. That makes sense. For completeness, may I ask, what are the labeling requirements for the rank:pairwise objective?

#8

As I said before, the label should be in the form of 0, 1, 2,…, i.e. non-negative integers. The higher number indicate the greater relevance, and 0 means “not relevant”

#9

Thank you Philip, the reason I asked was to clarify whether the rank:pairwise objective had the same label requirements as the rank:ndcg objective. I appreciate your patience and support.

#10

Yes, all ranking objectives have the same requirement.