MAP (Mean Average Precision) objective in python XGBoost ranking

In general I’m looking for technical detail/insight into how XGBoost implements the rank:map learning objective (maximize mean average precision) in python. For example:

  1. Does the predict method produce probabilities? Or scores? (as discussed in the "Evaluating XGBoost ranking" topic) In my implementation the predicted scores lie between 0 and 1.

  2. Also looking for example implementations of hyper-parameter optimization and cross validation for the python implementaton of xgboost pairwise ranking, as these don’t work in an obvious way with the standard gridsearchCV sklearn API, for example.

1 Like

I too was going to ask about using cross validation with XGBRanker. The only examples I could find are from the author of the object’s repo. Following their example, I think the only way to use sklearn CV objects is to make your own scorer because the argument group has to be passed to the fit function. Using a vanilla scorer from sklearn’s make_scorer function will raise this error:

grid_search.fit(X, y, groups=X_groups)
  File "/home/jon/dev/my-ltr/venv/lib/python3.7/site-packages/sklearn/model_selection/_search.py", line 739, in fit
    self.best_estimator_.fit(X, y, **fit_params)
TypeError: fit() missing 1 required positional argument: 'group'

You cannot assume the output would be proper probabilities. The predict() method will give you any real-valued outputs, which dictates ordering between items in the same query group. See Evaluating XGBoost Ranking