Hello. I’m new to XGBRanker models and I’m trying to use it for a ranking project.
I realized that; my predictions on the same test set, are different in every run.
I see that; when max_depth or n_estimators are very high, there is a small randomness in the predictions in every run - even when I set a constant random_state parameter
This is my code block;
x_train = x_train_backup.copy()
x_test = x_test_backup.copy()
y_train = y_train_backup.copy()
model = XGBRanker(tree_method=‘gpu_hist’,
objective=‘rank:pairwise’,
eval_metric = ‘ndcg’,
random_state = 1001,
eta = 1e-1,
max_depth = 10,
n_estimators = 320,
reg_alpha = 1e-4,
reg_lambda = 1e-1,
min_child_weight = 1,
n_jobs = 1
)
group_train = list(x_train.groupby(x_train.index)[‘group_eval’].sum())
model.fit(x_train, y_train, group = group_train)
prediction = model.predict(x_test)
However, when I reduce max_depth to 2-3; or n_estimators to 5, the problem disappear and the predictions in every run become identical.
Remember that max_depth and n_estimators are the parameters that increase overfitting.
So I have a sense that, in XGBRanker, coming close to overfitting cause some randomness in the predictions in enery run on the same test set.
Does anyone have any idea about this issue?