A problem when fitting XGBRegressor with xgb_model initialized

Hello! I want to continue the training of a model with parameter xgb_model assigned in the fit function as follows:

xgb_model_full_data = xgboost_model.fit(train_features, train_labels, eval_metric=custimized_error, eval_set=[(validation_features, validation_labels)], xgb_model=“test_xgb.bin”, early_stopping_rounds=1000)

The training process runs smoothly and the debug information show the final score assume 2.8 with early stopping.

When the fitting is finished, I do the following:
predictions = xgb_model_full_data.predict(validation_features)

The I calculate the score using the predictions and validation_labels, I got 3.23.

I do not understand why the scores are different since the predication is done on the same validation set.

I tried to train the regressor without initializing the xgb_model parameter, and repeat above procedure. The two scores show exactly same value.

What am I missing here?

Thanks and Happy new year.

Which XGBoost version are you using?

Hi hcho3,

the version is 1.2.0.

Should the training set used to continue training be exactly the same as the one used in test_xgb.bin?
Even a subset of the training set used in test_xgb.bin will lead to problem?

Some additional information:

  1. random seed is always the same for different run.