Interpretation of regression coefficients for Linear Booster

I am using XGBRegressor for multiple linear regression.

xgb_model = XGBRegressor(n_estimators=10, learning_rate=0.06, gamma=1, booster='gblinear', 
                     reg_lambda=0.0001, reg_alpha=0.0001,
                     n_jobs=-1)

I am getting the coefficients using xgb_model.coef_. Would the interpretation of the coefficients be the same as that of OLS Regression? That is, they represent “the mean change in the response variable for one unit of change in the predictor variable while holding other predictors in the model constant.”

Yes, since the model is of form y = X * beta.

1 Like

I had expected that X * coef_ + intercept_ would equal mdl.predict(X). Instead it’s systematically off by just under 0.5, with some variation around that. Any guess as to what I’m missing?

UPDATE: The mysterious 0.5 offset went away when I explicitly set base_score=0. Even though the parameter was saved as “None”, it appears to have been left at the default of 0.5 and added to X * coefs + intercept when fitting or predicting. I’m guessing this should probably be reported as a bug? Or at least documented somewhere?

I’m unclear on the source of (much smaller) the residual variation. Perhaps rounding or precision error? Has this been documented before?

@system I am trying to translate the linear learner of XGBoost in such way (like LR), X * coefs + intercept. I also face the base score problem. I think there is not a bug. Setting the base score help XGBoost quickly to get a good fitting point to avoid the sample shift problem.