Jointly train two XGBoost to predict one target [Regression]

Dear XGBoost folks,

I have 3 features, let’s say x1 , x2 , s ; and one target Y I want to predict . You can imagine x1 and x2 are true covariates, s is the variable we want to regress out.

In the linear model setting, people would like to have a Y = intercept + x_1\beta_1 + x_2\beta_2 + s\beta_s + eps. Then the role of intercept + x_1\beta_1 + x_2\beta_2 would know.

I want to apply XGB to capture some non-linear information of x1 and x2 corresponding to Y . Could I jointly optimize two XGBoost models such as Y = XGB_1(x_1, x_2) + XGB_2(s) ?

I am curious whether it is possible in XGBoost package or do you have any better suggestions?

Thanks!

1 Like