Prediction Intervals with XGBoost and Quantile regression

#1

Hi

I’m currently using a XGBoost regression model to output a single prediction. However, I want to try output prediction intervals instead. I have read online it is possible with XGBoost and Quantile regression, but I haven’t found any stable tutorials/materials online supporting this.

I know it is much easier to implement with LightGBM, however, my models performance drops when I switch over to LightGBM.

So my question is are Prediction Intervals with XGBoost possible and how? any information is much appreciated

#2

Hey there! I implemented it in XGBoost a couple years ago using a custom objective as described here: https://towardsdatascience.com/regression-prediction-intervals-with-xgboost-428e0a018b

I did try the LightGBM version as well and I got better results with it though, but this was like 3 years ago. Things may have changed by now

1 Like
#3

There’s an objective function called reg:quantileerror. The one in 1.7.x doesn’t consider learning rate, and is improved in the master branch (to be part of 2.0). The builtin version uses line search to handle zero Hessian value of the objective.

2 Likes
#4

You can also wrap xgboost in MAPIE:

https://mapie.readthedocs.io/en/latest/examples_regression/4-tutorials/plot_main-tutorial-regression.html

BR
E

1 Like