Hello,
This is related to:
MultiThreading not working with version 0.71.2 and
Big data will break the nthread setting in R-xgboost 0.71.2
Running the same model on R 3.5.0 and comparing 0.6.4.1 vs 0.71-2 results in very inferior timing performances for 0.71-2. I tested 0.6.4.1 using approx and exact and 0.71-2 by using approx, exact and hist. The machine has 2 CPUs (E5-2690), 14 cores each for a total of 56 threads.
In general, on 0.71-2 neither method, exact or approx manage to use the same number of threads as in 0.6.4.1. Interestingly hist does use a lot of cores but it is still slower than exact and approx.
Goodness-of-fit-wise in my applications, usual metrics (MAE, RMSE, etc.) are perfectly comparable. I do not observed any deterioration in GoF.
library(xgboost)
library(microbenchmark)
set.seed(222)
N <- 2*10^5
p <- 350
x <- matrix(rnorm(N * p), ncol = p)
y <- rnorm(N )
bnc_exact <- microbenchmark(
mymodel <- xgboost(data = x, label = y, nrounds = 5,
objective = "reg:linear", "tree_method" = "exact",
max_depth = 10, min_child_weight = 1, eta = 1,
subsample = 0.66, colsample_bytree = 0.33),
times = 6)
bnc_approx <- microbenchmark(
mymodel <- xgboost(data = x, label = y, nrounds = 5,
objective = "reg:linear", "tree_method" = "approx",
max_depth = 10, min_child_weight = 1, eta = 1,
subsample = 0.66, colsample_bytree = 0.33),
times = 6)
bnc_hist <- microbenchmark(
mymodel <- xgboost(data = x, label = y, nrounds = 5,
objective = "reg:linear", "tree_method" = "hist",
max_depth = 10, min_child_weight = 1, eta = 1,
subsample = 0.66, colsample_bytree = 0.33),
times = 6)
For the above example:
On 0.6.4.1, approx and exact required ~4.5" and ~3.0" for their fastest run respectively.
On 0.71-21, approx and exact were pretty comparable with ~14.4" and ~14.1" for their fastest run - hist's fastest run was 57.5". On other applications with ~1.5M points the performance hit is nearly 10x.
The observed performance deterioration on exact and approx is problematic. I appreciate that dual-CPUs multi-core systems are reasonably uncommon, but any help would be greatly appreciated. Are there any steps I could take to alleviate this performance hit?
I can down-grade my xgboost version (to 0.6.4.1) for the time being but can you please look into newer versions of xgboost becoming as fast as previously?
(0.71.1 shows a minor decrease in performance compared to 0.6.4.1.)
> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux
Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
locale:
[1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8
[5] LC_MONETARY=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8 LC_PAPER=en_GB.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] microbenchmark_1.4-4 xgboost_0.6.4.1