[XGBoost4J-Spark] Model training stuck at `foreachPartition at XGBoost.scala:565`

#1

We are trying to train a binary classification (0/1) model using XGBoost4J-Spark and it is getting stuck at foreachPartition at XGBoost.scala:565. The model training is working for different datasets of similar size, but for a particular dataset with the dependent variable distribution of

0: 49.09%
1: 50.91%

the training gets stuck at foreachPartition at XGBoost.scala:565.
Using xgboost4j and xgboost4j-spark version 1.0.0 on spark-2.4.0.

#2

I am having the same issue when using the tree method “approx”. It seems to work about 50% of the time for me using the exact same data and the exact same spark configurations.

I opened an issue here.