I am a bit confused about the xgboost gpu support, since I find a lot of old topics, so a few question accumulated:
What is the n_gpu parameter? I do not find it in the docs.
Is it possible to enable multi GPU support in scikit without dask? (I have a lot of different algorithms in a gridsearch and can not do a “special” dask gridsearch for xgboost)
How does xgboost behave in a gridsearch where n_jobs >1? What setting do you recommend for n_jobs?
My understanding is that setting n_jobs to lets say 4 will create 4 threads on my gpu when gpu_hist is enabled. However I found that by doing so, the training process is massively slowing down, even though als my process just accumulate for 50% of the GPU RAM. I do not know what the optimal setting is supposed to be. At n_jobs = 1 the GPU is at 40% processing load, so it feels like some potential is wasted here. The funny thing is when looking at catboost on gpu only n_jobs=1 is possibly in a gridsearch, which instantly fills the memory of the whole gpu - but it also only used 10-30% of the processing power. I use a Pascal Titan X for computing.