Doubt in xgboost GPU compilation

Hi community,

My doubt in GPU compilation is that, xgboost has these macro checks: #if defined(__CUDACC__) #else. Where exactly is the __CUDACC__ flag set? If the flag is not set is there any explicit way to set the flag?
Thanks.

The __CUDACC__ flag is defined whenever the nvcc compiler compiles a CUDA source file (.cu). And no, you cannot set the flag manually; nvcc sets it automatically.

So does it mean that it will be true only in .cu files and not in .cc, .h, .cuh files?

Yes. The __CUDACC__ flag is only defined in .cu files. This macro is useful in headers that get included in both .cc and .cu files, since we can add expressions that only apply when includded in .cu files.

Got it. One more thing, is it the case that for .cuh it won’t be set to true because, during compilation, just the .cu and and .cc files get compiled. So, CUDACC is set only if the header containing it is included in a .cu file and not a .cuh file?

Cuh file is also a header. The difference is that, as a matter of convention, cuh file only gets included into .cu files (and not into .cc files). As a result, the __CUDACC__ macro is always active in cuh files.

Thank you. I was able to resolve my issue with the help of your comments. I had another doubt, how do you ensure that XGBOOST_USE_CUDA flag is active, i.e. I can see that in the CMakeLists.txt you have set -DXGBOOST_USE_CUDA to 1. Is there a way to check to see how this flag is set, any message that can be printed in the CMakeLists.txt?

XGBOOST_USE_CUDA is defined everywhere if you set USE_CUDA=ON in the CMake build.

I am observing that it is defined in .cu, .cuh files but not in .h files. Also I see that it is defined in learner.cc (in ConfigureUpdaters() it calls the AssertGPUSupport where it is true) but not in cli_main.cc (checked by adding a similar ifdef XGBOOST_USE_CUDA) file. How exactly does this work, like where all is it supposed to be true?

The symbol is defined for all headers and sources that constitute the CMake target objxgboost:

You should also set CMAKE_VERBOSE_MAKEFILE option when building XGBoost, to inspect which macros get applied where.

mkdir build
cd build
cmake .. -DCMAKE_VERBOSE_MAKEFILE=ON -DUSE_CUDA=ON
make

Sure will check this out. Thanks