I’m getting GPU out of memory error on Windows. But the same code runs without error on Ubuntu on the same hardware.
The error is strange because it says free memory: 14 GB, requested memory: 9 GB.
Error:
Traceback (most recent call last):
File "C:\Users\Billy\Documents\numerai\test.py", line 9, in <module>
model.fit(X,y)
File "C:\Python39\lib\site-packages\xgboost\core.py", line 532, in inner_f
return f(**kwargs)
File "C:\Python39\lib\site-packages\xgboost\sklearn.py", line 961, in fit
self._Booster = train(
File "C:\Python39\lib\site-packages\xgboost\core.py", line 532, in inner_f
return f(**kwargs)
File "C:\Python39\lib\site-packages\xgboost\training.py", line 181, in train
bst.update(dtrain, i, obj)
File "C:\Python39\lib\site-packages\xgboost\core.py", line 1733, in update
_check_call(_LIB.XGBoosterUpdateOneIter(self.handle,
File "C:\Python39\lib\site-packages\xgboost\core.py", line 203, in _check_call
raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [00:40:36] C:/Users/Administrator/workspace/xgboost-win64_release_1.6.0/src/tree/updater_gpu_hist.cu:712: Exception in gpu_hist: [00:40:36] c:\users\administrator\workspace\xgboost-win64_release_1.6.0\src\data\../common/device_helpers.cuh:428: Memory allocation error on worker 0: Caching allocator
- Free memory: 14626127872
- Requested memory: 9755522303
Code to replicate:
from sklearn.datasets import make_regression
from xgboost import XGBRegressor
print(f"Making regression problem...")
X, y = make_regression(n_samples=4000000, n_features=1200, noise=0.5)
print(f"Fitting model...")
model = XGBRegressor(tree_method='gpu_hist')
model.fit(X,y)
print(f"Model fit!")
System specs:
XGBoost 1.6.1
CPU: Intel 8700k
RAM: 64 GB
GPU: NVIDIA 3090
GPU RAM: 24 GB
Windows 10
Nvidia driver: 512.77
CUDA: 11.6
Ubuntu 22.04 LTS