Some init parameters are ignored when initializing a DMatrix from a DataIter

fpeterek · February 3, 2022, 7:25am

Hello.

Recently, I had to deal with quite an unfortunate and difficult to figure out problem. At my place of work, we are currently trying to create XGB models on a Kubernetes cluster, and thus, certain limitations are imposed upon us (mostly memory limitations, to be more precise). To get around said limitations, I attempted to implement my own DataIter.

When attempting to teach the model using a DMatrix created from a DataIter, I kept receiving an exception saying feature names are not unique. We would pass the feature_names variable to the __init__ of the DMatrix, yet, as I figured out later, when initializing the DMatrix from an iterator, the feature_names parameter is ignored completely. The fix was rather simple, all I had to do was call set_info right after the DMatrix was initialized, which set feature_names to its proper value and I stopped receiving the aforementioned exception.

I know the DataIter feature is still considered experimental, yet, I wanted to ask. Is this behavior intended, or is it a bug? The fact that DMatrix.__init__ may ignore some parameters when initializing from an iterator is not documented anywhere. I did not notice anyone anywhere encountering the same issue. Thus, I was curious whether I was doing something wrong and feature_names are left uninitialized for a reason that is yet elusive to me, or whether I could at least file a bug report or submit a pull request.

Thanks a lot and sorry if I’m just missing something.

CurlyHairedGenius · December 21, 2022, 6:05am

I had a similar issue attempting to use the a custom DataIter with DeviceQuantileDMatrix.
You can pass the feature_names into the input_data call inside your DataIter implementation.
Take a look at the definition for input_data:

github.com

dmlc/xgboost/blob/e01639548a7faca9886fabe7201e7679fdd06b15/python-package/xgboost/core.py#L491




def _next_wrapper(self, this: None) -> int:  # pylint: disable=unused-argument
    """A wrapper for user defined `next` function.


    `this` is not used in Python.  ctypes can handle `self` of a Python
    member function automatically when converting it to c function
    pointer.


    """
    @require_keyword_args(True)
    def input_data(
        data: Any,
        *,
        feature_names: Optional[FeatureNames] = None,
        feature_types: Optional[FeatureTypes] = None,
        **kwargs: Any,
    ) -> None:
        from .data import _proxy_transform, dispatch_proxy_set_data


        new, cat_codes, feature_names, feature_types = _proxy_transform(
            data,

    input_data:
        A function with same data fields like `data`, `label` with
        `xgboost.DMatrix`.

With that said the other trick is to use parameter names when you call input_data.
e.i.: input_data(data=x, label=y) instead of input_data(x, y)

To get DeviceQuantileDMatrix to work, I also needed to pass a cupy array.

Take a look at the source for dispatch_proxy_set_data:

github.com

dmlc/xgboost/blob/c6a8754c62496e43452e6edf49eb0eb89ffcdc70/python-package/xgboost/data.py#L1220


        data = pd.DataFrame(data)
    if _is_pandas_df(data):
        arr, feature_names, feature_types = _transform_pandas_df(
            data, enable_categorical, feature_names, feature_types
        )
        arr, _ = _ensure_np_dtype(arr, arr.dtype)
        return arr, None, feature_names, feature_types
    raise TypeError("Value type is not supported for data iterator:" + str(type(data)))




def dispatch_proxy_set_data(
    proxy: _ProxyDMatrix,
    data: DataType,
    cat_codes: Optional[list],
    allow_host: bool,
) -> None:
    """Dispatch for DeviceQuantileDMatrix."""
    if not _is_cudf_ser(data) and not _is_pandas_series(data):
        _check_data_shape(data)


    if _is_cudf_df(data):

Some __init__ parameters are ignored when initializing a DMatrix from a DataIter

Some init parameters are ignored when initializing a DMatrix from a DataIter