Federated learning: Can I train using a histogram representation of data?

Hecramco · May 4, 2023, 5:20pm

Hello, I’m trying to aggregate some models for federated learning of a xgboost regressor. I managed to get an aggregated histogram representation of features and it’s gradients, but can I use this data to train the model given that I have the gradient and hessians? or is it not possible to re-use this data?

rongou · May 10, 2023, 5:55pm

You can try the custom boost function by passing your gradients and hessians:

github.com

dmlc/xgboost/blob/master/python-package/xgboost/core.py#L1930


        _check_call(
            _LIB.XGBoosterUpdateOneIter(
                self.handle, ctypes.c_int(iteration), dtrain.handle
            )
        )
    else:
        pred = self.predict(dtrain, output_margin=True, training=True)
        grad, hess = fobj(pred, dtrain)
        self.boost(dtrain, grad, hess)


def boost(self, dtrain: DMatrix, grad: np.ndarray, hess: np.ndarray) -> None:
    """Boost the booster for one iteration, with customized gradient
    statistics.  Like :py:func:`xgboost.Booster.update`, this
    function should not be called directly by users.


    Parameters
    ----------
    dtrain :
        The training DMatrix.
    grad :
        The first order of gradient.