Apparent bug when attempting to create custom evaluation metric in R

#1

No luck on stack overflow so trying here - not sure if this is the right place to report bugs… I am trying to create a custom poisson log likelihood evaluation metric for use in xgboost:

like_eval <- function(dat,preds)
  labels <- getinfo(dat,"label")
  err <- -log(dpois(x = labels, lambda = preds))
  return(list(metric = "error", value = err))
}

Everything I have seen so far indicates that this is the correct format. However, this raises an error ‘evaluation results must have names’. The source of this is in the function:

format.eval.string <- function(iter, eval_res, eval_err = NULL) {
  if (length(eval_res) == 0)
    stop('no evaluation results')
  enames <- names(eval_res)
  if (is.null(enames))
    stop('evaluation results must have names')
  iter <- sprintf('[%d]\t', iter)
  if (!is.null(eval_err)) {
    if (length(eval_res) != length(eval_err))
      stop('eval_res & eval_err lengths mismatch')
    res <- paste0(sprintf("%s:%f+%f", enames, eval_res, eval_err), collapse = '\t')
  } else {
    res <- paste0(sprintf("%s:%f", enames, eval_res), collapse = '\t')
  }
  return(paste0(iter, res))
}

To the best of my knowledge, having examined the source code for a few hours, this is a bug. Besides, this exception seems purely aesthetic in nature. The call to xgb.train is:

    mod <- xgb.train(
    objective = like_obj,
    feval = like_eval,
    maximize = FALSE,
    data = dtrain,
    nrounds = 1000,
    watchlist =  list(train=dtrain, test=dtest),
    early_stopping_rounds = 50,
    verbose = 2)

Is there a way to format the evaluation metric function to get around this issue (alternatively could this bug be fixed, or have I completely missed something?)

Thanks!

#2

Actually I just realised that this evaluation metric is implemented in base xgboost… but I think this is a bug still.

#3

@natudjurus The data matrix should be the second argument in the metric function. For example:

#4

ah yes of course, sorry that was a typo when I was transcribing the code. I think the error still occurs.

#5

If you have a reproducible example, please post it on GitHub.