Hello all –
There’s seems to be a trade-off between the maximum depth of the trees (max_depth) and the number of iterations (nrounds). Specifically, as one goes up the other tends to go down. In my analysis (70,000 rows, about 1000 predictors), I am attempting to use a grid search to find the optimal value for these parameters (as well as a few others). However, the cross validation scores suggest that the optimal model is obtained when max_depth exceeds 100, with a modest nrounds values (~200).
It seems to be like the opposite (i.e. high nrounds and lower max_depth) makes more sense for the ability to generalize the results to other databases, but it feel weird to just go with this feeling arbitrarily. Am I missing something? Is there a better criteria (other than CV that is) for setting max_depth?