Interaction Constraints not working as expected


#1

Hi,

I am trying to use interaction constraints in my exercise. I have give interaction_constraints="[[10,11],[0,1,2,3,4,5,6,7,8,9,10]]" as parameter to my model. Now based on this documentation (https://xgboost.readthedocs.io/en/latest/tutorials/feature_interaction_constraint.html),if there is a split on x11 then next split can only have either x10 or x11 as mentioned below.

0:[f11<23] yes=1,no=2,missing=2
** 1:[f10<1] yes=3,no=4,missing=3**

My question is now that the next split after f10 can split on any other features as well right? But i am not observing this trend. I have tried many iterations. And everytime, I am getting the whole tree developed on f10 and f11 without using any other variables.


#2

No, that’s not the case. Once you have f11 and f10 in first two splits, the subsequent splits are not allowed to choose any other feature.


#3

Hi, below is a snippet from the official documentation. Isn’t this contradicting your statement?

“”"
For one last example, we use [[0, 1], [1, 3, 4]] and choose feature 0 as split for root node. At the second layer of built tree, 1 is the only legitimate split candidate except for 0 itself, since they belong to the same constraint set. Following the grow path of our example tree below, the node at second layer splits at feature 1. But due to the fact that 1 also belongs to second constraint set [1, 3, 4], at third layer, we need to include all features as candidates to comply with its ascendants.
“”"


#4

Sorry, I was wrong. I didn’t see that 10 is in both constraint sets.

Just because features 0-9 are allowed do not mean that they have to be chosen. What is your max_depth? Can you increase it?


#5

Hi my max_depth is 7 and i am doing 50 boosting iterations. Should i increase both?


#6

Try increasing the number of boosting rounds.