K-fold cross validation feature importance

I’m using the python API and the xgb.cv() function for k-fold cross validation. Would like to measure the feature importance in each fold separately to determine how consistent the importances are across folds. How can I determine the feature importances? I don’t see the importance values exposed by the API, but maybe I missed it somewhere.

Boosters have the methods get_score and get_fscore which provide feature importances