![\"split-gain\"](\"https://i.imgur.com/yS4eQXF.png\")
Why do we evaluate feature importances? Because we want to select the good features and drop the bad ones. But which of the following three importances is the right one for this purpose?
\nIn this post I'll compare the three methods for feature selection and show that the last one is the right one.
\nThe former two methods are often seen in public gradient-boosting notebooks because people like to decorate their notebooks with a nice diagram, all the more as a single line of code suffices to generate the diagram. But the diagram alone doesn't give a better model! We need to interpret the importance values.
\nSplit feature importance counts how often a feature is used in the model. The model I'm using for this experiment has 1200 trees, and split feature importances are between 0 and 1082. If a feature has importance = 0, this means that the feature is not used in the model at all. If we omit the feature, model quality doesn't change, but the training takes less time. If split feature importance is high, we know that the feature is used often in the model and that it improves the model's training score.
\nGain feature importance is similar to split feature importance. It measures the gain which was achieved in all splits based on the feature, although the definition of this gain is hard to find in the documentation. The features of my model have importances between 0 and 3000000. Again, if a feature has importance = 0, this means that the feature is not used in the model at all.
\nThe following diagram shows that split and gain feature importance correlate quite well. The diagram is based on importance ranks, which are between 0 and 573 (my model has 574 features) - higher is better. The single dot at coordinates (30,30) represents 60 features which are not used in the model and have both importances equal to zero.
\n
Split and gain importance are cheap byproducts of the training process, but they have a massive shortcoming: They are based on the training data rather than the validation data and consequently cannot measure generalization. Would you ever evaluate a model by its training score? Of course not! Every child learns that models must be scored on a validation dataset so that we can evaluate its generalization capability. The same principle holds for feature importance.
\nPermutation feature importance is a model inspection technique that can be used for any fitted estimator when the data is tabular. The permutation feature importance is defined as the decrease in a model's validation score when a single feature value is randomly shuffled. In contrast to split and gain importance, permutation feature importance can be positive or negative:
\nIf a feature is complete noise, a decision tree can still use it to improve the training score, and the feature will get positive split/gain importances. Only permutation feature importance detects that such a feature doesn't generalize to the validation dataset.
\nThe following diagrams show how permutation feature importance compares to split and gain importance:
\n![\"split-gain-pfi\"](\"https://i.imgur.com/CBWXsbu.png\")
Conclusion: Although permutation feature importance is slower than the gradient boosters' built-in importance measures, it is much more useful because it measures generalization and distinguishes positive from negative importances. Features with negative importance should be dropped from the model.
\nSee here for a demo of the same effect on the Titanic dataset.
", "messages": [ { "id": 1821813, "postDate": "2022-06-15T21:12:23.507Z", "content": "Why do we evaluate feature importances? Because we want to select the good features and drop the bad ones. But which of the following three importances is the right one for this purpose?
\nIn this post I'll compare the three methods for feature selection and show that the last one is the right one.
\nThe former two methods are often seen in public gradient-boosting notebooks because people like to decorate their notebooks with a nice diagram, all the more as a single line of code suffices to generate the diagram. But the diagram alone doesn't give a better model! We need to interpret the importance values.
\nSplit feature importance counts how often a feature is used in the model. The model I'm using for this experiment has 1200 trees, and split feature importances are between 0 and 1082. If a feature has importance = 0, this means that the feature is not used in the model at all. If we omit the feature, model quality doesn't change, but the training takes less time. If split feature importance is high, we know that the feature is used often in the model and that it improves the model's training score.
\nGain feature importance is similar to split feature importance. It measures the gain which was achieved in all splits based on the feature, although the definition of this gain is hard to find in the documentation. The features of my model have importances between 0 and 3000000. Again, if a feature has importance = 0, this means that the feature is not used in the model at all.
\nThe following diagram shows that split and gain feature importance correlate quite well. The diagram is based on importance ranks, which are between 0 and 573 (my model has 574 features) - higher is better. The single dot at coordinates (30,30) represents 60 features which are not used in the model and have both importances equal to zero.
\n
Split and gain importance are cheap byproducts of the training process, but they have a massive shortcoming: They are based on the training data rather than the validation data and consequently cannot measure generalization. Would you ever evaluate a model by its training score? Of course not! Every child learns that models must be scored on a validation dataset so that we can evaluate its generalization capability. The same principle holds for feature importance.
\nPermutation feature importance is a model inspection technique that can be used for any fitted estimator when the data is tabular. The permutation feature importance is defined as the decrease in a model's validation score when a single feature value is randomly shuffled. In contrast to split and gain importance, permutation feature importance can be positive or negative:
\nIf a feature is complete noise, a decision tree can still use it to improve the training score, and the feature will get positive split/gain importances. Only permutation feature importance detects that such a feature doesn't generalize to the validation dataset.
\nThe following diagrams show how permutation feature importance compares to split and gain importance:
\n
Conclusion: Although permutation feature importance is slower than the gradient boosters' built-in importance measures, it is much more useful because it measures generalization and distinguishes positive from negative importances. Features with negative importance should be dropped from the model.
\nSee here for a demo of the same effect on the Titanic dataset.
", "rawMarkdown": "Why do we evaluate feature importances? Because we want to select the good features and drop the bad ones. But which of the following three importances is the right one for this purpose?\n\n1. Split feature importance?\n2. Gain feature importance?\n3. Permutation feature importance?\n\nIn this post I'll compare the three methods for feature selection and show that the last one is the right one.\n\nThe former two methods are often seen in public gradient-boosting notebooks because people like to decorate their notebooks with a nice diagram, all the more as a single line of code suffices to generate the diagram. But the diagram alone doesn't give a better model! We need to interpret the importance values.\n\n**Split feature importance** counts how often a feature is used in the model. The model I'm using for this experiment has 1200 trees, and split feature importances are between 0 and 1082. If a feature has importance = 0, this means that the feature is not used in the model at all. If we omit the feature, model quality doesn't change, but the training takes less time. If split feature importance is high, we know that the feature is used often in the model and that it improves the model's training score. \n\n**Gain feature importance** is similar to split feature importance. It measures the gain which was achieved in all splits based on the feature, although the definition of this gain is hard to find in the documentation. The features of my model have importances between 0 and 3000000. Again, if a feature has importance = 0, this means that the feature is not used in the model at all.\n\nThe following diagram shows that split and gain feature importance correlate quite well. The diagram is based on importance ranks, which are between 0 and 573 (my model has 574 features) - higher is better. The single dot at coordinates (30,30) represents 60 features which are not used in the model and have both importances equal to zero.\n\n\n\nSplit and gain importance are cheap byproducts of the training process, but they have a massive shortcoming: They are based on the training data rather than the validation data and consequently cannot measure generalization. Would you ever evaluate a model by its training score? Of course not! Every child learns that models must be scored on a validation dataset so that we can evaluate its generalization capability. The same principle holds for feature importance.\n\n**Permutation feature importance** is a model inspection technique that can be used for any fitted estimator when the data is tabular. The permutation feature importance is defined as the decrease in a model's validation score when a single feature value is randomly shuffled. In contrast to split and gain importance, permutation feature importance can be positive or negative:\n- Positive permutation feature importance means that the model profits from the feature and deteriorates if the feature is shuffled or removed.\n- Zero permutation feature importance means that the model doesn't change if we shuffle or remove the feature.\n- Negative permutation feature importance means that the model improves if the feature is removed.\n\nIf a feature is complete noise, a decision tree can still use it to improve the training score, and the feature will get positive split/gain importances. Only permutation feature importance detects that such a feature doesn't generalize to the validation dataset.\n\nThe following diagrams show how permutation feature importance compares to split and gain importance:\n- The green dots represent features with positive permutation feature importance. These are the good features.\n- The black dot at the left represents the 60 features which are not used in the model.\n- The red dots represent the features with negative permutation feature importance. These are the bad features although they may have high split or gain importance. D_43_min and S_26_min (represented by the two rightmost red dots) belong to the top 40 features according to split importance, but the model gets better if we drop them.\n\n\n\n**Conclusion:** Although permutation feature importance is slower than the gradient boosters' built-in importance measures, it is much more useful because it measures generalization and distinguishes positive from negative importances. Features with negative importance should be dropped from the model.\n\nSee [here](https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html) for a demo of the same effect on the Titanic dataset.", "votes": 209 }, { "id": 1822040, "postDate": "2022-06-16T03:14:20.233Z", "content": "I think you are very right, and your conclusion is very helpful. I have been using permutation feature importance for a while and I think it's a great tool.
\nI would like to add one thing though: when you shuffle a feature you dont take in cosideration the interaction it has with other features that might still pose a problem for feature selection.
\nFor example, if you have two columns that are the same: You can permute them all day long your model will always use one of them and the score won't change.
@yiwangnz see it working. Besides being compute-intensive I see the task in figuring out for which threshold to group two features. This dataset has a high dimensionality after the regular feature engineering done by top scoring notebooks. It would not be easy to group two features from the same category without removing important correlations.
", "rawMarkdown": "@yiwangnz see it working. Besides being compute-intensive I see the task in figuring out for which threshold to group two features. This dataset has a high dimensionality after the regular feature engineering done by top scoring notebooks. It would not be easy to group two features from the same category without removing important correlations. " }, { "id": 1863307, "postDate": "2022-07-20T09:36:03.200Z", "content": "i think if score don't change means the feature doesn't work and we can drop one. When we evaluate the other one we can get the real feature importance, is that a problem?
", "rawMarkdown": "i think if score don't change means the feature doesn't work and we can drop one. When we evaluate the other one we can get the real feature importance, is that a problem?", "votes": 1 } ] }, { "id": 1822914, "postDate": "2022-06-16T20:17:58.190Z", "content": "Contributing to the discussion, I think lofo (https://github.com/aerdem4/lofo-importance) is even a more robust approach for measuring feature importance, but it's computationally expensive as it needs to train a model for each feature in the dataset:
\n\n\nLOFO first evaluates the performance of the model with all the input features included, then iteratively removes one feature at a time, retrains the model, and evaluates its performance on a validation set.
\n
In the same package there also a fast-lofo (https://github.com/aerdem4/lofo-importance#flofo-importance) method, that is basically a supercharged permutation importance, that deals with the correlated feature overestimation problem.
", "rawMarkdown": "Contributing to the discussion, I think **lofo** (https://github.com/aerdem4/lofo-importance) is even a more robust approach for measuring feature importance, but it's computationally expensive as it needs to train a model for each feature in the dataset:\n> LOFO first evaluates the performance of the model with all the input features included, then iteratively removes one feature at a time, retrains the model, and evaluates its performance on a validation set.\n\nIn the same package there also a **fast-lofo** (https://github.com/aerdem4/lofo-importance#flofo-importance) method, that is basically a supercharged permutation importance, that deals with the correlated feature overestimation problem.\n", "votes": 9, "replies": [ { "id": 1844185, "postDate": "2022-07-05T11:59:46.507Z", "content": "If you have an estimator which provides information about feature importance you can basically use recursive feature elimination from sklearn instead:
\nhttps://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.RFE.html
Thanks @ambrosm for sharing! I implemented it and it helps a lot!
\nI would just annotate the following for others working with tree based algorithms. In tree based algorithms the order of your features matters a lot for your model! So if you give [A, B, C] to xgboost you will get a different result compared to [B, A, C].
\nI noticed that my permutation importance changes a lot depending on the feature order!
\nTherefore, if you are looking to find the \"general permutation importance\" of your features, independent of your model, fold or feature order, you might consider to perform several permutation importance valuations over different folds with different feature orders and average them. That helped me a lot! :)
", "rawMarkdown": "Thanks @ambrosm for sharing! I implemented it and it helps a lot!\n\nI would just annotate the following for others working with tree based algorithms. In tree based algorithms the order of your features matters a lot for your model! So if you give [A, B, C] to xgboost you will get a different result compared to [B, A, C]. \n\nI noticed that my permutation importance changes a lot depending on the feature order! \n\nTherefore, if you are looking to find the \"general permutation importance\" of your features, independent of your model, fold or feature order, you might consider to perform several permutation importance valuations over different folds with different feature orders and average them. That helped me a lot! :)", "votes": 5 }, { "id": 1880390, "postDate": "2022-08-01T17:08:30.933Z", "content": "Nice and interesting post, here is my 2cts:
\nI just think you need to be careful when saying :
\n\n\nWould you ever evaluate a model by its training score? Of course not! Every child learns that models must be scored on a validation dataset so that we can evaluate its generalization capability. The same principle holds for feature importance.
\n
In fact it is very debatable that a model could/should be explained by a validation set.
\nHere you are looking at feature importance only to remove some and improve your global CV score. But that's only one specific use case.
\nFeature importance and more broadly 'explainability' of a model are often used to explain what the model is doing and give trust to people relying on it.
\nWhen trying to explain a model, I think it makes sense to look at what it learnt during training and where the boundaries have been drawn, this only comes from the training data.
In the end, I think it's not black and white because methods based on validation set will have other problems like data shifts, sampling representativity etc…
\nCheers!
", "rawMarkdown": "Nice and interesting post, here is my 2cts:\n\nI just think you need to be careful when saying :\n> Would you ever evaluate a model by its training score? Of course not! Every child learns that models must be scored on a validation dataset so that we can evaluate its generalization capability. The same principle holds for feature importance.\n\nIn fact it is very debatable that a model could/should be explained by a validation set.\n\nHere you are looking at feature importance only to remove some and improve your global CV score. But that's only one specific use case.\n\nFeature importance and more broadly 'explainability' of a model are often used to explain what the model is doing and give trust to people relying on it.\nWhen trying to explain a model, I think it makes sense to look at what it learnt during training and where the boundaries have been drawn, this only comes from the training data.\n\nIn the end, I think it's not black and white because methods based on validation set will have other problems like data shifts, sampling representativity etc... \n\nCheers!", "votes": 6, "replies": [ { "id": 1880465, "postDate": "2022-08-01T18:37:29.360Z", "content": "Good point, @optimo!
", "rawMarkdown": "Good point, @optimo!", "votes": 2 }, { "id": 1908886, "postDate": "2022-08-22T05:53:19.813Z", "content": "Dear @optimo
\n\n\n\"look at what it learnt during training and where the boundaries have been drawn\"
\n
That is very relevant when it comes to tree based estimators. Such estimators cannot extrapolate (for example see the short notebook \"Extrapolation: Do not stray out of the forest!\"). This causes particular problems for permutation importance when they have been trained with correlated features; shuffling one of these features will create data-point pairs that are now either outside of the region that the estimator was trained on, or on the wrong side of a decision boundary, and the results for these points will become unreliable. There is an interesting (although somewhat heavy reading) paper on the subject: \"Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model or There Is No Free Variable Importance\".
\nAll the best,
\ncarl
Thank you @ambrosm nice demonstration !
\nThe only small limitations are a compatibility with scikit-learn API required for the model and the scoring that can not use (it seems) a custom metric but the scikit learn metrics. Amex scoring is so specific…
\nI noticed that you can get a gain when removing the worse score feature but if you do it again with the next one you may get a bad result. I guess the permutation can not take into account the cumulative result of 2 features deletion.
Hi @pourchot
\n\n\nI noticed that you can get a gain when removing the worse score feature but if you do it again with the next one…
\n
I did permutation and got a list of values corresponding to each feature.
\nNow, when we remove feature corresponding to worst score, we get a gain, If we now remove the Second Worse
\n feature from the list we may get a bad result.
\nDid you mean Second Worse when you tell next one or something else[like run permutation again and remove the worst].
Thanks,
", "rawMarkdown": "Hi @pourchot \n> I noticed that you can get a gain when removing the worse score feature but if you do it again with **the next one**...\n\nI did permutation and got a list of values corresponding to each feature. \nNow, when we remove feature corresponding to worst score, we get a gain, If we now remove the **Second Worse** \n feature from the list we may get a bad result.\nDid you mean **Second Worse** when you tell **next one** or something else[like run permutation again and remove the worst]. \n\nThanks," } ] }, { "id": 1826588, "postDate": "2022-06-20T13:19:52.497Z", "content": "hi @ambrosm, this is really great. From my own personal experience, the Permutation Feature Importance was great with text data where we wanted to identify the feature with right level of grams - usually the bigrams works better and the unigrams works the worst - Bigrams > Trigrams > Unigrams :)
", "rawMarkdown": "hi @ambrosm, this is really great. From my own personal experience, the Permutation Feature Importance was great with text data where we wanted to identify the feature with right level of grams - usually the bigrams works better and the unigrams works the worst - Bigrams > Trigrams > Unigrams :)\n", "votes": 4 }, { "id": 1992217, "postDate": "2022-10-17T15:25:22.097Z", "content": "Greetings, AmbrosM!
\nThanks for sharing your knoledge!
\nAnswer my questions, please.
Can we get a data leak (or something like data leak) if first we make permutation feature importance calculation on X,Y and then select some features and make model.fit(X[features],Y)?
Can we make model.fit(X,Y) and permutation_importance(model, X, Y) on full data?
\nOr we should use train_test_split and then model.fit(Xtrain,Ytrain), permutation_importance(model, Xtest, Ytest)?
Yes, it can be seen as something like a data leak: After you train on X_train and use X_val to calculate permutation feature importance and select features, the model will contain some leaked information about X_val. The situation is comparable to training on X_train and using X_val in Optuna to tune hyperparameters.
\nA possible solution is:
\nsklearn.inspection.permutation_importance on X_val1 and select featuresThank you so much!
\nI can't overestimate your sharing experience!
Thanks for posting this! I had no idea that features being shuffled had an impact on the model accuracy. This is a great explainer post and really thoughtfully put together.
", "rawMarkdown": "Thanks for posting this! I had no idea that features being shuffled had an impact on the model accuracy. This is a great explainer post and really thoughtfully put together.", "votes": 1 }, { "id": 1861467, "postDate": "2022-07-19T03:44:00.640Z", "content": "@ambrosm Thanks for sharing. About the feature selection, how about using the boruta method before conducting the xgb training?
", "rawMarkdown": " @ambrosm Thanks for sharing. About the feature selection, how about using the boruta method before conducting the xgb training? ", "votes": 1, "replies": [ { "id": 1861631, "postDate": "2022-07-19T06:18:33.513Z", "content": "Hi @alexyoyo I don't know - I have no experience with Boruta.
", "rawMarkdown": "Hi @alexyoyo I don't know - I have no experience with Boruta.", "votes": 2 } ] }, { "id": 1851459, "postDate": "2022-07-11T08:55:25.977Z", "content": "Thanks a lot, super usefull information!
", "rawMarkdown": "Thanks a lot, super usefull information!", "votes": 1 }, { "id": 1827137, "postDate": "2022-06-20T21:27:18.627Z", "content": "Looks great. very helpful
", "rawMarkdown": "Looks great. very helpful", "votes": 1 }, { "id": 1826142, "postDate": "2022-06-20T04:45:21.090Z", "content": "This is super helpful. Great post!
", "rawMarkdown": "This is super helpful. Great post!", "votes": 1 }, { "id": 1824173, "postDate": "2022-06-18T03:56:47.733Z", "content": "thx for sharing those insights!
", "rawMarkdown": "thx for sharing those insights!", "votes": 1 }, { "id": 1823801, "postDate": "2022-06-17T16:53:39.880Z", "content": "Very much insight and good summery as always @ambrosm
", "rawMarkdown": "Very much insight and good summery as always @ambrosm ", "votes": 1 }, { "id": 1823540, "postDate": "2022-06-17T13:21:12.887Z", "content": "Nice summary. Autoencoders are also a nice way to \"select\" (generate) important features out of the existing onces. Especially denosing autoencoders can be useful in this competition.
", "rawMarkdown": "Nice summary. Autoencoders are also a nice way to \"select\" (generate) important features out of the existing onces. Especially denosing autoencoders can be useful in this competition.", "votes": 1 }, { "id": 1822413, "postDate": "2022-06-16T10:55:27.583Z", "content": "your posts always bring so many insights. Thank you @ambrosm
", "rawMarkdown": "your posts always bring so many insights. Thank you @ambrosm ", "votes": 1 }, { "id": 1821844, "postDate": "2022-06-15T21:24:37.580Z", "content": "I always learn a lot with your posts, thanks @ambrosm
", "rawMarkdown": "I always learn a lot with your posts, thanks @ambrosm ", "votes": 1 }, { "id": 1927452, "postDate": "2022-09-05T16:21:21.790Z", "content": "Hey @ambrosm. Great post! Thank you for sharing the knowledge. I have been using Mutual Information (MI) score to select important features since it assess each feature on both linear and non-linear linkage with output variable. It is computationally expensive but are there any other shortcomings in it? Please let me know.
", "rawMarkdown": "Hey @ambrosm. Great post! Thank you for sharing the knowledge. I have been using Mutual Information (MI) score to select important features since it assess each feature on both linear and non-linear linkage with output variable. It is computationally expensive but are there any other shortcomings in it? Please let me know." }, { "id": 1892533, "postDate": "2022-08-10T06:17:26.210Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 1841937, "postDate": "2022-07-03T15:03:02.543Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 1825622, "postDate": "2022-06-19T14:59:04.757Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1825148, "postDate": "2022-06-19T02:28:50.487Z", "rawMarkdown": "", "votes": 2, "isDeleted": true, "replies": [ { "id": 1825175, "postDate": "2022-06-19T04:03:56.890Z", "content": "Hi @yiwangnz It seems that the gradient-boosting libraries provide these metrics only on the training set. So theoretically, you could measure these metrics on the validation set, but you would have to program them yourself. And this still wouldn't remedy the issue that they are always nonnegative.
", "rawMarkdown": "Hi @yiwangnz It seems that the gradient-boosting libraries provide these metrics only on the training set. So theoretically, you could measure these metrics on the validation set, but you would have to program them yourself. And this still wouldn't remedy the issue that they are always nonnegative.", "votes": 5 } ] }, { "id": 1877726, "postDate": "2022-07-31T01:08:51.050Z", "content": "Very helpful. Thanks for sharing👍
", "rawMarkdown": "Very helpful. Thanks for sharing👍", "votes": 1 }, { "id": 1877294, "postDate": "2022-07-30T14:02:47.213Z", "content": "Thanks for sharing this idea!
", "rawMarkdown": "Thanks for sharing this idea!", "votes": 1 }, { "id": 1842878, "postDate": "2022-07-04T11:13:12.867Z", "content": "Solid advice. Thank you!
", "rawMarkdown": "Solid advice. Thank you!", "votes": 1 }, { "id": 1842686, "postDate": "2022-07-04T07:54:21.453Z", "content": "Thanks. Very helpful !
", "rawMarkdown": "Thanks. Very helpful !", "votes": 1 }, { "id": 1842442, "postDate": "2022-07-04T02:27:28.627Z", "content": "good thanks for sharing
", "rawMarkdown": "good thanks for sharing", "votes": 1 }, { "id": 1837750, "postDate": "2022-06-29T21:24:32.750Z", "content": "Excellent post, thanks for sharing!
", "rawMarkdown": "Excellent post, thanks for sharing!", "votes": 1 }, { "id": 1831244, "postDate": "2022-06-24T04:35:21.857Z", "content": "great job!Thanks
", "rawMarkdown": "great job!Thanks", "votes": 1 }, { "id": 1829134, "postDate": "2022-06-22T10:47:53.717Z", "content": "thanks, sharing.
", "rawMarkdown": "thanks, sharing.", "votes": 1 }, { "id": 1825638, "postDate": "2022-06-19T15:23:31.213Z", "content": "Thanks for sharing.
", "rawMarkdown": "Thanks for sharing.", "votes": 1 }, { "id": 1824897, "postDate": "2022-06-18T17:53:40.110Z", "content": "Thanks for the head-up.
", "rawMarkdown": "Thanks for the head-up. ", "votes": 1 }, { "id": 1823636, "postDate": "2022-06-17T14:40:31.270Z", "content": "Thanks for sharing @ambrosm
", "rawMarkdown": "Thanks for sharing @ambrosm ", "votes": 1 }, { "id": 1822395, "postDate": "2022-06-16T10:32:36.960Z", "content": "Thanks for sharing the insights.
", "rawMarkdown": "Thanks for sharing the insights.", "votes": 1 }, { "id": 1909895, "postDate": "2022-08-23T03:24:27.937Z", "content": "Thanks for sharing! It helps a lot!
", "rawMarkdown": "Thanks for sharing! It helps a lot!", "votes": 2 }, { "id": 1821867, "postDate": "2022-06-15T21:35:04.473Z", "content": "Thanks for sharing @ambrosm
", "rawMarkdown": "Thanks for sharing @ambrosm ", "votes": 2 }, { "id": 1899624, "postDate": "2022-08-15T11:57:36.300Z", "content": "Very helpful. Thanks
", "rawMarkdown": "Very helpful. Thanks" } ], "comments": [ { "id": 1822040, "author_name": "The Devastator", "author_url": "", "post_date": "2022-06-16T03:14:20.233000", "content": "I think you are very right, and your conclusion is very helpful. I have been using permutation feature importance for a while and I think it's a great tool.
\nI would like to add one thing though: when you shuffle a feature you dont take in cosideration the interaction it has with other features that might still pose a problem for feature selection.
\nFor example, if you have two columns that are the same: You can permute them all day long your model will always use one of them and the score won't change.
@yiwangnz see it working. Besides being compute-intensive I see the task in figuring out for which threshold to group two features. This dataset has a high dimensionality after the regular feature engineering done by top scoring notebooks. It would not be easy to group two features from the same category without removing important correlations.
", "votes": 0, "replies": [] }, { "id": 1863307, "author_name": "Dou Fan", "author_url": "", "post_date": "2022-07-20T09:36:03.200000", "content": "i think if score don't change means the feature doesn't work and we can drop one. When we evaluate the other one we can get the real feature importance, is that a problem?
", "votes": 1, "replies": [] } ] }, { "id": 1822914, "author_name": "mavillan", "author_url": "", "post_date": "2022-06-16T20:17:58.190000", "content": "Contributing to the discussion, I think lofo (https://github.com/aerdem4/lofo-importance) is even a more robust approach for measuring feature importance, but it's computationally expensive as it needs to train a model for each feature in the dataset:
\n\n\nLOFO first evaluates the performance of the model with all the input features included, then iteratively removes one feature at a time, retrains the model, and evaluates its performance on a validation set.
\n
In the same package there also a fast-lofo (https://github.com/aerdem4/lofo-importance#flofo-importance) method, that is basically a supercharged permutation importance, that deals with the correlated feature overestimation problem.
", "votes": 9, "replies": [ { "id": 1844185, "author_name": "Ali Abdin", "author_url": "", "post_date": "2022-07-05T11:59:46.507000", "content": "If you have an estimator which provides information about feature importance you can basically use recursive feature elimination from sklearn instead:
\nhttps://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.RFE.html
Thanks @ambrosm for sharing! I implemented it and it helps a lot!
\nI would just annotate the following for others working with tree based algorithms. In tree based algorithms the order of your features matters a lot for your model! So if you give [A, B, C] to xgboost you will get a different result compared to [B, A, C].
\nI noticed that my permutation importance changes a lot depending on the feature order!
\nTherefore, if you are looking to find the \"general permutation importance\" of your features, independent of your model, fold or feature order, you might consider to perform several permutation importance valuations over different folds with different feature orders and average them. That helped me a lot! :)
", "votes": 5, "replies": [] }, { "id": 1880390, "author_name": "Optimo", "author_url": "", "post_date": "2022-08-01T17:08:30.933000", "content": "Nice and interesting post, here is my 2cts:
\nI just think you need to be careful when saying :
\n\n\nWould you ever evaluate a model by its training score? Of course not! Every child learns that models must be scored on a validation dataset so that we can evaluate its generalization capability. The same principle holds for feature importance.
\n
In fact it is very debatable that a model could/should be explained by a validation set.
\nHere you are looking at feature importance only to remove some and improve your global CV score. But that's only one specific use case.
\nFeature importance and more broadly 'explainability' of a model are often used to explain what the model is doing and give trust to people relying on it.
\nWhen trying to explain a model, I think it makes sense to look at what it learnt during training and where the boundaries have been drawn, this only comes from the training data.
In the end, I think it's not black and white because methods based on validation set will have other problems like data shifts, sampling representativity etc…
\nCheers!
", "votes": 6, "replies": [ { "id": 1880465, "author_name": "AmbrosM", "author_url": "", "post_date": "2022-08-01T18:37:29.360000", "content": "Good point, @optimo!
", "votes": 2, "replies": [] }, { "id": 1908886, "author_name": "Carl McBride Ellis", "author_url": "", "post_date": "2022-08-22T05:53:19.813000", "content": "Dear @optimo
\n\n\n\"look at what it learnt during training and where the boundaries have been drawn\"
\n
That is very relevant when it comes to tree based estimators. Such estimators cannot extrapolate (for example see the short notebook \"Extrapolation: Do not stray out of the forest!\"). This causes particular problems for permutation importance when they have been trained with correlated features; shuffling one of these features will create data-point pairs that are now either outside of the region that the estimator was trained on, or on the wrong side of a decision boundary, and the results for these points will become unreliable. There is an interesting (although somewhat heavy reading) paper on the subject: \"Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model or There Is No Free Variable Importance\".
\nAll the best,
\ncarl
Thank you @ambrosm nice demonstration !
\nThe only small limitations are a compatibility with scikit-learn API required for the model and the scoring that can not use (it seems) a custom metric but the scikit learn metrics. Amex scoring is so specific…
\nI noticed that you can get a gain when removing the worse score feature but if you do it again with the next one you may get a bad result. I guess the permutation can not take into account the cumulative result of 2 features deletion.
Hi @pourchot
\n\n\nI noticed that you can get a gain when removing the worse score feature but if you do it again with the next one…
\n
I did permutation and got a list of values corresponding to each feature.
\nNow, when we remove feature corresponding to worst score, we get a gain, If we now remove the Second Worse
\n feature from the list we may get a bad result.
\nDid you mean Second Worse when you tell next one or something else[like run permutation again and remove the worst].
Thanks,
", "votes": 0, "replies": [] } ] }, { "id": 1826588, "author_name": "MuralidharanM", "author_url": "", "post_date": "2022-06-20T13:19:52.497000", "content": "hi @ambrosm, this is really great. From my own personal experience, the Permutation Feature Importance was great with text data where we wanted to identify the feature with right level of grams - usually the bigrams works better and the unigrams works the worst - Bigrams > Trigrams > Unigrams :)
", "votes": 4, "replies": [] }, { "id": 1992217, "author_name": "Oleg Khudyakov", "author_url": "", "post_date": "2022-10-17T15:25:22.097000", "content": "Greetings, AmbrosM!
\nThanks for sharing your knoledge!
\nAnswer my questions, please.
Can we get a data leak (or something like data leak) if first we make permutation feature importance calculation on X,Y and then select some features and make model.fit(X[features],Y)?
Can we make model.fit(X,Y) and permutation_importance(model, X, Y) on full data?
\nOr we should use train_test_split and then model.fit(Xtrain,Ytrain), permutation_importance(model, Xtest, Ytest)?
Yes, it can be seen as something like a data leak: After you train on X_train and use X_val to calculate permutation feature importance and select features, the model will contain some leaked information about X_val. The situation is comparable to training on X_train and using X_val in Optuna to tune hyperparameters.
\nA possible solution is:
\nsklearn.inspection.permutation_importance on X_val1 and select featuresThank you so much!
\nI can't overestimate your sharing experience!
Thanks for posting this! I had no idea that features being shuffled had an impact on the model accuracy. This is a great explainer post and really thoughtfully put together.
", "votes": 1, "replies": [] }, { "id": 1861467, "author_name": "Alexyoyo", "author_url": "", "post_date": "2022-07-19T03:44:00.640000", "content": "@ambrosm Thanks for sharing. About the feature selection, how about using the boruta method before conducting the xgb training?
", "votes": 1, "replies": [ { "id": 1861631, "author_name": "AmbrosM", "author_url": "", "post_date": "2022-07-19T06:18:33.513000", "content": "Hi @alexyoyo I don't know - I have no experience with Boruta.
", "votes": 2, "replies": [] } ] }, { "id": 1851459, "author_name": "Oleg Khudyakov", "author_url": "", "post_date": "2022-07-11T08:55:25.977000", "content": "Thanks a lot, super usefull information!
", "votes": 1, "replies": [] }, { "id": 1827137, "author_name": "Muatasm Abdelsalam", "author_url": "", "post_date": "2022-06-20T21:27:18.627000", "content": "Looks great. very helpful
", "votes": 1, "replies": [] }, { "id": 1826142, "author_name": "Eric Inman", "author_url": "", "post_date": "2022-06-20T04:45:21.090000", "content": "This is super helpful. Great post!
", "votes": 1, "replies": [] }, { "id": 1824173, "author_name": "qilin chen", "author_url": "", "post_date": "2022-06-18T03:56:47.733000", "content": "thx for sharing those insights!
", "votes": 1, "replies": [] }, { "id": 1823801, "author_name": "Gaju Ahmed", "author_url": "", "post_date": "2022-06-17T16:53:39.880000", "content": "Very much insight and good summery as always @ambrosm
", "votes": 1, "replies": [] }, { "id": 1823540, "author_name": "Ali Abdin", "author_url": "", "post_date": "2022-06-17T13:21:12.887000", "content": "Nice summary. Autoencoders are also a nice way to \"select\" (generate) important features out of the existing onces. Especially denosing autoencoders can be useful in this competition.
", "votes": 1, "replies": [] }, { "id": 1822413, "author_name": "naiborhujosua", "author_url": "", "post_date": "2022-06-16T10:55:27.583000", "content": "your posts always bring so many insights. Thank you @ambrosm
", "votes": 1, "replies": [] }, { "id": 1821844, "author_name": "mavillan", "author_url": "", "post_date": "2022-06-15T21:24:37.580000", "content": "I always learn a lot with your posts, thanks @ambrosm
", "votes": 1, "replies": [] }, { "id": 1927452, "author_name": "Nishant Jairath", "author_url": "", "post_date": "2022-09-05T16:21:21.790000", "content": "Hey @ambrosm. Great post! Thank you for sharing the knowledge. I have been using Mutual Information (MI) score to select important features since it assess each feature on both linear and non-linear linkage with output variable. It is computationally expensive but are there any other shortcomings in it? Please let me know.
", "votes": 0, "replies": [] }, { "id": 1892533, "author_name": "", "author_url": "", "post_date": "2022-08-10T06:17:26.210000", "content": "", "votes": 1, "replies": [] }, { "id": 1841937, "author_name": "", "author_url": "", "post_date": "2022-07-03T15:03:02.543000", "content": "", "votes": 1, "replies": [] }, { "id": 1825622, "author_name": "", "author_url": "", "post_date": "2022-06-19T14:59:04.757000", "content": "", "votes": 0, "replies": [] }, { "id": 1825148, "author_name": "", "author_url": "", "post_date": "2022-06-19T02:28:50.487000", "content": "", "votes": 2, "replies": [ { "id": 1825175, "author_name": "AmbrosM", "author_url": "", "post_date": "2022-06-19T04:03:56.890000", "content": "Hi @yiwangnz It seems that the gradient-boosting libraries provide these metrics only on the training set. So theoretically, you could measure these metrics on the validation set, but you would have to program them yourself. And this still wouldn't remedy the issue that they are always nonnegative.
", "votes": 5, "replies": [] } ] }, { "id": 1877726, "author_name": "YASUKENN@Meiji_IMS_ND", "author_url": "", "post_date": "2022-07-31T01:08:51.050000", "content": "Very helpful. Thanks for sharing👍
", "votes": 1, "replies": [] }, { "id": 1877294, "author_name": "ChuanhaoLi2022", "author_url": "", "post_date": "2022-07-30T14:02:47.213000", "content": "Thanks for sharing this idea!
", "votes": 1, "replies": [] }, { "id": 1842878, "author_name": "Sohail Ahmed", "author_url": "", "post_date": "2022-07-04T11:13:12.867000", "content": "Solid advice. Thank you!
", "votes": 1, "replies": [] }, { "id": 1842686, "author_name": "andy jennings", "author_url": "", "post_date": "2022-07-04T07:54:21.453000", "content": "Thanks. Very helpful !
", "votes": 1, "replies": [] }, { "id": 1842442, "author_name": "hgn1102", "author_url": "", "post_date": "2022-07-04T02:27:28.627000", "content": "good thanks for sharing
", "votes": 1, "replies": [] }, { "id": 1837750, "author_name": "Adam Wurdits", "author_url": "", "post_date": "2022-06-29T21:24:32.750000", "content": "Excellent post, thanks for sharing!
", "votes": 1, "replies": [] }, { "id": 1831244, "author_name": "Kyrieshaohua", "author_url": "", "post_date": "2022-06-24T04:35:21.857000", "content": "great job!Thanks
", "votes": 1, "replies": [] }, { "id": 1829134, "author_name": "", "author_url": "", "post_date": "2022-06-22T10:47:53.717000", "content": "", "votes": 1, "replies": [] }, { "id": 1825638, "author_name": "", "author_url": "", "post_date": "2022-06-19T15:23:31.213000", "content": "", "votes": 1, "replies": [] }, { "id": 1824897, "author_name": "", "author_url": "", "post_date": "2022-06-18T17:53:40.110000", "content": "", "votes": 1, "replies": [] }, { "id": 1823636, "author_name": "", "author_url": "", "post_date": "2022-06-17T14:40:31.270000", "content": "", "votes": 1, "replies": [] }, { "id": 1822395, "author_name": "", "author_url": "", "post_date": "2022-06-16T10:32:36.960000", "content": "", "votes": 1, "replies": [] }, { "id": 1909895, "author_name": "", "author_url": "", "post_date": "2022-08-23T03:24:27.937000", "content": "", "votes": 2, "replies": [] }, { "id": 1821867, "author_name": "", "author_url": "", "post_date": "2022-06-15T21:35:04.473000", "content": "", "votes": 2, "replies": [] }, { "id": 1899624, "author_name": "", "author_url": "", "post_date": "2022-08-15T11:57:36.300000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1821813": "Why do we evaluate feature importances? Because we want to select the good features and drop the bad ones. But which of the following three importances is the right one for this purpose?\n\n1. Split feature importance?\n2. Gain feature importance?\n3. Permutation feature importance?\n\nIn this post I'll compare the three methods for feature selection and show that the last one is the right one.\n\nThe former two methods are often seen in public gradient-boosting notebooks because people like to decorate their notebooks with a nice diagram, all the more as a single line of code suffices to generate the diagram. But the diagram alone doesn't give a better model! We need to interpret the importance values.\n\n**Split feature importance** counts how often a feature is used in the model. The model I'm using for this experiment has 1200 trees, and split feature importances are between 0 and 1082. If a feature has importance = 0, this means that the feature is not used in the model at all. If we omit the feature, model quality doesn't change, but the training takes less time. If split feature importance is high, we know that the feature is used often in the model and that it improves the model's training score. \n\n**Gain feature importance** is similar to split feature importance. It measures the gain which was achieved in all splits based on the feature, although the definition of this gain is hard to find in the documentation. The features of my model have importances between 0 and 3000000. Again, if a feature has importance = 0, this means that the feature is not used in the model at all.\n\nThe following diagram shows that split and gain feature importance correlate quite well. The diagram is based on importance ranks, which are between 0 and 573 (my model has 574 features) - higher is better. The single dot at coordinates (30,30) represents 60 features which are not used in the model and have both importances equal to zero.\n\n\n\nSplit and gain importance are cheap byproducts of the training process, but they have a massive shortcoming: They are based on the training data rather than the validation data and consequently cannot measure generalization. Would you ever evaluate a model by its training score? Of course not! Every child learns that models must be scored on a validation dataset so that we can evaluate its generalization capability. The same principle holds for feature importance.\n\n**Permutation feature importance** is a model inspection technique that can be used for any fitted estimator when the data is tabular. The permutation feature importance is defined as the decrease in a model's validation score when a single feature value is randomly shuffled. In contrast to split and gain importance, permutation feature importance can be positive or negative:\n- Positive permutation feature importance means that the model profits from the feature and deteriorates if the feature is shuffled or removed.\n- Zero permutation feature importance means that the model doesn't change if we shuffle or remove the feature.\n- Negative permutation feature importance means that the model improves if the feature is removed.\n\nIf a feature is complete noise, a decision tree can still use it to improve the training score, and the feature will get positive split/gain importances. Only permutation feature importance detects that such a feature doesn't generalize to the validation dataset.\n\nThe following diagrams show how permutation feature importance compares to split and gain importance:\n- The green dots represent features with positive permutation feature importance. These are the good features.\n- The black dot at the left represents the 60 features which are not used in the model.\n- The red dots represent the features with negative permutation feature importance. These are the bad features although they may have high split or gain importance. D_43_min and S_26_min (represented by the two rightmost red dots) belong to the top 40 features according to split importance, but the model gets better if we drop them.\n\n\n\n**Conclusion:** Although permutation feature importance is slower than the gradient boosters' built-in importance measures, it is much more useful because it measures generalization and distinguishes positive from negative importances. Features with negative importance should be dropped from the model.\n\nSee [here](https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html) for a demo of the same effect on the Titanic dataset.", "1822040": "I think you are very right, and your conclusion is very helpful. I have been using permutation feature importance for a while and I think it's a great tool.\n\nI would like to add one thing though: when you shuffle a feature you dont take in cosideration the interaction it has with other features that might still pose a problem for feature selection. \nFor example, if you have two columns that are the same: You can permute them all day long your model will always use one of them and the score won't change. \n", "1822914": "Contributing to the discussion, I think **lofo** (https://github.com/aerdem4/lofo-importance) is even a more robust approach for measuring feature importance, but it's computationally expensive as it needs to train a model for each feature in the dataset:\n> LOFO first evaluates the performance of the model with all the input features included, then iteratively removes one feature at a time, retrains the model, and evaluates its performance on a validation set.\n\nIn the same package there also a **fast-lofo** (https://github.com/aerdem4/lofo-importance#flofo-importance) method, that is basically a supercharged permutation importance, that deals with the correlated feature overestimation problem.\n", "1885124": "Thanks @ambrosm for sharing! I implemented it and it helps a lot!\n\nI would just annotate the following for others working with tree based algorithms. In tree based algorithms the order of your features matters a lot for your model! So if you give [A, B, C] to xgboost you will get a different result compared to [B, A, C]. \n\nI noticed that my permutation importance changes a lot depending on the feature order! \n\nTherefore, if you are looking to find the \"general permutation importance\" of your features, independent of your model, fold or feature order, you might consider to perform several permutation importance valuations over different folds with different feature orders and average them. That helped me a lot! :)", "1880390": "Nice and interesting post, here is my 2cts:\n\nI just think you need to be careful when saying :\n> Would you ever evaluate a model by its training score? Of course not! Every child learns that models must be scored on a validation dataset so that we can evaluate its generalization capability. The same principle holds for feature importance.\n\nIn fact it is very debatable that a model could/should be explained by a validation set.\n\nHere you are looking at feature importance only to remove some and improve your global CV score. But that's only one specific use case.\n\nFeature importance and more broadly 'explainability' of a model are often used to explain what the model is doing and give trust to people relying on it.\nWhen trying to explain a model, I think it makes sense to look at what it learnt during training and where the boundaries have been drawn, this only comes from the training data.\n\nIn the end, I think it's not black and white because methods based on validation set will have other problems like data shifts, sampling representativity etc... \n\nCheers!", "1822610": "Thank you @ambrosm nice demonstration ! \nThe only small limitations are a compatibility with scikit-learn API required for the model and the scoring that can not use (it seems) a custom metric but the scikit learn metrics. Amex scoring is so specific...\nI noticed that you can get a gain when removing the worse score feature but if you do it again with the next one you may get a bad result. I guess the permutation can not take into account the cumulative result of 2 features deletion.", "1826588": "hi @ambrosm, this is really great. From my own personal experience, the Permutation Feature Importance was great with text data where we wanted to identify the feature with right level of grams - usually the bigrams works better and the unigrams works the worst - Bigrams > Trigrams > Unigrams :)\n", "1992217": "Greetings, AmbrosM!\n\nThanks for sharing your knoledge!\nAnswer my questions, please.\n\n1. Can we get a data leak (or something like data leak) if first we make permutation feature importance calculation on X,Y and then select some features and make model.fit(X[features],Y)?\n\n2. Can we make model.fit(X,Y) and permutation_importance(model, X, Y) on full data?\nOr we should use train_test_split and then model.fit(Xtrain,Ytrain), permutation_importance(model, Xtest, Ytest)?", "1873352": "Thanks for posting this! I had no idea that features being shuffled had an impact on the model accuracy. This is a great explainer post and really thoughtfully put together.", "1861467": " @ambrosm Thanks for sharing. About the feature selection, how about using the boruta method before conducting the xgb training? ", "1851459": "Thanks a lot, super usefull information!", "1827137": "Looks great. very helpful", "1826142": "This is super helpful. Great post!", "1824173": "thx for sharing those insights!", "1823801": "Very much insight and good summery as always @ambrosm ", "1823540": "Nice summary. Autoencoders are also a nice way to \"select\" (generate) important features out of the existing onces. Especially denosing autoencoders can be useful in this competition.", "1822413": "your posts always bring so many insights. Thank you @ambrosm ", "1821844": "I always learn a lot with your posts, thanks @ambrosm ", "1927452": "Hey @ambrosm. Great post! Thank you for sharing the knowledge. I have been using Mutual Information (MI) score to select important features since it assess each feature on both linear and non-linear linkage with output variable. It is computationally expensive but are there any other shortcomings in it? Please let me know.", "1892533": "", "1841937": "", "1825622": "", "1825148": "", "1877726": "Very helpful. Thanks for sharing👍", "1877294": "Thanks for sharing this idea!", "1842878": "Solid advice. Thank you!", "1842686": "Thanks. Very helpful !", "1842442": "good thanks for sharing", "1837750": "Excellent post, thanks for sharing!", "1831244": "great job!Thanks", "1829134": "thanks, sharing.", "1825638": "Thanks for sharing.", "1824897": "Thanks for the head-up. ", "1823636": "Thanks for sharing @ambrosm ", "1822395": "Thanks for sharing the insights.", "1909895": "Thanks for sharing! It helps a lot!", "1821867": "Thanks for sharing @ambrosm ", "1899624": "Very helpful. Thanks" } }