![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1182060%2F4cc45db867d9ec1b983c353c7f23b633%2Fshap.PNG?generation=1563462787163185&alt=media\")
Hi everyone I have made a kernel using SHAP model explainability techniques, that help to provide insights into what our models might be learning, check out Diabetic retinopathy - SHAP model explainability
\n\nHere is a sample image:\n
In the first image, we can see that on the first image the model gave high importance on areas that actually don't have anything, this may be a sign that the model is evaluating if the image has a full size or not, and this behavior may lead to overfitting.
", "messages": [ { "id": "579176", "postDate": "07/18/2019 15:18:47", "content": "Hi everyone I have made a kernel using SHAP model explainability techniques, that help to provide insights into what our models might be learning, check out Diabetic retinopathy - SHAP model explainability
\n\nHere is a sample image:\n
In the first image, we can see that on the first image the model gave high importance on areas that actually don't have anything, this may be a sign that the model is evaluating if the image has a full size or not, and this behavior may lead to overfitting.
", "rawMarkdown": "Hi everyone I have made a kernel using SHAP model explainability techniques, that help to provide insights into what our models might be learning, check out [Diabetic retinopathy - SHAP model explainability](https://www.kaggle.com/dimitreoliveira/diabetic-retinopathy-shap-model-explainability)\n\nHere is a sample image:\n\n\nIn the first image, we can see that on the first image the model gave high importance on areas that actually don't have anything, this may be a sign that the model is evaluating if the image has a full size or not, and this behavior may lead to overfitting.", "votes": null }, { "id": "580534", "postDate": "07/20/2019 10:07:01", "content": "This is great! Thank you for sharing.
", "rawMarkdown": "This is great! Thank you for sharing.", "votes": null }, { "id": "580590", "postDate": "07/20/2019 12:31:22", "content": "It's interesting that it is focusing on the black areas. If you look at img_size vs. label in the training data you will see that some image sizes are nearly always a particular label. The image size is probably related to the amount of black we see, hence why during training your model focuses on the black area outside the eye.
\n\nI think we need to be careful not to let this artefact of the training data impact our models :)
", "rawMarkdown": "It's interesting that it is focusing on the black areas. If you look at img_size vs. label in the training data you will see that some image sizes are nearly always a particular label. The image size is probably related to the amount of black we see, hence why during training your model focuses on the black area outside the eye. \n\nI think we need to be careful not to let this artefact of the training data impact our models :)", "votes": null }, { "id": "580604", "postDate": "07/20/2019 12:53:21", "content": "Yes @taindow , that's a very valid point, this is probably an indicator that croping the extra black area may be a good option.
", "rawMarkdown": "Yes @taindow , that's a very valid point, this is probably an indicator that croping the extra black area may be a good option.", "votes": null }, { "id": "2617801", "postDate": "01/24/2024 13:12:35", "content": "It's very interesting. Thank you for sharing.
", "rawMarkdown": "It's very interesting. Thank you for sharing.", "votes": null }, { "id": "3267984", "postDate": "08/12/2025 08:21:18", "content": "my shap is not working with tensorflow 2.8 and showing final error as a must be 1 dimensional or integer
", "rawMarkdown": "my shap is not working with tensorflow 2.8 and showing final error as a must be 1 dimensional or integer", "votes": null } ], "comments": [ { "id": 2617801, "author_name": "naoyafujishiro", "author_url": "", "post_date": "01/24/2024 13:12:35", "content": "It's very interesting. Thank you for sharing.
", "votes": null, "replies": [] }, { "id": 3267984, "author_name": "hamza202404", "author_url": "", "post_date": "08/12/2025 08:21:18", "content": "my shap is not working with tensorflow 2.8 and showing final error as a must be 1 dimensional or integer
", "votes": null, "replies": [] }, { "id": 580534, "author_name": "nwickman", "author_url": "", "post_date": "07/20/2019 10:07:01", "content": "This is great! Thank you for sharing.
", "votes": null, "replies": [] }, { "id": 580590, "author_name": "taindow", "author_url": "", "post_date": "07/20/2019 12:31:22", "content": "It's interesting that it is focusing on the black areas. If you look at img_size vs. label in the training data you will see that some image sizes are nearly always a particular label. The image size is probably related to the amount of black we see, hence why during training your model focuses on the black area outside the eye.
\n\nI think we need to be careful not to let this artefact of the training data impact our models :)
", "votes": null, "replies": [ { "id": 580604, "author_name": "dimitreoliveira", "author_url": "", "post_date": "07/20/2019 12:53:21", "content": "Yes @taindow , that's a very valid point, this is probably an indicator that croping the extra black area may be a good option.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "579176": "Hi everyone I have made a kernel using SHAP model explainability techniques, that help to provide insights into what our models might be learning, check out [Diabetic retinopathy - SHAP model explainability](https://www.kaggle.com/dimitreoliveira/diabetic-retinopathy-shap-model-explainability)\n\nHere is a sample image:\n\n\nIn the first image, we can see that on the first image the model gave high importance on areas that actually don't have anything, this may be a sign that the model is evaluating if the image has a full size or not, and this behavior may lead to overfitting.", "580534": "This is great! Thank you for sharing.", "580590": "It's interesting that it is focusing on the black areas. If you look at img_size vs. label in the training data you will see that some image sizes are nearly always a particular label. The image size is probably related to the amount of black we see, hence why during training your model focuses on the black area outside the eye. \n\nI think we need to be careful not to let this artefact of the training data impact our models :)", "580604": "Yes @taindow , that's a very valid point, this is probably an indicator that croping the extra black area may be a good option.", "2617801": "It's very interesting. Thank you for sharing.", "3267984": "my shap is not working with tensorflow 2.8 and showing final error as a must be 1 dimensional or integer" }, "source": "meta" }