![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1182060%2Fdcb42698bd2332c6522b9397c9f8383a%2FScreenshot%20from%202020-07-01%2022-39-18.png?generation=1593653982446044&alt=media\")
Hey everyone, machine learning explainability has been a hot topic for some time and this is even more important in the medical field, that end users really need to understand why a model made a specific prediction. One really cool lib that does a good job at this is SHAP, it also works for a lot of models.
\n\nTo demonstrate this I have just created a kernel that uses SHAP to explain the prediction of a model on data from this competition, this may also be a great tool to visualize and debug your model's prediction and search for possible improvements, to illustrate this let me show an example:
\n\n
If you look at this image the model is paying some attention to the mm scale on the left top of the image (because there are some pink dots there). I may be able to improve this model by removing that scale or maybe adding it to as data augmentation to other images.
And here is one that was correctly classified:
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1182060%2F1431826cb3d18c3f0ebcb1cec00d937b%2FScreenshot%20from%202020-07-01%2023-11-09.png?generation=1593655918403401&alt=media\")
The model does not seem to bother about the purple mark on the left. Anyway if you want to check out more examples take a look at the kernel!
\n\n\n", "messages": [ { "id": 911728, "postDate": "2020-07-02T01:42:40.743Z", "content": "ps: another example of SHAP used with deep learning but for Diabetic retinopathy
\n
Hey everyone, machine learning explainability has been a hot topic for some time and this is even more important in the medical field, that end users really need to understand why a model made a specific prediction. One really cool lib that does a good job at this is SHAP, it also works for a lot of models.
\n\nTo demonstrate this I have just created a kernel that uses SHAP to explain the prediction of a model on data from this competition, this may also be a great tool to visualize and debug your model's prediction and search for possible improvements, to illustrate this let me show an example:
\n\n
If you look at this image the model is paying some attention to the mm scale on the left top of the image (because there are some pink dots there). I may be able to improve this model by removing that scale or maybe adding it to as data augmentation to other images.
And here is one that was correctly classified:
\n\n
The model does not seem to bother about the purple mark on the left. Anyway if you want to check out more examples take a look at the kernel!
\n\n\n", "rawMarkdown": "Hey everyone, machine learning explainability has been a hot topic for some time and this is even more important in the medical field, that end users really need to understand why a model made a specific prediction. One really cool lib that does a good job at this is [SHAP](https://github.com/slundberg/shap), it also works for a lot of models.\n\nTo demonstrate this I have just created [a kernel](https://www.kaggle.com/dimitreoliveira/melanoma-classification-shap-model-explained) that uses SHAP to explain the prediction of a model on data from this competition, this may also be a great tool to visualize and debug your model's prediction and search for possible improvements, to illustrate this let me show an example:\n\n\n\nIf you look at this image the model is paying some attention to the `mm scale` on the left top of the image (because there are some pink dots there). I may be able to improve this model by removing that scale or maybe adding it to as data augmentation to other images. \n\nAnd here is one that was correctly classified:\n\n\n\nThe model does not seem to bother about the purple mark on the left. Anyway if you want to check out more examples take a look at the kernel!\n\n> ps: another example of SHAP used with deep learning but for [Diabetic retinopathy](https://www.kaggle.com/dimitreoliveira/diabetic-retinopathy-shap-model-explainability)", "votes": 29 }, { "id": 950175, "postDate": "2020-07-29T08:24:55.193Z", "content": "ps: another example of SHAP used with deep learning but for Diabetic retinopathy
\n
Thanks for this. \nBTW, dealing with the measurement leakage won't necessarily improve your model performance in this competition. It would help in the real world though :)
", "rawMarkdown": "Thanks for this. \nBTW, dealing with the measurement leakage won't necessarily improve your model performance in this competition. It would help in the real world though :)", "votes": 1, "replies": [ { "id": 950504, "postDate": "2020-07-29T12:42:47.093Z", "content": "You're welcome @danofer ,\nI agree with you, but being aware of things like measurement leakage may be important in many cases, especially with heavily unbalanced datasets, if the models see it very often on benign images it may associate it with the class, it may also have the possibility of it being associated more often with specific data (like 2018 samples), I don't know if it is the case here, but it is good to be aware.
", "rawMarkdown": "You're welcome @danofer ,\nI agree with you, but being aware of things like measurement leakage may be important in many cases, especially with heavily unbalanced datasets, if the models see it very often on benign images it may associate it with the class, it may also have the possibility of it being associated more often with specific data (like 2018 samples), I don't know if it is the case here, but it is good to be aware." } ] }, { "id": 948433, "postDate": "2020-07-27T23:51:53.760Z", "content": "Thank you, this is great work
", "rawMarkdown": "Thank you, this is great work", "votes": 1, "replies": [ { "id": 948446, "postDate": "2020-07-28T00:45:20.273Z", "content": "You are welcome @jacekpoplawski
", "rawMarkdown": "You are welcome @jacekpoplawski " } ] }, { "id": 946896, "postDate": "2020-07-26T23:40:46.827Z", "content": "@dimitreoliveira thanks for sharing.
", "rawMarkdown": "@dimitreoliveira thanks for sharing. ", "votes": 1, "replies": [ { "id": 946953, "postDate": "2020-07-27T01:13:38.483Z", "content": "You're welcome @sheriytm
", "rawMarkdown": "You're welcome @sheriytm " } ] }, { "id": 912008, "postDate": "2020-07-02T06:46:52.363Z", "content": "@dimitreoliveira An excellent idea for a future competition where prizes will be given based on a combination of accuracy & explainability!!
\n\nIn my office work, where we have regression and tree models on tabular data, we use SHAP values and column importance to try to explain predictions.
\n\nAs Chris mentioned above, I am also surprised that you could use SHAP with deep learning. In case of image data, we probably need to first identify superpixels in the image. Please check older ISIC competitions, where you may find Segmentation masks along with training images, and that may improve the results of your kernel.
", "rawMarkdown": "@dimitreoliveira An excellent idea for a **future competition** where prizes will be given based on a combination of accuracy & explainability!!\n\nIn my office work, where we have regression and tree models on tabular data, we use SHAP values and column importance to *try* to explain predictions.\n\nAs Chris mentioned above, I am also surprised that you could use SHAP with deep learning. In case of image data, we probably need to first identify superpixels in the image. Please check older ISIC competitions, where you may find **Segmentation** masks along with training images, and that may improve the results of your kernel.", "votes": 1, "replies": [ { "id": 912313, "postDate": "2020-07-02T11:51:36.687Z", "content": "@sirishks that sound like a good idea, but I think would be hard to evaluate the quality or level of explainability for the solutions, it may be very manual or subjective. But I can say that this kind of approach can definitely help tunning your model and to find possible biases.
\n\nYeah, using SHAP for SKlearn models is even easier and helps a lot. At my current company I used SHAP on a LSTM model once, and it helped me to find a bug at a model that was already in production.
", "rawMarkdown": "@sirishks that sound like a good idea, but I think would be hard to evaluate the quality or level of explainability for the solutions, it may be very manual or subjective. But I can say that this kind of approach can definitely help tunning your model and to find possible biases.\n\nYeah, using SHAP for SKlearn models is even easier and helps a lot. At my current company I used SHAP on a LSTM model once, and it helped me to find a bug at a model that was already in production.", "votes": 1 } ] }, { "id": 911926, "postDate": "2020-07-02T05:33:36.193Z", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!", "votes": 1, "replies": [ { "id": 912297, "postDate": "2020-07-02T11:46:06.470Z", "content": "You`re welcome @mariapushkareva !
", "rawMarkdown": "You`re welcome @mariapushkareva !", "votes": 1 } ] }, { "id": 951402, "postDate": "2020-07-30T06:02:06.787Z", "content": "FYI - Webinar today about leading edge explainable modeling methods such as Explainable Neural Networks (XNN) and GA2M, 30-July at 6pm GMT (12 hrs from now):\nModel Explainability with H2O Driverless AI 1.9
", "rawMarkdown": "FYI - Webinar today about leading edge **explainable modeling** methods such as Explainable Neural Networks (XNN) and GA2M, 30-July at 6pm GMT (12 hrs from now):\n[**Model Explainability with H2O Driverless AI 1.9**](https://www.brighttalk.com/webcast/16463/429733/further-exploration-into-model-explainability-with-h2o-driverless-ai-1-9)", "votes": 2, "replies": [ { "id": 951877, "postDate": "2020-07-30T13:32:25.817Z", "content": "Thanks for the head-up @sirishks !
", "rawMarkdown": "Thanks for the head-up @sirishks !", "votes": 1 } ] }, { "id": 946861, "postDate": "2020-07-26T22:44:05.743Z", "content": "awesome, thanks for sharing, I tried Grad-cam and find that mm scale seems to have some contribution to predict malignant on benign samples.
", "rawMarkdown": "awesome, thanks for sharing, I tried Grad-cam and find that mm scale seems to have some contribution to predict malignant on benign samples. ", "votes": 2, "replies": [ { "id": 946930, "postDate": "2020-07-27T00:44:54.920Z", "content": "Wow, that's very interesting.
", "rawMarkdown": "Wow, that's very interesting. " }, { "id": 946952, "postDate": "2020-07-27T01:13:02.913Z", "content": "Indeed @zhanglic , that is interesting, it may be because of the data augmentation you use, what is the CV/LB score of this model?
", "rawMarkdown": "Indeed @zhanglic , that is interesting, it may be because of the data augmentation you use, what is the CV/LB score of this model?" }, { "id": 949900, "postDate": "2020-07-29T03:56:10.990Z", "content": "maybe, I am also wondering how many malignant pictures have the mm scale, the model I tried the grad-cam was getting CV0.8873LB0.9107, not the best yet.
", "rawMarkdown": "maybe, I am also wondering how many malignant pictures have the mm scale, the model I tried the grad-cam was getting CV0.8873LB0.9107, not the best yet. ", "votes": 1 }, { "id": 951698, "postDate": "2020-07-30T10:17:48.447Z", "content": "I had the same experience. When I analyzed false positives/negatives in my OOF using the clustering approach shared by @cdeotte, I found out that many false positives of my model (LB .945) are due to the scale.\nAdding scales just as we add hairs could help improve the score, though I haven't tried it yet.
", "rawMarkdown": "I had the same experience. When I analyzed false positives/negatives in my OOF using the clustering approach shared by @cdeotte, I found out that many false positives of my model (LB .945) are due to the scale.\nAdding scales just as we add hairs could help improve the score, though I haven't tried it yet.", "votes": 1 }, { "id": 952154, "postDate": "2020-07-30T16:56:56.757Z", "content": "Thanks @zhanglic and @andrejrybr , those are valuable insights, I have not tested on my best models yet, but maybe it could be an option for data augmentation indeed.
", "rawMarkdown": "Thanks @zhanglic and @andrejrybr , those are valuable insights, I have not tested on my best models yet, but maybe it could be an option for data augmentation indeed." } ] }, { "id": 911777, "postDate": "2020-07-02T02:49:17.910Z", "content": "Awesome. I didn't realize you could use SHAP with deep learning. Thanks for sharing.
", "rawMarkdown": "Awesome. I didn't realize you could use SHAP with deep learning. Thanks for sharing.", "votes": 2, "replies": [ { "id": 912294, "postDate": "2020-07-02T11:45:53.510Z", "content": "Yeah they have a very friendly API, for deep learning, you can use both GradientExplainer or DeepExplainer the latter still have some problems with Tensorflow2
Outstanding!!!! Great!
\nIs any chance you can look into my notebook and give feedback: https://www.kaggle.com/remekkinas/tps5-is-about-sparsity-shap-extensive
Thanks @epocxy , yeah that is similar to activation maps or Grad-CAM, you can also take a look at LIME if you are interested.
", "rawMarkdown": "Thanks @epocxy , yeah that is similar to activation maps or Grad-CAM, you can also take a look at LIME if you are interested." } ] }, { "id": 949156, "postDate": "2020-07-28T12:58:43.930Z", "content": "Good one, thanks for sharing!๐
", "rawMarkdown": "Good one, thanks for sharing!๐ ", "votes": 1 } ], "comments": [ { "id": 950175, "author_name": "Dan Ofer", "author_url": "", "post_date": "2020-07-29T08:24:55.193000", "content": "Thanks for this. \nBTW, dealing with the measurement leakage won't necessarily improve your model performance in this competition. It would help in the real world though :)
", "votes": 1, "replies": [ { "id": 950504, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-29T12:42:47.093000", "content": "You're welcome @danofer ,\nI agree with you, but being aware of things like measurement leakage may be important in many cases, especially with heavily unbalanced datasets, if the models see it very often on benign images it may associate it with the class, it may also have the possibility of it being associated more often with specific data (like 2018 samples), I don't know if it is the case here, but it is good to be aware.
", "votes": 0, "replies": [] } ] }, { "id": 948433, "author_name": "Jacek Poplawski", "author_url": "", "post_date": "2020-07-27T23:51:53.760000", "content": "Thank you, this is great work
", "votes": 1, "replies": [ { "id": 948446, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-28T00:45:20.273000", "content": "You are welcome @jacekpoplawski
", "votes": 0, "replies": [] } ] }, { "id": 946896, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2020-07-26T23:40:46.827000", "content": "@dimitreoliveira thanks for sharing.
", "votes": 1, "replies": [ { "id": 946953, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-27T01:13:38.483000", "content": "You're welcome @sheriytm
", "votes": 0, "replies": [] } ] }, { "id": 912008, "author_name": "Sirish Somanchi", "author_url": "", "post_date": "2020-07-02T06:46:52.363000", "content": "@dimitreoliveira An excellent idea for a future competition where prizes will be given based on a combination of accuracy & explainability!!
\n\nIn my office work, where we have regression and tree models on tabular data, we use SHAP values and column importance to try to explain predictions.
\n\nAs Chris mentioned above, I am also surprised that you could use SHAP with deep learning. In case of image data, we probably need to first identify superpixels in the image. Please check older ISIC competitions, where you may find Segmentation masks along with training images, and that may improve the results of your kernel.
", "votes": 1, "replies": [ { "id": 912313, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-02T11:51:36.687000", "content": "@sirishks that sound like a good idea, but I think would be hard to evaluate the quality or level of explainability for the solutions, it may be very manual or subjective. But I can say that this kind of approach can definitely help tunning your model and to find possible biases.
\n\nYeah, using SHAP for SKlearn models is even easier and helps a lot. At my current company I used SHAP on a LSTM model once, and it helped me to find a bug at a model that was already in production.
", "votes": 1, "replies": [] } ] }, { "id": 911926, "author_name": "Marie", "author_url": "", "post_date": "2020-07-02T05:33:36.193000", "content": "Thanks for sharing!
", "votes": 1, "replies": [ { "id": 912297, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-02T11:46:06.470000", "content": "You`re welcome @mariapushkareva !
", "votes": 1, "replies": [] } ] }, { "id": 951402, "author_name": "Sirish Somanchi", "author_url": "", "post_date": "2020-07-30T06:02:06.787000", "content": "FYI - Webinar today about leading edge explainable modeling methods such as Explainable Neural Networks (XNN) and GA2M, 30-July at 6pm GMT (12 hrs from now):\nModel Explainability with H2O Driverless AI 1.9
", "votes": 2, "replies": [ { "id": 951877, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-30T13:32:25.817000", "content": "Thanks for the head-up @sirishks !
", "votes": 1, "replies": [] } ] }, { "id": 946861, "author_name": "Licheng Zhang", "author_url": "", "post_date": "2020-07-26T22:44:05.743000", "content": "awesome, thanks for sharing, I tried Grad-cam and find that mm scale seems to have some contribution to predict malignant on benign samples.
", "votes": 2, "replies": [ { "id": 946930, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-07-27T00:44:54.920000", "content": "Wow, that's very interesting.
", "votes": 0, "replies": [] }, { "id": 946952, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-27T01:13:02.913000", "content": "Indeed @zhanglic , that is interesting, it may be because of the data augmentation you use, what is the CV/LB score of this model?
", "votes": 0, "replies": [] }, { "id": 949900, "author_name": "Licheng Zhang", "author_url": "", "post_date": "2020-07-29T03:56:10.990000", "content": "maybe, I am also wondering how many malignant pictures have the mm scale, the model I tried the grad-cam was getting CV0.8873LB0.9107, not the best yet.
", "votes": 1, "replies": [] }, { "id": 951698, "author_name": "Andrej Rybรกr", "author_url": "", "post_date": "2020-07-30T10:17:48.447000", "content": "I had the same experience. When I analyzed false positives/negatives in my OOF using the clustering approach shared by @cdeotte, I found out that many false positives of my model (LB .945) are due to the scale.\nAdding scales just as we add hairs could help improve the score, though I haven't tried it yet.
", "votes": 1, "replies": [] }, { "id": 952154, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-30T16:56:56.757000", "content": "Thanks @zhanglic and @andrejrybr , those are valuable insights, I have not tested on my best models yet, but maybe it could be an option for data augmentation indeed.
", "votes": 0, "replies": [] } ] }, { "id": 911777, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-07-02T02:49:17.910000", "content": "Awesome. I didn't realize you could use SHAP with deep learning. Thanks for sharing.
", "votes": 2, "replies": [ { "id": 912294, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-07-02T11:45:53.510000", "content": "Yeah they have a very friendly API, for deep learning, you can use both GradientExplainer or DeepExplainer the latter still have some problems with Tensorflow2
Outstanding!!!! Great!
\nIs any chance you can look into my notebook and give feedback: https://www.kaggle.com/remekkinas/tps5-is-about-sparsity-shap-extensive
Thanks @epocxy , yeah that is similar to activation maps or Grad-CAM, you can also take a look at LIME if you are interested.
", "votes": 0, "replies": [] } ] }, { "id": 949156, "author_name": "Manish Kumar", "author_url": "", "post_date": "2020-07-28T12:58:43.930000", "content": "Good one, thanks for sharing!๐
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "911728": "Hey everyone, machine learning explainability has been a hot topic for some time and this is even more important in the medical field, that end users really need to understand why a model made a specific prediction. One really cool lib that does a good job at this is [SHAP](https://github.com/slundberg/shap), it also works for a lot of models.\n\nTo demonstrate this I have just created [a kernel](https://www.kaggle.com/dimitreoliveira/melanoma-classification-shap-model-explained) that uses SHAP to explain the prediction of a model on data from this competition, this may also be a great tool to visualize and debug your model's prediction and search for possible improvements, to illustrate this let me show an example:\n\n\n\nIf you look at this image the model is paying some attention to the `mm scale` on the left top of the image (because there are some pink dots there). I may be able to improve this model by removing that scale or maybe adding it to as data augmentation to other images. \n\nAnd here is one that was correctly classified:\n\n\n\nThe model does not seem to bother about the purple mark on the left. Anyway if you want to check out more examples take a look at the kernel!\n\n> ps: another example of SHAP used with deep learning but for [Diabetic retinopathy](https://www.kaggle.com/dimitreoliveira/diabetic-retinopathy-shap-model-explainability)", "950175": "Thanks for this. \nBTW, dealing with the measurement leakage won't necessarily improve your model performance in this competition. It would help in the real world though :)", "948433": "Thank you, this is great work", "946896": "@dimitreoliveira thanks for sharing. ", "912008": "@dimitreoliveira An excellent idea for a **future competition** where prizes will be given based on a combination of accuracy & explainability!!\n\nIn my office work, where we have regression and tree models on tabular data, we use SHAP values and column importance to *try* to explain predictions.\n\nAs Chris mentioned above, I am also surprised that you could use SHAP with deep learning. In case of image data, we probably need to first identify superpixels in the image. Please check older ISIC competitions, where you may find **Segmentation** masks along with training images, and that may improve the results of your kernel.", "911926": "Thanks for sharing!", "951402": "FYI - Webinar today about leading edge **explainable modeling** methods such as Explainable Neural Networks (XNN) and GA2M, 30-July at 6pm GMT (12 hrs from now):\n[**Model Explainability with H2O Driverless AI 1.9**](https://www.brighttalk.com/webcast/16463/429733/further-exploration-into-model-explainability-with-h2o-driverless-ai-1-9)", "946861": "awesome, thanks for sharing, I tried Grad-cam and find that mm scale seems to have some contribution to predict malignant on benign samples. ", "911777": "Awesome. I didn't realize you could use SHAP with deep learning. Thanks for sharing.", "1313599": "Outstanding!!!! Great!\nIs any chance you can look into my notebook and give feedback: https://www.kaggle.com/remekkinas/tps5-is-about-sparsity-shap-extensive", "915212": "", "949156": "Good one, thanks for sharing!๐ " } }