![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2605845%2F27c83e4d62b459e413c61386f766fd51%2FClass_activation_maps.png?generation=1581373475447334&alt=media\")
Hi!
\n\nEDIT: Added notebook with code
\n\nI've been inspired by the work of @pnussbaum on his reading the robot's mind notebook to look a bit more into how we can tackle some more error analysis methods, and getting a better understanding of where some of the errors might be occurring and where.
\n\nA while back, @cdeotte made a Class Activation Map notebook on how to make masks out of class activation maps for a unsupervised task. Ever since I saw that I wanted to try to implement it. So I did, I made it for a simple EfficientNetB0, and here is what it looks like:
\n\n
Those aren't great, because they are based on (3,3) filters that are up-scaled to the 75x75 size of the images here. @cdeotte was using an Xception model which had 7x7 filters on the last Conv2D layer. EfficientNet seems to have 3x3, from what I could tell and what I used.
\n\nAnyways, it's still some interesting info!
\n\nIf anybody is interested I'll clean the code up and post it, I'm sure there could be some interesting info to find with this! These things are so interesting once you start digging into them. EDIT: Voilà
", "messages": [ { "id": "741702", "postDate": "02/10/2020 22:28:14", "content": "Hi!
\n\nEDIT: Added notebook with code
\n\nI've been inspired by the work of @pnussbaum on his reading the robot's mind notebook to look a bit more into how we can tackle some more error analysis methods, and getting a better understanding of where some of the errors might be occurring and where.
\n\nA while back, @cdeotte made a Class Activation Map notebook on how to make masks out of class activation maps for a unsupervised task. Ever since I saw that I wanted to try to implement it. So I did, I made it for a simple EfficientNetB0, and here is what it looks like:
\n\n
Those aren't great, because they are based on (3,3) filters that are up-scaled to the 75x75 size of the images here. @cdeotte was using an Xception model which had 7x7 filters on the last Conv2D layer. EfficientNet seems to have 3x3, from what I could tell and what I used.
\n\nAnyways, it's still some interesting info!
\n\nIf anybody is interested I'll clean the code up and post it, I'm sure there could be some interesting info to find with this! These things are so interesting once you start digging into them. EDIT: Voilà
", "rawMarkdown": "Hi!\n\nEDIT: [Added notebook with code](https://www.kaggle.com/maxlenormand/multi-class-activation-map-with-efficientnetb0?scriptVersionId=28497228)\n\nI've been inspired by the work of @pnussbaum on his [reading the robot's mind notebook](https://www.kaggle.com/pnussbaum/grapheme-mind-reader-panv12-nogpu) to look a bit more into how we can tackle some more error analysis methods, and getting a better understanding of where some of the errors might be occurring and where.\n\nA while back, @cdeotte made a [Class Activation Map notebook](https://www.kaggle.com/cdeotte/unsupervised-masks-cv-0-60) on how to make masks out of class activation maps for a unsupervised task. Ever since I saw that I wanted to try to implement it. So I did, I made it for a simple EfficientNetB0, and here is what it looks like:\n\n\n\nThose aren't great, because they are based on (3,3) filters that are up-scaled to the 75x75 size of the images here. @cdeotte was using an Xception model which had 7x7 filters on the last Conv2D layer. EfficientNet seems to have 3x3, from what I could tell and what I used.\n\nAnyways, it's still some interesting info!\n\nIf anybody is interested I'll clean the code up and post it, I'm sure there could be some interesting info to find with this! These things are so interesting once you start digging into them. EDIT: [Voilà](https://www.kaggle.com/maxlenormand/multi-class-activation-map-with-efficientnetb0?scriptVersionId=28497228)", "votes": null }, { "id": "742180", "postDate": "02/11/2020 06:05:48", "content": "Thanks @maxlenormand . I really appreciate your work .
", "rawMarkdown": "Thanks @maxlenormand . I really appreciate your work .", "votes": null }, { "id": "742240", "postDate": "02/11/2020 07:16:53", "content": "Thanks!
", "rawMarkdown": "Thanks!", "votes": null }, { "id": "743856", "postDate": "02/12/2020 10:28:01", "content": "I'm sure this will give me insightful results when I am able to replicate this on my model.\nDo share the code once it's cleaned up. @maxlenormand
", "rawMarkdown": "I'm sure this will give me insightful results when I am able to replicate this on my model.\nDo share the code once it's cleaned up. @maxlenormand", "votes": null }, { "id": "743885", "postDate": "02/12/2020 11:14:19", "content": "I added it yesterday :)\nYou can find it here.
", "rawMarkdown": "I added it yesterday :)\nYou can find it [here](https://www.kaggle.com/maxlenormand/multi-class-activation-map-with-efficientnetb0?scriptVersionId=28497228).", "votes": null }, { "id": "1247612", "postDate": "03/21/2021 21:34:32", "content": "Hi @virajbagal - Thanks for the shout-out. How could I use these sorts of images to improve results?
", "rawMarkdown": "Hi @virajbagal - Thanks for the shout-out. How could I use these sorts of images to improve results?", "votes": null }, { "id": "1247961", "postDate": "03/22/2021 08:14:47", "content": "I think those are useful in order to find which images / specific classes lead to good or bad classifications
\nThis leads to a lot more granularity than just a value between 0 and 1 that an F1 score might provide. This is useful to know for example if a repeated specific class leads to wrong classification as it would be more pointed out in the GradCAM output. This was useful for us to get an understanding of such classes that might cause an issue
\nI hope this helps!
", "rawMarkdown": "I think those are useful in order to find which images / specific classes lead to good or bad classifications\n\nThis leads to a lot more granularity than just a value between 0 and 1 that an F1 score might provide. This is useful to know for example if a repeated specific class leads to wrong classification as it would be more pointed out in the GradCAM output. This was useful for us to get an understanding of such classes that might cause an issue\n\nI hope this helps!", "votes": null }, { "id": "1363617", "postDate": "06/24/2021 08:09:49", "content": "Hi @virajbagal - Thanks for the shout-out. How could I use these sorts of images to improve results?
", "votes": null, "replies": [ { "id": 1247961, "author_name": "maxlenormand", "author_url": "", "post_date": "03/22/2021 08:14:47", "content": "I think those are useful in order to find which images / specific classes lead to good or bad classifications
\nThis leads to a lot more granularity than just a value between 0 and 1 that an F1 score might provide. This is useful to know for example if a repeated specific class leads to wrong classification as it would be more pointed out in the GradCAM output. This was useful for us to get an understanding of such classes that might cause an issue
\nI hope this helps!
", "votes": null, "replies": [] } ] }, { "id": 1363617, "author_name": "fabienbenot", "author_url": "", "post_date": "06/24/2021 08:09:49", "content": "Thanks @maxlenormand . I really appreciate your work .
", "votes": null, "replies": [ { "id": 742240, "author_name": "maxlenormand", "author_url": "", "post_date": "02/11/2020 07:16:53", "content": "Thanks!
", "votes": null, "replies": [] } ] }, { "id": 743856, "author_name": "rohitagarwal", "author_url": "", "post_date": "02/12/2020 10:28:01", "content": "I'm sure this will give me insightful results when I am able to replicate this on my model.\nDo share the code once it's cleaned up. @maxlenormand
", "votes": null, "replies": [ { "id": 743885, "author_name": "maxlenormand", "author_url": "", "post_date": "02/12/2020 11:14:19", "content": "I added it yesterday :)\nYou can find it here.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "741702": "Hi!\n\nEDIT: [Added notebook with code](https://www.kaggle.com/maxlenormand/multi-class-activation-map-with-efficientnetb0?scriptVersionId=28497228)\n\nI've been inspired by the work of @pnussbaum on his [reading the robot's mind notebook](https://www.kaggle.com/pnussbaum/grapheme-mind-reader-panv12-nogpu) to look a bit more into how we can tackle some more error analysis methods, and getting a better understanding of where some of the errors might be occurring and where.\n\nA while back, @cdeotte made a [Class Activation Map notebook](https://www.kaggle.com/cdeotte/unsupervised-masks-cv-0-60) on how to make masks out of class activation maps for a unsupervised task. Ever since I saw that I wanted to try to implement it. So I did, I made it for a simple EfficientNetB0, and here is what it looks like:\n\n\n\nThose aren't great, because they are based on (3,3) filters that are up-scaled to the 75x75 size of the images here. @cdeotte was using an Xception model which had 7x7 filters on the last Conv2D layer. EfficientNet seems to have 3x3, from what I could tell and what I used.\n\nAnyways, it's still some interesting info!\n\nIf anybody is interested I'll clean the code up and post it, I'm sure there could be some interesting info to find with this! These things are so interesting once you start digging into them. EDIT: [Voilà](https://www.kaggle.com/maxlenormand/multi-class-activation-map-with-efficientnetb0?scriptVersionId=28497228)", "742180": "Thanks @maxlenormand . I really appreciate your work .", "742240": "Thanks!", "743856": "I'm sure this will give me insightful results when I am able to replicate this on my model.\nDo share the code once it's cleaned up. @maxlenormand", "743885": "I added it yesterday :)\nYou can find it [here](https://www.kaggle.com/maxlenormand/multi-class-activation-map-with-efficientnetb0?scriptVersionId=28497228).", "1247612": "Hi @virajbagal - Thanks for the shout-out. How could I use these sorts of images to improve results?", "1247961": "I think those are useful in order to find which images / specific classes lead to good or bad classifications\n\nThis leads to a lot more granularity than just a value between 0 and 1 that an F1 score might provide. This is useful to know for example if a repeated specific class leads to wrong classification as it would be more pointed out in the GradCAM output. This was useful for us to get an understanding of such classes that might cause an issue\n\nI hope this helps!", "1363617": "1. links\n1.1. https://www.kaggle.com/maxlenormand/multi-class-activation-map-with-efficientnetb0?scriptVersionId=28497228\n1.2. https://www.kaggle.com/c/bengaliai-cv19/code?competitionId=14897&sortBy=voteCount\n1.3. https://arxiv.org/pdf/2106.10472v1.pdf\n1.4. https://arxiv.org/pdf/1901.07683.pdf\n1.5. https://arxiv.org/pdf/1909.09839.pdf\n1.6. https://arxiv.org/pdf/2008.00299.pdf\n1.7. https://arxiv.org/pdf/1910.05518.pdf\n1.8. https://github.com/tyui592/class_activation_map\n1.9. https://github.com/nickbiso/Keras-Class-Activation-Map\n1.10. https://github.com/topics/class-activation-map\n1.11. https://github.com/frgfm/torch-cam\n1.12. https://github.com/KangBK0120/CAM\n1.13. https://arxiv.org/pdf/1512.04150.pdf\n2. Seems like Class Activation Map has been mentioned in a paper from 2015. The technique is basically about unsupervised localization. I wonder whether it has helped to get some improvement in practice." }, "source": "meta" }