![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11468/00ac0fca6.jpg\")
I created kernel with heatmaps for our best single model. HeatMap was created to illustrate how exactly neural networks recognize whales and where its main attention is focused. Output of the last BatchNormalization layer for Branch model was used to create HeatMap. Matrix for this layer for DenseNet121 and input size of 512 pixels has dimension (16, 16, 1024). Here 16x16 is the size of the feature map in pixels. In this case, total number of feature maps is 1024. This layer is followed by RELU activation, and then by GlobalMaxPooling layer, which selects the maximum pixel from each of the 1024 feature maps. Therefore, if we want to understand exactly what the model is focused at, we can count how many times each individual pixel takes the maximum value in the feature map. The second rather important parameter is how often the pixel value changes from one map to another; for this we can calculate the standard deviation. Next, we normalize the obtained values in the interval from 0 to 255 and increase 16x16 image to the size of the original image - 512x512
\n\n
![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11469/0041a9867.jpg\")
![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11470/006183fb4.jpg\")
![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11471/01288355d.jpg\")
Here, areas with the maximum standard deviation are marked in green. Pink is used for areas that have the maximum value on feature maps most often and thus determine the vector that goes to head model. White areas have two of these properties simultaneously. As we can see, the model very well marks the whales' visual features, scars, birthmarks and the shape of the tail.
\n\nKernel: https://www.kaggle.com/zfturbo/visualisation-of-siamese-net/
\n\nUPD: Also added small video with single feature map visualization. But it's not so interesting as heatmap images: https://www.youtube.com/watch?v=_csyNsSGbRI
", "messages": [ { "id": "481075", "postDate": "03/01/2019 02:57:17", "content": "I created kernel with heatmaps for our best single model. HeatMap was created to illustrate how exactly neural networks recognize whales and where its main attention is focused. Output of the last BatchNormalization layer for Branch model was used to create HeatMap. Matrix for this layer for DenseNet121 and input size of 512 pixels has dimension (16, 16, 1024). Here 16x16 is the size of the feature map in pixels. In this case, total number of feature maps is 1024. This layer is followed by RELU activation, and then by GlobalMaxPooling layer, which selects the maximum pixel from each of the 1024 feature maps. Therefore, if we want to understand exactly what the model is focused at, we can count how many times each individual pixel takes the maximum value in the feature map. The second rather important parameter is how often the pixel value changes from one map to another; for this we can calculate the standard deviation. Next, we normalize the obtained values in the interval from 0 to 255 and increase 16x16 image to the size of the original image - 512x512
\n\n
Here, areas with the maximum standard deviation are marked in green. Pink is used for areas that have the maximum value on feature maps most often and thus determine the vector that goes to head model. White areas have two of these properties simultaneously. As we can see, the model very well marks the whales' visual features, scars, birthmarks and the shape of the tail.
\n\nKernel: https://www.kaggle.com/zfturbo/visualisation-of-siamese-net/
\n\nUPD: Also added small video with single feature map visualization. But it's not so interesting as heatmap images: https://www.youtube.com/watch?v=_csyNsSGbRI
", "rawMarkdown": "I created kernel with heatmaps for our best single model. HeatMap was created to illustrate how exactly neural networks recognize whales and where its main attention is focused. Output of the last BatchNormalization layer for Branch model was used to create HeatMap. Matrix for this layer for DenseNet121 and input size of 512 pixels has dimension (16, 16, 1024). Here 16x16 is the size of the feature map in pixels. In this case, total number of feature maps is 1024. This layer is followed by RELU activation, and then by GlobalMaxPooling layer, which selects the maximum pixel from each of the 1024 feature maps. Therefore, if we want to understand exactly what the model is focused at, we can count how many times each individual pixel takes the maximum value in the feature map. The second rather important parameter is how often the pixel value changes from one map to another; for this we can calculate the standard deviation. Next, we normalize the obtained values in the interval from 0 to 255 and increase 16x16 image to the size of the original image - 512x512\n\n![enter image description here][1]\n\n![enter image description here][2]\n\n![enter image description here][3]\n\n![enter image description here][4]\n\nHere, areas with the maximum standard deviation are marked in green. Pink is used for areas that have the maximum value on feature maps most often and thus determine the vector that goes to head model. White areas have two of these properties simultaneously. As we can see, the model very well marks the whales' visual features, scars, birthmarks and the shape of the tail.\n\nKernel: https://www.kaggle.com/zfturbo/visualisation-of-siamese-net/\n\n**UPD**: Also added small video with single feature map visualization. But it's not so interesting as heatmap images: https://www.youtube.com/watch?v=_csyNsSGbRI\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11468/00ac0fca6.jpg\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11469/0041a9867.jpg\n [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11470/006183fb4.jpg\n [4]: https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11471/01288355d.jpg", "votes": null }, { "id": "481148", "postDate": "03/01/2019 05:02:40", "content": "Beautiful
", "rawMarkdown": "Beautiful", "votes": null }, { "id": "481237", "postDate": "03/01/2019 07:27:18", "content": "Thanks for sharing @zfturbo
", "rawMarkdown": "Thanks for sharing @zfturbo", "votes": null }, { "id": "481280", "postDate": "03/01/2019 08:06:30", "content": "Cool! I'm reading your kernel
", "rawMarkdown": "Cool! I'm reading your kernel", "votes": null }, { "id": "481309", "postDate": "03/01/2019 08:29:50", "content": "@zfturbo congrats to you and team for another strong finish. Thanks for sharing.
", "rawMarkdown": "zfturbo congrats to you and team for another strong finish. Thanks for sharing.", "votes": null }, { "id": "481345", "postDate": "03/01/2019 09:36:48", "content": "wonderful sharing, thank you!
", "rawMarkdown": "wonderful sharing, thank you!", "votes": null }, { "id": "481456", "postDate": "03/01/2019 12:28:56", "content": "Very cool!
", "rawMarkdown": "Very cool!", "votes": null }, { "id": "482816", "postDate": "03/03/2019 17:37:49", "content": "Also added small video with single feature map visualization. But it's not so interesting as heatmap images:\nhttps://www.youtube.com/watch?v=_csyNsSGbRI
", "rawMarkdown": "Also added small video with single feature map visualization. But it's not so interesting as heatmap images:\nhttps://www.youtube.com/watch?v=_csyNsSGbRI", "votes": null } ], "comments": [ { "id": 481148, "author_name": "interneuron", "author_url": "", "post_date": "03/01/2019 05:02:40", "content": "Beautiful
", "votes": null, "replies": [] }, { "id": 481237, "author_name": "karthik7395", "author_url": "", "post_date": "03/01/2019 07:27:18", "content": "Thanks for sharing @zfturbo
", "votes": null, "replies": [] }, { "id": 481280, "author_name": "benwu232", "author_url": "", "post_date": "03/01/2019 08:06:30", "content": "Cool! I'm reading your kernel
", "votes": null, "replies": [] }, { "id": 481309, "author_name": "sheriytm", "author_url": "", "post_date": "03/01/2019 08:29:50", "content": "@zfturbo congrats to you and team for another strong finish. Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 481345, "author_name": "soonhwankwon", "author_url": "", "post_date": "03/01/2019 09:36:48", "content": "wonderful sharing, thank you!
", "votes": null, "replies": [] }, { "id": 481456, "author_name": "noonv13", "author_url": "", "post_date": "03/01/2019 12:28:56", "content": "Very cool!
", "votes": null, "replies": [] }, { "id": 482816, "author_name": "zfturbo", "author_url": "", "post_date": "03/03/2019 17:37:49", "content": "Also added small video with single feature map visualization. But it's not so interesting as heatmap images:\nhttps://www.youtube.com/watch?v=_csyNsSGbRI
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "481075": "I created kernel with heatmaps for our best single model. HeatMap was created to illustrate how exactly neural networks recognize whales and where its main attention is focused. Output of the last BatchNormalization layer for Branch model was used to create HeatMap. Matrix for this layer for DenseNet121 and input size of 512 pixels has dimension (16, 16, 1024). Here 16x16 is the size of the feature map in pixels. In this case, total number of feature maps is 1024. This layer is followed by RELU activation, and then by GlobalMaxPooling layer, which selects the maximum pixel from each of the 1024 feature maps. Therefore, if we want to understand exactly what the model is focused at, we can count how many times each individual pixel takes the maximum value in the feature map. The second rather important parameter is how often the pixel value changes from one map to another; for this we can calculate the standard deviation. Next, we normalize the obtained values in the interval from 0 to 255 and increase 16x16 image to the size of the original image - 512x512\n\n![enter image description here][1]\n\n![enter image description here][2]\n\n![enter image description here][3]\n\n![enter image description here][4]\n\nHere, areas with the maximum standard deviation are marked in green. Pink is used for areas that have the maximum value on feature maps most often and thus determine the vector that goes to head model. White areas have two of these properties simultaneously. As we can see, the model very well marks the whales' visual features, scars, birthmarks and the shape of the tail.\n\nKernel: https://www.kaggle.com/zfturbo/visualisation-of-siamese-net/\n\n**UPD**: Also added small video with single feature map visualization. But it's not so interesting as heatmap images: https://www.youtube.com/watch?v=_csyNsSGbRI\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11468/00ac0fca6.jpg\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11469/0041a9867.jpg\n [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11470/006183fb4.jpg\n [4]: https://storage.googleapis.com/kaggle-forum-message-attachments/481075/11471/01288355d.jpg", "481148": "Beautiful", "481237": "Thanks for sharing @zfturbo", "481280": "Cool! I'm reading your kernel", "481309": "zfturbo congrats to you and team for another strong finish. Thanks for sharing.", "481345": "wonderful sharing, thank you!", "481456": "Very cool!", "482816": "Also added small video with single feature map visualization. But it's not so interesting as heatmap images:\nhttps://www.youtube.com/watch?v=_csyNsSGbRI" }, "source": "meta" }