{
  "id": 38633,
  "title": "Visualizing hidden layer activations",
  "url": "/competitions/carvana-image-masking-challenge/discussion/38633",
  "author_name": "",
  "post_date": "2017-08-28T00:57:41.058962600Z",
  "votes": 23,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Visualizing feature maps from CNN shows some interesting stuff is being learnt by neural networks to aid segmentation. Looks like some feature maps highlight shadows, edge detectors, e.t.c. The last one is really weird, it highlights the brake-light and front windows, go figure!!</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7209/6_1.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7219/11_4_OTHER.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7207/30_1.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7208/16_29.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7212/10_5.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7211/30_0.jpg\" alt=\"enter image description here\" title=\"\"></p>\n\n<p>Feature maps after more training\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7217/2_4.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7206/11_4.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7213/28_5.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7214/28_8.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7215/28_14.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7216/29_15.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7218/28_13.jpg\" alt=\"enter image description here\" title=\"\"></p>",
  "messages": [
    {
      "id": "216748",
      "postDate": "08/28/2017 00:57:41",
      "content": "<p>Visualizing feature maps from CNN shows some interesting stuff is being learnt by neural networks to aid segmentation. Looks like some feature maps highlight shadows, edge detectors, e.t.c. The last one is really weird, it highlights the brake-light and front windows, go figure!!</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7209/6_1.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7219/11_4_OTHER.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7207/30_1.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7208/16_29.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7212/10_5.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7211/30_0.jpg\" alt=\"enter image description here\" title=\"\"></p>\n\n<p>Feature maps after more training\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7217/2_4.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7206/11_4.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7213/28_5.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7214/28_8.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7215/28_14.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7216/29_15.jpg\" alt=\"enter image description here\" title=\"\">\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/216748/7218/28_13.jpg\" alt=\"enter image description here\" title=\"\"></p>",
      "rawMarkdown": "Visualizing feature maps from CNN shows some interesting stuff is being learnt by neural networks to aid segmentation. Looks like some feature maps highlight shadows, edge detectors, e.t.c. The last one is really weird, it highlights the brake-light and front windows, go figure!!\n\n![enter image description here][1]\n![enter image description here][2]\n![enter image description here][3]\n![enter image description here][4]\n![enter image description here][5]\n![enter image description here][6]\n\nFeature maps after more training\n![enter image description here][7]\n![enter image description here][8]\n![enter image description here][9]\n![enter image description here][10]\n![enter image description here][11]\n![enter image description here][12]\n![enter image description here][13]\n\n\n  [1]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7209/6_1.jpg\n  [2]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7219/11_4_OTHER.jpg\n  [3]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7207/30_1.jpg\n  [4]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7208/16_29.jpg\n  [5]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7212/10_5.jpg\n  [6]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7211/30_0.jpg\n  [7]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7217/2_4.jpg\n  [8]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7206/11_4.jpg\n  [9]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7213/28_5.jpg\n  [10]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7214/28_8.jpg\n  [11]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7215/28_14.jpg\n  [12]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7216/29_15.jpg\n  [13]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7218/28_13.jpg",
      "votes": null
    },
    {
      "id": "217173",
      "postDate": "08/29/2017 16:41:10",
      "content": "<p>Cool visualizations ;)</p>",
      "rawMarkdown": "Cool visualizations ;)",
      "votes": null
    },
    {
      "id": "217185",
      "postDate": "08/29/2017 17:18:46",
      "content": "<p>yep!! neural nets are artists deep down inside, </p>",
      "rawMarkdown": "yep!! neural nets are artists deep down inside,",
      "votes": null
    },
    {
      "id": "217399",
      "postDate": "08/30/2017 12:51:27",
      "content": "<p>Can you share the visualisation code? It's pretty cool :)</p>",
      "rawMarkdown": "Can you share the visualisation code? It's pretty cool :)",
      "votes": null
    },
    {
      "id": "217423",
      "postDate": "08/30/2017 15:47:51",
      "content": "<p>I will share when the competition is over, I think a lot of insight can be gained by looking at the activations e.g there is a lot of redundancy, and I am still planning on entering this competition with slim margins :)</p>",
      "rawMarkdown": "I will share when the competition is over, I think a lot of insight can be gained by looking at the activations e.g there is a lot of redundancy, and I am still planning on entering this competition with slim margins :)",
      "votes": null
    },
    {
      "id": "217605",
      "postDate": "08/31/2017 09:56:42",
      "content": "<p>cool cool cool</p>",
      "rawMarkdown": "cool cool cool",
      "votes": null
    },
    {
      "id": "218119",
      "postDate": "09/02/2017 10:59:58",
      "content": "<p>What activation function should be used？when i use relu to all the conv layer ,and the output layer use the sigmoid \n but the loss of crossentropy  is nan.</p>",
      "rawMarkdown": "What activation function should be used？when i use relu to all the conv layer ,and the output layer use the sigmoid \n but the loss of crossentropy  is nan.",
      "votes": null
    },
    {
      "id": "218120",
      "postDate": "09/02/2017 11:19:37",
      "content": "<p>I think your problem is unrelated to this topic.</p>\n\n<p>Also, I guess there is a problem in your code/model. Your activation functions shouldn't cause a nan-loss.</p>",
      "rawMarkdown": "I think your problem is unrelated to this topic.\n\nAlso, I guess there is a problem in your code/model. Your activation functions shouldn't cause a nan-loss.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 217173,
      "author_name": "fpaboim",
      "author_url": "",
      "post_date": "08/29/2017 16:41:10",
      "content": "<p>Cool visualizations ;)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 217185,
      "author_name": "godaibo",
      "author_url": "",
      "post_date": "08/29/2017 17:18:46",
      "content": "<p>yep!! neural nets are artists deep down inside, </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 217399,
      "author_name": "craigglastonbury",
      "author_url": "",
      "post_date": "08/30/2017 12:51:27",
      "content": "<p>Can you share the visualisation code? It's pretty cool :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 217423,
      "author_name": "godaibo",
      "author_url": "",
      "post_date": "08/30/2017 15:47:51",
      "content": "<p>I will share when the competition is over, I think a lot of insight can be gained by looking at the activations e.g there is a lot of redundancy, and I am still planning on entering this competition with slim margins :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 217605,
      "author_name": "zhihang",
      "author_url": "",
      "post_date": "08/31/2017 09:56:42",
      "content": "<p>cool cool cool</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 218119,
      "author_name": "nobodylj",
      "author_url": "",
      "post_date": "09/02/2017 10:59:58",
      "content": "<p>What activation function should be used？when i use relu to all the conv layer ,and the output layer use the sigmoid \n but the loss of crossentropy  is nan.</p>",
      "votes": null,
      "replies": [
        {
          "id": 218120,
          "author_name": "depthfirstsearch",
          "author_url": "",
          "post_date": "09/02/2017 11:19:37",
          "content": "<p>I think your problem is unrelated to this topic.</p>\n\n<p>Also, I guess there is a problem in your code/model. Your activation functions shouldn't cause a nan-loss.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "216748": "Visualizing feature maps from CNN shows some interesting stuff is being learnt by neural networks to aid segmentation. Looks like some feature maps highlight shadows, edge detectors, e.t.c. The last one is really weird, it highlights the brake-light and front windows, go figure!!\n\n![enter image description here][1]\n![enter image description here][2]\n![enter image description here][3]\n![enter image description here][4]\n![enter image description here][5]\n![enter image description here][6]\n\nFeature maps after more training\n![enter image description here][7]\n![enter image description here][8]\n![enter image description here][9]\n![enter image description here][10]\n![enter image description here][11]\n![enter image description here][12]\n![enter image description here][13]\n\n\n  [1]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7209/6_1.jpg\n  [2]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7219/11_4_OTHER.jpg\n  [3]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7207/30_1.jpg\n  [4]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7208/16_29.jpg\n  [5]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7212/10_5.jpg\n  [6]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7211/30_0.jpg\n  [7]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7217/2_4.jpg\n  [8]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7206/11_4.jpg\n  [9]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7213/28_5.jpg\n  [10]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7214/28_8.jpg\n  [11]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7215/28_14.jpg\n  [12]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7216/29_15.jpg\n  [13]: http://kaggle2.blob.core.windows.net/forum-message-attachments/216748/7218/28_13.jpg",
    "217173": "Cool visualizations ;)",
    "217185": "yep!! neural nets are artists deep down inside,",
    "217399": "Can you share the visualisation code? It's pretty cool :)",
    "217423": "I will share when the competition is over, I think a lot of insight can be gained by looking at the activations e.g there is a lot of redundancy, and I am still planning on entering this competition with slim margins :)",
    "217605": "cool cool cool",
    "218119": "What activation function should be used？when i use relu to all the conv layer ,and the output layer use the sigmoid \n but the loss of crossentropy  is nan.",
    "218120": "I think your problem is unrelated to this topic.\n\nAlso, I guess there is a problem in your code/model. Your activation functions shouldn't cause a nan-loss."
  },
  "source": "meta"
}