{
  "id": 20661,
  "title": "Visualizing what the Keras+VGG16  net is looking for",
  "url": "/competitions/state-farm-distracted-driver-detection/discussion/20661",
  "author_name": "",
  "post_date": "2016-05-03T10:11:23.720Z",
  "votes": 6,
  "comment_count": 5,
  "views": 3194,
  "content": "<p>Hey guys,</p>\n\n<p>I have implemented the deconvolution method of <a href=\"http://arxiv.org/abs/1311.2901\">Zeiler and Fergus</a> with Keras and Theano.</p>\n\n<p>This yields pretty interesting figures which tell us the location and nature of the patterns that a specific layer/feature map pair responds most strongly to.</p>\n\n<p><a href=\"https://github.com/tdeboissiere/DeepLearningImplementations/tree/master/DeconvNet\">Code here</a></p>\n\n<p>The KerasDeconv class should adapt to any Keras model (made of ZeroPadding/Conv2d/MaxPooling2D) but I have not tested it extensively : contributions welcome !</p>",
  "messages": [
    {
      "id": "118331",
      "postDate": "05/03/2016 10:11:23",
      "content": "<p>Hey guys,</p>\n\n<p>I have implemented the deconvolution method of <a href=\"http://arxiv.org/abs/1311.2901\">Zeiler and Fergus</a> with Keras and Theano.</p>\n\n<p>This yields pretty interesting figures which tell us the location and nature of the patterns that a specific layer/feature map pair responds most strongly to.</p>\n\n<p><a href=\"https://github.com/tdeboissiere/DeepLearningImplementations/tree/master/DeconvNet\">Code here</a></p>\n\n<p>The KerasDeconv class should adapt to any Keras model (made of ZeroPadding/Conv2d/MaxPooling2D) but I have not tested it extensively : contributions welcome !</p>",
      "rawMarkdown": "Hey guys,\r\n\r\nI have implemented the deconvolution method of [Zeiler and Fergus][1] with Keras and Theano.\r\n\r\nThis yields pretty interesting figures which tell us the location and nature of the patterns that a specific layer/feature map pair responds most strongly to.\r\n\r\n[Code here][2]\r\n\r\nThe KerasDeconv class should adapt to any Keras model (made of ZeroPadding/Conv2d/MaxPooling2D) but I have not tested it extensively : contributions welcome !\r\n\r\n  [1]: http://arxiv.org/abs/1311.2901\r\n  [2]: https://github.com/tdeboissiere/DeepLearningImplementations/tree/master/DeconvNet",
      "votes": null
    },
    {
      "id": "118407",
      "postDate": "05/03/2016 15:19:08",
      "content": "<p>Do I understand correctly that the VGG net wasn't finetuned on this dataset? If that's true this shouldn't really tell us much about what a ConvNet would look for when trying to classify the State Farm images. Instead it shows us what's maximally activated w.r.t. ImageNet.</p>",
      "rawMarkdown": "Do I understand correctly that the VGG net wasn't finetuned on this dataset? If that's true this shouldn't really tell us much about what a ConvNet would look for when trying to classify the State Farm images. Instead it shows us what's maximally activated w.r.t. ImageNet.",
      "votes": null
    },
    {
      "id": "118492",
      "postDate": "05/03/2016 23:29:07",
      "content": "<p>That is absolutely correct, the title is a bit misleading.</p>\n\n<p>However, see it this way:</p>\n\n<p>When trained on ImageNet, VGG was not taught to recognize everything (I reckon the steering wheel is not a class of the corresponding ImageNet set) and yet it strongly activates on the steering wheel (top right corner of convolution2d_13).</p>\n\n<p>This shows the generalisation power of a pre-trained net (and why it makes sense to use it on a different dataset):</p>\n\n<p>It has learnt to recognize complex abstractions (in that case, the steering wheel) despite not being explicitly trained to do so. The reason for that is that this abstraction is probably correlated to a real class of the ImageNet set. For instance, if you want to learn to detect a tie, it makes sense to learn how to detect a face because the two often go together in the same picture.</p>",
      "rawMarkdown": "That is absolutely correct, the title is a bit misleading.\r\n\r\nHowever, see it this way:\r\n\r\nWhen trained on ImageNet, VGG was not taught to recognize everything (I reckon the steering wheel is not a class of the corresponding ImageNet set) and yet it strongly activates on the steering wheel (top right corner of convolution2d_13).\r\n\r\nThis shows the generalisation power of a pre-trained net (and why it makes sense to use it on a different dataset):\r\n\r\nIt has learnt to recognize complex abstractions (in that case, the steering wheel) despite not being explicitly trained to do so. The reason for that is that this abstraction is probably correlated to a real class of the ImageNet set. For instance, if you want to learn to detect a tie, it makes sense to learn how to detect a face because the two often go together in the same picture.",
      "votes": null
    },
    {
      "id": "139771",
      "postDate": "10/16/2016 13:50:24",
      "content": "<p>Hi,\nI am not able to get the code for deconvolution. Can you please re-link the page address here if you have changed it?\nThanks</p>",
      "rawMarkdown": "Hi,\r\nI am not able to get the code for deconvolution. Can you please re-link the page address here if you have changed it?\r\nThanks",
      "votes": null
    },
    {
      "id": "139850",
      "postDate": "10/16/2016 23:19:07",
      "content": "<p>Updated the repository with a new link </p>",
      "rawMarkdown": "Updated the repository with a new link",
      "votes": null
    },
    {
      "id": "148470",
      "postDate": "12/05/2016 05:42:45",
      "content": "",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 118407,
      "author_name": "florianm",
      "author_url": "",
      "post_date": "05/03/2016 15:19:08",
      "content": "<p>Do I understand correctly that the VGG net wasn't finetuned on this dataset? If that's true this shouldn't really tell us much about what a ConvNet would look for when trying to classify the State Farm images. Instead it shows us what's maximally activated w.r.t. ImageNet.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 118492,
      "author_name": "thibmain",
      "author_url": "",
      "post_date": "05/03/2016 23:29:07",
      "content": "<p>That is absolutely correct, the title is a bit misleading.</p>\n\n<p>However, see it this way:</p>\n\n<p>When trained on ImageNet, VGG was not taught to recognize everything (I reckon the steering wheel is not a class of the corresponding ImageNet set) and yet it strongly activates on the steering wheel (top right corner of convolution2d_13).</p>\n\n<p>This shows the generalisation power of a pre-trained net (and why it makes sense to use it on a different dataset):</p>\n\n<p>It has learnt to recognize complex abstractions (in that case, the steering wheel) despite not being explicitly trained to do so. The reason for that is that this abstraction is probably correlated to a real class of the ImageNet set. For instance, if you want to learn to detect a tie, it makes sense to learn how to detect a face because the two often go together in the same picture.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 139771,
      "author_name": "vivekbawa",
      "author_url": "",
      "post_date": "10/16/2016 13:50:24",
      "content": "<p>Hi,\nI am not able to get the code for deconvolution. Can you please re-link the page address here if you have changed it?\nThanks</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 139850,
      "author_name": "thibmain",
      "author_url": "",
      "post_date": "10/16/2016 23:19:07",
      "content": "<p>Updated the repository with a new link </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 148470,
      "author_name": "zhangyundong",
      "author_url": "",
      "post_date": "12/05/2016 05:42:45",
      "content": "",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "118331": "Hey guys,\r\n\r\nI have implemented the deconvolution method of [Zeiler and Fergus][1] with Keras and Theano.\r\n\r\nThis yields pretty interesting figures which tell us the location and nature of the patterns that a specific layer/feature map pair responds most strongly to.\r\n\r\n[Code here][2]\r\n\r\nThe KerasDeconv class should adapt to any Keras model (made of ZeroPadding/Conv2d/MaxPooling2D) but I have not tested it extensively : contributions welcome !\r\n\r\n  [1]: http://arxiv.org/abs/1311.2901\r\n  [2]: https://github.com/tdeboissiere/DeepLearningImplementations/tree/master/DeconvNet",
    "118407": "Do I understand correctly that the VGG net wasn't finetuned on this dataset? If that's true this shouldn't really tell us much about what a ConvNet would look for when trying to classify the State Farm images. Instead it shows us what's maximally activated w.r.t. ImageNet.",
    "118492": "That is absolutely correct, the title is a bit misleading.\r\n\r\nHowever, see it this way:\r\n\r\nWhen trained on ImageNet, VGG was not taught to recognize everything (I reckon the steering wheel is not a class of the corresponding ImageNet set) and yet it strongly activates on the steering wheel (top right corner of convolution2d_13).\r\n\r\nThis shows the generalisation power of a pre-trained net (and why it makes sense to use it on a different dataset):\r\n\r\nIt has learnt to recognize complex abstractions (in that case, the steering wheel) despite not being explicitly trained to do so. The reason for that is that this abstraction is probably correlated to a real class of the ImageNet set. For instance, if you want to learn to detect a tie, it makes sense to learn how to detect a face because the two often go together in the same picture.",
    "139771": "Hi,\r\nI am not able to get the code for deconvolution. Can you please re-link the page address here if you have changed it?\r\nThanks",
    "139850": "Updated the repository with a new link",
    "148470": ""
  },
  "source": "meta"
}