{
  "id": 17421,
  "title": "Complete Train Set  Head Annotations",
  "url": "/competitions/noaa-right-whale-recognition/discussion/17421",
  "author_name": "",
  "post_date": "2015-11-15T01:34:16.827Z",
  "votes": 27,
  "comment_count": 9,
  "views": 3689,
  "content": "<p>Inspired by the annotation project by @Smerity</p>\n\n<p><a href=\"https://www.kaggle.com/c/noaa-right-whale-recognition/forums/t/16353/annotated-faces-for-noaa-right-whale\">https://www.kaggle.com/c/noaa-right-whale-recognition/forums/t/16353/annotated-faces-for-noaa-right-whale</a></p>\n\n<p>I have personally annotated all whale faces in the train set.  I hope every one can make good use of this.</p>\n\n<p>Intriguingly, using just the heads for training a deep convnet did not yield better results than the net I trained with the original images (scaled to 128x128), with which I got a local CV score of around 5.5, which translates to a LB score of 5.2</p>\n\n<p>I guess the network did not learn any useful whale features, but mostly made the predictions based on irrelevant features, such as water colour (as there are many whale pictures taken in similar illuminating conditions with distinct water color variations). </p>",
  "messages": [
    {
      "id": "98846",
      "postDate": "11/15/2015 01:34:16",
      "content": "<p>Inspired by the annotation project by @Smerity</p>\n\n<p><a href=\"https://www.kaggle.com/c/noaa-right-whale-recognition/forums/t/16353/annotated-faces-for-noaa-right-whale\">https://www.kaggle.com/c/noaa-right-whale-recognition/forums/t/16353/annotated-faces-for-noaa-right-whale</a></p>\n\n<p>I have personally annotated all whale faces in the train set.  I hope every one can make good use of this.</p>\n\n<p>Intriguingly, using just the heads for training a deep convnet did not yield better results than the net I trained with the original images (scaled to 128x128), with which I got a local CV score of around 5.5, which translates to a LB score of 5.2</p>\n\n<p>I guess the network did not learn any useful whale features, but mostly made the predictions based on irrelevant features, such as water colour (as there are many whale pictures taken in similar illuminating conditions with distinct water color variations). </p>",
      "rawMarkdown": "Inspired by the annotation project by @Smerity\r\n\r\nhttps://www.kaggle.com/c/noaa-right-whale-recognition/forums/t/16353/annotated-faces-for-noaa-right-whale\r\n\r\nI have personally annotated all whale faces in the train set.  I hope every one can make good use of this.\r\n\r\nIntriguingly, using just the heads for training a deep convnet did not yield better results than the net I trained with the original images (scaled to 128x128), with which I got a local CV score of around 5.5, which translates to a LB score of 5.2\r\n\r\nI guess the network did not learn any useful whale features, but mostly made the predictions based on irrelevant features, such as water colour (as there are many whale pictures taken in similar illuminating conditions with distinct water color variations).",
      "votes": null
    },
    {
      "id": "98866",
      "postDate": "11/15/2015 15:39:02",
      "content": "<p>Hi Vinh, </p>\n\n<p>Thank you very much for sharing this! </p>",
      "rawMarkdown": "Hi Vinh, \r\n\r\nThank you very much for sharing this!",
      "votes": null
    },
    {
      "id": "98954",
      "postDate": "11/17/2015 02:04:33",
      "content": "<p>@Vinh Nguyen, Did you use the heads to train a  head detector? So  you used the head detector to detecet head  in test image.</p>",
      "rawMarkdown": "Vinh Nguyen, Did you use the heads to train a  head detector? So  you used the head detector to detecet head  in test image.",
      "votes": null
    },
    {
      "id": "98955",
      "postDate": "11/17/2015 02:19:17",
      "content": "<p>@CHiQ: this set was manually annotated.  Hopefully you can use it to train an effective head detector.</p>\n\n<p>I trained a head detector with Matlab, but the performance wasn't satisfactory.</p>",
      "rawMarkdown": "CHiQ: this set was manually annotated.  Hopefully you can use it to train an effective head detector.\r\n\r\nI trained a head detector with Matlab, but the performance wasn't satisfactory.",
      "votes": null
    },
    {
      "id": "98956",
      "postDate": "11/17/2015 03:19:24",
      "content": "<p>@Vinh Nguyen: I am curious of how you manage to get a LB score of 5.2 with deep convnet trained on original images. I have tried VGG19 and alexnet on original images but both didn't work will :( ( somewhere around 27 in LB score ) They seems to overfit severely. Any suggestion? </p>",
      "rawMarkdown": "Vinh Nguyen: I am curious of how you manage to get a LB score of 5.2 with deep convnet trained on original images. I have tried VGG19 and alexnet on original images but both didn't work will :( ( somewhere around 27 in LB score ) They seems to overfit severely. Any suggestion?",
      "votes": null
    },
    {
      "id": "98958",
      "postDate": "11/17/2015 03:21:58",
      "content": "<p>Wow Vinh, you're brilliant! ^_^ I'm looking forward to playing with this trove of new data soon!</p>\n\n<p>From my preliminary exploration, just the heads resulted in improved accuracy but the number of whale faces was smaller than the full set so that could be an artefact. I presume you were performing dataset augmentation (rotations and flips) during training?</p>\n\n<p>Can you pull request (or if you'd prefer I could just add) the mass annotations to the Right Whale Hunt repository?</p>",
      "rawMarkdown": "Wow Vinh, you're brilliant! ^_^ I'm looking forward to playing with this trove of new data soon!\r\n\r\nFrom my preliminary exploration, just the heads resulted in improved accuracy but the number of whale faces was smaller than the full set so that could be an artefact. I presume you were performing dataset augmentation (rotations and flips) during training?\r\n\r\nCan you pull request (or if you'd prefer I could just add) the mass annotations to the Right Whale Hunt repository?",
      "votes": null
    },
    {
      "id": "98959",
      "postDate": "11/17/2015 03:26:10",
      "content": "<p>@andy: Surely they overfit a lot. I used lots of different regularization strategies:</p>\n\n<ul>\n<li>dropout at 0.8 rate   --- seem to help :D</li>\n<li>batch normalization</li>\n<li>L2</li>\n<li>reduce the network size to about 10^4-10^5 parameters</li>\n<li>data augmentation: rotation, flip, stretch, color perturbation, etc...</li>\n<li><p>ensemble over several architectures</p></li>\n<li><p>network in network architecture + average pooling: doesn't seems to help</p></li>\n<li>yet to try maxout</li>\n</ul>\n\n<p>but yes, after all these, the results are still very crappy</p>",
      "rawMarkdown": "andy: Surely they overfit a lot. I used lots of different regularization strategies:\r\n\r\n+ dropout at 0.8 rate   --- seem to help :D\r\n+ batch normalization\r\n+ L2\r\n+ reduce the network size to about 10^4-10^5 parameters\r\n+ data augmentation: rotation, flip, stretch, color perturbation, etc...\r\n+ ensemble over several architectures\r\n\r\n+ network in network architecture + average pooling: doesn't seems to help\r\n+ yet to try maxout\r\n\r\nbut yes, after all these, the results are still very crappy",
      "votes": null
    },
    {
      "id": "98960",
      "postDate": "11/17/2015 03:27:23",
      "content": "<p>@smerity: yes pls do add those annotations to your repo.</p>",
      "rawMarkdown": "smerity: yes pls do add those annotations to your repo.",
      "votes": null
    },
    {
      "id": "99084",
      "postDate": "11/19/2015 09:10:26",
      "content": "<p>Hi!</p>\n\n<p>First of all, @Vinh Nguyen thanks a lot for sharing this! </p>\n\n<p>I tried to read today your annotation file and run some statistics on it. It seems that some whales have two bounding rects on them (ex: w_9200.jpg). Any reason for that? I noticed that there are 2 whales, but only one head. Well, this is the first time my assertion fails, I will come back as I will gather more data about the faces ;)</p>\n\n<p>Thanks a lot again!</p>\n\n<p><strong>edit:</strong></p>\n\n<pre><code>Not exactly 1 annotation: w_9200.jpg :2\nNot exactly 1 annotation: w_1300.jpg :2\nNot exactly 1 annotation: w_7077.jpg :0\nNot exactly 1 annotation: w_4876.jpg :2\n\nNot in labeled json but in training:[]\nNot in train.csv but in labeled json['w_1300.jpg' 'w_4876.jpg' 'w_9200.jpg']\n</code></pre>\n\n<p>Well, we can live with that! </p>",
      "rawMarkdown": "Hi!\r\n\r\nFirst of all, @Vinh Nguyen thanks a lot for sharing this! \r\n\r\nI tried to read today your annotation file and run some statistics on it. It seems that some whales have two bounding rects on them (ex: w_9200.jpg). Any reason for that? I noticed that there are 2 whales, but only one head. Well, this is the first time my assertion fails, I will come back as I will gather more data about the faces ;)\r\n\r\nThanks a lot again!\r\n\r\n**edit:**\r\n\r\n    Not exactly 1 annotation: w_9200.jpg :2\r\n    Not exactly 1 annotation: w_1300.jpg :2\r\n    Not exactly 1 annotation: w_7077.jpg :0\r\n    Not exactly 1 annotation: w_4876.jpg :2\r\n\r\n    Not in labeled json but in training:[]\r\n    Not in train.csv but in labeled json['w_1300.jpg' 'w_4876.jpg' 'w_9200.jpg']\r\n\r\nWell, we can live with that!",
      "votes": null
    },
    {
      "id": "99092",
      "postDate": "11/19/2015 10:31:20",
      "content": "<p>hi @visoft, good catch, for image with 2 whales, I just assume one is the 'main' whale that belongs to a class, but that's no guarantee... \nYou can visually check the annotations with sloth, as in @smerity 's thread.</p>",
      "rawMarkdown": "hi @visoft, good catch, for image with 2 whales, I just assume one is the 'main' whale that belongs to a class, but that's no guarantee... \r\nYou can visually check the annotations with sloth, as in @smerity 's thread.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 98866,
      "author_name": "lamdang",
      "author_url": "",
      "post_date": "11/15/2015 15:39:02",
      "content": "<p>Hi Vinh, </p>\n\n<p>Thank you very much for sharing this! </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 98954,
      "author_name": "wuqiangchiq",
      "author_url": "",
      "post_date": "11/17/2015 02:04:33",
      "content": "<p>@Vinh Nguyen, Did you use the heads to train a  head detector? So  you used the head detector to detecet head  in test image.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 98955,
      "author_name": "vinhnguyen",
      "author_url": "",
      "post_date": "11/17/2015 02:19:17",
      "content": "<p>@CHiQ: this set was manually annotated.  Hopefully you can use it to train an effective head detector.</p>\n\n<p>I trained a head detector with Matlab, but the performance wasn't satisfactory.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 98956,
      "author_name": "andytsai",
      "author_url": "",
      "post_date": "11/17/2015 03:19:24",
      "content": "<p>@Vinh Nguyen: I am curious of how you manage to get a LB score of 5.2 with deep convnet trained on original images. I have tried VGG19 and alexnet on original images but both didn't work will :( ( somewhere around 27 in LB score ) They seems to overfit severely. Any suggestion? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 98958,
      "author_name": "smerity",
      "author_url": "",
      "post_date": "11/17/2015 03:21:58",
      "content": "<p>Wow Vinh, you're brilliant! ^_^ I'm looking forward to playing with this trove of new data soon!</p>\n\n<p>From my preliminary exploration, just the heads resulted in improved accuracy but the number of whale faces was smaller than the full set so that could be an artefact. I presume you were performing dataset augmentation (rotations and flips) during training?</p>\n\n<p>Can you pull request (or if you'd prefer I could just add) the mass annotations to the Right Whale Hunt repository?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 98959,
      "author_name": "vinhnguyen",
      "author_url": "",
      "post_date": "11/17/2015 03:26:10",
      "content": "<p>@andy: Surely they overfit a lot. I used lots of different regularization strategies:</p>\n\n<ul>\n<li>dropout at 0.8 rate   --- seem to help :D</li>\n<li>batch normalization</li>\n<li>L2</li>\n<li>reduce the network size to about 10^4-10^5 parameters</li>\n<li>data augmentation: rotation, flip, stretch, color perturbation, etc...</li>\n<li><p>ensemble over several architectures</p></li>\n<li><p>network in network architecture + average pooling: doesn't seems to help</p></li>\n<li>yet to try maxout</li>\n</ul>\n\n<p>but yes, after all these, the results are still very crappy</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 98960,
      "author_name": "vinhnguyen",
      "author_url": "",
      "post_date": "11/17/2015 03:27:23",
      "content": "<p>@smerity: yes pls do add those annotations to your repo.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 99084,
      "author_name": "visoft",
      "author_url": "",
      "post_date": "11/19/2015 09:10:26",
      "content": "<p>Hi!</p>\n\n<p>First of all, @Vinh Nguyen thanks a lot for sharing this! </p>\n\n<p>I tried to read today your annotation file and run some statistics on it. It seems that some whales have two bounding rects on them (ex: w_9200.jpg). Any reason for that? I noticed that there are 2 whales, but only one head. Well, this is the first time my assertion fails, I will come back as I will gather more data about the faces ;)</p>\n\n<p>Thanks a lot again!</p>\n\n<p><strong>edit:</strong></p>\n\n<pre><code>Not exactly 1 annotation: w_9200.jpg :2\nNot exactly 1 annotation: w_1300.jpg :2\nNot exactly 1 annotation: w_7077.jpg :0\nNot exactly 1 annotation: w_4876.jpg :2\n\nNot in labeled json but in training:[]\nNot in train.csv but in labeled json['w_1300.jpg' 'w_4876.jpg' 'w_9200.jpg']\n</code></pre>\n\n<p>Well, we can live with that! </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 99092,
      "author_name": "vinhnguyen",
      "author_url": "",
      "post_date": "11/19/2015 10:31:20",
      "content": "<p>hi @visoft, good catch, for image with 2 whales, I just assume one is the 'main' whale that belongs to a class, but that's no guarantee... \nYou can visually check the annotations with sloth, as in @smerity 's thread.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "98846": "Inspired by the annotation project by @Smerity\r\n\r\nhttps://www.kaggle.com/c/noaa-right-whale-recognition/forums/t/16353/annotated-faces-for-noaa-right-whale\r\n\r\nI have personally annotated all whale faces in the train set.  I hope every one can make good use of this.\r\n\r\nIntriguingly, using just the heads for training a deep convnet did not yield better results than the net I trained with the original images (scaled to 128x128), with which I got a local CV score of around 5.5, which translates to a LB score of 5.2\r\n\r\nI guess the network did not learn any useful whale features, but mostly made the predictions based on irrelevant features, such as water colour (as there are many whale pictures taken in similar illuminating conditions with distinct water color variations).",
    "98866": "Hi Vinh, \r\n\r\nThank you very much for sharing this!",
    "98954": "Vinh Nguyen, Did you use the heads to train a  head detector? So  you used the head detector to detecet head  in test image.",
    "98955": "CHiQ: this set was manually annotated.  Hopefully you can use it to train an effective head detector.\r\n\r\nI trained a head detector with Matlab, but the performance wasn't satisfactory.",
    "98956": "Vinh Nguyen: I am curious of how you manage to get a LB score of 5.2 with deep convnet trained on original images. I have tried VGG19 and alexnet on original images but both didn't work will :( ( somewhere around 27 in LB score ) They seems to overfit severely. Any suggestion?",
    "98958": "Wow Vinh, you're brilliant! ^_^ I'm looking forward to playing with this trove of new data soon!\r\n\r\nFrom my preliminary exploration, just the heads resulted in improved accuracy but the number of whale faces was smaller than the full set so that could be an artefact. I presume you were performing dataset augmentation (rotations and flips) during training?\r\n\r\nCan you pull request (or if you'd prefer I could just add) the mass annotations to the Right Whale Hunt repository?",
    "98959": "andy: Surely they overfit a lot. I used lots of different regularization strategies:\r\n\r\n+ dropout at 0.8 rate   --- seem to help :D\r\n+ batch normalization\r\n+ L2\r\n+ reduce the network size to about 10^4-10^5 parameters\r\n+ data augmentation: rotation, flip, stretch, color perturbation, etc...\r\n+ ensemble over several architectures\r\n\r\n+ network in network architecture + average pooling: doesn't seems to help\r\n+ yet to try maxout\r\n\r\nbut yes, after all these, the results are still very crappy",
    "98960": "smerity: yes pls do add those annotations to your repo.",
    "99084": "Hi!\r\n\r\nFirst of all, @Vinh Nguyen thanks a lot for sharing this! \r\n\r\nI tried to read today your annotation file and run some statistics on it. It seems that some whales have two bounding rects on them (ex: w_9200.jpg). Any reason for that? I noticed that there are 2 whales, but only one head. Well, this is the first time my assertion fails, I will come back as I will gather more data about the faces ;)\r\n\r\nThanks a lot again!\r\n\r\n**edit:**\r\n\r\n    Not exactly 1 annotation: w_9200.jpg :2\r\n    Not exactly 1 annotation: w_1300.jpg :2\r\n    Not exactly 1 annotation: w_7077.jpg :0\r\n    Not exactly 1 annotation: w_4876.jpg :2\r\n\r\n    Not in labeled json but in training:[]\r\n    Not in train.csv but in labeled json['w_1300.jpg' 'w_4876.jpg' 'w_9200.jpg']\r\n\r\nWell, we can live with that!",
    "99092": "hi @visoft, good catch, for image with 2 whales, I just assume one is the 'main' whale that belongs to a class, but that's no guarantee... \r\nYou can visually check the annotations with sloth, as in @smerity 's thread."
  },
  "source": "meta"
}