{
  "id": 79524,
  "title": "Analysing the results: some difficult test cases",
  "url": "/competitions/humpback-whale-identification/discussion/79524",
  "author_name": "",
  "post_date": "2019-02-05T04:00:03.538194600Z",
  "votes": 21,
  "comment_count": 24,
  "views": 0,
  "content": "<p>here are some difficult tests. It gives you  an idea how to design your algorithm to correctly match these cases:</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466291/11174/diff1.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466291/11175/diff2.png\" alt=\"enter image description here\"></p>",
  "messages": [
    {
      "id": "466291",
      "postDate": "02/05/2019 04:00:03",
      "content": "<p>here are some difficult tests. It gives you  an idea how to design your algorithm to correctly match these cases:</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466291/11174/diff1.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466291/11175/diff2.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "here are some difficult tests. It gives you  an idea how to design your algorithm to correctly match these cases:\n\n  ![enter image description here][1]\n\n  ![enter image description here][2]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466291/11174/diff1.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/466291/11175/diff2.png",
      "votes": null
    },
    {
      "id": "466293",
      "postDate": "02/05/2019 04:08:50",
      "content": "<p>i am still tweaking my pipeline. but it think the following is promising:</p>\n\n<p>1)  train a 5004-class classifier (e.g. using heavy augmentation/alignment)\n2) retrieve say top 50 results\n3) use metric learning to refine the results. (on aligned images?). Metric learning will determine if the whale is \"new_whale\" or not. You can (and maybe should?) use classifier pretrained weights to initialise network for metric learning.</p>\n\n<p>The same pipleline is used for training as well (i.e.  classifier is used to mine hard negative samples for metric learning )\n I am thinking of how to do it end-to-end.</p>\n\n<p>Also how to use the test data as unlabelled data for training. if metric learning can improve results of classifier, they are give \"pseudo\" labels of test images)?</p>",
      "rawMarkdown": "i am still tweaking my pipeline. but it think the following is promising:\n\n1)  train a 5004-class classifier (e.g. using heavy augmentation/alignment)\n2) retrieve say top 50 results\n3) use metric learning to refine the results. (on aligned images?). Metric learning will determine if the whale is \"new_whale\" or not. You can (and maybe should?) use classifier pretrained weights to initialise network for metric learning.\n\nThe same pipleline is used for training as well (i.e.  classifier is used to mine hard negative samples for metric learning )\n I am thinking of how to do it end-to-end.\n\nAlso how to use the test data as unlabelled data for training. if metric learning can improve results of classifier, they are give \"pseudo\" labels of test images)?",
      "votes": null
    },
    {
      "id": "466297",
      "postDate": "02/05/2019 04:30:03",
      "content": "<ul>\n<li><p>the following are seem relevant to this challenge: one-shot learning, metric learning, image retrieval, multi-class classification.  All can be used, but one has to note the ease of training different systems. e.g. is it easier to train multi-classifier or one-shot classifier  or  distance predictor via metric learning?</p></li>\n<li><p>finally, combinations of techniques should work better, but what is the key principal method to use?</p></li>\n</ul>\n\n<p>But on a closer look, i note that:</p>\n\n<ol>\n<li>although some class has only one sample, but the variation can be effectively modeled. this is because whale fluke are planar, you can easily create massive train samples  (and these synthesized samples can be close to the test samples). In fact you can also do heavy augmentation on the test too.</li>\n</ol>\n\n<p>Hence it is \"not really a one sample problem\" , i.e. not really one-shot.</p>\n\n<ol>\n<li>The test samples must definitely in one of the 5004 train classes. hence it is not really metric learning. (in real metric learning, you test the distance of 2 given samples, \"both\" not seen in the train set).  Nevertheless, metric learning is useful to refine results since the metric is ranking based finally.</li>\n</ol>\n\n<p>Further if metric learning is used, is it sample-to-sample or sample-to-cluster? (each train id may have more than one images) if the image are not pose-aligned, sample-to-sample is difficult. I would say aligned sample-to-sample is the best in this case.</p>",
      "rawMarkdown": "- the following are seem relevant to this challenge: one-shot learning, metric learning, image retrieval, multi-class classification.  All can be used, but one has to note the ease of training different systems. e.g. is it easier to train multi-classifier or one-shot classifier  or  distance predictor via metric learning?\n\n- finally, combinations of techniques should work better, but what is the key principal method to use?\n\nBut on a closer look, i note that:\n\n1. although some class has only one sample, but the variation can be effectively modeled. this is because whale fluke are planar, you can easily create massive train samples  (and these synthesized samples can be close to the test samples). In fact you can also do heavy augmentation on the test too.\n\nHence it is \"not really a one sample problem\" , i.e. not really one-shot.\n\n2. The test samples must definitely in one of the 5004 train classes. hence it is not really metric learning. (in real metric learning, you test the distance of 2 given samples, \"both\" not seen in the train set).  Nevertheless, metric learning is useful to refine results since the metric is ranking based finally.\n\nFurther if metric learning is used, is it sample-to-sample or sample-to-cluster? (each train id may have more than one images) if the image are not pose-aligned, sample-to-sample is difficult. I would say aligned sample-to-sample is the best in this case.",
      "votes": null
    },
    {
      "id": "466299",
      "postDate": "02/05/2019 04:34:04",
      "content": "<p>Thank Heng, grate work</p>",
      "rawMarkdown": "Thank Heng, grate work",
      "votes": null
    },
    {
      "id": "466303",
      "postDate": "02/05/2019 04:44:00",
      "content": "<p>...</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466303/11176/diff3.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "...\n\n  ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466303/11176/diff3.png",
      "votes": null
    },
    {
      "id": "466308",
      "postDate": "02/05/2019 05:01:13",
      "content": "<p>check this image:</p>\n\n<p>Differences Between Softmax-based Classification and Metric Learning</p>\n\n<p><a href=\"https://www.groundai.com/media/arxiv_projects/325403/fig/fig2/fig2.jpg\">https://www.groundai.com/media/arxiv_projects/325403/fig/fig2/fig2.jpg</a></p>\n\n<p><a href=\"https://arxiv.org/abs/1712.10151v1\">https://arxiv.org/abs/1712.10151v1</a></p>",
      "rawMarkdown": "check this image:\n\nDifferences Between Softmax-based Classification and Metric Learning\n\nhttps://www.groundai.com/media/arxiv_projects/325403/fig/fig2/fig2.jpg\n\nhttps://arxiv.org/abs/1712.10151v1",
      "votes": null
    },
    {
      "id": "466387",
      "postDate": "02/05/2019 09:30:48",
      "content": "<p>...\n  <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466387/11179/diff4.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "...\n  ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466387/11179/diff4.png",
      "votes": null
    },
    {
      "id": "466400",
      "postDate": "02/05/2019 10:24:31",
      "content": "<p>...\n  <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466400/11181/diff5.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "...\n  ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466400/11181/diff5.png",
      "votes": null
    },
    {
      "id": "466430",
      "postDate": "02/05/2019 11:10:15",
      "content": "<p>Thanks Heng! May I ask how/what you use to analyse the results in so much detail?</p>",
      "rawMarkdown": "Thanks Heng! May I ask how/what you use to analyse the results in so much detail?",
      "votes": null
    },
    {
      "id": "466486",
      "postDate": "02/05/2019 13:32:09",
      "content": "<p>not all marks are present at all time ...</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466486/11182/diff6.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466486/11183/diff7.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "not all marks are present at all time ...\n\n  ![enter image description here][1]\n\n  ![enter image description here][2]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466486/11182/diff6.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/466486/11183/diff7.png",
      "votes": null
    },
    {
      "id": "466488",
      "postDate": "02/05/2019 13:34:40",
      "content": "<p>edges help! use this to decide to use tight or loss bounding box ...</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466488/11184/diff8.png\" alt=\"enter image description here\">\n   <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466488/11185/diff9.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "edges help! use this to decide to use tight or loss bounding box ...\n\n   ![enter image description here][1]\n   ![enter image description here][2]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466488/11184/diff8.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/466488/11185/diff9.png",
      "votes": null
    },
    {
      "id": "466533",
      "postDate": "02/05/2019 15:15:07",
      "content": "<p>based on your prediction scores, you can write a code to output the k-nearest neighbors (e.g. top 15 predicted whale id)</p>",
      "rawMarkdown": "based on your prediction scores, you can write a code to output the k-nearest neighbors (e.g. top 15 predicted whale id)",
      "votes": null
    },
    {
      "id": "466546",
      "postDate": "02/05/2019 15:42:08",
      "content": "<p>a couple of low resolution test images:</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466546/11186/diff10.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "a couple of low resolution test images:\n\n   ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466546/11186/diff10.png",
      "votes": null
    },
    {
      "id": "466941",
      "postDate": "02/06/2019 08:40:39",
      "content": "<p>Nice findings ! Blur filter can be used as the data augmetation which simulate these law resolution images.</p>",
      "rawMarkdown": "Nice findings ! Blur filter can be used as the data augmetation which simulate these law resolution images.",
      "votes": null
    },
    {
      "id": "467977",
      "postDate": "02/08/2019 04:20:36",
      "content": "<p>a couple of wrongly flipped test images</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/467977/11205/diff11.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "a couple of wrongly flipped test images\n\n   ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/467977/11205/diff11.png",
      "votes": null
    },
    {
      "id": "468200",
      "postDate": "02/08/2019 13:20:22",
      "content": "<p>This is amazing. I'm currently learning Data Science and by looking at this it seems the lecture will be more interesting.😊</p>",
      "rawMarkdown": "This is amazing. I'm currently learning Data Science and by looking at this it seems the lecture will be more interesting.😊",
      "votes": null
    },
    {
      "id": "468747",
      "postDate": "02/09/2019 15:26:46",
      "content": "<p>let's check the class activation map (class-wise feature heatmap) to see if the model is learning correctly or not!</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11227/cam1.png\" alt=\"enter image description here\">\n  <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11228/cam2.png\" alt=\"enter image description here\">\n <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11229/cam3.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "let's check the class activation map (class-wise feature heatmap) to see if the model is learning correctly or not!\n\n  ![enter image description here][1]\n  ![enter image description here][2]\n ![enter image description here][3]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11227/cam1.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11228/cam2.png\n  [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11229/cam3.png",
      "votes": null
    },
    {
      "id": "468862",
      "postDate": "02/09/2019 22:10:01",
      "content": "<p>I really must thank you for offering these peeks into some of the deeper and more sophisticated ways to interrogate these complex and (to a novice like me at least) confusing models. </p>\n\n<p>May I ask about those gray-scale grids (class activation map/class-wise feature heatmap as you put it)? Are those outputs from some layer in your model represented as a square grid? Are you sliding the red box across the images and seeing where activation is highest? Is this done manually? Or do you give the entire image and the red box is the location of the strongest activation? Thanks again!</p>",
      "rawMarkdown": "I really must thank you for offering these peeks into some of the deeper and more sophisticated ways to interrogate these complex and (to a novice like me at least) confusing models. \n\nMay I ask about those gray-scale grids (class activation map/class-wise feature heatmap as you put it)? Are those outputs from some layer in your model represented as a square grid? Are you sliding the red box across the images and seeing where activation is highest? Is this done manually? Or do you give the entire image and the red box is the location of the strongest activation? Thanks again!",
      "votes": null
    },
    {
      "id": "469447",
      "postDate": "02/11/2019 08:30:00",
      "content": "<p>they are the last feature map (1/32 scaled of input), based on this paper:\n<a href=\"http://cnnlocalization.csail.mit.edu/\">http://cnnlocalization.csail.mit.edu/</a></p>\n\n<p>this is an old paper. you can use grad-CAM for better results</p>",
      "rawMarkdown": "they are the last feature map (1/32 scaled of input), based on this paper:\nhttp://cnnlocalization.csail.mit.edu/\n\n\nthis is an old paper. you can use grad-CAM for better results",
      "votes": null
    },
    {
      "id": "469449",
      "postDate": "02/11/2019 08:33:57",
      "content": "<p>the network needs some imagination to project the shape to the correct view\n(there are at least two instances of such case in test)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/469449/11255/super05.png\" alt=\"enter image description here\"></p>\n\n<p>my feeling is that today, deep learning has good memorization. The next step towards is imagination, how to generate new instances of A after seeing many instances of B, assuming A and B have common transform .</p>\n\n<p>If a network  has imagination, then it can explore and that is when real intelligence comes .</p>",
      "rawMarkdown": "the network needs some imagination to project the shape to the correct view\n(there are at least two instances of such case in test)\n\n\n  ![enter image description here][1]\n\nmy feeling is that today, deep learning has good memorization. The next step towards is imagination, how to generate new instances of A after seeing many instances of B, assuming A and B have common transform .\n\nIf a network  has imagination, then it can explore and that is when real intelligence comes .\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/469449/11255/super05.png",
      "votes": null
    },
    {
      "id": "469451",
      "postDate": "02/11/2019 08:34:34",
      "content": "<p>Hi Heng, the model u used for class activation map visualization is trained on imgs without bbox cropping, right?</p>",
      "rawMarkdown": "Hi Heng, the model u used for class activation map visualization is trained on imgs without bbox cropping, right?",
      "votes": null
    },
    {
      "id": "469452",
      "postDate": "02/11/2019 08:38:50",
      "content": "<p>it is with box cropping. but my augmentation has resale up to 0.25</p>",
      "rawMarkdown": "it is with box cropping. but my augmentation has resale up to 0.25",
      "votes": null
    },
    {
      "id": "469520",
      "postDate": "02/11/2019 11:18:35",
      "content": "<p>Thanks!</p>",
      "rawMarkdown": "Thanks!",
      "votes": null
    },
    {
      "id": "474365",
      "postDate": "02/19/2019 09:12:01",
      "content": "<p>here are the cases you need to get correct if you want to have lb score &gt;0.900.</p>\n\n<p>Markings can be deceiving. You have to look at the edge too. Slide.3 is an example of this</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11349/Slide1.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11348/Slide3.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11352/Slide6.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11353/Slide4.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "here are the cases you need to get correct if you want to have lb score &gt;0.900.\n\nMarkings can be deceiving. You have to look at the edge too. Slide.3 is an example of this\n\n \n\n  ![enter image description here][1]\n\n  ![enter image description here][2]\n\n  ![enter image description here][3]\n\n  ![enter image description here][4]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11349/Slide1.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11348/Slide3.png\n  [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11352/Slide6.png\n  [4]: https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11353/Slide4.png",
      "votes": null
    },
    {
      "id": "474527",
      "postDate": "02/19/2019 14:07:48",
      "content": "<p>what a match!</p>\n\n<p>The trick is really to enlarge your image and \"look for one or two strong match\", which is often just a very small region.</p>\n\n<p>In fact for for a single image, you can create multiple crop of a high resolution image. Even if you have a single image per class, you actually have many samples (one sample for one distinct region)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11355/difficult_match.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11357/diffcult2.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11358/difficult1.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "what a match!\n\nThe trick is really to enlarge your image and \"look for one or two strong match\", which is often just a very small region.\n\nIn fact for for a single image, you can create multiple crop of a high resolution image. Even if you have a single image per class, you actually have many samples (one sample for one distinct region)\n\n  ![enter image description here][1]\n\n  ![enter image description here][2]\n\n  ![enter image description here][3]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11355/difficult_match.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11357/diffcult2.png\n  [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11358/difficult1.png",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 466293,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/05/2019 04:08:50",
      "content": "<p>i am still tweaking my pipeline. but it think the following is promising:</p>\n\n<p>1)  train a 5004-class classifier (e.g. using heavy augmentation/alignment)\n2) retrieve say top 50 results\n3) use metric learning to refine the results. (on aligned images?). Metric learning will determine if the whale is \"new_whale\" or not. You can (and maybe should?) use classifier pretrained weights to initialise network for metric learning.</p>\n\n<p>The same pipleline is used for training as well (i.e.  classifier is used to mine hard negative samples for metric learning )\n I am thinking of how to do it end-to-end.</p>\n\n<p>Also how to use the test data as unlabelled data for training. if metric learning can improve results of classifier, they are give \"pseudo\" labels of test images)?</p>",
      "votes": null,
      "replies": [
        {
          "id": 466299,
          "author_name": "hadxu123",
          "author_url": "",
          "post_date": "02/05/2019 04:34:04",
          "content": "<p>Thank Heng, grate work</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 466308,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "02/05/2019 05:01:13",
          "content": "<p>check this image:</p>\n\n<p>Differences Between Softmax-based Classification and Metric Learning</p>\n\n<p><a href=\"https://www.groundai.com/media/arxiv_projects/325403/fig/fig2/fig2.jpg\">https://www.groundai.com/media/arxiv_projects/325403/fig/fig2/fig2.jpg</a></p>\n\n<p><a href=\"https://arxiv.org/abs/1712.10151v1\">https://arxiv.org/abs/1712.10151v1</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 466297,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/05/2019 04:30:03",
      "content": "<ul>\n<li><p>the following are seem relevant to this challenge: one-shot learning, metric learning, image retrieval, multi-class classification.  All can be used, but one has to note the ease of training different systems. e.g. is it easier to train multi-classifier or one-shot classifier  or  distance predictor via metric learning?</p></li>\n<li><p>finally, combinations of techniques should work better, but what is the key principal method to use?</p></li>\n</ul>\n\n<p>But on a closer look, i note that:</p>\n\n<ol>\n<li>although some class has only one sample, but the variation can be effectively modeled. this is because whale fluke are planar, you can easily create massive train samples  (and these synthesized samples can be close to the test samples). In fact you can also do heavy augmentation on the test too.</li>\n</ol>\n\n<p>Hence it is \"not really a one sample problem\" , i.e. not really one-shot.</p>\n\n<ol>\n<li>The test samples must definitely in one of the 5004 train classes. hence it is not really metric learning. (in real metric learning, you test the distance of 2 given samples, \"both\" not seen in the train set).  Nevertheless, metric learning is useful to refine results since the metric is ranking based finally.</li>\n</ol>\n\n<p>Further if metric learning is used, is it sample-to-sample or sample-to-cluster? (each train id may have more than one images) if the image are not pose-aligned, sample-to-sample is difficult. I would say aligned sample-to-sample is the best in this case.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 466303,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/05/2019 04:44:00",
      "content": "<p>...</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466303/11176/diff3.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 466387,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/05/2019 09:30:48",
      "content": "<p>...\n  <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466387/11179/diff4.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 466400,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/05/2019 10:24:31",
      "content": "<p>...\n  <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466400/11181/diff5.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 466430,
      "author_name": "anjum48",
      "author_url": "",
      "post_date": "02/05/2019 11:10:15",
      "content": "<p>Thanks Heng! May I ask how/what you use to analyse the results in so much detail?</p>",
      "votes": null,
      "replies": [
        {
          "id": 466533,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "02/05/2019 15:15:07",
          "content": "<p>based on your prediction scores, you can write a code to output the k-nearest neighbors (e.g. top 15 predicted whale id)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 466486,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/05/2019 13:32:09",
      "content": "<p>not all marks are present at all time ...</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466486/11182/diff6.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466486/11183/diff7.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 466488,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/05/2019 13:34:40",
      "content": "<p>edges help! use this to decide to use tight or loss bounding box ...</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466488/11184/diff8.png\" alt=\"enter image description here\">\n   <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466488/11185/diff9.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 466546,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/05/2019 15:42:08",
      "content": "<p>a couple of low resolution test images:</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/466546/11186/diff10.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 466941,
          "author_name": "toshik",
          "author_url": "",
          "post_date": "02/06/2019 08:40:39",
          "content": "<p>Nice findings ! Blur filter can be used as the data augmetation which simulate these law resolution images.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 467977,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/08/2019 04:20:36",
      "content": "<p>a couple of wrongly flipped test images</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/467977/11205/diff11.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 468200,
      "author_name": "saurabhkawli",
      "author_url": "",
      "post_date": "02/08/2019 13:20:22",
      "content": "<p>This is amazing. I'm currently learning Data Science and by looking at this it seems the lecture will be more interesting.😊</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 468747,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/09/2019 15:26:46",
      "content": "<p>let's check the class activation map (class-wise feature heatmap) to see if the model is learning correctly or not!</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11227/cam1.png\" alt=\"enter image description here\">\n  <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11228/cam2.png\" alt=\"enter image description here\">\n <img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11229/cam3.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 469451,
          "author_name": "shentao",
          "author_url": "",
          "post_date": "02/11/2019 08:34:34",
          "content": "<p>Hi Heng, the model u used for class activation map visualization is trained on imgs without bbox cropping, right?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 469452,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "02/11/2019 08:38:50",
          "content": "<p>it is with box cropping. but my augmentation has resale up to 0.25</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 468862,
      "author_name": "interneuron",
      "author_url": "",
      "post_date": "02/09/2019 22:10:01",
      "content": "<p>I really must thank you for offering these peeks into some of the deeper and more sophisticated ways to interrogate these complex and (to a novice like me at least) confusing models. </p>\n\n<p>May I ask about those gray-scale grids (class activation map/class-wise feature heatmap as you put it)? Are those outputs from some layer in your model represented as a square grid? Are you sliding the red box across the images and seeing where activation is highest? Is this done manually? Or do you give the entire image and the red box is the location of the strongest activation? Thanks again!</p>",
      "votes": null,
      "replies": [
        {
          "id": 469447,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "02/11/2019 08:30:00",
          "content": "<p>they are the last feature map (1/32 scaled of input), based on this paper:\n<a href=\"http://cnnlocalization.csail.mit.edu/\">http://cnnlocalization.csail.mit.edu/</a></p>\n\n<p>this is an old paper. you can use grad-CAM for better results</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 469520,
          "author_name": "interneuron",
          "author_url": "",
          "post_date": "02/11/2019 11:18:35",
          "content": "<p>Thanks!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 469449,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/11/2019 08:33:57",
      "content": "<p>the network needs some imagination to project the shape to the correct view\n(there are at least two instances of such case in test)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/469449/11255/super05.png\" alt=\"enter image description here\"></p>\n\n<p>my feeling is that today, deep learning has good memorization. The next step towards is imagination, how to generate new instances of A after seeing many instances of B, assuming A and B have common transform .</p>\n\n<p>If a network  has imagination, then it can explore and that is when real intelligence comes .</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 474365,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/19/2019 09:12:01",
      "content": "<p>here are the cases you need to get correct if you want to have lb score &gt;0.900.</p>\n\n<p>Markings can be deceiving. You have to look at the edge too. Slide.3 is an example of this</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11349/Slide1.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11348/Slide3.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11352/Slide6.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11353/Slide4.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 474527,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/19/2019 14:07:48",
      "content": "<p>what a match!</p>\n\n<p>The trick is really to enlarge your image and \"look for one or two strong match\", which is often just a very small region.</p>\n\n<p>In fact for for a single image, you can create multiple crop of a high resolution image. Even if you have a single image per class, you actually have many samples (one sample for one distinct region)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11355/difficult_match.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11357/diffcult2.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11358/difficult1.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "466291": "here are some difficult tests. It gives you  an idea how to design your algorithm to correctly match these cases:\n\n  ![enter image description here][1]\n\n  ![enter image description here][2]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466291/11174/diff1.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/466291/11175/diff2.png",
    "466293": "i am still tweaking my pipeline. but it think the following is promising:\n\n1)  train a 5004-class classifier (e.g. using heavy augmentation/alignment)\n2) retrieve say top 50 results\n3) use metric learning to refine the results. (on aligned images?). Metric learning will determine if the whale is \"new_whale\" or not. You can (and maybe should?) use classifier pretrained weights to initialise network for metric learning.\n\nThe same pipleline is used for training as well (i.e.  classifier is used to mine hard negative samples for metric learning )\n I am thinking of how to do it end-to-end.\n\nAlso how to use the test data as unlabelled data for training. if metric learning can improve results of classifier, they are give \"pseudo\" labels of test images)?",
    "466297": "- the following are seem relevant to this challenge: one-shot learning, metric learning, image retrieval, multi-class classification.  All can be used, but one has to note the ease of training different systems. e.g. is it easier to train multi-classifier or one-shot classifier  or  distance predictor via metric learning?\n\n- finally, combinations of techniques should work better, but what is the key principal method to use?\n\nBut on a closer look, i note that:\n\n1. although some class has only one sample, but the variation can be effectively modeled. this is because whale fluke are planar, you can easily create massive train samples  (and these synthesized samples can be close to the test samples). In fact you can also do heavy augmentation on the test too.\n\nHence it is \"not really a one sample problem\" , i.e. not really one-shot.\n\n2. The test samples must definitely in one of the 5004 train classes. hence it is not really metric learning. (in real metric learning, you test the distance of 2 given samples, \"both\" not seen in the train set).  Nevertheless, metric learning is useful to refine results since the metric is ranking based finally.\n\nFurther if metric learning is used, is it sample-to-sample or sample-to-cluster? (each train id may have more than one images) if the image are not pose-aligned, sample-to-sample is difficult. I would say aligned sample-to-sample is the best in this case.",
    "466299": "Thank Heng, grate work",
    "466303": "...\n\n  ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466303/11176/diff3.png",
    "466308": "check this image:\n\nDifferences Between Softmax-based Classification and Metric Learning\n\nhttps://www.groundai.com/media/arxiv_projects/325403/fig/fig2/fig2.jpg\n\nhttps://arxiv.org/abs/1712.10151v1",
    "466387": "...\n  ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466387/11179/diff4.png",
    "466400": "...\n  ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466400/11181/diff5.png",
    "466430": "Thanks Heng! May I ask how/what you use to analyse the results in so much detail?",
    "466486": "not all marks are present at all time ...\n\n  ![enter image description here][1]\n\n  ![enter image description here][2]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466486/11182/diff6.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/466486/11183/diff7.png",
    "466488": "edges help! use this to decide to use tight or loss bounding box ...\n\n   ![enter image description here][1]\n   ![enter image description here][2]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466488/11184/diff8.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/466488/11185/diff9.png",
    "466533": "based on your prediction scores, you can write a code to output the k-nearest neighbors (e.g. top 15 predicted whale id)",
    "466546": "a couple of low resolution test images:\n\n   ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/466546/11186/diff10.png",
    "466941": "Nice findings ! Blur filter can be used as the data augmetation which simulate these law resolution images.",
    "467977": "a couple of wrongly flipped test images\n\n   ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/467977/11205/diff11.png",
    "468200": "This is amazing. I'm currently learning Data Science and by looking at this it seems the lecture will be more interesting.😊",
    "468747": "let's check the class activation map (class-wise feature heatmap) to see if the model is learning correctly or not!\n\n  ![enter image description here][1]\n  ![enter image description here][2]\n ![enter image description here][3]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11227/cam1.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11228/cam2.png\n  [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/468747/11229/cam3.png",
    "468862": "I really must thank you for offering these peeks into some of the deeper and more sophisticated ways to interrogate these complex and (to a novice like me at least) confusing models. \n\nMay I ask about those gray-scale grids (class activation map/class-wise feature heatmap as you put it)? Are those outputs from some layer in your model represented as a square grid? Are you sliding the red box across the images and seeing where activation is highest? Is this done manually? Or do you give the entire image and the red box is the location of the strongest activation? Thanks again!",
    "469447": "they are the last feature map (1/32 scaled of input), based on this paper:\nhttp://cnnlocalization.csail.mit.edu/\n\n\nthis is an old paper. you can use grad-CAM for better results",
    "469449": "the network needs some imagination to project the shape to the correct view\n(there are at least two instances of such case in test)\n\n\n  ![enter image description here][1]\n\nmy feeling is that today, deep learning has good memorization. The next step towards is imagination, how to generate new instances of A after seeing many instances of B, assuming A and B have common transform .\n\nIf a network  has imagination, then it can explore and that is when real intelligence comes .\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/469449/11255/super05.png",
    "469451": "Hi Heng, the model u used for class activation map visualization is trained on imgs without bbox cropping, right?",
    "469452": "it is with box cropping. but my augmentation has resale up to 0.25",
    "469520": "Thanks!",
    "474365": "here are the cases you need to get correct if you want to have lb score &gt;0.900.\n\nMarkings can be deceiving. You have to look at the edge too. Slide.3 is an example of this\n\n \n\n  ![enter image description here][1]\n\n  ![enter image description here][2]\n\n  ![enter image description here][3]\n\n  ![enter image description here][4]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11349/Slide1.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11348/Slide3.png\n  [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11352/Slide6.png\n  [4]: https://storage.googleapis.com/kaggle-forum-message-attachments/474365/11353/Slide4.png",
    "474527": "what a match!\n\nThe trick is really to enlarge your image and \"look for one or two strong match\", which is often just a very small region.\n\nIn fact for for a single image, you can create multiple crop of a high resolution image. Even if you have a single image per class, you actually have many samples (one sample for one distinct region)\n\n  ![enter image description here][1]\n\n  ![enter image description here][2]\n\n  ![enter image description here][3]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11355/difficult_match.png\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11357/diffcult2.png\n  [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/474527/11358/difficult1.png"
  },
  "source": "meta"
}