{
  "id": 67286,
  "title": "Incorrect masks",
  "url": "/competitions/airbus-ship-detection/discussion/67286",
  "author_name": "Iafoss",
  "post_date": "2018-10-01T02:51:42.727000",
  "votes": 11,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Greetings,</p>\n\n<p>When I performed error analysis, I found that some \"ground truth\" masks are quite off. Below I put several examples. The 1-st and 3-d rows are the full size images, the 2-nd and 4-th ones are zoomed in parts with ships. The first column contains original images, the second one contains the \"ground truth\" mask, and the last one shows the prediction of the model. The green boxes outline the positions of \"ground truth\" masks.  The image names are 5d4992896.jpg and 59c2b8090.jpg.\n<img src=\"https://image.ibb.co/i9SBmz/download_9.png\" alt=\"enter image description here\"><img src=\"https://image.ibb.co/bCn0eK/download_10.png\" alt=\"enter image description here\"></p>\n\n<p>The above examples are the most pronounced ones I found after checking several predictions with low score. However, it is quite common to see something like the following image, when the \"ground truth\" mask (green box) is quite tilted, and therefore the prediction of the model (red box) cannot get high score even if it fits the ship much better.\n<img src=\"https://image.ibb.co/gJtKnz/ship_detection_box.png\" alt=\"enter image description here\"></p>",
  "messages": [
    {
      "id": 396576,
      "postDate": "2018-10-01T02:51:42.727Z",
      "content": "<p>Greetings,</p>\n\n<p>When I performed error analysis, I found that some \"ground truth\" masks are quite off. Below I put several examples. The 1-st and 3-d rows are the full size images, the 2-nd and 4-th ones are zoomed in parts with ships. The first column contains original images, the second one contains the \"ground truth\" mask, and the last one shows the prediction of the model. The green boxes outline the positions of \"ground truth\" masks.  The image names are 5d4992896.jpg and 59c2b8090.jpg.\n<img src=\"https://image.ibb.co/i9SBmz/download_9.png\" alt=\"enter image description here\"><img src=\"https://image.ibb.co/bCn0eK/download_10.png\" alt=\"enter image description here\"></p>\n\n<p>The above examples are the most pronounced ones I found after checking several predictions with low score. However, it is quite common to see something like the following image, when the \"ground truth\" mask (green box) is quite tilted, and therefore the prediction of the model (red box) cannot get high score even if it fits the ship much better.\n<img src=\"https://image.ibb.co/gJtKnz/ship_detection_box.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "Greetings,\n\nWhen I performed error analysis, I found that some \"ground truth\" masks are quite off. Below I put several examples. The 1-st and 3-d rows are the full size images, the 2-nd and 4-th ones are zoomed in parts with ships. The first column contains original images, the second one contains the \"ground truth\" mask, and the last one shows the prediction of the model. The green boxes outline the positions of \"ground truth\" masks.  The image names are 5d4992896.jpg and 59c2b8090.jpg.\n![enter image description here][1]![enter image description here][2]\n\nThe above examples are the most pronounced ones I found after checking several predictions with low score. However, it is quite common to see something like the following image, when the \"ground truth\" mask (green box) is quite tilted, and therefore the prediction of the model (red box) cannot get high score even if it fits the ship much better.\n![enter image description here][3]\n\n\n  [1]: https://image.ibb.co/i9SBmz/download_9.png\n  [2]: https://image.ibb.co/bCn0eK/download_10.png\n  [3]: https://image.ibb.co/gJtKnz/ship_detection_box.png",
      "votes": 10
    },
    {
      "id": 406705,
      "postDate": "2018-10-19T17:30:07.013Z",
      "content": "<p>Hi!\nI am encountering the same issue!</p>",
      "rawMarkdown": "Hi!\nI am encountering the same issue!\n\n",
      "votes": 1
    },
    {
      "id": 398018,
      "postDate": "2018-10-03T12:50:34.913Z",
      "content": "<p>Thanks for this analysis, I think it is a huge problem and I'm surprised that nobody answer to you despite you show it in another thread long time ago.</p>",
      "rawMarkdown": "Thanks for this analysis, I think it is a huge problem and I'm surprised that nobody answer to you despite you show it in another thread long time ago.",
      "votes": 2,
      "replies": [
        {
          "id": 398138,
          "postDate": "2018-10-03T15:01:41.140Z",
          "content": "<p>I just afraid that the new test set will contain similar problems( In this case Unet based solutions will be penalized even more(  In addition to the prediction the mask instead of pixelized rotating bounding boxes and splitting the single mask into individual ones, some boxes are also shifted, and it may be difficult to train Unet to do the same shifting as in the training/test dataset since the predicted mask is strongly connected to particular pixels in the input image(</p>",
          "rawMarkdown": "I just afraid that the new test set will contain similar problems( In this case Unet based solutions will be penalized even more(  In addition to the prediction the mask instead of pixelized rotating bounding boxes and splitting the single mask into individual ones, some boxes are also shifted, and it may be difficult to train Unet to do the same shifting as in the training/test dataset since the predicted mask is strongly connected to particular pixels in the input image("
        }
      ]
    },
    {
      "id": 398781,
      "postDate": "2018-10-04T15:23:41.070Z",
      "content": "<p>Several additional examples of poor masks were posted in <a href=\"https://www.kaggle.com/c/airbus-ship-detection/discussion/64730\">https://www.kaggle.com/c/airbus-ship-detection/discussion/64730</a> by Rüdiger Jungbeck.</p>",
      "rawMarkdown": "Several additional examples of poor masks were posted in https://www.kaggle.com/c/airbus-ship-detection/discussion/64730 by Rüdiger Jungbeck."
    },
    {
      "id": 400915,
      "postDate": "2018-10-09T06:23:59.670Z",
      "content": "<p>Thanks for this.  Note that there are also some wholly mislabelled images.  See the attachment for an example.  (The second column is ground-truth.)</p>",
      "rawMarkdown": "Thanks for this.  Note that there are also some wholly mislabelled images.  See the attachment for an example.  (The second column is ground-truth.)",
      "votes": 1,
      "isDeleted": true,
      "replies": [
        {
          "id": 400933,
          "postDate": "2018-10-09T06:45:43.303Z",
          "content": "<p>I saw several images where my model recognized small ships not present in the \"ground truth\" masks, but they were really tiny. You example is much clearer.</p>",
          "rawMarkdown": "I saw several images where my model recognized small ships not present in the \"ground truth\" masks, but they were really tiny. You example is much clearer."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 406705,
      "author_name": "Borys Tymchenko",
      "author_url": "",
      "post_date": "2018-10-19T17:30:07.013000",
      "content": "<p>Hi!\nI am encountering the same issue!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 398018,
      "author_name": "Benoit Courty",
      "author_url": "",
      "post_date": "2018-10-03T12:50:34.913000",
      "content": "<p>Thanks for this analysis, I think it is a huge problem and I'm surprised that nobody answer to you despite you show it in another thread long time ago.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 398138,
          "author_name": "Iafoss",
          "author_url": "",
          "post_date": "2018-10-03T15:01:41.140000",
          "content": "<p>I just afraid that the new test set will contain similar problems( In this case Unet based solutions will be penalized even more(  In addition to the prediction the mask instead of pixelized rotating bounding boxes and splitting the single mask into individual ones, some boxes are also shifted, and it may be difficult to train Unet to do the same shifting as in the training/test dataset since the predicted mask is strongly connected to particular pixels in the input image(</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 398781,
      "author_name": "Iafoss",
      "author_url": "",
      "post_date": "2018-10-04T15:23:41.070000",
      "content": "<p>Several additional examples of poor masks were posted in <a href=\"https://www.kaggle.com/c/airbus-ship-detection/discussion/64730\">https://www.kaggle.com/c/airbus-ship-detection/discussion/64730</a> by Rüdiger Jungbeck.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 400915,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-10-09T06:23:59.670000",
      "content": "<p>Thanks for this.  Note that there are also some wholly mislabelled images.  See the attachment for an example.  (The second column is ground-truth.)</p>",
      "votes": 1,
      "replies": [
        {
          "id": 400933,
          "author_name": "Iafoss",
          "author_url": "",
          "post_date": "2018-10-09T06:45:43.303000",
          "content": "<p>I saw several images where my model recognized small ships not present in the \"ground truth\" masks, but they were really tiny. You example is much clearer.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "396576": "Greetings,\n\nWhen I performed error analysis, I found that some \"ground truth\" masks are quite off. Below I put several examples. The 1-st and 3-d rows are the full size images, the 2-nd and 4-th ones are zoomed in parts with ships. The first column contains original images, the second one contains the \"ground truth\" mask, and the last one shows the prediction of the model. The green boxes outline the positions of \"ground truth\" masks.  The image names are 5d4992896.jpg and 59c2b8090.jpg.\n![enter image description here][1]![enter image description here][2]\n\nThe above examples are the most pronounced ones I found after checking several predictions with low score. However, it is quite common to see something like the following image, when the \"ground truth\" mask (green box) is quite tilted, and therefore the prediction of the model (red box) cannot get high score even if it fits the ship much better.\n![enter image description here][3]\n\n\n  [1]: https://image.ibb.co/i9SBmz/download_9.png\n  [2]: https://image.ibb.co/bCn0eK/download_10.png\n  [3]: https://image.ibb.co/gJtKnz/ship_detection_box.png",
    "406705": "Hi!\nI am encountering the same issue!\n\n",
    "398018": "Thanks for this analysis, I think it is a huge problem and I'm surprised that nobody answer to you despite you show it in another thread long time ago.",
    "398781": "Several additional examples of poor masks were posted in https://www.kaggle.com/c/airbus-ship-detection/discussion/64730 by Rüdiger Jungbeck.",
    "400915": "Thanks for this.  Note that there are also some wholly mislabelled images.  See the attachment for an example.  (The second column is ground-truth.)"
  }
}