{
  "id": 71595,
  "title": "9th place solution",
  "url": "/competitions/airbus-ship-detection/writeups/kensho-mosaic-9th-place-solution",
  "author_name": "",
  "post_date": "2018-11-15T02:27:28.730Z",
  "votes": 55,
  "comment_count": 24,
  "views": 0,
  "content": "<p>Congratulations to the winners, especially Victor and Selim of topcoders, whose 2018 DSB winning <a href=\"https://github.com/selimsef/dsb2018_topcoders\">solution</a> we adopted for this competition.</p>\n\n<h2>Finding overlap</h2>\n\n<p>Due to the overlapping issue, we wrote an algorithm to find all the overlapping pairs of images in train set, then ran the <a href=\"https://leetcode.com/problems/friend-circles/description/\">friend circles</a> algorithm to find all the 57877 groups of images. Images in each group is supposedly from the same acquisition and images from different groups do not overlap.</p>\n\n<p>We then did the train/val split to have 2920 groups (10442 images) in val set while making sure train and val have same distribution of number of ships per image. We did this split before test_v2 was released and used it for the majority of the completion but switched to a balanced 4 fold towards the end which have distribution (in ship number and ship size) closer to test set.</p>\n\n<h2>Binary classification model</h2>\n\n<p>An ensemble of fastai resnet34 and resnext50 models trained on our split based on Iafoss’s <a href=\"https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook\">kernel</a>.</p>\n\n<h2>Labels</h2>\n\n<p>We used topcoder’s dsb2018 winning solution code to generate contour (i.e. divider) layer and used 3 channel labels: ship, contour and background. Contour layers are generated in order to help the model separating close ships. An example:\n<img src=\"https://i.imgur.com/KanhOpD.png\" alt=\"label\"></p>\n\n<h2>Model</h2>\n\n<p>Best model is Unet densenet169 implemented in Keras by topcoders. We also tried resnet152, resnet101, inception resnet v2</p>\n\n<h2>Loss</h2>\n\n<p>Double head loss (i.e. half binary cross entropy, half dice loss) for ship layer. For contour layer, we tried both double head loss and only dice loss since the pixels are too imbalanced.</p>\n\n<h2>Input size</h2>\n\n<p>512 and 768</p>\n\n<h2>Thresholds</h2>\n\n<ol>\n<li>Best classifier threshold is 0.5–0.7 depending on the model. Images below this threshold have empty prediction in submission; images above this threshold go to Unet model which may still end up having empty predictions from Unet. </li>\n<li>Unet pixelwise threshold\nWe noticed that for all our models, best pixelwise threshold on local validation (usually around 0.5) is never the best on public LB. After some simulations and LB probing, we highly suspected discrepancy between train and test label standards: test set masks are more “tight”, i.e. higher threshold on test set (0.6-0.8 depending on model) works out better on public LB. This turned out to  be the case just for the public LB.</li>\n</ol>\n\n<h2>Training techniques</h2>\n\n<p>All the standard ones: data augmentation, cyclic learning rates, TTA</p>\n\n<h2>Postprocessing (mostly using skimage)</h2>\n\n<ul>\n<li>Algo to separate \"weakly connected\" masks, with erosion and watershed. We start from an intuition that generally masks are rectangular and when the model predicted ship boundaries are 'weak', they touch along the longer side of rectangles. To separate this case, we first create a thin rectangular structuring element that is aligned with the major axis orientation of the original mask. Applying erosion with this structuring element will help separate the boundary. Since erosion reduces the pixels from the original mask, we try to make this up by applying a watershed algorithm on top of erosion. Example:\n<img src=\"https://i.imgur.com/rYRXeLQ.jpg\" alt=\"weakly\"></li>\n<li>to cut corners (probing the possibility that test set labels may be made as actual ship shape instead of rectangles) </li>\n<li>to “rectanglize” unet output masks by finding minimal bounding box and shrink it to same size</li>\n</ul>\n\n<h2>Final models</h2>\n\n<p>Best one in local validation and best on public LB. The best on public LB severely overfitted, like quite some teams did. The best one in local validation turned out to be 9th place. </p>\n\n<h2>Other things tried</h2>\n\n<ul>\n<li>Ensembling different models by averaging predicted probability matrixes – not working well </li>\n<li>Combining different models based on number of ships per image and ship size (some models work better for certain images or ships) – may overfit public LB</li>\n<li>MaskRcnn – much worse than unet</li>\n<li>Adding ships predicted in one model to the results of another model – slightly improved but too complicated</li>\n</ul>\n\n<h2>Team and hardwares</h2>\n\n<p>We are full time data scientists and coworkers. We don’t do image tasks at our job. This is the first deep learning competition for most of the team members. We teamed up very early. We have 6 Nvidia 1080Ti or equivalent and 2 smaller ones</p>\n\n<h2>Questions for other top teams</h2>\n\n<ul>\n<li>How do you deal with competitions like this where the public LB is too small and delta between local CV and public LB too unstable? Do you always faithfully trust your local CV no matter what? </li>\n<li>What if the delta is because of difference in distribution (or even labeling standards as discussed <a href=\"https://www.kaggle.com/c/airbus-ship-detection/discussion/70221\">here</a>) between train and test sets? </li>\n<li>I noticed bestfitting submitted many times, so there's must be some value in public LB?</li>\n</ul>",
  "messages": [
    {
      "id": "421431",
      "postDate": "11/15/2018 02:07:57",
      "content": "<p>Congratulations to the winners, especially Victor and Selim of topcoders, whose 2018 DSB winning <a href=\"https://github.com/selimsef/dsb2018_topcoders\">solution</a> we adopted for this competition.</p>\n\n<h2>Finding overlap</h2>\n\n<p>Due to the overlapping issue, we wrote an algorithm to find all the overlapping pairs of images in train set, then ran the <a href=\"https://leetcode.com/problems/friend-circles/description/\">friend circles</a> algorithm to find all the 57877 groups of images. Images in each group is supposedly from the same acquisition and images from different groups do not overlap.</p>\n\n<p>We then did the train/val split to have 2920 groups (10442 images) in val set while making sure train and val have same distribution of number of ships per image. We did this split before test_v2 was released and used it for the majority of the completion but switched to a balanced 4 fold towards the end which have distribution (in ship number and ship size) closer to test set.</p>\n\n<h2>Binary classification model</h2>\n\n<p>An ensemble of fastai resnet34 and resnext50 models trained on our split based on Iafoss’s <a href=\"https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook\">kernel</a>.</p>\n\n<h2>Labels</h2>\n\n<p>We used topcoder’s dsb2018 winning solution code to generate contour (i.e. divider) layer and used 3 channel labels: ship, contour and background. Contour layers are generated in order to help the model separating close ships. An example:\n<img src=\"https://i.imgur.com/KanhOpD.png\" alt=\"label\"></p>\n\n<h2>Model</h2>\n\n<p>Best model is Unet densenet169 implemented in Keras by topcoders. We also tried resnet152, resnet101, inception resnet v2</p>\n\n<h2>Loss</h2>\n\n<p>Double head loss (i.e. half binary cross entropy, half dice loss) for ship layer. For contour layer, we tried both double head loss and only dice loss since the pixels are too imbalanced.</p>\n\n<h2>Input size</h2>\n\n<p>512 and 768</p>\n\n<h2>Thresholds</h2>\n\n<ol>\n<li>Best classifier threshold is 0.5–0.7 depending on the model. Images below this threshold have empty prediction in submission; images above this threshold go to Unet model which may still end up having empty predictions from Unet. </li>\n<li>Unet pixelwise threshold\nWe noticed that for all our models, best pixelwise threshold on local validation (usually around 0.5) is never the best on public LB. After some simulations and LB probing, we highly suspected discrepancy between train and test label standards: test set masks are more “tight”, i.e. higher threshold on test set (0.6-0.8 depending on model) works out better on public LB. This turned out to  be the case just for the public LB.</li>\n</ol>\n\n<h2>Training techniques</h2>\n\n<p>All the standard ones: data augmentation, cyclic learning rates, TTA</p>\n\n<h2>Postprocessing (mostly using skimage)</h2>\n\n<ul>\n<li>Algo to separate \"weakly connected\" masks, with erosion and watershed. We start from an intuition that generally masks are rectangular and when the model predicted ship boundaries are 'weak', they touch along the longer side of rectangles. To separate this case, we first create a thin rectangular structuring element that is aligned with the major axis orientation of the original mask. Applying erosion with this structuring element will help separate the boundary. Since erosion reduces the pixels from the original mask, we try to make this up by applying a watershed algorithm on top of erosion. Example:\n<img src=\"https://i.imgur.com/rYRXeLQ.jpg\" alt=\"weakly\"></li>\n<li>to cut corners (probing the possibility that test set labels may be made as actual ship shape instead of rectangles) </li>\n<li>to “rectanglize” unet output masks by finding minimal bounding box and shrink it to same size</li>\n</ul>\n\n<h2>Final models</h2>\n\n<p>Best one in local validation and best on public LB. The best on public LB severely overfitted, like quite some teams did. The best one in local validation turned out to be 9th place. </p>\n\n<h2>Other things tried</h2>\n\n<ul>\n<li>Ensembling different models by averaging predicted probability matrixes – not working well </li>\n<li>Combining different models based on number of ships per image and ship size (some models work better for certain images or ships) – may overfit public LB</li>\n<li>MaskRcnn – much worse than unet</li>\n<li>Adding ships predicted in one model to the results of another model – slightly improved but too complicated</li>\n</ul>\n\n<h2>Team and hardwares</h2>\n\n<p>We are full time data scientists and coworkers. We don’t do image tasks at our job. This is the first deep learning competition for most of the team members. We teamed up very early. We have 6 Nvidia 1080Ti or equivalent and 2 smaller ones</p>\n\n<h2>Questions for other top teams</h2>\n\n<ul>\n<li>How do you deal with competitions like this where the public LB is too small and delta between local CV and public LB too unstable? Do you always faithfully trust your local CV no matter what? </li>\n<li>What if the delta is because of difference in distribution (or even labeling standards as discussed <a href=\"https://www.kaggle.com/c/airbus-ship-detection/discussion/70221\">here</a>) between train and test sets? </li>\n<li>I noticed bestfitting submitted many times, so there's must be some value in public LB?</li>\n</ul>",
      "rawMarkdown": "Congratulations to the winners, especially Victor and Selim of topcoders, whose 2018 DSB winning [solution][1] we adopted for this competition.\n\n## Finding overlap\nDue to the overlapping issue, we wrote an algorithm to find all the overlapping pairs of images in train set, then ran the [friend circles][2] algorithm to find all the 57877 groups of images. Images in each group is supposedly from the same acquisition and images from different groups do not overlap.\n\nWe then did the train/val split to have 2920 groups (10442 images) in val set while making sure train and val have same distribution of number of ships per image. We did this split before test_v2 was released and used it for the majority of the completion but switched to a balanced 4 fold towards the end which have distribution (in ship number and ship size) closer to test set.\n\n## Binary classification model\nAn ensemble of fastai resnet34 and resnext50 models trained on our split based on Iafoss’s [kernel][3].\n\n## Labels\nWe used topcoder’s dsb2018 winning solution code to generate contour (i.e. divider) layer and used 3 channel labels: ship, contour and background. Contour layers are generated in order to help the model separating close ships. An example:\n![label][4]\n\n## Model\nBest model is Unet densenet169 implemented in Keras by topcoders. We also tried resnet152, resnet101, inception resnet v2\n\n## Loss\nDouble head loss (i.e. half binary cross entropy, half dice loss) for ship layer. For contour layer, we tried both double head loss and only dice loss since the pixels are too imbalanced.\n\n## Input size\n512 and 768\n\n## Thresholds\n1. Best classifier threshold is 0.5–0.7 depending on the model. Images below this threshold have empty prediction in submission; images above this threshold go to Unet model which may still end up having empty predictions from Unet. \n2. Unet pixelwise threshold\nWe noticed that for all our models, best pixelwise threshold on local validation (usually around 0.5) is never the best on public LB. After some simulations and LB probing, we highly suspected discrepancy between train and test label standards: test set masks are more “tight”, i.e. higher threshold on test set (0.6-0.8 depending on model) works out better on public LB. This turned out to  be the case just for the public LB.\n\n## Training techniques\n All the standard ones: data augmentation, cyclic learning rates, TTA\n\n## Postprocessing (mostly using skimage)\n-\tAlgo to separate \"weakly connected\" masks, with erosion and watershed. We start from an intuition that generally masks are rectangular and when the model predicted ship boundaries are 'weak', they touch along the longer side of rectangles. To separate this case, we first create a thin rectangular structuring element that is aligned with the major axis orientation of the original mask. Applying erosion with this structuring element will help separate the boundary. Since erosion reduces the pixels from the original mask, we try to make this up by applying a watershed algorithm on top of erosion. Example:\n![weakly][5]\n-\tto cut corners (probing the possibility that test set labels may be made as actual ship shape instead of rectangles) \n-\tto “rectanglize” unet output masks by finding minimal bounding box and shrink it to same size\n\n## Final models\nBest one in local validation and best on public LB. The best on public LB severely overfitted, like quite some teams did. The best one in local validation turned out to be 9th place. \n\n## Other things tried\n-\tEnsembling different models by averaging predicted probability matrixes – not working well \n-\tCombining different models based on number of ships per image and ship size (some models work better for certain images or ships) – may overfit public LB\n-\tMaskRcnn – much worse than unet\n-\tAdding ships predicted in one model to the results of another model – slightly improved but too complicated\n\n## Team and hardwares\nWe are full time data scientists and coworkers. We don’t do image tasks at our job. This is the first deep learning competition for most of the team members. We teamed up very early. We have 6 Nvidia 1080Ti or equivalent and 2 smaller ones\n\n\n## Questions for other top teams\n- How do you deal with competitions like this where the public LB is too small and delta between local CV and public LB too unstable? Do you always faithfully trust your local CV no matter what? \n- What if the delta is because of difference in distribution (or even labeling standards as discussed [here][6]) between train and test sets? \n- I noticed bestfitting submitted many times, so there's must be some value in public LB?\n\n\n  [1]: https://github.com/selimsef/dsb2018_topcoders\n  [2]: https://leetcode.com/problems/friend-circles/description/\n  [3]: https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook\n  [4]: https://i.imgur.com/KanhOpD.png\n  [5]: https://i.imgur.com/rYRXeLQ.jpg\n  [6]: https://www.kaggle.com/c/airbus-ship-detection/discussion/70221",
      "votes": null
    },
    {
      "id": "421435",
      "postDate": "11/15/2018 02:18:17",
      "content": "<p>Nice summary! Thanks a lot!</p>",
      "rawMarkdown": "Nice summary! Thanks a lot!",
      "votes": null
    },
    {
      "id": "421436",
      "postDate": "11/15/2018 02:22:57",
      "content": "<p>congratulations to the team! this is a solid and great work!</p>",
      "rawMarkdown": "congratulations to the team! this is a solid and great work!",
      "votes": null
    },
    {
      "id": "421456",
      "postDate": "11/15/2018 03:14:21",
      "content": "<p>wow 3 channel label approach seems so neat! wonderful!</p>",
      "rawMarkdown": "wow 3 channel label approach seems so neat! wonderful!",
      "votes": null
    },
    {
      "id": "421467",
      "postDate": "11/15/2018 03:38:21",
      "content": "<p>Congrats on your great work and idea on 3 channels for ship, contour and background!</p>",
      "rawMarkdown": "Congrats on your great work and idea on 3 channels for ship, contour and background!",
      "votes": null
    },
    {
      "id": "421470",
      "postDate": "11/15/2018 03:43:20",
      "content": "<p>Congratulation!! 3 channel label approach is amazing\nGreat Work ! Thanks your share</p>",
      "rawMarkdown": "Congratulation!! 3 channel label approach is amazing\nGreat Work ! Thanks your share",
      "votes": null
    },
    {
      "id": "421503",
      "postDate": "11/15/2018 04:37:12",
      "content": "<p>Congratulations @Bo and team. Thanks for sharing your solution.</p>\n\n<p>I do not have a GPU machine and relied on Kaggle kernel and a cloud account for competitions. In this one I must admit I was discouraged by the leak earlier on so decided not to spend too much time on it. I did something similar to what @Lafoss did in <a href=\"https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook\">this notebook</a> before the leak. So I adapted it to the new data and submitted.</p>",
      "rawMarkdown": "Congratulations @Bo and team. Thanks for sharing your solution.\n\nI do not have a GPU machine and relied on Kaggle kernel and a cloud account for competitions. In this one I must admit I was discouraged by the leak earlier on so decided not to spend too much time on it. I did something similar to what @Lafoss did in [this notebook][1] before the leak. So I adapted it to the new data and submitted.\n\n\n  [1]: https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook",
      "votes": null
    },
    {
      "id": "421687",
      "postDate": "11/15/2018 09:12:04",
      "content": "<p>Congratulations and thank you for sharing your strategy! I really like the idea of three channels labels</p>",
      "rawMarkdown": "Congratulations and thank you for sharing your strategy! I really like the idea of three channels labels",
      "votes": null
    },
    {
      "id": "421819",
      "postDate": "11/15/2018 13:05:08",
      "content": "<p>Awesome! thanks for sharing!</p>",
      "rawMarkdown": "Awesome! thanks for sharing!",
      "votes": null
    },
    {
      "id": "421824",
      "postDate": "11/15/2018 13:12:20",
      "content": "<p>Congrats, Thanks for sharing your ideas, I learnt about applying the 3 channel label from your notes and it is interesting..</p>",
      "rawMarkdown": "Congrats, Thanks for sharing your ideas, I learnt about applying the 3 channel label from your notes and it is interesting..",
      "votes": null
    },
    {
      "id": "422015",
      "postDate": "11/15/2018 17:11:37",
      "content": "<p>hi could u throw some more light on this..\nused 3 channel labels: ship, contour and background. Contour layers are generated in order to help the model separating close ships. An example:\nHow is this achieved any pointer ?</p>\n\n<p>2) What was ensembling architecture FA resnet-&gt;resnet 101,m unable to visualize it</p>",
      "rawMarkdown": "hi could u throw some more light on this..\nused 3 channel labels: ship, contour and background. Contour layers are generated in order to help the model separating close ships. An example:\nHow is this achieved any pointer ?\n\n2) What was ensembling architecture FA resnet-&gt;resnet 101,m unable to visualize it",
      "votes": null
    },
    {
      "id": "422120",
      "postDate": "11/15/2018 20:04:06",
      "content": "<p>Thanks Terence. Hope you have better luck with private LB next time!</p>",
      "rawMarkdown": "Thanks Terence. Hope you have better luck with private LB next time!",
      "votes": null
    },
    {
      "id": "422121",
      "postDate": "11/15/2018 20:04:33",
      "content": "<p>Thanks Ray</p>",
      "rawMarkdown": "Thanks Ray",
      "votes": null
    },
    {
      "id": "422125",
      "postDate": "11/15/2018 20:07:33",
      "content": "<p>We haven't cleaned up the code yet, but you can check out topcoder's implementation for a similar task for a previous competition: <a href=\"https://github.com/selimsef/dsb2018_topcoders/blob/master/victor/create_masks.py\">https://github.com/selimsef/dsb2018_topcoders/blob/master/victor/create_masks.py</a></p>\n\n<p>I don't understand your second question.</p>",
      "rawMarkdown": "We haven't cleaned up the code yet, but you can check out topcoder's implementation for a similar task for a previous competition: https://github.com/selimsef/dsb2018_topcoders/blob/master/victor/create_masks.py\n\nI don't understand your second question.",
      "votes": null
    },
    {
      "id": "422144",
      "postDate": "11/15/2018 20:48:07",
      "content": "<p>Thanks so much for sharing your solution!</p>",
      "rawMarkdown": "Thanks so much for sharing your solution!",
      "votes": null
    },
    {
      "id": "422251",
      "postDate": "11/16/2018 01:11:45",
      "content": "<p>Thank you so much for sharing your great idea.</p>",
      "rawMarkdown": "Thank you so much for sharing your great idea.",
      "votes": null
    },
    {
      "id": "422457",
      "postDate": "11/16/2018 08:49:49",
      "content": "<p>i meant as part of ensembling what you did...and how</p>",
      "rawMarkdown": "i meant as part of ensembling what you did...and how",
      "votes": null
    },
    {
      "id": "422488",
      "postDate": "11/16/2018 09:44:18",
      "content": "<p>Thanks so much for sharing your solution! Learned a lot from it.</p>",
      "rawMarkdown": "Thanks so much for sharing your solution! Learned a lot from it.",
      "votes": null
    },
    {
      "id": "422740",
      "postDate": "11/16/2018 17:49:55",
      "content": "<p>Very well written explanation. Nice work and congrats!</p>",
      "rawMarkdown": "Very well written explanation. Nice work and congrats!",
      "votes": null
    },
    {
      "id": "422774",
      "postDate": "11/16/2018 19:07:41",
      "content": "<p>You mean \"An ensemble of fastai resnet34 and resnext50\" ?</p>\n\n<p>For each image, resnet34 has a predicted probability of being a ship; resnext50 also has a predicted probability of being a ship. Take a weighted average would be the ensemble's output.</p>",
      "rawMarkdown": "You mean \"An ensemble of fastai resnet34 and resnext50\" ?\n\nFor each image, resnet34 has a predicted probability of being a ship; resnext50 also has a predicted probability of being a ship. Take a weighted average would be the ensemble's output.",
      "votes": null
    },
    {
      "id": "422950",
      "postDate": "11/17/2018 05:54:25",
      "content": "<p>THanks this what i was looking.. so this averaging  was done before  applying thresholding &amp; labelling  to build the masks?\nsay pixels predicted prob will look\n0.1,.98,.80..... in array\nso we will average this outputs from other NN ?</p>",
      "rawMarkdown": "THanks this what i was looking.. so this averaging  was done before  applying thresholding &amp; labelling  to build the masks?\nsay pixels predicted prob will look\n0.1,.98,.80..... in array\nso we will average this outputs from other NN ?",
      "votes": null
    },
    {
      "id": "456799",
      "postDate": "01/16/2019 15:10:38",
      "content": "<p>can you share your code?</p>",
      "rawMarkdown": "can you share your code?",
      "votes": null
    },
    {
      "id": "457550",
      "postDate": "01/17/2019 17:07:42",
      "content": "<p>congratulations! Can you share your code?</p>",
      "rawMarkdown": "congratulations! Can you share your code?",
      "votes": null
    },
    {
      "id": "762909",
      "postDate": "03/03/2020 23:44:29",
      "content": "<p>Do you have a code for more detail with Postprocessing (mostly using skimage)\n?</p>",
      "rawMarkdown": "Do you have a code for more detail with Postprocessing (mostly using skimage)\n?",
      "votes": null
    },
    {
      "id": "1684472",
      "postDate": "02/10/2022 14:11:51",
      "content": "<p>Hi everyone, I want to know do I need to label the training dataset using CVAT and then use a faster RCNN ?</p>",
      "rawMarkdown": "Hi everyone, I want to know do I need to label the training dataset using CVAT and then use a faster RCNN ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1684472,
      "author_name": "sayalisalunkh",
      "author_url": "",
      "post_date": "02/10/2022 14:11:51",
      "content": "<p>Hi everyone, I want to know do I need to label the training dataset using CVAT and then use a faster RCNN ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 421435,
      "author_name": "ucdhyan",
      "author_url": "",
      "post_date": "11/15/2018 02:18:17",
      "content": "<p>Nice summary! Thanks a lot!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 421436,
      "author_name": "terenceliu4444",
      "author_url": "",
      "post_date": "11/15/2018 02:22:57",
      "content": "<p>congratulations to the team! this is a solid and great work!</p>",
      "votes": null,
      "replies": [
        {
          "id": 422120,
          "author_name": "boliu0",
          "author_url": "",
          "post_date": "11/15/2018 20:04:06",
          "content": "<p>Thanks Terence. Hope you have better luck with private LB next time!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 421456,
      "author_name": "soonhwankwon",
      "author_url": "",
      "post_date": "11/15/2018 03:14:21",
      "content": "<p>wow 3 channel label approach seems so neat! wonderful!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 421467,
      "author_name": "markpeng",
      "author_url": "",
      "post_date": "11/15/2018 03:38:21",
      "content": "<p>Congrats on your great work and idea on 3 channels for ship, contour and background!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 421470,
      "author_name": "super13579",
      "author_url": "",
      "post_date": "11/15/2018 03:43:20",
      "content": "<p>Congratulation!! 3 channel label approach is amazing\nGreat Work ! Thanks your share</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 421503,
      "author_name": "sheriytm",
      "author_url": "",
      "post_date": "11/15/2018 04:37:12",
      "content": "<p>Congratulations @Bo and team. Thanks for sharing your solution.</p>\n\n<p>I do not have a GPU machine and relied on Kaggle kernel and a cloud account for competitions. In this one I must admit I was discouraged by the leak earlier on so decided not to spend too much time on it. I did something similar to what @Lafoss did in <a href=\"https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook\">this notebook</a> before the leak. So I adapted it to the new data and submitted.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 421687,
      "author_name": "enricsarle",
      "author_url": "",
      "post_date": "11/15/2018 09:12:04",
      "content": "<p>Congratulations and thank you for sharing your strategy! I really like the idea of three channels labels</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 421819,
      "author_name": "rhgrossm",
      "author_url": "",
      "post_date": "11/15/2018 13:05:08",
      "content": "<p>Awesome! thanks for sharing!</p>",
      "votes": null,
      "replies": [
        {
          "id": 422121,
          "author_name": "boliu0",
          "author_url": "",
          "post_date": "11/15/2018 20:04:33",
          "content": "<p>Thanks Ray</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 421824,
      "author_name": "viswanathravindran",
      "author_url": "",
      "post_date": "11/15/2018 13:12:20",
      "content": "<p>Congrats, Thanks for sharing your ideas, I learnt about applying the 3 channel label from your notes and it is interesting..</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 422015,
      "author_name": "jaideepvalani",
      "author_url": "",
      "post_date": "11/15/2018 17:11:37",
      "content": "<p>hi could u throw some more light on this..\nused 3 channel labels: ship, contour and background. Contour layers are generated in order to help the model separating close ships. An example:\nHow is this achieved any pointer ?</p>\n\n<p>2) What was ensembling architecture FA resnet-&gt;resnet 101,m unable to visualize it</p>",
      "votes": null,
      "replies": [
        {
          "id": 422125,
          "author_name": "boliu0",
          "author_url": "",
          "post_date": "11/15/2018 20:07:33",
          "content": "<p>We haven't cleaned up the code yet, but you can check out topcoder's implementation for a similar task for a previous competition: <a href=\"https://github.com/selimsef/dsb2018_topcoders/blob/master/victor/create_masks.py\">https://github.com/selimsef/dsb2018_topcoders/blob/master/victor/create_masks.py</a></p>\n\n<p>I don't understand your second question.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 422457,
          "author_name": "jaideepvalani",
          "author_url": "",
          "post_date": "11/16/2018 08:49:49",
          "content": "<p>i meant as part of ensembling what you did...and how</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 422774,
          "author_name": "boliu0",
          "author_url": "",
          "post_date": "11/16/2018 19:07:41",
          "content": "<p>You mean \"An ensemble of fastai resnet34 and resnext50\" ?</p>\n\n<p>For each image, resnet34 has a predicted probability of being a ship; resnext50 also has a predicted probability of being a ship. Take a weighted average would be the ensemble's output.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 422950,
          "author_name": "jaideepvalani",
          "author_url": "",
          "post_date": "11/17/2018 05:54:25",
          "content": "<p>THanks this what i was looking.. so this averaging  was done before  applying thresholding &amp; labelling  to build the masks?\nsay pixels predicted prob will look\n0.1,.98,.80..... in array\nso we will average this outputs from other NN ?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 422144,
      "author_name": "zhangchi9",
      "author_url": "",
      "post_date": "11/15/2018 20:48:07",
      "content": "<p>Thanks so much for sharing your solution!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 422251,
      "author_name": "leekltw1",
      "author_url": "",
      "post_date": "11/16/2018 01:11:45",
      "content": "<p>Thank you so much for sharing your great idea.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 422488,
      "author_name": "deepak525",
      "author_url": "",
      "post_date": "11/16/2018 09:44:18",
      "content": "<p>Thanks so much for sharing your solution! Learned a lot from it.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 422740,
      "author_name": "kingrandy",
      "author_url": "",
      "post_date": "11/16/2018 17:49:55",
      "content": "<p>Very well written explanation. Nice work and congrats!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 456799,
      "author_name": "xiaojidan",
      "author_url": "",
      "post_date": "01/16/2019 15:10:38",
      "content": "<p>can you share your code?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 457550,
      "author_name": "xiaojidan",
      "author_url": "",
      "post_date": "01/17/2019 17:07:42",
      "content": "<p>congratulations! Can you share your code?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 762909,
      "author_name": "",
      "author_url": "",
      "post_date": "03/03/2020 23:44:29",
      "content": "<p>Do you have a code for more detail with Postprocessing (mostly using skimage)\n?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "421431": "Congratulations to the winners, especially Victor and Selim of topcoders, whose 2018 DSB winning [solution][1] we adopted for this competition.\n\n## Finding overlap\nDue to the overlapping issue, we wrote an algorithm to find all the overlapping pairs of images in train set, then ran the [friend circles][2] algorithm to find all the 57877 groups of images. Images in each group is supposedly from the same acquisition and images from different groups do not overlap.\n\nWe then did the train/val split to have 2920 groups (10442 images) in val set while making sure train and val have same distribution of number of ships per image. We did this split before test_v2 was released and used it for the majority of the completion but switched to a balanced 4 fold towards the end which have distribution (in ship number and ship size) closer to test set.\n\n## Binary classification model\nAn ensemble of fastai resnet34 and resnext50 models trained on our split based on Iafoss’s [kernel][3].\n\n## Labels\nWe used topcoder’s dsb2018 winning solution code to generate contour (i.e. divider) layer and used 3 channel labels: ship, contour and background. Contour layers are generated in order to help the model separating close ships. An example:\n![label][4]\n\n## Model\nBest model is Unet densenet169 implemented in Keras by topcoders. We also tried resnet152, resnet101, inception resnet v2\n\n## Loss\nDouble head loss (i.e. half binary cross entropy, half dice loss) for ship layer. For contour layer, we tried both double head loss and only dice loss since the pixels are too imbalanced.\n\n## Input size\n512 and 768\n\n## Thresholds\n1. Best classifier threshold is 0.5–0.7 depending on the model. Images below this threshold have empty prediction in submission; images above this threshold go to Unet model which may still end up having empty predictions from Unet. \n2. Unet pixelwise threshold\nWe noticed that for all our models, best pixelwise threshold on local validation (usually around 0.5) is never the best on public LB. After some simulations and LB probing, we highly suspected discrepancy between train and test label standards: test set masks are more “tight”, i.e. higher threshold on test set (0.6-0.8 depending on model) works out better on public LB. This turned out to  be the case just for the public LB.\n\n## Training techniques\n All the standard ones: data augmentation, cyclic learning rates, TTA\n\n## Postprocessing (mostly using skimage)\n-\tAlgo to separate \"weakly connected\" masks, with erosion and watershed. We start from an intuition that generally masks are rectangular and when the model predicted ship boundaries are 'weak', they touch along the longer side of rectangles. To separate this case, we first create a thin rectangular structuring element that is aligned with the major axis orientation of the original mask. Applying erosion with this structuring element will help separate the boundary. Since erosion reduces the pixels from the original mask, we try to make this up by applying a watershed algorithm on top of erosion. Example:\n![weakly][5]\n-\tto cut corners (probing the possibility that test set labels may be made as actual ship shape instead of rectangles) \n-\tto “rectanglize” unet output masks by finding minimal bounding box and shrink it to same size\n\n## Final models\nBest one in local validation and best on public LB. The best on public LB severely overfitted, like quite some teams did. The best one in local validation turned out to be 9th place. \n\n## Other things tried\n-\tEnsembling different models by averaging predicted probability matrixes – not working well \n-\tCombining different models based on number of ships per image and ship size (some models work better for certain images or ships) – may overfit public LB\n-\tMaskRcnn – much worse than unet\n-\tAdding ships predicted in one model to the results of another model – slightly improved but too complicated\n\n## Team and hardwares\nWe are full time data scientists and coworkers. We don’t do image tasks at our job. This is the first deep learning competition for most of the team members. We teamed up very early. We have 6 Nvidia 1080Ti or equivalent and 2 smaller ones\n\n\n## Questions for other top teams\n- How do you deal with competitions like this where the public LB is too small and delta between local CV and public LB too unstable? Do you always faithfully trust your local CV no matter what? \n- What if the delta is because of difference in distribution (or even labeling standards as discussed [here][6]) between train and test sets? \n- I noticed bestfitting submitted many times, so there's must be some value in public LB?\n\n\n  [1]: https://github.com/selimsef/dsb2018_topcoders\n  [2]: https://leetcode.com/problems/friend-circles/description/\n  [3]: https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook\n  [4]: https://i.imgur.com/KanhOpD.png\n  [5]: https://i.imgur.com/rYRXeLQ.jpg\n  [6]: https://www.kaggle.com/c/airbus-ship-detection/discussion/70221",
    "421435": "Nice summary! Thanks a lot!",
    "421436": "congratulations to the team! this is a solid and great work!",
    "421456": "wow 3 channel label approach seems so neat! wonderful!",
    "421467": "Congrats on your great work and idea on 3 channels for ship, contour and background!",
    "421470": "Congratulation!! 3 channel label approach is amazing\nGreat Work ! Thanks your share",
    "421503": "Congratulations @Bo and team. Thanks for sharing your solution.\n\nI do not have a GPU machine and relied on Kaggle kernel and a cloud account for competitions. In this one I must admit I was discouraged by the leak earlier on so decided not to spend too much time on it. I did something similar to what @Lafoss did in [this notebook][1] before the leak. So I adapted it to the new data and submitted.\n\n\n  [1]: https://www.kaggle.com/iafoss/fine-tuning-resnet34-on-ship-detection/notebook",
    "421687": "Congratulations and thank you for sharing your strategy! I really like the idea of three channels labels",
    "421819": "Awesome! thanks for sharing!",
    "421824": "Congrats, Thanks for sharing your ideas, I learnt about applying the 3 channel label from your notes and it is interesting..",
    "422015": "hi could u throw some more light on this..\nused 3 channel labels: ship, contour and background. Contour layers are generated in order to help the model separating close ships. An example:\nHow is this achieved any pointer ?\n\n2) What was ensembling architecture FA resnet-&gt;resnet 101,m unable to visualize it",
    "422120": "Thanks Terence. Hope you have better luck with private LB next time!",
    "422121": "Thanks Ray",
    "422125": "We haven't cleaned up the code yet, but you can check out topcoder's implementation for a similar task for a previous competition: https://github.com/selimsef/dsb2018_topcoders/blob/master/victor/create_masks.py\n\nI don't understand your second question.",
    "422144": "Thanks so much for sharing your solution!",
    "422251": "Thank you so much for sharing your great idea.",
    "422457": "i meant as part of ensembling what you did...and how",
    "422488": "Thanks so much for sharing your solution! Learned a lot from it.",
    "422740": "Very well written explanation. Nice work and congrats!",
    "422774": "You mean \"An ensemble of fastai resnet34 and resnext50\" ?\n\nFor each image, resnet34 has a predicted probability of being a ship; resnext50 also has a predicted probability of being a ship. Take a weighted average would be the ensemble's output.",
    "422950": "THanks this what i was looking.. so this averaging  was done before  applying thresholding &amp; labelling  to build the masks?\nsay pixels predicted prob will look\n0.1,.98,.80..... in array\nso we will average this outputs from other NN ?",
    "456799": "can you share your code?",
    "457550": "congratulations! Can you share your code?",
    "762909": "Do you have a code for more detail with Postprocessing (mostly using skimage)\n?",
    "1684472": "Hi everyone, I want to know do I need to label the training dataset using CVAT and then use a faster RCNN ?"
  },
  "source": "meta"
}