{
  "id": 118184,
  "title": "22th Place - Lessons learned from a beginner",
  "url": "/competitions/understanding_cloud_organization/writeups/harold-team-brazil-22th-place-lessons-learned-from",
  "author_name": "",
  "post_date": "2019-11-19T23:20:13.358345200Z",
  "votes": 11,
  "comment_count": 8,
  "views": 0,
  "content": "<p>First congrats to all the winners.\nI would like to thank Kaggle for hosting this competition which was the first one I could dedicate myself and won my first medal.</p>\n\n<p><strong>What didn't work for me</strong>\n- Lovasz loss\n- Deeper encoders (efficientb7,senet)\n- Pseudo labeling</p>\n\n<p><strong>Our solution</strong>\nOur solution is basically emsemble of segmentation models with post processing to remove masks</p>\n\n<p>Models (6 folds each):\n- ResNet34 - Unet*\n- EfficientNetB2 - Unet*\n- EfficientNetB2 - FPN\n- EfficientNetB2 - LinkNet\n- EfficientNetB5 - Unet</p>\n\n<p>Loss: BCE + Dice\n* Those models was trained with different image size (320x480, 384x576, 512x512, 704x1056)</p>\n\n<p>Post Processing:\ntriplet threshold searching for binarization, remove small masks and binarization again for the remaining masks after the first two steps. All this was done with the validation data from all 6 folds.</p>\n\n<p>CV: 0.6651\nPublic: 0.67556\nPrivate: 0.66498</p>\n\n<p><strong>The Good Lesson</strong>\nI didn't know much about image segmentation, so this competition was a great learning.</p>\n\n<ul>\n<li><p>Read all comments and try to get the tips.</p></li>\n<li><p>Build a good validation set\ntuning post processing parameters was only possible without overfitting because of that</p></li>\n</ul>\n\n<p><strong>The Bad Lesson</strong>\n- Trust in your CV</p>\n\n<p>I had a better model that scored:</p>\n\n<p>CV: 0.6681\nPublic: 0.66759\nPrivate: 0.66824</p>\n\n<p>Why didn't I choose it? because of the second lesson ...</p>\n\n<ul>\n<li>Trust in you\nMy best model was something different. I trained one model for each mask type, predicted one by one and put it in original format (4 masks stacked) before applying post processing. </li>\n</ul>\n\n<p>This allowed me to compare with the same out of fold data I had so far. \nA simple blend of ResNet34-Unet + EfficientNetB2-Unet got 0.668 on CV.</p>\n\n<p>But I read that some kagglers didn't get good results with this method, I was afraid of having a leak in my validation and public LB was worse. So I gave up on this idea...</p>\n\n<p><strong>Acknowledgment</strong>\nI would like to thank my team and all those who shared in some way.</p>\n\n<p>Sharing is a very good thing, but I think it should be done at the right time. As I said I am a beginner, but also someone who worked hard on this competition reading past competition solutions. So I think everyone can do the same.</p>",
  "messages": [
    {
      "id": "677207",
      "postDate": "11/19/2019 23:20:13",
      "content": "<p>First congrats to all the winners.\nI would like to thank Kaggle for hosting this competition which was the first one I could dedicate myself and won my first medal.</p>\n\n<p><strong>What didn't work for me</strong>\n- Lovasz loss\n- Deeper encoders (efficientb7,senet)\n- Pseudo labeling</p>\n\n<p><strong>Our solution</strong>\nOur solution is basically emsemble of segmentation models with post processing to remove masks</p>\n\n<p>Models (6 folds each):\n- ResNet34 - Unet*\n- EfficientNetB2 - Unet*\n- EfficientNetB2 - FPN\n- EfficientNetB2 - LinkNet\n- EfficientNetB5 - Unet</p>\n\n<p>Loss: BCE + Dice\n* Those models was trained with different image size (320x480, 384x576, 512x512, 704x1056)</p>\n\n<p>Post Processing:\ntriplet threshold searching for binarization, remove small masks and binarization again for the remaining masks after the first two steps. All this was done with the validation data from all 6 folds.</p>\n\n<p>CV: 0.6651\nPublic: 0.67556\nPrivate: 0.66498</p>\n\n<p><strong>The Good Lesson</strong>\nI didn't know much about image segmentation, so this competition was a great learning.</p>\n\n<ul>\n<li><p>Read all comments and try to get the tips.</p></li>\n<li><p>Build a good validation set\ntuning post processing parameters was only possible without overfitting because of that</p></li>\n</ul>\n\n<p><strong>The Bad Lesson</strong>\n- Trust in your CV</p>\n\n<p>I had a better model that scored:</p>\n\n<p>CV: 0.6681\nPublic: 0.66759\nPrivate: 0.66824</p>\n\n<p>Why didn't I choose it? because of the second lesson ...</p>\n\n<ul>\n<li>Trust in you\nMy best model was something different. I trained one model for each mask type, predicted one by one and put it in original format (4 masks stacked) before applying post processing. </li>\n</ul>\n\n<p>This allowed me to compare with the same out of fold data I had so far. \nA simple blend of ResNet34-Unet + EfficientNetB2-Unet got 0.668 on CV.</p>\n\n<p>But I read that some kagglers didn't get good results with this method, I was afraid of having a leak in my validation and public LB was worse. So I gave up on this idea...</p>\n\n<p><strong>Acknowledgment</strong>\nI would like to thank my team and all those who shared in some way.</p>\n\n<p>Sharing is a very good thing, but I think it should be done at the right time. As I said I am a beginner, but also someone who worked hard on this competition reading past competition solutions. So I think everyone can do the same.</p>",
      "rawMarkdown": "First congrats to all the winners.\nI would like to thank Kaggle for hosting this competition which was the first one I could dedicate myself and won my first medal.\n\n**What didn't work for me**\n- Lovasz loss\n- Deeper encoders (efficientb7,senet)\n- Pseudo labeling\n\n**Our solution**\nOur solution is basically emsemble of segmentation models with post processing to remove masks\n\nModels (6 folds each):\n- ResNet34 - Unet*\n- EfficientNetB2 - Unet*\n- EfficientNetB2 - FPN\n- EfficientNetB2 - LinkNet\n- EfficientNetB5 - Unet\n\nLoss: BCE + Dice\n* Those models was trained with different image size (320x480, 384x576, 512x512, 704x1056)\n\nPost Processing:\ntriplet threshold searching for binarization, remove small masks and binarization again for the remaining masks after the first two steps. All this was done with the validation data from all 6 folds.\n\nCV: 0.6651\nPublic: 0.67556\nPrivate: 0.66498\n\n**The Good Lesson**\nI didn't know much about image segmentation, so this competition was a great learning.\n\n- Read all comments and try to get the tips.\n\n- Build a good validation set\ntuning post processing parameters was only possible without overfitting because of that\n\n\n**The Bad Lesson**\n- Trust in your CV\n\nI had a better model that scored:\n\nCV: 0.6681\nPublic: 0.66759\nPrivate: 0.66824\n\nWhy didn't I choose it? because of the second lesson ...\n\n-  Trust in you\nMy best model was something different. I trained one model for each mask type, predicted one by one and put it in original format (4 masks stacked) before applying post processing. \n\nThis allowed me to compare with the same out of fold data I had so far. \nA simple blend of ResNet34-Unet + EfficientNetB2-Unet got 0.668 on CV.\n\nBut I read that some kagglers didn't get good results with this method, I was afraid of having a leak in my validation and public LB was worse. So I gave up on this idea...\n\n\n**Acknowledgment**\nI would like to thank my team and all those who shared in some way.\n\nSharing is a very good thing, but I think it should be done at the right time. As I said I am a beginner, but also someone who worked hard on this competition reading past competition solutions. So I think everyone can do the same.",
      "votes": null
    },
    {
      "id": "677217",
      "postDate": "11/19/2019 23:36:07",
      "content": "<p>Nice writeup. So your best model has CV the best too, and LB score the same too. If you trust it you should have gold. That’s nice. Congrats on your first medal, you missed gold by only a slight margin, like me :D </p>",
      "rawMarkdown": "Nice writeup. So your best model has CV the best too, and LB score the same too. If you trust it you should have gold. That’s nice. Congrats on your first medal, you missed gold by only a slight margin, like me :D",
      "votes": null
    },
    {
      "id": "677228",
      "postDate": "11/19/2019 23:57:48",
      "content": "<p>First at all, congrats to all.\nThanks to Kaggle and the hosting team.</p>\n\n<p>I'm a complete beginner, but i learn a lot on this competition, <a href=\"/igormunizims\">@igormunizims</a> taught me a lot.\nAnd thanks, to all kaggles, who share his Notebooks and his ideas.</p>\n\n<p>But now, complementing everything Igor said.</p>\n\n<p><strong>Augumentation</strong></p>\n\n<p>Our best augumentation was:\n- RandomSizedCrop;\n- Horizontal/Vertical Flip;\n- ShiftScaleRotate;\n- ElasticTransform;\n- GridDistortion;\n- OpticalDistortion;\n- RandomBrightnessContrast.</p>\n\n<p>We try a lot of combinations, but far from test every combination, there was no time to get the best on this.</p>\n\n<p><strong>Losses</strong></p>\n\n<p>We try many losses, but then wasn't better result than <strong>BCE + Dice</strong>.</p>\n\n<p>Somes anothers losses used:\n- tversky;\n- jaccard;\n- lovasz.</p>\n\n<p><strong>Classification</strong></p>\n\n<p>Classification was try only on the final week, looking on  the results on Public and Private leadeboard, the results was pretty good. But, we dont use this on choosen submissios, because in that moment we dont have some way to validate the classification.</p>",
      "rawMarkdown": "First at all, congrats to all.\nThanks to Kaggle and the hosting team.\n\nI'm a complete beginner, but i learn a lot on this competition, @igormunizims taught me a lot.\nAnd thanks, to all kaggles, who share his Notebooks and his ideas.\n\nBut now, complementing everything Igor said.\n\n**Augumentation**\n\nOur best augumentation was:\n- RandomSizedCrop;\n- Horizontal/Vertical Flip;\n- ShiftScaleRotate;\n- ElasticTransform;\n- GridDistortion;\n- OpticalDistortion;\n- RandomBrightnessContrast.\n\nWe try a lot of combinations, but far from test every combination, there was no time to get the best on this.\n\n**Losses**\n\nWe try many losses, but then wasn't better result than **BCE + Dice**.\n\nSomes anothers losses used:\n- tversky;\n- jaccard;\n- lovasz.\n\n**Classification**\n\nClassification was try only on the final week, looking on  the results on Public and Private leadeboard, the results was pretty good. But, we dont use this on choosen submissios, because in that moment we dont have some way to validate the classification.",
      "votes": null
    },
    {
      "id": "677281",
      "postDate": "11/20/2019 01:42:23",
      "content": "<p>Congrats to the team! Well deserved Silver medal.</p>",
      "rawMarkdown": "Congrats to the team! Well deserved Silver medal.",
      "votes": null
    },
    {
      "id": "677309",
      "postDate": "11/20/2019 03:04:41",
      "content": "<p>Congrats  <a href=\"/igormunizims\">@igormunizims</a> </p>",
      "rawMarkdown": "Congrats  @igormunizims",
      "votes": null
    },
    {
      "id": "677408",
      "postDate": "11/20/2019 05:36:08",
      "content": "<p>Congratulations\nThanks for Sharing your Approach &amp; Insights! <a href=\"/igormunizims\">@igormunizims</a> </p>",
      "rawMarkdown": "Congratulations\nThanks for Sharing your Approach &amp; Insights! @igormunizims",
      "votes": null
    },
    {
      "id": "677659",
      "postDate": "11/20/2019 13:18:08",
      "content": "<p>Thanks Giba!</p>",
      "rawMarkdown": "Thanks Giba!",
      "votes": null
    },
    {
      "id": "677660",
      "postDate": "11/20/2019 13:19:21",
      "content": "<p>Better luck for us next time ;D</p>",
      "rawMarkdown": "Better luck for us next time ;D",
      "votes": null
    },
    {
      "id": "677782",
      "postDate": "11/20/2019 15:31:57",
      "content": "<p>Congratulations🎉 Thanks for sharing.</p>\n\n<blockquote>\n  <p>As I said I am a beginner, but also someone who worked hard on this competition reading past competition solutions. So I think everyone can do the same.</p>\n</blockquote>\n\n<p>I also do my best👍 </p>",
      "rawMarkdown": "Congratulations🎉 Thanks for sharing.\n&gt; As I said I am a beginner, but also someone who worked hard on this competition reading past competition solutions. So I think everyone can do the same.\n\nI also do my best👍",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 677217,
      "author_name": "khahuras",
      "author_url": "",
      "post_date": "11/19/2019 23:36:07",
      "content": "<p>Nice writeup. So your best model has CV the best too, and LB score the same too. If you trust it you should have gold. That’s nice. Congrats on your first medal, you missed gold by only a slight margin, like me :D </p>",
      "votes": null,
      "replies": [
        {
          "id": 677660,
          "author_name": "igormunizims",
          "author_url": "",
          "post_date": "11/20/2019 13:19:21",
          "content": "<p>Better luck for us next time ;D</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 677228,
      "author_name": "kaleus",
      "author_url": "",
      "post_date": "11/19/2019 23:57:48",
      "content": "<p>First at all, congrats to all.\nThanks to Kaggle and the hosting team.</p>\n\n<p>I'm a complete beginner, but i learn a lot on this competition, <a href=\"/igormunizims\">@igormunizims</a> taught me a lot.\nAnd thanks, to all kaggles, who share his Notebooks and his ideas.</p>\n\n<p>But now, complementing everything Igor said.</p>\n\n<p><strong>Augumentation</strong></p>\n\n<p>Our best augumentation was:\n- RandomSizedCrop;\n- Horizontal/Vertical Flip;\n- ShiftScaleRotate;\n- ElasticTransform;\n- GridDistortion;\n- OpticalDistortion;\n- RandomBrightnessContrast.</p>\n\n<p>We try a lot of combinations, but far from test every combination, there was no time to get the best on this.</p>\n\n<p><strong>Losses</strong></p>\n\n<p>We try many losses, but then wasn't better result than <strong>BCE + Dice</strong>.</p>\n\n<p>Somes anothers losses used:\n- tversky;\n- jaccard;\n- lovasz.</p>\n\n<p><strong>Classification</strong></p>\n\n<p>Classification was try only on the final week, looking on  the results on Public and Private leadeboard, the results was pretty good. But, we dont use this on choosen submissios, because in that moment we dont have some way to validate the classification.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 677281,
      "author_name": "titericz",
      "author_url": "",
      "post_date": "11/20/2019 01:42:23",
      "content": "<p>Congrats to the team! Well deserved Silver medal.</p>",
      "votes": null,
      "replies": [
        {
          "id": 677659,
          "author_name": "igormunizims",
          "author_url": "",
          "post_date": "11/20/2019 13:18:08",
          "content": "<p>Thanks Giba!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 677309,
      "author_name": "karthik7395",
      "author_url": "",
      "post_date": "11/20/2019 03:04:41",
      "content": "<p>Congrats  <a href=\"/igormunizims\">@igormunizims</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 677408,
      "author_name": "veeralakrishna",
      "author_url": "",
      "post_date": "11/20/2019 05:36:08",
      "content": "<p>Congratulations\nThanks for Sharing your Approach &amp; Insights! <a href=\"/igormunizims\">@igormunizims</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 677782,
      "author_name": "mashlyn",
      "author_url": "",
      "post_date": "11/20/2019 15:31:57",
      "content": "<p>Congratulations🎉 Thanks for sharing.</p>\n\n<blockquote>\n  <p>As I said I am a beginner, but also someone who worked hard on this competition reading past competition solutions. So I think everyone can do the same.</p>\n</blockquote>\n\n<p>I also do my best👍 </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "677207": "First congrats to all the winners.\nI would like to thank Kaggle for hosting this competition which was the first one I could dedicate myself and won my first medal.\n\n**What didn't work for me**\n- Lovasz loss\n- Deeper encoders (efficientb7,senet)\n- Pseudo labeling\n\n**Our solution**\nOur solution is basically emsemble of segmentation models with post processing to remove masks\n\nModels (6 folds each):\n- ResNet34 - Unet*\n- EfficientNetB2 - Unet*\n- EfficientNetB2 - FPN\n- EfficientNetB2 - LinkNet\n- EfficientNetB5 - Unet\n\nLoss: BCE + Dice\n* Those models was trained with different image size (320x480, 384x576, 512x512, 704x1056)\n\nPost Processing:\ntriplet threshold searching for binarization, remove small masks and binarization again for the remaining masks after the first two steps. All this was done with the validation data from all 6 folds.\n\nCV: 0.6651\nPublic: 0.67556\nPrivate: 0.66498\n\n**The Good Lesson**\nI didn't know much about image segmentation, so this competition was a great learning.\n\n- Read all comments and try to get the tips.\n\n- Build a good validation set\ntuning post processing parameters was only possible without overfitting because of that\n\n\n**The Bad Lesson**\n- Trust in your CV\n\nI had a better model that scored:\n\nCV: 0.6681\nPublic: 0.66759\nPrivate: 0.66824\n\nWhy didn't I choose it? because of the second lesson ...\n\n-  Trust in you\nMy best model was something different. I trained one model for each mask type, predicted one by one and put it in original format (4 masks stacked) before applying post processing. \n\nThis allowed me to compare with the same out of fold data I had so far. \nA simple blend of ResNet34-Unet + EfficientNetB2-Unet got 0.668 on CV.\n\nBut I read that some kagglers didn't get good results with this method, I was afraid of having a leak in my validation and public LB was worse. So I gave up on this idea...\n\n\n**Acknowledgment**\nI would like to thank my team and all those who shared in some way.\n\nSharing is a very good thing, but I think it should be done at the right time. As I said I am a beginner, but also someone who worked hard on this competition reading past competition solutions. So I think everyone can do the same.",
    "677217": "Nice writeup. So your best model has CV the best too, and LB score the same too. If you trust it you should have gold. That’s nice. Congrats on your first medal, you missed gold by only a slight margin, like me :D",
    "677228": "First at all, congrats to all.\nThanks to Kaggle and the hosting team.\n\nI'm a complete beginner, but i learn a lot on this competition, @igormunizims taught me a lot.\nAnd thanks, to all kaggles, who share his Notebooks and his ideas.\n\nBut now, complementing everything Igor said.\n\n**Augumentation**\n\nOur best augumentation was:\n- RandomSizedCrop;\n- Horizontal/Vertical Flip;\n- ShiftScaleRotate;\n- ElasticTransform;\n- GridDistortion;\n- OpticalDistortion;\n- RandomBrightnessContrast.\n\nWe try a lot of combinations, but far from test every combination, there was no time to get the best on this.\n\n**Losses**\n\nWe try many losses, but then wasn't better result than **BCE + Dice**.\n\nSomes anothers losses used:\n- tversky;\n- jaccard;\n- lovasz.\n\n**Classification**\n\nClassification was try only on the final week, looking on  the results on Public and Private leadeboard, the results was pretty good. But, we dont use this on choosen submissios, because in that moment we dont have some way to validate the classification.",
    "677281": "Congrats to the team! Well deserved Silver medal.",
    "677309": "Congrats  @igormunizims",
    "677408": "Congratulations\nThanks for Sharing your Approach &amp; Insights! @igormunizims",
    "677659": "Thanks Giba!",
    "677660": "Better luck for us next time ;D",
    "677782": "Congratulations🎉 Thanks for sharing.\n&gt; As I said I am a beginner, but also someone who worked hard on this competition reading past competition solutions. So I think everyone can do the same.\n\nI also do my best👍"
  },
  "source": "meta"
}