{
  "id": 95282,
  "title": "6th place simple solution with code",
  "url": "/competitions/imet-2019-fgvc6/writeups/starlight-6th-place-simple-solution-with-code",
  "author_name": "",
  "post_date": "2019-06-11T08:22:14.623Z",
  "votes": 61,
  "comment_count": 11,
  "views": 0,
  "content": "<p>Congratulations to all the teams got the medal.\nCompared to other teams, my solution is quite simple (though I tried other tricks, none of them worked).</p>\n\n<h2>Solution</h2>\n\n<p>My code is available at <a href=\"https://github.com/YU1ut/imet-6th-soltuion\">https://github.com/YU1ut/imet-6th-soltuion</a>.\nMy solution is based on this kernel <a href=\"https://www.kaggle.com/lopuhin/imet-2019-submission\">https://www.kaggle.com/lopuhin/imet-2019-submission</a>, and I modified it as follows.</p>\n\n<ol>\n<li>Change the input size to 320 by <code>RandomCrop(320, pad_if_needed=True)</code>.</li>\n<li>Add Mixup and RandomErasing during training.</li>\n<li>Add pre-trained models from <a href=\"https://github.com/Cadene/pretrained-models.pytorch\">https://github.com/Cadene/pretrained-models.pytorch</a>.</li>\n<li>Make a 40-fold CV and use some of them to train models.</li>\n<li>Train se_resnext101 by 10 fold (1~10), inceptionresnetv2 by 5 folds (6~10) and pnasnet5large by 5 folds (1~5). As a result, 20 models are trained.</li>\n<li>Use the average output of all models (#TTA 2). And adjust the threshold for each image according to the max probability of that image. Supposing we have a probability matrix whose shape is (N_SAMPLES x N_CLASSES), the binary results can be calculated by \n<code>\nprob_mask = []\nfor prob in probabilities:\n    prob_mask.append(prob &amp;gt; prob.max()/7)\n</code>\nThis threshold calculation can boost the LB score about 0.005.</li>\n</ol>\n\n<h2>What didn't work for me:</h2>\n\n<ol>\n<li>Graph Convolutional Networks <a href=\"https://arxiv.org/abs/1904.03582\">https://arxiv.org/abs/1904.03582</a></li>\n<li>Soft Label and Pseudo Labeling</li>\n<li>EfficientNet</li>\n</ol>\n\n<p>Thanks.</p>",
  "messages": [
    {
      "id": "550000",
      "postDate": "06/11/2019 07:57:56",
      "content": "<p>Congratulations to all the teams got the medal.\nCompared to other teams, my solution is quite simple (though I tried other tricks, none of them worked).</p>\n\n<h2>Solution</h2>\n\n<p>My code is available at <a href=\"https://github.com/YU1ut/imet-6th-soltuion\">https://github.com/YU1ut/imet-6th-soltuion</a>.\nMy solution is based on this kernel <a href=\"https://www.kaggle.com/lopuhin/imet-2019-submission\">https://www.kaggle.com/lopuhin/imet-2019-submission</a>, and I modified it as follows.</p>\n\n<ol>\n<li>Change the input size to 320 by <code>RandomCrop(320, pad_if_needed=True)</code>.</li>\n<li>Add Mixup and RandomErasing during training.</li>\n<li>Add pre-trained models from <a href=\"https://github.com/Cadene/pretrained-models.pytorch\">https://github.com/Cadene/pretrained-models.pytorch</a>.</li>\n<li>Make a 40-fold CV and use some of them to train models.</li>\n<li>Train se_resnext101 by 10 fold (1~10), inceptionresnetv2 by 5 folds (6~10) and pnasnet5large by 5 folds (1~5). As a result, 20 models are trained.</li>\n<li>Use the average output of all models (#TTA 2). And adjust the threshold for each image according to the max probability of that image. Supposing we have a probability matrix whose shape is (N_SAMPLES x N_CLASSES), the binary results can be calculated by \n<code>\nprob_mask = []\nfor prob in probabilities:\n    prob_mask.append(prob &amp;gt; prob.max()/7)\n</code>\nThis threshold calculation can boost the LB score about 0.005.</li>\n</ol>\n\n<h2>What didn't work for me:</h2>\n\n<ol>\n<li>Graph Convolutional Networks <a href=\"https://arxiv.org/abs/1904.03582\">https://arxiv.org/abs/1904.03582</a></li>\n<li>Soft Label and Pseudo Labeling</li>\n<li>EfficientNet</li>\n</ol>\n\n<p>Thanks.</p>",
      "rawMarkdown": "Congratulations to all the teams got the medal.\nCompared to other teams, my solution is quite simple (though I tried other tricks, none of them worked).\n\n## Solution\nMy code is available at https://github.com/YU1ut/imet-6th-soltuion.\nMy solution is based on this kernel https://www.kaggle.com/lopuhin/imet-2019-submission, and I modified it as follows.\n\n1. Change the input size to 320 by ```RandomCrop(320, pad_if_needed=True)```.\n2. Add Mixup and RandomErasing during training.\n3. Add pre-trained models from https://github.com/Cadene/pretrained-models.pytorch.\n4. Make a 40-fold CV and use some of them to train models.\n5. Train se_resnext101 by 10 fold (1~10), inceptionresnetv2 by 5 folds (6~10) and pnasnet5large by 5 folds (1~5). As a result, 20 models are trained.\n6. Use the average output of all models (#TTA 2). And adjust the threshold for each image according to the max probability of that image. Supposing we have a probability matrix whose shape is (N\\_SAMPLES x N\\_CLASSES), the binary results can be calculated by \n```\nprob_mask = []\nfor prob in probabilities:\n        prob_mask.append(prob &gt; prob.max()/7)\n```\nThis threshold calculation can boost the LB score about 0.005.\n\n## What didn't work for me:\n1. Graph Convolutional Networks https://arxiv.org/abs/1904.03582\n2. Soft Label and Pseudo Labeling\n3. EfficientNet\n\nThanks.",
      "votes": null
    },
    {
      "id": "550005",
      "postDate": "06/11/2019 08:02:32",
      "content": "<p>Congratulations.. \nThe  operation of adjusting the threshold for each image is awesome..</p>",
      "rawMarkdown": "Congratulations.. \nThe  operation of adjusting the threshold for each image is awesome..",
      "votes": null
    },
    {
      "id": "550077",
      "postDate": "06/11/2019 09:25:40",
      "content": "<p>Congrats!!!  And thanks for sharing.</p>",
      "rawMarkdown": "Congrats!!!  And thanks for sharing.",
      "votes": null
    },
    {
      "id": "550092",
      "postDate": "06/11/2019 09:46:58",
      "content": "<p>Congratulations!!</p>",
      "rawMarkdown": "Congratulations!!",
      "votes": null
    },
    {
      "id": "550165",
      "postDate": "06/11/2019 11:22:58",
      "content": "<p>Congratulations!  but I didn't see mixup in your code?</p>",
      "rawMarkdown": "Congratulations!  but I didn't see mixup in your code?",
      "votes": null
    },
    {
      "id": "550166",
      "postDate": "06/11/2019 11:24:36",
      "content": "<p>Mixup is implemented here:\n<a href=\"https://github.com/YU1ut/imet-6th-soltuion/blob/master/main.py#L235-L244\">https://github.com/YU1ut/imet-6th-soltuion/blob/master/main.py#L235-L244</a></p>",
      "rawMarkdown": "Mixup is implemented here:\nhttps://github.com/YU1ut/imet-6th-soltuion/blob/master/main.py#L235-L244",
      "votes": null
    },
    {
      "id": "550286",
      "postDate": "06/11/2019 13:18:48",
      "content": "<p>Thanks and congrats ! This is really interesting solution. Why is it masked dividing by 7 ? (prob &gt; prob.max()/7)</p>",
      "rawMarkdown": "Thanks and congrats ! This is really interesting solution. Why is it masked dividing by 7 ? (prob &gt; prob.max()/7)",
      "votes": null
    },
    {
      "id": "550858",
      "postDate": "06/12/2019 05:01:57",
      "content": "<p>congrats! gcn didn't work for me, too. thanks for sharing</p>",
      "rawMarkdown": "congrats! gcn didn't work for me, too. thanks for sharing",
      "votes": null
    },
    {
      "id": "550869",
      "postDate": "06/12/2019 05:26:53",
      "content": "<p>I tried different values on my local OOF validation data and 7 is the best.</p>",
      "rawMarkdown": "I tried different values on my local OOF validation data and 7 is the best.",
      "votes": null
    },
    {
      "id": "550915",
      "postDate": "06/12/2019 06:19:12",
      "content": "<p>Conratulations and thanks for sharing! \nIdea of thresholding each image is interesting : )</p>",
      "rawMarkdown": "Conratulations and thanks for sharing! \nIdea of thresholding each image is interesting : )",
      "votes": null
    },
    {
      "id": "550969",
      "postDate": "06/12/2019 07:27:54",
      "content": "<p>Thanks !</p>",
      "rawMarkdown": "Thanks !",
      "votes": null
    },
    {
      "id": "799801",
      "postDate": "04/06/2020 19:06:55",
      "content": "<p>Congrats ! Thanks for sharing!</p>",
      "rawMarkdown": "Congrats ! Thanks for sharing!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 550005,
      "author_name": "garybios",
      "author_url": "",
      "post_date": "06/11/2019 08:02:32",
      "content": "<p>Congratulations.. \nThe  operation of adjusting the threshold for each image is awesome..</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 550077,
      "author_name": "guiyuan320",
      "author_url": "",
      "post_date": "06/11/2019 09:25:40",
      "content": "<p>Congrats!!!  And thanks for sharing.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 550092,
      "author_name": "phalanx",
      "author_url": "",
      "post_date": "06/11/2019 09:46:58",
      "content": "<p>Congratulations!!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 550165,
      "author_name": "soywugzm",
      "author_url": "",
      "post_date": "06/11/2019 11:22:58",
      "content": "<p>Congratulations!  but I didn't see mixup in your code?</p>",
      "votes": null,
      "replies": [
        {
          "id": 550166,
          "author_name": "angelecarre",
          "author_url": "",
          "post_date": "06/11/2019 11:24:36",
          "content": "<p>Mixup is implemented here:\n<a href=\"https://github.com/YU1ut/imet-6th-soltuion/blob/master/main.py#L235-L244\">https://github.com/YU1ut/imet-6th-soltuion/blob/master/main.py#L235-L244</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 550286,
      "author_name": "harshthaker",
      "author_url": "",
      "post_date": "06/11/2019 13:18:48",
      "content": "<p>Thanks and congrats ! This is really interesting solution. Why is it masked dividing by 7 ? (prob &gt; prob.max()/7)</p>",
      "votes": null,
      "replies": [
        {
          "id": 550869,
          "author_name": "angelecarre",
          "author_url": "",
          "post_date": "06/12/2019 05:26:53",
          "content": "<p>I tried different values on my local OOF validation data and 7 is the best.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 550969,
          "author_name": "harshthaker",
          "author_url": "",
          "post_date": "06/12/2019 07:27:54",
          "content": "<p>Thanks !</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 550858,
      "author_name": "kevin0401",
      "author_url": "",
      "post_date": "06/12/2019 05:01:57",
      "content": "<p>congrats! gcn didn't work for me, too. thanks for sharing</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 550915,
      "author_name": "shiron8bit",
      "author_url": "",
      "post_date": "06/12/2019 06:19:12",
      "content": "<p>Conratulations and thanks for sharing! \nIdea of thresholding each image is interesting : )</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 799801,
      "author_name": "mielek",
      "author_url": "",
      "post_date": "04/06/2020 19:06:55",
      "content": "<p>Congrats ! Thanks for sharing!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "550000": "Congratulations to all the teams got the medal.\nCompared to other teams, my solution is quite simple (though I tried other tricks, none of them worked).\n\n## Solution\nMy code is available at https://github.com/YU1ut/imet-6th-soltuion.\nMy solution is based on this kernel https://www.kaggle.com/lopuhin/imet-2019-submission, and I modified it as follows.\n\n1. Change the input size to 320 by ```RandomCrop(320, pad_if_needed=True)```.\n2. Add Mixup and RandomErasing during training.\n3. Add pre-trained models from https://github.com/Cadene/pretrained-models.pytorch.\n4. Make a 40-fold CV and use some of them to train models.\n5. Train se_resnext101 by 10 fold (1~10), inceptionresnetv2 by 5 folds (6~10) and pnasnet5large by 5 folds (1~5). As a result, 20 models are trained.\n6. Use the average output of all models (#TTA 2). And adjust the threshold for each image according to the max probability of that image. Supposing we have a probability matrix whose shape is (N\\_SAMPLES x N\\_CLASSES), the binary results can be calculated by \n```\nprob_mask = []\nfor prob in probabilities:\n        prob_mask.append(prob &gt; prob.max()/7)\n```\nThis threshold calculation can boost the LB score about 0.005.\n\n## What didn't work for me:\n1. Graph Convolutional Networks https://arxiv.org/abs/1904.03582\n2. Soft Label and Pseudo Labeling\n3. EfficientNet\n\nThanks.",
    "550005": "Congratulations.. \nThe  operation of adjusting the threshold for each image is awesome..",
    "550077": "Congrats!!!  And thanks for sharing.",
    "550092": "Congratulations!!",
    "550165": "Congratulations!  but I didn't see mixup in your code?",
    "550166": "Mixup is implemented here:\nhttps://github.com/YU1ut/imet-6th-soltuion/blob/master/main.py#L235-L244",
    "550286": "Thanks and congrats ! This is really interesting solution. Why is it masked dividing by 7 ? (prob &gt; prob.max()/7)",
    "550858": "congrats! gcn didn't work for me, too. thanks for sharing",
    "550869": "I tried different values on my local OOF validation data and 7 is the best.",
    "550915": "Conratulations and thanks for sharing! \nIdea of thresholding each image is interesting : )",
    "550969": "Thanks !",
    "799801": "Congrats ! Thanks for sharing!"
  },
  "source": "meta"
}