{
  "id": 309582,
  "title": "Things that improve performance",
  "url": "/competitions/happy-whale-and-dolphin/discussion/309582",
  "author_name": "",
  "post_date": "2022-02-24T09:56:31.278272500Z",
  "votes": 22,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Using Ref <a href=\"https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop\" target=\"_blank\">https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop</a><br>\nGetting LB score 0.679 in 20 epochs with a small trick</p>\n<p><strong>Boosters:</strong></p>\n<ol>\n<li>Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]</li>\n<li>Introduced Multi-sample dropout before embedding layer</li>\n</ol>\n<p><strong>Failures: {Performance reduced}</strong></p>\n<ol>\n<li>Concat [GAP, GEM]</li>\n<li>Concat [GAP, GMP] results in nan loss</li>\n<li>Cutout</li>\n<li>Gridmask was not very effective</li>\n<li>Cutmix [Unable to implement properly]</li>\n</ol>\n<p><strong>Notes:</strong></p>\n<ol>\n<li>Usual augmentation schemes don't seem to work well on the challenge. Or am I missing something</li>\n<li>There's good amount of overfitting. Need to fix this</li>\n<li><strong>Data augment is applied for the validation dataset, eval, and test dataset. Kindly fix that in the notebook</strong></li>\n</ol>\n<p><strong>Reference Notebooks:</strong></p>\n<ol>\n<li><a href=\"https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu\" target=\"_blank\">https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu</a></li>\n<li><a href=\"https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop\" target=\"_blank\">https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop</a></li>\n</ol>\n<p><strong>Best Score:</strong></p>\n<ul>\n<li>~45K images in the training</li>\n<li>40 epochs</li>\n<li>Single Model</li>\n<li>Eff-B7 </li>\n<li>600</li>\n<li>LB: 0.813</li>\n</ul>\n<p><strong>Dataset Analysis</strong><br>\nFull Body annotation &gt; Detic &gt; Yolov5 </p>",
  "messages": [
    {
      "id": "1703218",
      "postDate": "02/24/2022 09:56:31",
      "content": "<p>Using Ref <a href=\"https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop\" target=\"_blank\">https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop</a><br>\nGetting LB score 0.679 in 20 epochs with a small trick</p>\n<p><strong>Boosters:</strong></p>\n<ol>\n<li>Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]</li>\n<li>Introduced Multi-sample dropout before embedding layer</li>\n</ol>\n<p><strong>Failures: {Performance reduced}</strong></p>\n<ol>\n<li>Concat [GAP, GEM]</li>\n<li>Concat [GAP, GMP] results in nan loss</li>\n<li>Cutout</li>\n<li>Gridmask was not very effective</li>\n<li>Cutmix [Unable to implement properly]</li>\n</ol>\n<p><strong>Notes:</strong></p>\n<ol>\n<li>Usual augmentation schemes don't seem to work well on the challenge. Or am I missing something</li>\n<li>There's good amount of overfitting. Need to fix this</li>\n<li><strong>Data augment is applied for the validation dataset, eval, and test dataset. Kindly fix that in the notebook</strong></li>\n</ol>\n<p><strong>Reference Notebooks:</strong></p>\n<ol>\n<li><a href=\"https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu\" target=\"_blank\">https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu</a></li>\n<li><a href=\"https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop\" target=\"_blank\">https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop</a></li>\n</ol>\n<p><strong>Best Score:</strong></p>\n<ul>\n<li>~45K images in the training</li>\n<li>40 epochs</li>\n<li>Single Model</li>\n<li>Eff-B7 </li>\n<li>600</li>\n<li>LB: 0.813</li>\n</ul>\n<p><strong>Dataset Analysis</strong><br>\nFull Body annotation &gt; Detic &gt; Yolov5 </p>",
      "rawMarkdown": "Using Ref https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop\nGetting LB score 0.679 in 20 epochs with a small trick\n\n**Boosters:**\n1. Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]\n2. Introduced Multi-sample dropout before embedding layer\n\n**Failures: {Performance reduced}**\n1. Concat [GAP, GEM]\n2. Concat [GAP, GMP] results in nan loss\n3. Cutout\n4. Gridmask was not very effective\n5. Cutmix [Unable to implement properly]\n\n**Notes:**\n1. Usual augmentation schemes don't seem to work well on the challenge. Or am I missing something\n2. There's good amount of overfitting. Need to fix this\n3. **Data augment is applied for the validation dataset, eval, and test dataset. Kindly fix that in the notebook**\n\n\n**Reference Notebooks:**\n1. https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu\n2. https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop\n\n\n**Best Score:**\n- ~45K images in the training\n- 40 epochs\n- Single Model\n- Eff-B7 \n- 600\n- LB: 0.813\n\n**Dataset Analysis**\nFull Body annotation > Detic > Yolov5",
      "votes": null
    },
    {
      "id": "1750233",
      "postDate": "04/09/2022 12:44:37",
      "content": "<p>Thank you for your great sharing ！</p>\n<p>I have 1 question .(Maybe  this is stupid question)(Because Im beginner of deeplearning…)</p>\n<blockquote>\n  <p>Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]</p>\n</blockquote>\n<p>What does it mean?<br>\nIs bn wrong in the notebook ?</p>",
      "rawMarkdown": "Thank you for your great sharing ！\n\nI have 1 question .(Maybe  this is stupid question)(Because Im beginner of deeplearning…)\n\n> Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]\n\nWhat does it mean?\nIs bn wrong in the notebook ?",
      "votes": null
    },
    {
      "id": "1751454",
      "postDate": "04/10/2022 19:14:47",
      "content": "<blockquote>\n  <ul>\n  <li>Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]</li>\n  <li>Introduced Multi-sample dropout before embedding layer</li>\n  </ul>\n</blockquote>\n<p>I agree. In notebook function does not freeze BN layers (actually this function does not freeze anything).  Today I implemented freeze BN for Efnet - for me it does not give any gain for score ☹️ </p>\n<p>Could you plot model with multi-sample dropout? Why? Because I implemented is as well. Now it requires more computational power, memory … and … time to train model. As a result I have not scored better so far. This is really cool idea so I am afraid I have not implemented it properly. </p>",
      "rawMarkdown": "> * Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]\n* Introduced Multi-sample dropout before embedding layer\n\nI agree. In notebook function does not freeze BN layers (actually this function does not freeze anything).  Today I implemented freeze BN for Efnet - for me it does not give any gain for score ☹️ \n\nCould you plot model with multi-sample dropout? Why? Because I implemented is as well. Now it requires more computational power, memory ... and ... time to train model. As a result I have not scored better so far. This is really cool idea so I am afraid I have not implemented it properly.",
      "votes": null
    },
    {
      "id": "1751469",
      "postDate": "04/10/2022 19:43:22",
      "content": "<p>Once the competition is over. I will share my model design.<br>\nYou can have a look and share your suggestion </p>",
      "rawMarkdown": "Once the competition is over. I will share my model design.\nYou can have a look and share your suggestion",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1750233,
      "author_name": "whitelily",
      "author_url": "",
      "post_date": "04/09/2022 12:44:37",
      "content": "<p>Thank you for your great sharing ！</p>\n<p>I have 1 question .(Maybe  this is stupid question)(Because Im beginner of deeplearning…)</p>\n<blockquote>\n  <p>Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]</p>\n</blockquote>\n<p>What does it mean?<br>\nIs bn wrong in the notebook ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1751454,
      "author_name": "remekkinas",
      "author_url": "",
      "post_date": "04/10/2022 19:14:47",
      "content": "<blockquote>\n  <ul>\n  <li>Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]</li>\n  <li>Introduced Multi-sample dropout before embedding layer</li>\n  </ul>\n</blockquote>\n<p>I agree. In notebook function does not freeze BN layers (actually this function does not freeze anything).  Today I implemented freeze BN for Efnet - for me it does not give any gain for score ☹️ </p>\n<p>Could you plot model with multi-sample dropout? Why? Because I implemented is as well. Now it requires more computational power, memory … and … time to train model. As a result I have not scored better so far. This is really cool idea so I am afraid I have not implemented it properly. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1751469,
          "author_name": "dhakshiin1601",
          "author_url": "",
          "post_date": "04/10/2022 19:43:22",
          "content": "<p>Once the competition is over. I will share my model design.<br>\nYou can have a look and share your suggestion </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1703218": "Using Ref https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop\nGetting LB score 0.679 in 20 epochs with a small trick\n\n**Boosters:**\n1. Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]\n2. Introduced Multi-sample dropout before embedding layer\n\n**Failures: {Performance reduced}**\n1. Concat [GAP, GEM]\n2. Concat [GAP, GMP] results in nan loss\n3. Cutout\n4. Gridmask was not very effective\n5. Cutmix [Unable to implement properly]\n\n**Notes:**\n1. Usual augmentation schemes don't seem to work well on the challenge. Or am I missing something\n2. There's good amount of overfitting. Need to fix this\n3. **Data augment is applied for the validation dataset, eval, and test dataset. Kindly fix that in the notebook**\n\n\n**Reference Notebooks:**\n1. https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu\n2. https://www.kaggle.com/dragonzhang/happywhale-effnet-b7-fork-with-detic-crop\n\n\n**Best Score:**\n- ~45K images in the training\n- 40 epochs\n- Single Model\n- Eff-B7 \n- 600\n- LB: 0.813\n\n**Dataset Analysis**\nFull Body annotation > Detic > Yolov5",
    "1750233": "Thank you for your great sharing ！\n\nI have 1 question .(Maybe  this is stupid question)(Because Im beginner of deeplearning…)\n\n> Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]\n\nWhat does it mean?\nIs bn wrong in the notebook ?",
    "1751454": "> * Freeze Batchnorm [The freeze_bn in the notebook is not correct. Modify the notebook properly]\n* Introduced Multi-sample dropout before embedding layer\n\nI agree. In notebook function does not freeze BN layers (actually this function does not freeze anything).  Today I implemented freeze BN for Efnet - for me it does not give any gain for score ☹️ \n\nCould you plot model with multi-sample dropout? Why? Because I implemented is as well. Now it requires more computational power, memory ... and ... time to train model. As a result I have not scored better so far. This is really cool idea so I am afraid I have not implemented it properly.",
    "1751469": "Once the competition is over. I will share my model design.\nYou can have a look and share your suggestion"
  },
  "source": "meta"
}