{
  "id": 220813,
  "title": "35th place solution",
  "url": "/competitions/cassava-leaf-disease-classification/writeups/trust-cv-35th-place-solution",
  "author_name": "",
  "post_date": "2021-02-19T16:14:24.857Z",
  "votes": 9,
  "comment_count": 3,
  "views": 0,
  "content": "<p>First, thanks to my great teammates for the hard work!</p>\n<p>In our solution we trained a B0, and tried several approaches and measured the CV improvement. </p>\n<h2>Filter Noisy Label</h2>\n<p>We checked the best model out of fold results with the labels and remove those label, which confidence difference with the ground truth was large. We used this Filter Noisy Label dataset for training, but we validated it on the original dataset.</p>\n<h2>Models</h2>\n<p>We had 8 categories during our training. </p>\n<p><strong>Model Category 1</strong></p>\n<ul>\n<li>Trained in Full Training Data</li>\n<li>Filter Noisy Label</li>\n<li>No external data</li>\n<li>No Cutmix</li>\n<li>Taylor + Cross Entropy Loss</li>\n<li>Image Size 666 500 -&gt; cropped to 500 500</li>\n<li>Noisy Student</li>\n</ul>\n<p><strong>Model Category 2</strong> </p>\n<ul>\n<li><p>The difference with Category 1:</p></li>\n<li><p><em>Image Size 600 800 -&gt; cropped to 600 600</em></p>\n<ul>\n<li>pretrained weights: Imagenet</li></ul></li>\n</ul>\n<p><strong>Model Category 3</strong></p>\n<ul>\n<li><p>The difference with Category 1:</p></li>\n<li><p>Using cutmix</p>\n<ul>\n<li>pretrained weights: Imagenet</li></ul></li>\n</ul>\n<p><strong>Model Category 4</strong></p>\n<ul>\n<li><p>The difference with Category 2:</p></li>\n<li><p>Using cutmix</p>\n<ul>\n<li>pretrained weights: Imagenet</li></ul></li>\n</ul>\n<p>With the 4 ensembled categories, we generated pseudo labels.</p>\n<p><strong>Category 5</strong>: same as Categor 1 + External data (2019 dataset) + Pseudolabels </p>\n<p><strong>Category 6</strong>: same as Categor 2 + External data (2019 dataset) + Pseudolabels </p>\n<p><strong>Category 7</strong>: same as Categor 3 + External data (2019 dataset) + Pseudolabels </p>\n<p><strong>Category 8</strong>: same as Categor 4 + External data (2019 dataset) + Pseudolabels</p>\n<p>We trained most of EfficentNet models (B0 to B4), and ResNext50 for each categories. We calculated the inference time for each model, and the accuracy for 8 TTA = ['rotate0', 'rotate90', 'rotate180','rotate270', 'rotate0_lr', 'rotate90_lr', 'rotate180_lr','rotate270_lr']. Based on the models out of fold results per TTA, we optimize their soft voting weights, and TTA numbers with a BayesianOptimization. </p>\n<p>In the end, we added a ViT model from a public notebook and ensembled. </p>",
  "messages": [
    {
      "id": "1210706",
      "postDate": "02/19/2021 16:10:36",
      "content": "<p>First, thanks to my great teammates for the hard work!</p>\n<p>In our solution we trained a B0, and tried several approaches and measured the CV improvement. </p>\n<h2>Filter Noisy Label</h2>\n<p>We checked the best model out of fold results with the labels and remove those label, which confidence difference with the ground truth was large. We used this Filter Noisy Label dataset for training, but we validated it on the original dataset.</p>\n<h2>Models</h2>\n<p>We had 8 categories during our training. </p>\n<p><strong>Model Category 1</strong></p>\n<ul>\n<li>Trained in Full Training Data</li>\n<li>Filter Noisy Label</li>\n<li>No external data</li>\n<li>No Cutmix</li>\n<li>Taylor + Cross Entropy Loss</li>\n<li>Image Size 666 500 -&gt; cropped to 500 500</li>\n<li>Noisy Student</li>\n</ul>\n<p><strong>Model Category 2</strong> </p>\n<ul>\n<li><p>The difference with Category 1:</p></li>\n<li><p><em>Image Size 600 800 -&gt; cropped to 600 600</em></p>\n<ul>\n<li>pretrained weights: Imagenet</li></ul></li>\n</ul>\n<p><strong>Model Category 3</strong></p>\n<ul>\n<li><p>The difference with Category 1:</p></li>\n<li><p>Using cutmix</p>\n<ul>\n<li>pretrained weights: Imagenet</li></ul></li>\n</ul>\n<p><strong>Model Category 4</strong></p>\n<ul>\n<li><p>The difference with Category 2:</p></li>\n<li><p>Using cutmix</p>\n<ul>\n<li>pretrained weights: Imagenet</li></ul></li>\n</ul>\n<p>With the 4 ensembled categories, we generated pseudo labels.</p>\n<p><strong>Category 5</strong>: same as Categor 1 + External data (2019 dataset) + Pseudolabels </p>\n<p><strong>Category 6</strong>: same as Categor 2 + External data (2019 dataset) + Pseudolabels </p>\n<p><strong>Category 7</strong>: same as Categor 3 + External data (2019 dataset) + Pseudolabels </p>\n<p><strong>Category 8</strong>: same as Categor 4 + External data (2019 dataset) + Pseudolabels</p>\n<p>We trained most of EfficentNet models (B0 to B4), and ResNext50 for each categories. We calculated the inference time for each model, and the accuracy for 8 TTA = ['rotate0', 'rotate90', 'rotate180','rotate270', 'rotate0_lr', 'rotate90_lr', 'rotate180_lr','rotate270_lr']. Based on the models out of fold results per TTA, we optimize their soft voting weights, and TTA numbers with a BayesianOptimization. </p>\n<p>In the end, we added a ViT model from a public notebook and ensembled. </p>",
      "rawMarkdown": "First, thanks to my great teammates for the hard work!\n\nIn our solution we trained a B0, and tried several approaches and measured the CV improvement. \n\n## Filter Noisy Label\n\nWe checked the best model out of fold results with the labels and remove those label, which confidence difference with the ground truth was large. We used this Filter Noisy Label dataset for training, but we validated it on the original dataset.\n\n## Models\n\nWe had 8 categories during our training. \n\n**Model Category 1**\n\n- Trained in Full Training Data\n- Filter Noisy Label\n- No external data\n- No Cutmix\n- Taylor + Cross Entropy Loss\n- Image Size 666 500 -> cropped to 500 500\n- Noisy Student\n\n**Model Category 2** \n\n- The difference with Category 1:\n\n- *Image Size 600 800 -> cropped to 600 600*\n  - pretrained weights: Imagenet\n\n**Model Category 3**\n\n- The difference with Category 1:\n\n- Using cutmix\n  - pretrained weights: Imagenet\n\n**Model Category 4**\n\n- The difference with Category 2:\n\n- Using cutmix\n  - pretrained weights: Imagenet\n\nWith the 4 ensembled categories, we generated pseudo labels.\n\n**Category 5**: same as Categor 1 + External data (2019 dataset) + Pseudolabels \n\n**Category 6**: same as Categor 2 + External data (2019 dataset) + Pseudolabels \n\n**Category 7**: same as Categor 3 + External data (2019 dataset) + Pseudolabels \n\n**Category 8**: same as Categor 4 + External data (2019 dataset) + Pseudolabels\n\nWe trained most of EfficentNet models (B0 to B4), and ResNext50 for each categories. We calculated the inference time for each model, and the accuracy for 8 TTA = ['rotate0', 'rotate90', 'rotate180','rotate270', 'rotate0_lr', 'rotate90_lr', 'rotate180_lr','rotate270_lr']. Based on the models out of fold results per TTA, we optimize their soft voting weights, and TTA numbers with a BayesianOptimization. \n\nIn the end, we added a ViT model from a public notebook and ensembled.",
      "votes": null
    },
    {
      "id": "1210750",
      "postDate": "02/19/2021 16:46:39",
      "content": "<p>You're true gem. Can you share your complete notebook link?</p>",
      "rawMarkdown": "You're true gem. Can you share your complete notebook link?",
      "votes": null
    },
    {
      "id": "1211982",
      "postDate": "02/20/2021 18:07:28",
      "content": "<p><a href=\"https://www.kaggle.com/bessenyeiszilrd/final-submission-private-lb-0-901\" target=\"_blank\">https://www.kaggle.com/bessenyeiszilrd/final-submission-private-lb-0-901</a></p>",
      "rawMarkdown": "https://www.kaggle.com/bessenyeiszilrd/final-submission-private-lb-0-901",
      "votes": null
    },
    {
      "id": "1212001",
      "postDate": "02/20/2021 18:49:17",
      "content": "<p>Thank you so much </p>",
      "rawMarkdown": "Thank you so much",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1210750,
      "author_name": "mnavaidd",
      "author_url": "",
      "post_date": "02/19/2021 16:46:39",
      "content": "<p>You're true gem. Can you share your complete notebook link?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1211982,
          "author_name": "bessenyeiszilrd",
          "author_url": "",
          "post_date": "02/20/2021 18:07:28",
          "content": "<p><a href=\"https://www.kaggle.com/bessenyeiszilrd/final-submission-private-lb-0-901\" target=\"_blank\">https://www.kaggle.com/bessenyeiszilrd/final-submission-private-lb-0-901</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1212001,
          "author_name": "mnavaidd",
          "author_url": "",
          "post_date": "02/20/2021 18:49:17",
          "content": "<p>Thank you so much </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1210706": "First, thanks to my great teammates for the hard work!\n\nIn our solution we trained a B0, and tried several approaches and measured the CV improvement. \n\n## Filter Noisy Label\n\nWe checked the best model out of fold results with the labels and remove those label, which confidence difference with the ground truth was large. We used this Filter Noisy Label dataset for training, but we validated it on the original dataset.\n\n## Models\n\nWe had 8 categories during our training. \n\n**Model Category 1**\n\n- Trained in Full Training Data\n- Filter Noisy Label\n- No external data\n- No Cutmix\n- Taylor + Cross Entropy Loss\n- Image Size 666 500 -> cropped to 500 500\n- Noisy Student\n\n**Model Category 2** \n\n- The difference with Category 1:\n\n- *Image Size 600 800 -> cropped to 600 600*\n  - pretrained weights: Imagenet\n\n**Model Category 3**\n\n- The difference with Category 1:\n\n- Using cutmix\n  - pretrained weights: Imagenet\n\n**Model Category 4**\n\n- The difference with Category 2:\n\n- Using cutmix\n  - pretrained weights: Imagenet\n\nWith the 4 ensembled categories, we generated pseudo labels.\n\n**Category 5**: same as Categor 1 + External data (2019 dataset) + Pseudolabels \n\n**Category 6**: same as Categor 2 + External data (2019 dataset) + Pseudolabels \n\n**Category 7**: same as Categor 3 + External data (2019 dataset) + Pseudolabels \n\n**Category 8**: same as Categor 4 + External data (2019 dataset) + Pseudolabels\n\nWe trained most of EfficentNet models (B0 to B4), and ResNext50 for each categories. We calculated the inference time for each model, and the accuracy for 8 TTA = ['rotate0', 'rotate90', 'rotate180','rotate270', 'rotate0_lr', 'rotate90_lr', 'rotate180_lr','rotate270_lr']. Based on the models out of fold results per TTA, we optimize their soft voting weights, and TTA numbers with a BayesianOptimization. \n\nIn the end, we added a ViT model from a public notebook and ensembled.",
    "1210750": "You're true gem. Can you share your complete notebook link?",
    "1211982": "https://www.kaggle.com/bessenyeiszilrd/final-submission-private-lb-0-901",
    "1212001": "Thank you so much"
  },
  "source": "meta"
}