{
  "id": 220875,
  "title": "Top %8 solution - (with unselected top %4)",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/220875",
  "author_name": "Sinan Calisir",
  "post_date": "2021-02-19T23:21:25.568000",
  "votes": 6,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hi everyone, </p>\n<p>First of all, congrats to all the winners. It is a fantastic achievement. Also thanks to Kaggle and the competition host. This was quite an interesting challenge with multiple aspects. The noise and the class imbalance were hard to tackle. </p>\n<p>Our team consists of two people (me and my great teammate <a href=\"https://www.kaggle.com/emirkocak\" target=\"_blank\">@emirkocak</a>). We finished the competition in the top %8. But like many other teams, we had a higher private LB submission which we did not select :) Here, we will summarize both of our solutions:</p>\n<p><strong>Our final submission (Top %8 - private: 0.8984):</strong></p>\n<p>We started this competition by visualizing and exploring the data. At first glance, it was somewhat confusing because of the wrong labels. With this in mind, we passed to the modeling stage.</p>\n<p>To set up a baseline score, we used this <a href=\"https://www.kaggle.com/yasufuminakama/cassava-resnext50-32x4d-starter-training/\" target=\"_blank\">notebook</a> (thanks a lot <a href=\"https://www.kaggle.com/yasufuminakama\" target=\"_blank\">@yasufuminakama</a> 🙏) and used the following augmentations. Here <code>HueSaturationValue</code> and <code>RandomBrightnessContrast</code> was optional. We experimented with the different versions of them.</p>\n<pre><code>def get_train_transforms():\n    return albumentations.Compose([\n        # Optionally only resize\n        # albumentations.Resize(512, 512),\n        albumentations.RandomResizedCrop(\n            512, 512, \n            scale=(0.6, 1.0), ratio=(3/5, 5/3)),\n        albumentations.Transpose(),\n        albumentations.HorizontalFlip(),\n        albumentations.VerticalFlip(),\n        albumentations.ShiftScaleRotate(),\n        albumentations.HueSaturationValue(\n            hue_shift_limit=0.2,\n            sat_shift_limit=0.2,\n            val_shift_limit=0.2,\n            p=0.5\n        ),\n        albumentations.RandomBrightnessContrast(\n            brightness_limit=(-0.1, 0.1),\n            contrast_limit=(-0.1, 0.1),\n            p=0.5\n        ),\n        albumentations.Normalize(\n            mean=[0.485, 0.456, 0.406], \n            std=[0.229, 0.224, 0.225]),\n        albumentations.Cutout(num_holes=8, max_h_size=32, max_w_size=32, fill_value=0, p=0.5),\n        ToTensorV2(),\n    ])\n</code></pre>\n<p><strong>Model Details:</strong></p>\n<ul>\n<li>Backbone: resnext50, efficient_net_b3, ViT</li>\n<li>2019 data</li>\n<li>Image size: <ul>\n<li>(resnext50, efficient_net_b3): 512</li>\n<li>(ViT): 384</li></ul></li>\n<li>Folds: 5</li>\n<li>Epoch: 10 (for each fold)</li>\n<li>Optimizer: Adam(lr=0.001)</li>\n<li>Scheduler: Cosine + Warmup (1 warmup + 9 cosine annealing)</li>\n<li>TTA: 5 for resnext50 and ViT, 8 for the efficient_net_b3<ul>\n<li>Here we applied both light and heavy tta. Heavy one performed better.</li></ul></li>\n</ul>\n<p><strong>Top %4 submission: (private: 0.8993)</strong></p>\n<p>This submission looks pretty much the same as above but a little bit lighter. I guess \"The Less is More\" philosophy was correct here :)</p>\n<ul>\n<li>Backbone: resnext50, efficient_net_b1</li>\n<li>Image size: 512</li>\n<li>Folds: 5</li>\n<li>Other parameters are the same</li>\n<li>TTA: 5<ul>\n<li>Opposite to the model above, here light TTA worked much better (only transpose and flip).</li></ul></li>\n</ul>\n<p><strong>Things did not work:</strong></p>\n<ul>\n<li>SnapMix and CutMix</li>\n<li>Pseudo Labelling</li>\n</ul>\n<p>It was funny and challenging competition for both of us. We learned a lot. Congratulations to the top teams again. ✌️</p>\n<p>Sinan &amp; Emir</p>",
  "messages": [
    {
      "id": 1211049,
      "postDate": "2021-02-19T23:21:25.570Z",
      "content": "<p>Hi everyone, </p>\n<p>First of all, congrats to all the winners. It is a fantastic achievement. Also thanks to Kaggle and the competition host. This was quite an interesting challenge with multiple aspects. The noise and the class imbalance were hard to tackle. </p>\n<p>Our team consists of two people (me and my great teammate <a href=\"https://www.kaggle.com/emirkocak\" target=\"_blank\">@emirkocak</a>). We finished the competition in the top %8. But like many other teams, we had a higher private LB submission which we did not select :) Here, we will summarize both of our solutions:</p>\n<p><strong>Our final submission (Top %8 - private: 0.8984):</strong></p>\n<p>We started this competition by visualizing and exploring the data. At first glance, it was somewhat confusing because of the wrong labels. With this in mind, we passed to the modeling stage.</p>\n<p>To set up a baseline score, we used this <a href=\"https://www.kaggle.com/yasufuminakama/cassava-resnext50-32x4d-starter-training/\" target=\"_blank\">notebook</a> (thanks a lot <a href=\"https://www.kaggle.com/yasufuminakama\" target=\"_blank\">@yasufuminakama</a> 🙏) and used the following augmentations. Here <code>HueSaturationValue</code> and <code>RandomBrightnessContrast</code> was optional. We experimented with the different versions of them.</p>\n<pre><code>def get_train_transforms():\n    return albumentations.Compose([\n        # Optionally only resize\n        # albumentations.Resize(512, 512),\n        albumentations.RandomResizedCrop(\n            512, 512, \n            scale=(0.6, 1.0), ratio=(3/5, 5/3)),\n        albumentations.Transpose(),\n        albumentations.HorizontalFlip(),\n        albumentations.VerticalFlip(),\n        albumentations.ShiftScaleRotate(),\n        albumentations.HueSaturationValue(\n            hue_shift_limit=0.2,\n            sat_shift_limit=0.2,\n            val_shift_limit=0.2,\n            p=0.5\n        ),\n        albumentations.RandomBrightnessContrast(\n            brightness_limit=(-0.1, 0.1),\n            contrast_limit=(-0.1, 0.1),\n            p=0.5\n        ),\n        albumentations.Normalize(\n            mean=[0.485, 0.456, 0.406], \n            std=[0.229, 0.224, 0.225]),\n        albumentations.Cutout(num_holes=8, max_h_size=32, max_w_size=32, fill_value=0, p=0.5),\n        ToTensorV2(),\n    ])\n</code></pre>\n<p><strong>Model Details:</strong></p>\n<ul>\n<li>Backbone: resnext50, efficient_net_b3, ViT</li>\n<li>2019 data</li>\n<li>Image size: <ul>\n<li>(resnext50, efficient_net_b3): 512</li>\n<li>(ViT): 384</li></ul></li>\n<li>Folds: 5</li>\n<li>Epoch: 10 (for each fold)</li>\n<li>Optimizer: Adam(lr=0.001)</li>\n<li>Scheduler: Cosine + Warmup (1 warmup + 9 cosine annealing)</li>\n<li>TTA: 5 for resnext50 and ViT, 8 for the efficient_net_b3<ul>\n<li>Here we applied both light and heavy tta. Heavy one performed better.</li></ul></li>\n</ul>\n<p><strong>Top %4 submission: (private: 0.8993)</strong></p>\n<p>This submission looks pretty much the same as above but a little bit lighter. I guess \"The Less is More\" philosophy was correct here :)</p>\n<ul>\n<li>Backbone: resnext50, efficient_net_b1</li>\n<li>Image size: 512</li>\n<li>Folds: 5</li>\n<li>Other parameters are the same</li>\n<li>TTA: 5<ul>\n<li>Opposite to the model above, here light TTA worked much better (only transpose and flip).</li></ul></li>\n</ul>\n<p><strong>Things did not work:</strong></p>\n<ul>\n<li>SnapMix and CutMix</li>\n<li>Pseudo Labelling</li>\n</ul>\n<p>It was funny and challenging competition for both of us. We learned a lot. Congratulations to the top teams again. ✌️</p>\n<p>Sinan &amp; Emir</p>",
      "rawMarkdown": "Hi everyone, \n\nFirst of all, congrats to all the winners. It is a fantastic achievement. Also thanks to Kaggle and the competition host. This was quite an interesting challenge with multiple aspects. The noise and the class imbalance were hard to tackle. \n\nOur team consists of two people (me and my great teammate @emirkocak). We finished the competition in the top %8. But like many other teams, we had a higher private LB submission which we did not select :) Here, we will summarize both of our solutions:\n\n**Our final submission (Top %8 - private: 0.8984):**\n\nWe started this competition by visualizing and exploring the data. At first glance, it was somewhat confusing because of the wrong labels. With this in mind, we passed to the modeling stage.\n\nTo set up a baseline score, we used this [notebook](https://www.kaggle.com/yasufuminakama/cassava-resnext50-32x4d-starter-training/) (thanks a lot @yasufuminakama 🙏) and used the following augmentations. Here `HueSaturationValue` and `RandomBrightnessContrast` was optional. We experimented with the different versions of them.\n\n```python\ndef get_train_transforms():\n    return albumentations.Compose([\n        # Optionally only resize\n        # albumentations.Resize(512, 512),\n        albumentations.RandomResizedCrop(\n            512, 512, \n            scale=(0.6, 1.0), ratio=(3/5, 5/3)),\n        albumentations.Transpose(),\n        albumentations.HorizontalFlip(),\n        albumentations.VerticalFlip(),\n        albumentations.ShiftScaleRotate(),\n        albumentations.HueSaturationValue(\n            hue_shift_limit=0.2,\n            sat_shift_limit=0.2,\n            val_shift_limit=0.2,\n            p=0.5\n        ),\n        albumentations.RandomBrightnessContrast(\n            brightness_limit=(-0.1, 0.1),\n            contrast_limit=(-0.1, 0.1),\n            p=0.5\n        ),\n        albumentations.Normalize(\n            mean=[0.485, 0.456, 0.406], \n            std=[0.229, 0.224, 0.225]),\n        albumentations.Cutout(num_holes=8, max_h_size=32, max_w_size=32, fill_value=0, p=0.5),\n        ToTensorV2(),\n    ])\n```\n\n**Model Details:**\n* Backbone: resnext50, efficient_net_b3, ViT\n* 2019 data\n* Image size: \n    * (resnext50, efficient_net_b3): 512\n    * (ViT): 384\n* Folds: 5\n* Epoch: 10 (for each fold)\n* Optimizer: Adam(lr=0.001)\n* Scheduler: Cosine + Warmup (1 warmup + 9 cosine annealing)\n* TTA: 5 for resnext50 and ViT, 8 for the efficient_net_b3\n  * Here we applied both light and heavy tta. Heavy one performed better.\n\n**Top %4 submission: (private: 0.8993)**\n\nThis submission looks pretty much the same as above but a little bit lighter. I guess \"The Less is More\" philosophy was correct here :)\n\n* Backbone: resnext50, efficient_net_b1\n* Image size: 512\n* Folds: 5\n* Other parameters are the same\n* TTA: 5\n  * Opposite to the model above, here light TTA worked much better (only transpose and flip).\n\n**Things did not work:**\n* SnapMix and CutMix\n* Pseudo Labelling\n\nIt was funny and challenging competition for both of us. We learned a lot. Congratulations to the top teams again. ✌️\n\nSinan & Emir",
      "votes": 6
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1211049": "Hi everyone, \n\nFirst of all, congrats to all the winners. It is a fantastic achievement. Also thanks to Kaggle and the competition host. This was quite an interesting challenge with multiple aspects. The noise and the class imbalance were hard to tackle. \n\nOur team consists of two people (me and my great teammate @emirkocak). We finished the competition in the top %8. But like many other teams, we had a higher private LB submission which we did not select :) Here, we will summarize both of our solutions:\n\n**Our final submission (Top %8 - private: 0.8984):**\n\nWe started this competition by visualizing and exploring the data. At first glance, it was somewhat confusing because of the wrong labels. With this in mind, we passed to the modeling stage.\n\nTo set up a baseline score, we used this [notebook](https://www.kaggle.com/yasufuminakama/cassava-resnext50-32x4d-starter-training/) (thanks a lot @yasufuminakama 🙏) and used the following augmentations. Here `HueSaturationValue` and `RandomBrightnessContrast` was optional. We experimented with the different versions of them.\n\n```python\ndef get_train_transforms():\n    return albumentations.Compose([\n        # Optionally only resize\n        # albumentations.Resize(512, 512),\n        albumentations.RandomResizedCrop(\n            512, 512, \n            scale=(0.6, 1.0), ratio=(3/5, 5/3)),\n        albumentations.Transpose(),\n        albumentations.HorizontalFlip(),\n        albumentations.VerticalFlip(),\n        albumentations.ShiftScaleRotate(),\n        albumentations.HueSaturationValue(\n            hue_shift_limit=0.2,\n            sat_shift_limit=0.2,\n            val_shift_limit=0.2,\n            p=0.5\n        ),\n        albumentations.RandomBrightnessContrast(\n            brightness_limit=(-0.1, 0.1),\n            contrast_limit=(-0.1, 0.1),\n            p=0.5\n        ),\n        albumentations.Normalize(\n            mean=[0.485, 0.456, 0.406], \n            std=[0.229, 0.224, 0.225]),\n        albumentations.Cutout(num_holes=8, max_h_size=32, max_w_size=32, fill_value=0, p=0.5),\n        ToTensorV2(),\n    ])\n```\n\n**Model Details:**\n* Backbone: resnext50, efficient_net_b3, ViT\n* 2019 data\n* Image size: \n    * (resnext50, efficient_net_b3): 512\n    * (ViT): 384\n* Folds: 5\n* Epoch: 10 (for each fold)\n* Optimizer: Adam(lr=0.001)\n* Scheduler: Cosine + Warmup (1 warmup + 9 cosine annealing)\n* TTA: 5 for resnext50 and ViT, 8 for the efficient_net_b3\n  * Here we applied both light and heavy tta. Heavy one performed better.\n\n**Top %4 submission: (private: 0.8993)**\n\nThis submission looks pretty much the same as above but a little bit lighter. I guess \"The Less is More\" philosophy was correct here :)\n\n* Backbone: resnext50, efficient_net_b1\n* Image size: 512\n* Folds: 5\n* Other parameters are the same\n* TTA: 5\n  * Opposite to the model above, here light TTA worked much better (only transpose and flip).\n\n**Things did not work:**\n* SnapMix and CutMix\n* Pseudo Labelling\n\nIt was funny and challenging competition for both of us. We learned a lot. Congratulations to the top teams again. ✌️\n\nSinan & Emir"
  }
}