{
  "id": 220672,
  "title": "Private LB 105th / Public LB 970th solution and first medal 🥳",
  "url": "/competitions/cassava-leaf-disease-classification/writeups/samsan-tech-private-lb-105th-public-lb-970th-solut",
  "author_name": "",
  "post_date": "2021-03-31T11:03:37.563Z",
  "votes": 12,
  "comment_count": 12,
  "views": 0,
  "content": "<p>First of all, congratulations to all the competitors. Huge thanks to Kaggle, Makerere University AI Lab for organising this competition.  <br>\nI'd also like to thank my amazing teammate, <a href=\"https://www.kaggle.com/kjkr73\" target=\"_blank\">@kjkr73</a>. He did most of the heavy lifting. He'll be publishing a kernel soon.</p>\n<p>⚠  We actually jumped to 105. :)</p>\n<h2>Data</h2>\n<ul>\n<li>We only used the competition data.</li>\n</ul>\n<h2>Validation Strategy</h2>\n<ul>\n<li>Stratified 5-fold.</li>\n<li>My CV score and LB score were correlated from the very start so I didn't worry about it too much.</li>\n<li>Trusted our CV but didn't disregard Public LB entirely.</li>\n</ul>\n<h2>Image Preprocessing And Augmentations</h2>\n<ul>\n<li>We trained models on 224*224, 256*256, 384*384 and 512*512 image size. Larger image size always had better scores.</li>\n<li>We tried multiple combinations of augmentations. Following combinations performed the best:<ul>\n<li>Cutout, HorizontalFlip, VerticalFlip, ShiftScaleRotate, HueSaturationValue, RandomBrightnessContrast.</li>\n<li>Cutout, CoarseDropout,  HorizontalFlip, VerticalFlip.</li></ul></li>\n</ul>\n<h2>Neural Network Architectures</h2>\n<ul>\n<li>We tried ResNet34, ResNext50_32x4d, EfficientNet B1 NS, EfficientNet B3 NS, EfficientNet B4 NS, ViT-Base patch 16, and DeiT-Base patch 16 224.</li>\n</ul>\n<h2>Miscellaneous</h2>\n<ul>\n<li>Loss(es):<ul>\n<li>Bi-tempered loss with label smoothing</li>\n<li>Taylor Crossentropy with label smoothing</li></ul></li>\n<li>Optimizer:<ul>\n<li>Adam</li></ul></li>\n<li>LR Scheduler(s):<ul>\n<li>OneCycleLR</li>\n<li>CosineAnnealingWarmRestarts</li></ul></li>\n<li>Hardware/ platform:<ul>\n<li>Kaggle kernels</li></ul></li>\n</ul>\n<h2>Final Submissions</h2>\n<ul>\n<li>Selected Submission 1 (Private 0.8999, Public 0.9009) is a blend of:<ul>\n<li>EfficientNet B1 NS on 512*512 image size [1xTTA].</li>\n<li>ResNext50_32x4d on 512*512 image size [1xTTA].</li>\n<li>ResNext50_32x4d on 512*512 image size [No TTA].</li>\n<li>EfficientNet B3 NS on 512*512 image size [No TTA].   </li></ul></li>\n<li>Selected Submission 2 (Private 0.8981, Public 0.9002) is a blend of:<ul>\n<li>EfficientNet B1 NS on 512*512 image size [1xTTA].</li>\n<li>ResNext50_32x4d on 512*512 image size [No TTA].</li>\n<li>EfficientNet B3 NS on 512*512 image size [No TTA].   </li></ul></li>\n</ul>\n<h2>Some observations/ tips for competing in a CV comp with Kaggle Kernels only:</h2>\n<ul>\n<li>Data preprocessing != data augmentation. Resize images to the intended size before training, it will save you some GPU quota and speed up your experiments. Credits: <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a>; read more <a href=\"https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/161446\" target=\"_blank\">here</a>.</li>\n<li>Create a proof of concept kernel which is essentially a smaller version of the training kernel for fast experimentation or to check the feasibility of an idea. This kernel should run quickly and return an OOF CV score. You can make this kernel fast with a smaller number of CV folds/ smaller number of epochs. If you see improvement in this CV score, copy the config and run it on your training kernel.</li>\n<li>Write modular, maintainable, and readable code.</li>\n</ul>\n<h2>Credits</h2>\n<ul>\n<li><code>pandas</code> <a href=\"https://github.com/pandas-dev/pandas\" target=\"_blank\">https://github.com/pandas-dev/pandas</a></li>\n<li><code>torch</code> <a href=\"https://github.com/pytorch/pytorch\" target=\"_blank\">https://github.com/pytorch/pytorch</a></li>\n<li><code>timm</code> <a href=\"https://github.com/rwightman/pytorch-image-models\" target=\"_blank\">https://github.com/rwightman/pytorch-image-models</a>.</li>\n<li><code>albumentations</code> <a href=\"https://github.com/albumentations-team/albumentations\" target=\"_blank\">https://github.com/albumentations-team/albumentations</a></li>\n<li><code>lycon</code> <a href=\"https://github.com/ethereon/lycon\" target=\"_blank\">https://github.com/ethereon/lycon</a></li>\n</ul>\n<p>PS: This is our (my and <a href=\"https://www.kaggle.com/kjkr73\" target=\"_blank\">@kjkr73</a>'s) first medal 🥳<br>\nEdit: Updated LB scores and rank.</p>",
  "messages": [
    {
      "id": "1210009",
      "postDate": "02/19/2021 06:47:21",
      "content": "<p>First of all, congratulations to all the competitors. Huge thanks to Kaggle, Makerere University AI Lab for organising this competition.  <br>\nI'd also like to thank my amazing teammate, <a href=\"https://www.kaggle.com/kjkr73\" target=\"_blank\">@kjkr73</a>. He did most of the heavy lifting. He'll be publishing a kernel soon.</p>\n<p>⚠  We actually jumped to 105. :)</p>\n<h2>Data</h2>\n<ul>\n<li>We only used the competition data.</li>\n</ul>\n<h2>Validation Strategy</h2>\n<ul>\n<li>Stratified 5-fold.</li>\n<li>My CV score and LB score were correlated from the very start so I didn't worry about it too much.</li>\n<li>Trusted our CV but didn't disregard Public LB entirely.</li>\n</ul>\n<h2>Image Preprocessing And Augmentations</h2>\n<ul>\n<li>We trained models on 224*224, 256*256, 384*384 and 512*512 image size. Larger image size always had better scores.</li>\n<li>We tried multiple combinations of augmentations. Following combinations performed the best:<ul>\n<li>Cutout, HorizontalFlip, VerticalFlip, ShiftScaleRotate, HueSaturationValue, RandomBrightnessContrast.</li>\n<li>Cutout, CoarseDropout,  HorizontalFlip, VerticalFlip.</li></ul></li>\n</ul>\n<h2>Neural Network Architectures</h2>\n<ul>\n<li>We tried ResNet34, ResNext50_32x4d, EfficientNet B1 NS, EfficientNet B3 NS, EfficientNet B4 NS, ViT-Base patch 16, and DeiT-Base patch 16 224.</li>\n</ul>\n<h2>Miscellaneous</h2>\n<ul>\n<li>Loss(es):<ul>\n<li>Bi-tempered loss with label smoothing</li>\n<li>Taylor Crossentropy with label smoothing</li></ul></li>\n<li>Optimizer:<ul>\n<li>Adam</li></ul></li>\n<li>LR Scheduler(s):<ul>\n<li>OneCycleLR</li>\n<li>CosineAnnealingWarmRestarts</li></ul></li>\n<li>Hardware/ platform:<ul>\n<li>Kaggle kernels</li></ul></li>\n</ul>\n<h2>Final Submissions</h2>\n<ul>\n<li>Selected Submission 1 (Private 0.8999, Public 0.9009) is a blend of:<ul>\n<li>EfficientNet B1 NS on 512*512 image size [1xTTA].</li>\n<li>ResNext50_32x4d on 512*512 image size [1xTTA].</li>\n<li>ResNext50_32x4d on 512*512 image size [No TTA].</li>\n<li>EfficientNet B3 NS on 512*512 image size [No TTA].   </li></ul></li>\n<li>Selected Submission 2 (Private 0.8981, Public 0.9002) is a blend of:<ul>\n<li>EfficientNet B1 NS on 512*512 image size [1xTTA].</li>\n<li>ResNext50_32x4d on 512*512 image size [No TTA].</li>\n<li>EfficientNet B3 NS on 512*512 image size [No TTA].   </li></ul></li>\n</ul>\n<h2>Some observations/ tips for competing in a CV comp with Kaggle Kernels only:</h2>\n<ul>\n<li>Data preprocessing != data augmentation. Resize images to the intended size before training, it will save you some GPU quota and speed up your experiments. Credits: <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a>; read more <a href=\"https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/161446\" target=\"_blank\">here</a>.</li>\n<li>Create a proof of concept kernel which is essentially a smaller version of the training kernel for fast experimentation or to check the feasibility of an idea. This kernel should run quickly and return an OOF CV score. You can make this kernel fast with a smaller number of CV folds/ smaller number of epochs. If you see improvement in this CV score, copy the config and run it on your training kernel.</li>\n<li>Write modular, maintainable, and readable code.</li>\n</ul>\n<h2>Credits</h2>\n<ul>\n<li><code>pandas</code> <a href=\"https://github.com/pandas-dev/pandas\" target=\"_blank\">https://github.com/pandas-dev/pandas</a></li>\n<li><code>torch</code> <a href=\"https://github.com/pytorch/pytorch\" target=\"_blank\">https://github.com/pytorch/pytorch</a></li>\n<li><code>timm</code> <a href=\"https://github.com/rwightman/pytorch-image-models\" target=\"_blank\">https://github.com/rwightman/pytorch-image-models</a>.</li>\n<li><code>albumentations</code> <a href=\"https://github.com/albumentations-team/albumentations\" target=\"_blank\">https://github.com/albumentations-team/albumentations</a></li>\n<li><code>lycon</code> <a href=\"https://github.com/ethereon/lycon\" target=\"_blank\">https://github.com/ethereon/lycon</a></li>\n</ul>\n<p>PS: This is our (my and <a href=\"https://www.kaggle.com/kjkr73\" target=\"_blank\">@kjkr73</a>'s) first medal 🥳<br>\nEdit: Updated LB scores and rank.</p>",
      "rawMarkdown": "First of all, congratulations to all the competitors. Huge thanks to Kaggle, Makerere University AI Lab for organising this competition.  \nI'd also like to thank my amazing teammate, @kjkr73. He did most of the heavy lifting. He'll be publishing a kernel soon.\n\n⚠ ~~The Private Leaderboard is not finalised yet. In the worst case, we'll fall to 245.~~ We actually jumped to 105. :)\n\n## Data\n- We only used the competition data.\n\n## Validation Strategy\n- Stratified 5-fold.\n- My CV score and LB score were correlated from the very start so I didn't worry about it too much.\n- Trusted our CV but didn't disregard Public LB entirely.\n\n## Image Preprocessing And Augmentations\n- We trained models on 224\\*224, 256\\*256, 384\\*384 and 512\\*512 image size. Larger image size always had better scores.\n- We tried multiple combinations of augmentations. Following combinations performed the best:\n    - Cutout, HorizontalFlip, VerticalFlip, ShiftScaleRotate, HueSaturationValue, RandomBrightnessContrast.\n    - Cutout, CoarseDropout,  HorizontalFlip, VerticalFlip.\n\n## Neural Network Architectures\n- We tried ResNet34, ResNext50_32x4d, EfficientNet B1 NS, EfficientNet B3 NS, EfficientNet B4 NS, ViT-Base patch 16, and DeiT-Base patch 16 224.\n\n## Miscellaneous\n- Loss(es):\n    - Bi-tempered loss with label smoothing\n    - Taylor Crossentropy with label smoothing\n- Optimizer:\n    - Adam\n- LR Scheduler(s):\n    - OneCycleLR\n    - CosineAnnealingWarmRestarts\n- Hardware/ platform:\n    - Kaggle kernels\n\n## Final Submissions\n- Selected Submission 1 (Private 0.8999, Public 0.9009) is a blend of:\n    - EfficientNet B1 NS on 512*512 image size [1xTTA].\n    - ResNext50_32x4d on 512*512 image size [1xTTA].\n    - ResNext50_32x4d on 512*512 image size [No TTA].\n    - EfficientNet B3 NS on 512*512 image size [No TTA].   \n- Selected Submission 2 (Private 0.8981, Public 0.9002) is a blend of:\n    - EfficientNet B1 NS on 512*512 image size [1xTTA].\n    - ResNext50_32x4d on 512*512 image size [No TTA].\n    - EfficientNet B3 NS on 512*512 image size [No TTA].   \n\n## Some observations/ tips for competing in a CV comp with Kaggle Kernels only: \n- Data preprocessing != data augmentation. Resize images to the intended size before training, it will save you some GPU quota and speed up your experiments. Credits: @cdeotte; read more [here](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/161446).\n- Create a proof of concept kernel which is essentially a smaller version of the training kernel for fast experimentation or to check the feasibility of an idea. This kernel should run quickly and return an OOF CV score. You can make this kernel fast with a smaller number of CV folds/ smaller number of epochs. If you see improvement in this CV score, copy the config and run it on your training kernel.\n- Write modular, maintainable, and readable code.\n\n## Credits \n- `pandas` https://github.com/pandas-dev/pandas\n- `torch` https://github.com/pytorch/pytorch\n- `timm` https://github.com/rwightman/pytorch-image-models.\n- `albumentations` https://github.com/albumentations-team/albumentations\n- `lycon` https://github.com/ethereon/lycon\n\nPS: This is our (my and @kjkr73's) first medal 🥳\nEdit: Updated LB scores and rank.",
      "votes": null
    },
    {
      "id": "1210024",
      "postDate": "02/19/2021 06:54:19",
      "content": "<p>Congrats on your first medal. :)</p>",
      "rawMarkdown": "Congrats on your first medal. :)",
      "votes": null
    },
    {
      "id": "1210041",
      "postDate": "02/19/2021 07:05:45",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/piantic\" target=\"_blank\">@piantic</a>, your <a href=\"https://www.kaggle.com/piantic/no-tta-cassava-resnext50-32x4d-inference-lb0-903\" target=\"_blank\">no tta kernel</a> was super helpful. Definitely my favourite kernel from this competition.</p>",
      "rawMarkdown": "Thank you @piantic, your [no tta kernel](https://www.kaggle.com/piantic/no-tta-cassava-resnext50-32x4d-inference-lb0-903) was super helpful. Definitely my favourite kernel from this competition.",
      "votes": null
    },
    {
      "id": "1210055",
      "postDate": "02/19/2021 07:13:36",
      "content": "<p>Congrats! I personally found the label smoothening and taylor crossentropy benefitted the most. When I implemented bi tempered, it relegated the model instead and wasn't as useful than taylor. What's your experience on this? Did you use both?</p>",
      "rawMarkdown": "Congrats! I personally found the label smoothening and taylor crossentropy benefitted the most. When I implemented bi tempered, it relegated the model instead and wasn't as useful than taylor. What's your experience on this? Did you use both?",
      "votes": null
    },
    {
      "id": "1210057",
      "postDate": "02/19/2021 07:15:01",
      "content": "<p>Hey! Where's the info that you might fall to 245 place from?</p>",
      "rawMarkdown": "Hey! Where's the info that you might fall to 245 place from?",
      "votes": null
    },
    {
      "id": "1210077",
      "postDate": "02/19/2021 07:38:26",
      "content": "<p>Yes, 2 models out of 4 in the best submission are based on the Taylor Cross-Entropy Loss and the rest 2 are based on Bi-Tempered Loss. In the case of Bi-tempered loss, I tweaked (lots of trails were done) the values of T_1 and T_2 and smoothing so as to give the best performance.</p>",
      "rawMarkdown": "Yes, 2 models out of 4 in the best submission are based on the Taylor Cross-Entropy Loss and the rest 2 are based on Bi-Tempered Loss. In the case of Bi-tempered loss, I tweaked (lots of trails were done) the values of T_1 and T_2 and smoothing so as to give the best performance.",
      "votes": null
    },
    {
      "id": "1210083",
      "postDate": "02/19/2021 07:44:35",
      "content": "<p>Congratulations !! Thanks for this writeup. </p>",
      "rawMarkdown": "Congratulations !! Thanks for this writeup.",
      "votes": null
    },
    {
      "id": "1210114",
      "postDate": "02/19/2021 07:59:29",
      "content": "<p>Since the final submission will also consider the fourth decimal place digit (to be announced) of the score and in the worst case our submission will tie with teams having 0.8990 and will be latest submission of all, in that case we will fall to 245.  You can read more <a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220602\" target=\"_blank\">here</a>.</p>",
      "rawMarkdown": "Since the final submission will also consider the fourth decimal place digit (to be announced) of the score and in the worst case our submission will tie with teams having 0.8990 and will be latest submission of all, in that case we will fall to 245.  You can read more [here](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220602).",
      "votes": null
    },
    {
      "id": "1210159",
      "postDate": "02/19/2021 08:29:53",
      "content": "<p>Thank you!</p>",
      "rawMarkdown": "Thank you!",
      "votes": null
    },
    {
      "id": "1210161",
      "postDate": "02/19/2021 08:31:35",
      "content": "<p>Thanks, <a href=\"https://www.kaggle.com/piantic\" target=\"_blank\">@piantic</a> sir learned a lot from you this competition :) and it took me 1.5 years to get the first medal 😁.</p>",
      "rawMarkdown": "Thanks, @piantic sir learned a lot from you this competition :) and it took me 1.5 years to get the first medal 😁.",
      "votes": null
    },
    {
      "id": "1210167",
      "postDate": "02/19/2021 08:33:01",
      "content": "<p>Thank you </p>",
      "rawMarkdown": "Thank you",
      "votes": null
    },
    {
      "id": "1210198",
      "postDate": "02/19/2021 08:57:19",
      "content": "<p>Congrats! I'm amazed how you achieved this while only using Kaggle kernels and thank you for those tips.<br>\nJust a few questions if you don't mind:</p>\n<p>Did you exhaust your weekly kaggle allocations?</p>\n<blockquote>\n  <p>Resize images to intended size before training</p>\n</blockquote>\n<p>Does it mean something like creating a dataset that was resized to save processing time?</p>",
      "rawMarkdown": "Congrats! I'm amazed how you achieved this while only using Kaggle kernels and thank you for those tips.\nJust a few questions if you don't mind:\n\nDid you exhaust your weekly kaggle allocations?\n\n> Resize images to intended size before training\n\nDoes it mean something like creating a dataset that was resized to save processing time?",
      "votes": null
    },
    {
      "id": "1210248",
      "postDate": "02/19/2021 09:40:08",
      "content": "<p>Thank you!</p>\n<blockquote>\n  <p>Did you exhaust your weekly kaggle allocations?</p>\n</blockquote>\n<p>Yes, but rarely. I used to exhaust weekly quotas frequently in the past competitions (like SIIM Melanoma Classification) but ever since Kaggle started allocating <a href=\"https://www.kaggle.com/product-feedback/173129\" target=\"_blank\">'floating' quota</a> I rarely run out of GPU hours. I guess limited resources force you to come up with <em>creative</em> solutions. :)</p>\n<blockquote>\n  <p>Does it mean something like creating a dataset that was resized to save processing time?</p>\n</blockquote>\n<p>Yes. I'm sure there are better ways to do it but <a href=\"https://www.kaggle.com/spoon69/spoon-cassava-resize-384\" target=\"_blank\">here</a>'s how I do it.</p>",
      "rawMarkdown": "Thank you!\n\n> Did you exhaust your weekly kaggle allocations?\n\nYes, but rarely. I used to exhaust weekly quotas frequently in the past competitions (like SIIM Melanoma Classification) but ever since Kaggle started allocating ['floating' quota](https://www.kaggle.com/product-feedback/173129) I rarely run out of GPU hours. I guess limited resources force you to come up with _creative_ solutions. :)\n\n> Does it mean something like creating a dataset that was resized to save processing time?\n\nYes. I'm sure there are better ways to do it but [here](https://www.kaggle.com/spoon69/spoon-cassava-resize-384)'s how I do it.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1210024,
      "author_name": "piantic",
      "author_url": "",
      "post_date": "02/19/2021 06:54:19",
      "content": "<p>Congrats on your first medal. :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 1210041,
          "author_name": "spoon69",
          "author_url": "",
          "post_date": "02/19/2021 07:05:45",
          "content": "<p>Thank you <a href=\"https://www.kaggle.com/piantic\" target=\"_blank\">@piantic</a>, your <a href=\"https://www.kaggle.com/piantic/no-tta-cassava-resnext50-32x4d-inference-lb0-903\" target=\"_blank\">no tta kernel</a> was super helpful. Definitely my favourite kernel from this competition.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1210161,
          "author_name": "kjkr73",
          "author_url": "",
          "post_date": "02/19/2021 08:31:35",
          "content": "<p>Thanks, <a href=\"https://www.kaggle.com/piantic\" target=\"_blank\">@piantic</a> sir learned a lot from you this competition :) and it took me 1.5 years to get the first medal 😁.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1210055,
      "author_name": "andyjianzhou",
      "author_url": "",
      "post_date": "02/19/2021 07:13:36",
      "content": "<p>Congrats! I personally found the label smoothening and taylor crossentropy benefitted the most. When I implemented bi tempered, it relegated the model instead and wasn't as useful than taylor. What's your experience on this? Did you use both?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1210077,
          "author_name": "kjkr73",
          "author_url": "",
          "post_date": "02/19/2021 07:38:26",
          "content": "<p>Yes, 2 models out of 4 in the best submission are based on the Taylor Cross-Entropy Loss and the rest 2 are based on Bi-Tempered Loss. In the case of Bi-tempered loss, I tweaked (lots of trails were done) the values of T_1 and T_2 and smoothing so as to give the best performance.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1210057,
      "author_name": "vadimtimakin",
      "author_url": "",
      "post_date": "02/19/2021 07:15:01",
      "content": "<p>Hey! Where's the info that you might fall to 245 place from?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1210114,
          "author_name": "spoon69",
          "author_url": "",
          "post_date": "02/19/2021 07:59:29",
          "content": "<p>Since the final submission will also consider the fourth decimal place digit (to be announced) of the score and in the worst case our submission will tie with teams having 0.8990 and will be latest submission of all, in that case we will fall to 245.  You can read more <a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220602\" target=\"_blank\">here</a>.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1210083,
      "author_name": "saurabh2mishra",
      "author_url": "",
      "post_date": "02/19/2021 07:44:35",
      "content": "<p>Congratulations !! Thanks for this writeup. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1210159,
          "author_name": "spoon69",
          "author_url": "",
          "post_date": "02/19/2021 08:29:53",
          "content": "<p>Thank you!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1210167,
          "author_name": "kjkr73",
          "author_url": "",
          "post_date": "02/19/2021 08:33:01",
          "content": "<p>Thank you </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1210198,
      "author_name": "jasondolorso",
      "author_url": "",
      "post_date": "02/19/2021 08:57:19",
      "content": "<p>Congrats! I'm amazed how you achieved this while only using Kaggle kernels and thank you for those tips.<br>\nJust a few questions if you don't mind:</p>\n<p>Did you exhaust your weekly kaggle allocations?</p>\n<blockquote>\n  <p>Resize images to intended size before training</p>\n</blockquote>\n<p>Does it mean something like creating a dataset that was resized to save processing time?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1210248,
          "author_name": "spoon69",
          "author_url": "",
          "post_date": "02/19/2021 09:40:08",
          "content": "<p>Thank you!</p>\n<blockquote>\n  <p>Did you exhaust your weekly kaggle allocations?</p>\n</blockquote>\n<p>Yes, but rarely. I used to exhaust weekly quotas frequently in the past competitions (like SIIM Melanoma Classification) but ever since Kaggle started allocating <a href=\"https://www.kaggle.com/product-feedback/173129\" target=\"_blank\">'floating' quota</a> I rarely run out of GPU hours. I guess limited resources force you to come up with <em>creative</em> solutions. :)</p>\n<blockquote>\n  <p>Does it mean something like creating a dataset that was resized to save processing time?</p>\n</blockquote>\n<p>Yes. I'm sure there are better ways to do it but <a href=\"https://www.kaggle.com/spoon69/spoon-cassava-resize-384\" target=\"_blank\">here</a>'s how I do it.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1210009": "First of all, congratulations to all the competitors. Huge thanks to Kaggle, Makerere University AI Lab for organising this competition.  \nI'd also like to thank my amazing teammate, @kjkr73. He did most of the heavy lifting. He'll be publishing a kernel soon.\n\n⚠ ~~The Private Leaderboard is not finalised yet. In the worst case, we'll fall to 245.~~ We actually jumped to 105. :)\n\n## Data\n- We only used the competition data.\n\n## Validation Strategy\n- Stratified 5-fold.\n- My CV score and LB score were correlated from the very start so I didn't worry about it too much.\n- Trusted our CV but didn't disregard Public LB entirely.\n\n## Image Preprocessing And Augmentations\n- We trained models on 224\\*224, 256\\*256, 384\\*384 and 512\\*512 image size. Larger image size always had better scores.\n- We tried multiple combinations of augmentations. Following combinations performed the best:\n    - Cutout, HorizontalFlip, VerticalFlip, ShiftScaleRotate, HueSaturationValue, RandomBrightnessContrast.\n    - Cutout, CoarseDropout,  HorizontalFlip, VerticalFlip.\n\n## Neural Network Architectures\n- We tried ResNet34, ResNext50_32x4d, EfficientNet B1 NS, EfficientNet B3 NS, EfficientNet B4 NS, ViT-Base patch 16, and DeiT-Base patch 16 224.\n\n## Miscellaneous\n- Loss(es):\n    - Bi-tempered loss with label smoothing\n    - Taylor Crossentropy with label smoothing\n- Optimizer:\n    - Adam\n- LR Scheduler(s):\n    - OneCycleLR\n    - CosineAnnealingWarmRestarts\n- Hardware/ platform:\n    - Kaggle kernels\n\n## Final Submissions\n- Selected Submission 1 (Private 0.8999, Public 0.9009) is a blend of:\n    - EfficientNet B1 NS on 512*512 image size [1xTTA].\n    - ResNext50_32x4d on 512*512 image size [1xTTA].\n    - ResNext50_32x4d on 512*512 image size [No TTA].\n    - EfficientNet B3 NS on 512*512 image size [No TTA].   \n- Selected Submission 2 (Private 0.8981, Public 0.9002) is a blend of:\n    - EfficientNet B1 NS on 512*512 image size [1xTTA].\n    - ResNext50_32x4d on 512*512 image size [No TTA].\n    - EfficientNet B3 NS on 512*512 image size [No TTA].   \n\n## Some observations/ tips for competing in a CV comp with Kaggle Kernels only: \n- Data preprocessing != data augmentation. Resize images to the intended size before training, it will save you some GPU quota and speed up your experiments. Credits: @cdeotte; read more [here](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/161446).\n- Create a proof of concept kernel which is essentially a smaller version of the training kernel for fast experimentation or to check the feasibility of an idea. This kernel should run quickly and return an OOF CV score. You can make this kernel fast with a smaller number of CV folds/ smaller number of epochs. If you see improvement in this CV score, copy the config and run it on your training kernel.\n- Write modular, maintainable, and readable code.\n\n## Credits \n- `pandas` https://github.com/pandas-dev/pandas\n- `torch` https://github.com/pytorch/pytorch\n- `timm` https://github.com/rwightman/pytorch-image-models.\n- `albumentations` https://github.com/albumentations-team/albumentations\n- `lycon` https://github.com/ethereon/lycon\n\nPS: This is our (my and @kjkr73's) first medal 🥳\nEdit: Updated LB scores and rank.",
    "1210024": "Congrats on your first medal. :)",
    "1210041": "Thank you @piantic, your [no tta kernel](https://www.kaggle.com/piantic/no-tta-cassava-resnext50-32x4d-inference-lb0-903) was super helpful. Definitely my favourite kernel from this competition.",
    "1210055": "Congrats! I personally found the label smoothening and taylor crossentropy benefitted the most. When I implemented bi tempered, it relegated the model instead and wasn't as useful than taylor. What's your experience on this? Did you use both?",
    "1210057": "Hey! Where's the info that you might fall to 245 place from?",
    "1210077": "Yes, 2 models out of 4 in the best submission are based on the Taylor Cross-Entropy Loss and the rest 2 are based on Bi-Tempered Loss. In the case of Bi-tempered loss, I tweaked (lots of trails were done) the values of T_1 and T_2 and smoothing so as to give the best performance.",
    "1210083": "Congratulations !! Thanks for this writeup.",
    "1210114": "Since the final submission will also consider the fourth decimal place digit (to be announced) of the score and in the worst case our submission will tie with teams having 0.8990 and will be latest submission of all, in that case we will fall to 245.  You can read more [here](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/220602).",
    "1210159": "Thank you!",
    "1210161": "Thanks, @piantic sir learned a lot from you this competition :) and it took me 1.5 years to get the first medal 😁.",
    "1210167": "Thank you",
    "1210198": "Congrats! I'm amazed how you achieved this while only using Kaggle kernels and thank you for those tips.\nJust a few questions if you don't mind:\n\nDid you exhaust your weekly kaggle allocations?\n\n> Resize images to intended size before training\n\nDoes it mean something like creating a dataset that was resized to save processing time?",
    "1210248": "Thank you!\n\n> Did you exhaust your weekly kaggle allocations?\n\nYes, but rarely. I used to exhaust weekly quotas frequently in the past competitions (like SIIM Melanoma Classification) but ever since Kaggle started allocating ['floating' quota](https://www.kaggle.com/product-feedback/173129) I rarely run out of GPU hours. I guess limited resources force you to come up with _creative_ solutions. :)\n\n> Does it mean something like creating a dataset that was resized to save processing time?\n\nYes. I'm sure there are better ways to do it but [here](https://www.kaggle.com/spoon69/spoon-cassava-resize-384)'s how I do it."
  },
  "source": "meta"
}