{
  "id": 218984,
  "title": "Any tips on getting .90+ ?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/218984",
  "author_name": "",
  "post_date": "2021-02-12T23:13:22.121614500Z",
  "votes": 17,
  "comment_count": 16,
  "views": 0,
  "content": "<p>Hey guys,<br>\nI joined this competition 3 weeks before deadline and been experimenting different stuffs since then, but still can't even crack the .90 mark on both CV and LB.<br>\nHere's the things me and my teammates have experimented so far….<br>\n</p><ul><br>\n<li>Pretrained models on Imagenet and NoisyStudents.</li><li>Simple/Heavy Augmentation.</li><li>Different Loss Functions.<br> <ul><li>Categorical Cross Entropy with Label Smoothing</li><li>Focal Cosine Loss</li><li>Symmetric Cross Entropy Loss</li><li>Bi Tempered Logistic Loss</li><li>Taylor's Cross Entropy Loss with Label Smoothing</li></ul></li></ul><p></p>\n<ul>\n<li>External Data from 2019 Competition.</li>\n<li>Pretraining on PlantVillage Dataset</li>\n<li>Different Learning Rate and Schedulers</li>\n<li>Epoch Threshold for applying Fancy Loss and Augmentations(i.e. mixup/cutmix)</li>\n<li>Upsampling Poor Scoring Classes.</li>\n<li>Fancy Heads</li>\n<li>Adding GAN generated Images.</li>\n<li>Mixup without Hesitation.</li>\n<li>Bigger Image Size</li>\n<li>Ensemble</li>\n<li>TTA</li>\n</ul>\n<p>We could only manage to get 0.896 on CV and 0.895 on LB.<br>\nAny tips on improving CV and LB?<br>\nThanks In Advance. </p>",
  "messages": [
    {
      "id": "1198321",
      "postDate": "02/12/2021 23:13:22",
      "content": "<p>Hey guys,<br>\nI joined this competition 3 weeks before deadline and been experimenting different stuffs since then, but still can't even crack the .90 mark on both CV and LB.<br>\nHere's the things me and my teammates have experimented so far….<br>\n</p><ul><br>\n<li>Pretrained models on Imagenet and NoisyStudents.</li><li>Simple/Heavy Augmentation.</li><li>Different Loss Functions.<br> <ul><li>Categorical Cross Entropy with Label Smoothing</li><li>Focal Cosine Loss</li><li>Symmetric Cross Entropy Loss</li><li>Bi Tempered Logistic Loss</li><li>Taylor's Cross Entropy Loss with Label Smoothing</li></ul></li></ul><p></p>\n<ul>\n<li>External Data from 2019 Competition.</li>\n<li>Pretraining on PlantVillage Dataset</li>\n<li>Different Learning Rate and Schedulers</li>\n<li>Epoch Threshold for applying Fancy Loss and Augmentations(i.e. mixup/cutmix)</li>\n<li>Upsampling Poor Scoring Classes.</li>\n<li>Fancy Heads</li>\n<li>Adding GAN generated Images.</li>\n<li>Mixup without Hesitation.</li>\n<li>Bigger Image Size</li>\n<li>Ensemble</li>\n<li>TTA</li>\n</ul>\n<p>We could only manage to get 0.896 on CV and 0.895 on LB.<br>\nAny tips on improving CV and LB?<br>\nThanks In Advance. </p>",
      "rawMarkdown": "Hey guys,\nI joined this competition 3 weeks before deadline and been experimenting different stuffs since then, but still can't even crack the .90 mark on both CV and LB.\nHere's the things me and my teammates have experimented so far....\n<ul>\n<li>Pretrained models on Imagenet and NoisyStudents.</li><li>Simple/Heavy Augmentation.</li><li>Different Loss Functions.<br> <ul><li>Categorical Cross Entropy with Label Smoothing</li><li>Focal Cosine Loss</li><li>Symmetric Cross Entropy Loss</li><li>Bi Tempered Logistic Loss</li><li>Taylor's Cross Entropy Loss with Label Smoothing</li><ul></li>\n<li>External Data from 2019 Competition.</li>\n<li>Pretraining on PlantVillage Dataset</li>\n<li>Different Learning Rate and Schedulers</li>\n<li>Epoch Threshold for applying Fancy Loss and Augmentations(i.e. mixup/cutmix)</li>\n<li>Upsampling Poor Scoring Classes.</li>\n<li>Fancy Heads</li>\n<li>Adding GAN generated Images.</li>\n<li>Mixup without Hesitation.</li>\n<li>Bigger Image Size</li>\n<li>Ensemble</li>\n<li>TTA</li>\n</ul>\nWe could only manage to get 0.896 on CV and 0.895 on LB.\nAny tips on improving CV and LB?\nThanks In Advance.",
      "votes": null
    },
    {
      "id": "1198503",
      "postDate": "02/13/2021 05:47:59",
      "content": "<p>Following…<br>\nI have also tried most of the things you did, am at 0.895 LB right now.<br>\nI will try doing TTA and using the 2019 comp data as well(these are pending for me).<br>\nwould like to know how to get out of this slump as well.</p>",
      "rawMarkdown": "Following...\nI have also tried most of the things you did, am at 0.895 LB right now.\nI will try doing TTA and using the 2019 comp data as well(these are pending for me).\nwould like to know how to get out of this slump as well.",
      "votes": null
    },
    {
      "id": "1198544",
      "postDate": "02/13/2021 06:05:15",
      "content": "<p>With ensemble It's easy to get close to 0.905 but yeah private lb can be different Though :(</p>",
      "rawMarkdown": "With ensemble It's easy to get close to 0.905 but yeah private lb can be different Though :(",
      "votes": null
    },
    {
      "id": "1198604",
      "postDate": "02/13/2021 07:05:54",
      "content": "<p>have you tried most of them together? using cumix &amp; mixup, larger image sizes(512x512), Effb5 ,5 folds with noisy student weights with a minimal TTA  gives me .900. I am hoping that ensemble would get me up to bronze. </p>",
      "rawMarkdown": "have you tried most of them together? using cumix & mixup, larger image sizes(512x512), Effb5 ,5 folds with noisy student weights with a minimal TTA  gives me .900. I am hoping that ensemble would get me up to bronze.",
      "votes": null
    },
    {
      "id": "1198747",
      "postDate": "02/13/2021 09:09:37",
      "content": "<p>The same for me. I've used images from 2018 competitions (even test and extra images for upsampling less frequent classes), tried resnext50, resnext101, efficientnetb0, efficientnetb4, ensembling, tta, and I'm not able to score above LB 0.897 (single models and ensembles score up to 0.905 locally).</p>",
      "rawMarkdown": "The same for me. I've used images from 2018 competitions (even test and extra images for upsampling less frequent classes), tried resnext50, resnext101, efficientnetb0, efficientnetb4, ensembling, tta, and I'm not able to score above LB 0.897 (single models and ensembles score up to 0.905 locally).",
      "votes": null
    },
    {
      "id": "1199215",
      "postDate": "02/13/2021 16:06:35",
      "content": "<p>yeah, it's kinda frustrating  :( </p>",
      "rawMarkdown": "yeah, it's kinda frustrating  :(",
      "votes": null
    },
    {
      "id": "1199220",
      "postDate": "02/13/2021 16:08:58",
      "content": "<p>yeah I have done that. From our experiments on pytorch, bigger efficient-net models didn't help that much. I am still trying to find suitable TTA's. Thank you BTW.</p>",
      "rawMarkdown": "yeah I have done that. From our experiments on pytorch, bigger efficient-net models didn't help that much. I am still trying to find suitable TTA's. Thank you BTW.",
      "votes": null
    },
    {
      "id": "1199224",
      "postDate": "02/13/2021 16:13:32",
      "content": "<p>We tried ensembling different models too.<br>\nIn this competition people's been fighting for score at 3 decimal point which kinda gives Tweet Sentiment Extraction competition's type of vibe. Vai, what is your prediction on this one to end up in a similar way?</p>",
      "rawMarkdown": "We tried ensembling different models too.\nIn this competition people's been fighting for score at 3 decimal point which kinda gives Tweet Sentiment Extraction competition's type of vibe. Vai, what is your prediction on this one to end up in a similar way?",
      "votes": null
    },
    {
      "id": "1199228",
      "postDate": "02/13/2021 16:16:54",
      "content": "<p><a href=\"https://www.kaggle.com/zaber666\" target=\"_blank\">@zaber666</a> <strong>Vai, what is your prediction on this one to end up in a similar way?</strong> <br>\nyes brother i think so 😄😄😄 <br>\nhope private lb is with you this time ✔️❤️🎉</p>",
      "rawMarkdown": "zaber666 **Vai, what is your prediction on this one to end up in a similar way?** \nyes brother i think so 😄😄😄 \nhope private lb is with you this time ✔️❤️🎉",
      "votes": null
    },
    {
      "id": "1199254",
      "postDate": "02/13/2021 16:50:44",
      "content": "<p>We are lagging behind too much. I don't a comeback from this. 😓<br>\nHopefully we will get our 3rd competition master from BD this time. Best of luck vai 😃</p>",
      "rawMarkdown": "We are lagging behind too much. I don't a comeback from this. 😓\nHopefully we will get our 3rd competition master from BD this time. Best of luck vai 😃",
      "votes": null
    },
    {
      "id": "1199259",
      "postDate": "02/13/2021 16:58:58",
      "content": "<p>Hello!</p>\n<p>I see it's been quite a long way you did. </p>\n<p>Nevertheless all of the things listed above can be pretty useful standalone, the main point (and the hardest one) is to find the proper combination of those. I can suggest a few more things to check out before the competition ends to keep you inspired (just as these things inspired me):</p>\n<ul>\n<li><strong><a href=\"https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-training-with-tpu-v2-pods\" target=\"_blank\">this notebook</a></strong> by <a href=\"https://www.kaggle.com/dimitreoliveira\" target=\"_blank\">@dimitreoliveira</a> is a fancy 0.900 pipeline with tips on what worked and what didn't work</li>\n<li>some detailed descriptions of label smoothing, learning rate schedules, knowledge distillation and other tricks can be found in <strong><a href=\"https://arxiv.org/abs/1812.01187\" target=\"_blank\">this paper</a></strong> </li>\n<li>knowledge distillation or drop threshold can be helpful for this noisy data, so check out <strong><a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215607\" target=\"_blank\">this topic</a></strong> where I share my labels as well</li>\n</ul>",
      "rawMarkdown": "Hello!\n\nI see it's been quite a long way you did. \n\nNevertheless all of the things listed above can be pretty useful standalone, the main point (and the hardest one) is to find the proper combination of those. I can suggest a few more things to check out before the competition ends to keep you inspired (just as these things inspired me):\n* **[this notebook](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-training-with-tpu-v2-pods)** by @dimitreoliveira is a fancy 0.900 pipeline with tips on what worked and what didn't work\n* some detailed descriptions of label smoothing, learning rate schedules, knowledge distillation and other tricks can be found in **[this paper](https://arxiv.org/abs/1812.01187)** \n* knowledge distillation or drop threshold can be helpful for this noisy data, so check out **[this topic](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215607)** where I share my labels as well",
      "votes": null
    },
    {
      "id": "1199284",
      "postDate": "02/13/2021 17:23:33",
      "content": "<p>Thank you !!! <br>\nI will try these stuffs.</p>",
      "rawMarkdown": "Thank you !!! \nI will try these stuffs.",
      "votes": null
    },
    {
      "id": "1199310",
      "postDate": "02/13/2021 17:49:36",
      "content": "<p>For me cutmix with low amount of additionnal regularization/augmentations + center cropping gives me 0.902 CV single model but cant seem to push the lb more than 0.9 even when ensembling</p>\n<p>Good luck :)</p>",
      "rawMarkdown": "For me cutmix with low amount of additionnal regularization/augmentations + center cropping gives me 0.902 CV single model but cant seem to push the lb more than 0.9 even when ensembling\n\nGood luck :)",
      "votes": null
    },
    {
      "id": "1199371",
      "postDate": "02/13/2021 19:20:57",
      "content": "<p>Well, that's pretty high CV. I haven't tried center cropping yet but cutmix/mixup helped in my case. And single model validation score vs LB wasn't correlated in my case either.  You should compare it with your CV.</p>",
      "rawMarkdown": "Well, that's pretty high CV. I haven't tried center cropping yet but cutmix/mixup helped in my case. And single model validation score vs LB wasn't correlated in my case either.  You should compare it with your CV.",
      "votes": null
    },
    {
      "id": "1199492",
      "postDate": "02/13/2021 20:53:30",
      "content": "<p>Regarding few useful tips for getting closer to 0.90 you can refer link <a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215359\" target=\"_blank\">here</a></p>",
      "rawMarkdown": "Regarding few useful tips for getting closer to 0.90 you can refer link [here](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215359)",
      "votes": null
    },
    {
      "id": "1200138",
      "postDate": "02/14/2021 12:49:26",
      "content": "<p>I tried resenet50 with low augmentation and efficientnet_b4 with heavy augmentation. Ensemble the results and able to get LB 0.902.</p>",
      "rawMarkdown": "I tried resenet50 with low augmentation and efficientnet_b4 with heavy augmentation. Ensemble the results and able to get LB 0.902.",
      "votes": null
    },
    {
      "id": "1209179",
      "postDate": "02/18/2021 18:34:08",
      "content": "<p>I was able to train an efficientnet-b4 model using Adam optimiser, label smoothing cross entropy loss, test time augmentation to get  to a public score of 0.901. Ensembling with a resnet-50 model further boosted the public score to 0.903.</p>",
      "rawMarkdown": "I was able to train an efficientnet-b4 model using Adam optimiser, label smoothing cross entropy loss, test time augmentation to get  to a public score of 0.901. Ensembling with a resnet-50 model further boosted the public score to 0.903.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1198503,
      "author_name": "mohneesh7",
      "author_url": "",
      "post_date": "02/13/2021 05:47:59",
      "content": "<p>Following…<br>\nI have also tried most of the things you did, am at 0.895 LB right now.<br>\nI will try doing TTA and using the 2019 comp data as well(these are pending for me).<br>\nwould like to know how to get out of this slump as well.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1198544,
      "author_name": "mobassir",
      "author_url": "",
      "post_date": "02/13/2021 06:05:15",
      "content": "<p>With ensemble It's easy to get close to 0.905 but yeah private lb can be different Though :(</p>",
      "votes": null,
      "replies": [
        {
          "id": 1199224,
          "author_name": "zaber666",
          "author_url": "",
          "post_date": "02/13/2021 16:13:32",
          "content": "<p>We tried ensembling different models too.<br>\nIn this competition people's been fighting for score at 3 decimal point which kinda gives Tweet Sentiment Extraction competition's type of vibe. Vai, what is your prediction on this one to end up in a similar way?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1199228,
          "author_name": "mobassir",
          "author_url": "",
          "post_date": "02/13/2021 16:16:54",
          "content": "<p><a href=\"https://www.kaggle.com/zaber666\" target=\"_blank\">@zaber666</a> <strong>Vai, what is your prediction on this one to end up in a similar way?</strong> <br>\nyes brother i think so 😄😄😄 <br>\nhope private lb is with you this time ✔️❤️🎉</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1199254,
          "author_name": "zaber666",
          "author_url": "",
          "post_date": "02/13/2021 16:50:44",
          "content": "<p>We are lagging behind too much. I don't a comeback from this. 😓<br>\nHopefully we will get our 3rd competition master from BD this time. Best of luck vai 😃</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1198604,
      "author_name": "deepakbhatp",
      "author_url": "",
      "post_date": "02/13/2021 07:05:54",
      "content": "<p>have you tried most of them together? using cumix &amp; mixup, larger image sizes(512x512), Effb5 ,5 folds with noisy student weights with a minimal TTA  gives me .900. I am hoping that ensemble would get me up to bronze. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1199220,
          "author_name": "zaber666",
          "author_url": "",
          "post_date": "02/13/2021 16:08:58",
          "content": "<p>yeah I have done that. From our experiments on pytorch, bigger efficient-net models didn't help that much. I am still trying to find suitable TTA's. Thank you BTW.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1198747,
      "author_name": "jcesquiveld",
      "author_url": "",
      "post_date": "02/13/2021 09:09:37",
      "content": "<p>The same for me. I've used images from 2018 competitions (even test and extra images for upsampling less frequent classes), tried resnext50, resnext101, efficientnetb0, efficientnetb4, ensembling, tta, and I'm not able to score above LB 0.897 (single models and ensembles score up to 0.905 locally).</p>",
      "votes": null,
      "replies": [
        {
          "id": 1199215,
          "author_name": "zaber666",
          "author_url": "",
          "post_date": "02/13/2021 16:06:35",
          "content": "<p>yeah, it's kinda frustrating  :( </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1200138,
          "author_name": "adg1822",
          "author_url": "",
          "post_date": "02/14/2021 12:49:26",
          "content": "<p>I tried resenet50 with low augmentation and efficientnet_b4 with heavy augmentation. Ensemble the results and able to get LB 0.902.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1199259,
      "author_name": "nickuzmenkov",
      "author_url": "",
      "post_date": "02/13/2021 16:58:58",
      "content": "<p>Hello!</p>\n<p>I see it's been quite a long way you did. </p>\n<p>Nevertheless all of the things listed above can be pretty useful standalone, the main point (and the hardest one) is to find the proper combination of those. I can suggest a few more things to check out before the competition ends to keep you inspired (just as these things inspired me):</p>\n<ul>\n<li><strong><a href=\"https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-training-with-tpu-v2-pods\" target=\"_blank\">this notebook</a></strong> by <a href=\"https://www.kaggle.com/dimitreoliveira\" target=\"_blank\">@dimitreoliveira</a> is a fancy 0.900 pipeline with tips on what worked and what didn't work</li>\n<li>some detailed descriptions of label smoothing, learning rate schedules, knowledge distillation and other tricks can be found in <strong><a href=\"https://arxiv.org/abs/1812.01187\" target=\"_blank\">this paper</a></strong> </li>\n<li>knowledge distillation or drop threshold can be helpful for this noisy data, so check out <strong><a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215607\" target=\"_blank\">this topic</a></strong> where I share my labels as well</li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 1199284,
          "author_name": "zaber666",
          "author_url": "",
          "post_date": "02/13/2021 17:23:33",
          "content": "<p>Thank you !!! <br>\nI will try these stuffs.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1199310,
      "author_name": "yannmajewski",
      "author_url": "",
      "post_date": "02/13/2021 17:49:36",
      "content": "<p>For me cutmix with low amount of additionnal regularization/augmentations + center cropping gives me 0.902 CV single model but cant seem to push the lb more than 0.9 even when ensembling</p>\n<p>Good luck :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 1199371,
          "author_name": "zaber666",
          "author_url": "",
          "post_date": "02/13/2021 19:20:57",
          "content": "<p>Well, that's pretty high CV. I haven't tried center cropping yet but cutmix/mixup helped in my case. And single model validation score vs LB wasn't correlated in my case either.  You should compare it with your CV.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1199492,
      "author_name": "vickygoyal",
      "author_url": "",
      "post_date": "02/13/2021 20:53:30",
      "content": "<p>Regarding few useful tips for getting closer to 0.90 you can refer link <a href=\"https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215359\" target=\"_blank\">here</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1209179,
      "author_name": "roshanbaliga",
      "author_url": "",
      "post_date": "02/18/2021 18:34:08",
      "content": "<p>I was able to train an efficientnet-b4 model using Adam optimiser, label smoothing cross entropy loss, test time augmentation to get  to a public score of 0.901. Ensembling with a resnet-50 model further boosted the public score to 0.903.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1198321": "Hey guys,\nI joined this competition 3 weeks before deadline and been experimenting different stuffs since then, but still can't even crack the .90 mark on both CV and LB.\nHere's the things me and my teammates have experimented so far....\n<ul>\n<li>Pretrained models on Imagenet and NoisyStudents.</li><li>Simple/Heavy Augmentation.</li><li>Different Loss Functions.<br> <ul><li>Categorical Cross Entropy with Label Smoothing</li><li>Focal Cosine Loss</li><li>Symmetric Cross Entropy Loss</li><li>Bi Tempered Logistic Loss</li><li>Taylor's Cross Entropy Loss with Label Smoothing</li><ul></li>\n<li>External Data from 2019 Competition.</li>\n<li>Pretraining on PlantVillage Dataset</li>\n<li>Different Learning Rate and Schedulers</li>\n<li>Epoch Threshold for applying Fancy Loss and Augmentations(i.e. mixup/cutmix)</li>\n<li>Upsampling Poor Scoring Classes.</li>\n<li>Fancy Heads</li>\n<li>Adding GAN generated Images.</li>\n<li>Mixup without Hesitation.</li>\n<li>Bigger Image Size</li>\n<li>Ensemble</li>\n<li>TTA</li>\n</ul>\nWe could only manage to get 0.896 on CV and 0.895 on LB.\nAny tips on improving CV and LB?\nThanks In Advance.",
    "1198503": "Following...\nI have also tried most of the things you did, am at 0.895 LB right now.\nI will try doing TTA and using the 2019 comp data as well(these are pending for me).\nwould like to know how to get out of this slump as well.",
    "1198544": "With ensemble It's easy to get close to 0.905 but yeah private lb can be different Though :(",
    "1198604": "have you tried most of them together? using cumix & mixup, larger image sizes(512x512), Effb5 ,5 folds with noisy student weights with a minimal TTA  gives me .900. I am hoping that ensemble would get me up to bronze.",
    "1198747": "The same for me. I've used images from 2018 competitions (even test and extra images for upsampling less frequent classes), tried resnext50, resnext101, efficientnetb0, efficientnetb4, ensembling, tta, and I'm not able to score above LB 0.897 (single models and ensembles score up to 0.905 locally).",
    "1199215": "yeah, it's kinda frustrating  :(",
    "1199220": "yeah I have done that. From our experiments on pytorch, bigger efficient-net models didn't help that much. I am still trying to find suitable TTA's. Thank you BTW.",
    "1199224": "We tried ensembling different models too.\nIn this competition people's been fighting for score at 3 decimal point which kinda gives Tweet Sentiment Extraction competition's type of vibe. Vai, what is your prediction on this one to end up in a similar way?",
    "1199228": "zaber666 **Vai, what is your prediction on this one to end up in a similar way?** \nyes brother i think so 😄😄😄 \nhope private lb is with you this time ✔️❤️🎉",
    "1199254": "We are lagging behind too much. I don't a comeback from this. 😓\nHopefully we will get our 3rd competition master from BD this time. Best of luck vai 😃",
    "1199259": "Hello!\n\nI see it's been quite a long way you did. \n\nNevertheless all of the things listed above can be pretty useful standalone, the main point (and the hardest one) is to find the proper combination of those. I can suggest a few more things to check out before the competition ends to keep you inspired (just as these things inspired me):\n* **[this notebook](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-training-with-tpu-v2-pods)** by @dimitreoliveira is a fancy 0.900 pipeline with tips on what worked and what didn't work\n* some detailed descriptions of label smoothing, learning rate schedules, knowledge distillation and other tricks can be found in **[this paper](https://arxiv.org/abs/1812.01187)** \n* knowledge distillation or drop threshold can be helpful for this noisy data, so check out **[this topic](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215607)** where I share my labels as well",
    "1199284": "Thank you !!! \nI will try these stuffs.",
    "1199310": "For me cutmix with low amount of additionnal regularization/augmentations + center cropping gives me 0.902 CV single model but cant seem to push the lb more than 0.9 even when ensembling\n\nGood luck :)",
    "1199371": "Well, that's pretty high CV. I haven't tried center cropping yet but cutmix/mixup helped in my case. And single model validation score vs LB wasn't correlated in my case either.  You should compare it with your CV.",
    "1199492": "Regarding few useful tips for getting closer to 0.90 you can refer link [here](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/215359)",
    "1200138": "I tried resenet50 with low augmentation and efficientnet_b4 with heavy augmentation. Ensemble the results and able to get LB 0.902.",
    "1209179": "I was able to train an efficientnet-b4 model using Adam optimiser, label smoothing cross entropy loss, test time augmentation to get  to a public score of 0.901. Ensembling with a resnet-50 model further boosted the public score to 0.903."
  },
  "source": "meta"
}