After experimenting with my code and also looking at other's work, mainly through Kaggle discussions, here is what i found which may be a reason for low score :
\n\n\nIf anyone has implemented tempered loss in your kernel please share ! 👀
\n
After experimenting with my code and also looking at other's work, mainly through Kaggle discussions, here is what i found which may be a reason for low score :
\n\n\nIf anyone has implemented tempered loss in your kernel please share ! 👀
\n
Albumentation was fast and gave good results for this dataset.
\nVariable brightness, contrast and hue didn't give significant improvement. It is good to go with light augmentations like randomcrop, resize and flips.
\nI haven't tested cutout or advanced techniques like snapmix. If anyone tried it, is it helping.
", "rawMarkdown": "Albumentation was fast and gave good results for this dataset.\n\nVariable brightness, contrast and hue didn't give significant improvement. It is good to go with light augmentations like randomcrop, resize and flips. \n\nI haven't tested cutout or advanced techniques like snapmix. If anyone tried it, is it helping.", "votes": null }, { "id": "1131965", "postDate": "12/30/2020 05:12:47", "content": "Thanks for your reply!
\nHave you tried tempered loss ?
Some time ago I had published a kernel with bi-tempered loss here
\nPlease note that this implementation is entirely borrowed from https://github.com/fhopfmueller/bi-tempered-loss-pytorch/blob/master/bi_tempered_loss_pytorch.py
Thanks! This will help!
", "rawMarkdown": "Thanks! This will help!", "votes": null }, { "id": "1132036", "postDate": "12/30/2020 06:11:58", "content": "Yes I had tried bi-tempered loss which @debarshichanda recommended. But focalcosineloss worked better than this.
", "rawMarkdown": "Yes I had tried bi-tempered loss which @debarshichanda recommended. But focalcosineloss worked better than this.", "votes": null }, { "id": "1132055", "postDate": "12/30/2020 06:23:21", "content": "Where can i find focalcosineloss documentation?
", "rawMarkdown": "Where can i find **focalcosineloss** documentation?", "votes": null }, { "id": "1132061", "postDate": "12/30/2020 06:30:38", "content": "This discussion has implementation of FocalCosineLoss
\nhttps://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/203271
@sandeepganesh049 How did you find out that the dataset has large margin noise?
", "rawMarkdown": "sandeepganesh049 How did you find out that the dataset has large margin noise?", "votes": null }, { "id": "1132544", "postDate": "12/30/2020 13:38:03", "content": "https://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html
\nWell looking from this, it must be either large margin noise or random noise
Also check out this : https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/202017
", "rawMarkdown": "https://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html\nWell looking from this, it must be either large margin noise or random noise\n\nAlso check out this : https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/202017", "votes": null } ], "comments": [ { "id": 1131957, "author_name": "tamilselvanmoorthy", "author_url": "", "post_date": "12/30/2020 05:08:15", "content": "Albumentation was fast and gave good results for this dataset.
\nVariable brightness, contrast and hue didn't give significant improvement. It is good to go with light augmentations like randomcrop, resize and flips.
\nI haven't tested cutout or advanced techniques like snapmix. If anyone tried it, is it helping.
", "votes": null, "replies": [ { "id": 1131965, "author_name": "sandeepganesh049", "author_url": "", "post_date": "12/30/2020 05:12:47", "content": "Thanks for your reply!
\nHave you tried tempered loss ?
Yes I had tried bi-tempered loss which @debarshichanda recommended. But focalcosineloss worked better than this.
", "votes": null, "replies": [] }, { "id": 1132055, "author_name": "sandeepganesh049", "author_url": "", "post_date": "12/30/2020 06:23:21", "content": "Where can i find focalcosineloss documentation?
", "votes": null, "replies": [] }, { "id": 1132061, "author_name": "debarshichanda", "author_url": "", "post_date": "12/30/2020 06:30:38", "content": "This discussion has implementation of FocalCosineLoss
\nhttps://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/203271
Some time ago I had published a kernel with bi-tempered loss here
\nPlease note that this implementation is entirely borrowed from https://github.com/fhopfmueller/bi-tempered-loss-pytorch/blob/master/bi_tempered_loss_pytorch.py
Thanks! This will help!
", "votes": null, "replies": [] } ] }, { "id": 1132457, "author_name": "debarshichanda", "author_url": "", "post_date": "12/30/2020 12:23:01", "content": "@sandeepganesh049 How did you find out that the dataset has large margin noise?
", "votes": null, "replies": [ { "id": 1132544, "author_name": "sandeepganesh049", "author_url": "", "post_date": "12/30/2020 13:38:03", "content": "https://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html
\nWell looking from this, it must be either large margin noise or random noise
Also check out this : https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/202017
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1131933": "After experimenting with my code and also looking at other's work, mainly through Kaggle discussions, here is what i found which may be a reason for low score :\n1. ***Loss***- normal loss vs tempered loss (for noisy data)\nhttps://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html\nOur cassava dataset has large margin noise. So using normal losses won't help with these large noises. Tempered Loss is designed to deal with noisy datasets.\n> If anyone has implemented **tempered loss** in your kernel please share ! 👀\n\n2. ***Label*** - normal labels vs label smoothing\n3. ***Data augmentation*** - ImageGenerator vs Augmentation vs Albumentation\nI have used ImageGenerator so far to get only 0.87 lb. Should i use aug or alb ?", "1131957": "Albumentation was fast and gave good results for this dataset.\n\nVariable brightness, contrast and hue didn't give significant improvement. It is good to go with light augmentations like randomcrop, resize and flips. \n\nI haven't tested cutout or advanced techniques like snapmix. If anyone tried it, is it helping.", "1131965": "Thanks for your reply!\nHave you tried ***tempered loss*** ?", "1131979": "Some time ago I had published a kernel with bi-tempered loss [here](https://www.kaggle.com/debarshichanda/cassava-bitempered-logistic-loss)\nPlease note that this implementation is entirely borrowed from [https://github.com/fhopfmueller/bi-tempered-loss-pytorch/blob/master/bi_tempered_loss_pytorch.py](https://github.com/fhopfmueller/bi-tempered-loss-pytorch/blob/master/bi_tempered_loss_pytorch.py)", "1131983": "Thanks! This will help!", "1132036": "Yes I had tried bi-tempered loss which @debarshichanda recommended. But focalcosineloss worked better than this.", "1132055": "Where can i find **focalcosineloss** documentation?", "1132061": "This discussion has implementation of FocalCosineLoss\n[https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/203271](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/203271)", "1132457": "sandeepganesh049 How did you find out that the dataset has large margin noise?", "1132544": "https://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html\nWell looking from this, it must be either large margin noise or random noise\n\nAlso check out this : https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/202017" }, "source": "meta" }