I have used CutMix, SnapMix, and Fmix with TaylorCrossEntropy, TaylorSmoothed, LabelSmoothed, and CrossEntropy Loss with the Alpha value of 1, LabelSmooth prob 0.5, and decay power of 5. Also used different types of optimizers like Adam, AdamW, AdamP, and Ranger with efficient net b3, b4, b5, resnext50_32x4d, and ViT. Using these augmentations techniques resulted in poor accuracy while training than without using them. Or maybe I'm using it the wrong way. If you want to see the training notebook let me know in the comments.
", "messages": [ { "id": "1190124", "postDate": "02/07/2021 14:01:52", "content": "I have used CutMix, SnapMix, and Fmix with TaylorCrossEntropy, TaylorSmoothed, LabelSmoothed, and CrossEntropy Loss with the Alpha value of 1, LabelSmooth prob 0.5, and decay power of 5. Also used different types of optimizers like Adam, AdamW, AdamP, and Ranger with efficient net b3, b4, b5, resnext50_32x4d, and ViT. Using these augmentations techniques resulted in poor accuracy while training than without using them. Or maybe I'm using it the wrong way. If you want to see the training notebook let me know in the comments.
", "rawMarkdown": "I have used CutMix, SnapMix, and Fmix with TaylorCrossEntropy, TaylorSmoothed, LabelSmoothed, and CrossEntropy Loss with the Alpha value of 1, LabelSmooth prob 0.5, and decay power of 5. Also used different types of optimizers like Adam, AdamW, AdamP, and Ranger with efficient net b3, b4, b5, resnext50_32x4d, and ViT. Using these augmentations techniques resulted in poor accuracy while training than without using them. Or maybe I'm using it the wrong way. If you want to see the training notebook let me know in the comments.", "votes": null }, { "id": "1190316", "postDate": "02/07/2021 16:33:09", "content": "check This Topic: CutMix and MixUp help? please.
", "rawMarkdown": "check [This Topic: CutMix and MixUp help?](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/214225) please.", "votes": null }, { "id": "1190496", "postDate": "02/07/2021 18:38:23", "content": "I tried cutmix, snapmix, fmix, Miuxup, etc. none of them work but general corsedropout works.
\nI think it depends on your general transformation and model.
", "rawMarkdown": "I tried cutmix, snapmix, fmix, Miuxup, etc. none of them work but general corsedropout works.\n\nI think it depends on your general transformation and model.", "votes": null }, { "id": "1190644", "postDate": "02/07/2021 21:39:47", "content": "There's also two more recent threads: Combination of cutmix and other augmentations and Cutmix & Fmix reduce the accuracy??!. These have even more speculation, theories and people sharing their experiences.
", "rawMarkdown": "There's also two more recent threads: [Combination of cutmix and other augmentations](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/217150) and [Cutmix & Fmix reduce the accuracy??!](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/217129). These have even more speculation, theories and people sharing their experiences.", "votes": null }, { "id": "1191477", "postDate": "02/08/2021 13:32:15", "content": "cutmix and labelsmooth both help me
", "rawMarkdown": "cutmix and labelsmooth both help me", "votes": null }, { "id": "1191859", "postDate": "02/08/2021 18:15:29", "content": "CutMix did not help me improve my CV/LB
", "rawMarkdown": "CutMix did not help me improve my CV/LB", "votes": null }, { "id": "1192973", "postDate": "02/09/2021 11:44:30", "content": "While training CutMix and label smooth were giving lower accuracy and more loss than the one without them. Optimizer used was AdamW.
", "rawMarkdown": "While training CutMix and label smooth were giving lower accuracy and more loss than the one without them. Optimizer used was AdamW.", "votes": null }, { "id": "1192978", "postDate": "02/09/2021 11:46:13", "content": "Is there any proper tutorial or illustration on how to increase accuracy using CutMix.
", "rawMarkdown": "Is there any proper tutorial or illustration on how to increase accuracy using CutMix.", "votes": null }, { "id": "1193818", "postDate": "02/09/2021 21:48:14", "content": "I have noticed, that CutMix did improve my CV/LB with a different Optimizer (I was using RAdam before, swapping to AdamW increased my score)
", "rawMarkdown": "I have noticed, that CutMix did improve my CV/LB with a different Optimizer (I was using RAdam before, swapping to AdamW increased my score)", "votes": null }, { "id": "1194487", "postDate": "02/10/2021 08:04:15", "content": "How are you using CutMix can you give me some help or a tutorial link. I'm skeptical about my code if it is correct or not.
", "rawMarkdown": "How are you using CutMix can you give me some help or a tutorial link. I'm skeptical about my code if it is correct or not.", "votes": null }, { "id": "1194596", "postDate": "02/10/2021 09:09:52", "content": "Cutmix is helpful.
", "rawMarkdown": "Cutmix is helpful.", "votes": null }, { "id": "1195514", "postDate": "02/10/2021 21:17:15", "content": "I tried cutmix but it didn't improved my LB/CV. Cutouts was useful for me.
", "rawMarkdown": "I tried cutmix but it didn't improved my LB/CV. Cutouts was useful for me.", "votes": null }, { "id": "1196573", "postDate": "02/11/2021 13:33:43", "content": "@vickygoyal How are you using it, can you help me. Probably I might be using it wrong.
", "rawMarkdown": "vickygoyal How are you using it, can you help me. Probably I might be using it wrong.", "votes": null }, { "id": "1196773", "postDate": "02/11/2021 16:12:32", "content": "you can refer this link
", "rawMarkdown": "you can refer this [link](https://www.kaggle.com/cdeotte/cutmix-and-mixup-on-gpu-tpu)", "votes": null }, { "id": "1196905", "postDate": "02/11/2021 18:45:30", "content": "@raufmomin can you elaborate on the technique you mentioned or any link for reference.
", "rawMarkdown": "raufmomin can you elaborate on the technique you mentioned or any link for reference.", "votes": null }, { "id": "1196973", "postDate": "02/11/2021 19:56:21", "content": "Sure! You can check out this links.
\nLink: https://medium.com/@ombelinelag/cutout-regularization-for-cnns-62670d86bc33
\nPaper: https://arxiv.org/abs/1708.04552
Cutmix helps me, and other augmentation and general training tric , too.
\nI think the difference of cutmix effictiveness among people is largely depend on other parameter settings like learning rate, loss
\n and so on.
check This Topic: CutMix and MixUp help? please.
", "votes": null, "replies": [ { "id": 1190644, "author_name": "bjoernholzhauer", "author_url": "", "post_date": "02/07/2021 21:39:47", "content": "There's also two more recent threads: Combination of cutmix and other augmentations and Cutmix & Fmix reduce the accuracy??!. These have even more speculation, theories and people sharing their experiences.
", "votes": null, "replies": [] }, { "id": 1192978, "author_name": "sanchitvj", "author_url": "", "post_date": "02/09/2021 11:46:13", "content": "Is there any proper tutorial or illustration on how to increase accuracy using CutMix.
", "votes": null, "replies": [] } ] }, { "id": 1190496, "author_name": "tom88jerry", "author_url": "", "post_date": "02/07/2021 18:38:23", "content": "I tried cutmix, snapmix, fmix, Miuxup, etc. none of them work but general corsedropout works.
\nI think it depends on your general transformation and model.
", "votes": null, "replies": [] }, { "id": 1191477, "author_name": "hks520110", "author_url": "", "post_date": "02/08/2021 13:32:15", "content": "cutmix and labelsmooth both help me
", "votes": null, "replies": [ { "id": 1192973, "author_name": "sanchitvj", "author_url": "", "post_date": "02/09/2021 11:44:30", "content": "While training CutMix and label smooth were giving lower accuracy and more loss than the one without them. Optimizer used was AdamW.
", "votes": null, "replies": [] } ] }, { "id": 1191859, "author_name": "aliabdin1", "author_url": "", "post_date": "02/08/2021 18:15:29", "content": "CutMix did not help me improve my CV/LB
", "votes": null, "replies": [ { "id": 1193818, "author_name": "aliabdin1", "author_url": "", "post_date": "02/09/2021 21:48:14", "content": "I have noticed, that CutMix did improve my CV/LB with a different Optimizer (I was using RAdam before, swapping to AdamW increased my score)
", "votes": null, "replies": [] }, { "id": 1194487, "author_name": "sanchitvj", "author_url": "", "post_date": "02/10/2021 08:04:15", "content": "How are you using CutMix can you give me some help or a tutorial link. I'm skeptical about my code if it is correct or not.
", "votes": null, "replies": [] } ] }, { "id": 1194596, "author_name": "vickygoyal", "author_url": "", "post_date": "02/10/2021 09:09:52", "content": "Cutmix is helpful.
", "votes": null, "replies": [ { "id": 1196573, "author_name": "sanchitvj", "author_url": "", "post_date": "02/11/2021 13:33:43", "content": "@vickygoyal How are you using it, can you help me. Probably I might be using it wrong.
", "votes": null, "replies": [] }, { "id": 1196773, "author_name": "vickygoyal", "author_url": "", "post_date": "02/11/2021 16:12:32", "content": "you can refer this link
", "votes": null, "replies": [] } ] }, { "id": 1195514, "author_name": "raufmomin", "author_url": "", "post_date": "02/10/2021 21:17:15", "content": "I tried cutmix but it didn't improved my LB/CV. Cutouts was useful for me.
", "votes": null, "replies": [ { "id": 1196905, "author_name": "sanchitvj", "author_url": "", "post_date": "02/11/2021 18:45:30", "content": "@raufmomin can you elaborate on the technique you mentioned or any link for reference.
", "votes": null, "replies": [] }, { "id": 1196973, "author_name": "raufmomin", "author_url": "", "post_date": "02/11/2021 19:56:21", "content": "Sure! You can check out this links.
\nLink: https://medium.com/@ombelinelag/cutout-regularization-for-cnns-62670d86bc33
\nPaper: https://arxiv.org/abs/1708.04552
Cutmix helps me, and other augmentation and general training tric , too.
\nI think the difference of cutmix effictiveness among people is largely depend on other parameter settings like learning rate, loss
\n and so on.