{ "id": 206724, "title": "Light augs to 0.906", "url": "/competitions/cassava-leaf-disease-classification/discussion/206724", "author_name": "", "post_date": "2020-12-26T08:24:35.662027700Z", "votes": 114, "comment_count": 27, "views": 0, "content": "

Yesterday, I shared the opinion here https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/206489 that we might need carefully select TTA augs or even augs while training.

\n

I rarely train a model with almost no augs in CV task because regular augs always help.

\n

However, I noticed light augs in this task leads to an obvious boost on LB (did not boost that much on CV).

\n

Therefore, I trained a light aug version model today, and it boosts my LB to the 1st.

\n

Never been such a high rank before. Although the CV score indicates that it is likely to be overfitting on public LB. I think it still worth sharing for those who wanna climb a higher place.

\n

Edit: date 1.27
\nI shall highlight again, it boosts a bit on public LB, but not with a good CV. For unseen data, enough augs are still needed to be robust. Do not pay too much attention on climbing public LB, trust your CV.

", "messages": [ { "id": "1127092", "postDate": "12/26/2020 08:24:35", "content": "

Yesterday, I shared the opinion here https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/206489 that we might need carefully select TTA augs or even augs while training.

\n

I rarely train a model with almost no augs in CV task because regular augs always help.

\n

However, I noticed light augs in this task leads to an obvious boost on LB (did not boost that much on CV).

\n

Therefore, I trained a light aug version model today, and it boosts my LB to the 1st.

\n

Never been such a high rank before. Although the CV score indicates that it is likely to be overfitting on public LB. I think it still worth sharing for those who wanna climb a higher place.

\n

Edit: date 1.27
\nI shall highlight again, it boosts a bit on public LB, but not with a good CV. For unseen data, enough augs are still needed to be robust. Do not pay too much attention on climbing public LB, trust your CV.

", "rawMarkdown": "Yesterday, I shared the opinion here [https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/206489](url) that we might need carefully select TTA augs or even augs while training.\n\nI rarely train a model with almost no augs in CV task because regular augs always help.\n\nHowever, I noticed light augs in this task leads to an obvious boost on LB (did not boost that much on CV).\n\nTherefore, I trained a light aug version model today, and it boosts my LB to the 1st.\n\nNever been such a high rank before. Although the CV score indicates that **it is likely to be overfitting on public LB**. I think it still worth sharing for those who wanna climb a higher place.\n\nEdit: date 1.27\n**I shall highlight again, it boosts a bit on public LB, but not with a good CV. For unseen data, enough augs are still needed to be robust. Do not pay too much attention on climbing public LB, trust your CV.**", "votes": null }, { "id": "1127368", "postDate": "12/26/2020 13:32:14", "content": "

Hello,What means \"Light\" ?

", "rawMarkdown": "Hello,What means \"Light\" ?", "votes": null }, { "id": "1127380", "postDate": "12/26/2020 13:49:25", "content": "

So are you doing just crop, resize and normalisation? No TTA? Is 0.906 the result of an ensemble or a single model (5 folds)? What image size are you cropping?

", "rawMarkdown": "So are you doing just crop, resize and normalisation? No TTA? Is 0.906 the result of an ensemble or a single model (5 folds)? What image size are you cropping?", "votes": null }, { "id": "1127416", "postDate": "12/26/2020 14:25:38", "content": "

less is more!

", "rawMarkdown": "less is more!", "votes": null }, { "id": "1127466", "postDate": "12/26/2020 15:05:37", "content": "

Details of the light seems interesting here :)

", "rawMarkdown": "Details of the light seems interesting here :)", "votes": null }, { "id": "1127477", "postDate": "12/26/2020 15:12:06", "content": "

Btw, can you share how many TTA steps do you apply? It's perfectly fine if you don't want to share :)

", "rawMarkdown": "Btw, can you share how many TTA steps do you apply? It's perfectly fine if you don't want to share :)", "votes": null }, { "id": "1127531", "postDate": "12/26/2020 16:00:30", "content": "

4 for the best score currently

", "rawMarkdown": "4 for the best score currently", "votes": null }, { "id": "1127534", "postDate": "12/26/2020 16:05:39", "content": "

Just crop, resize and normalisation while TTA and a bit more flip & cutout for training. I did not start to test more augs yet.
\n0.906 is the result of a ViT model and a self-ensemble effnet. You could see all you want at this great notebook https://www.kaggle.com/szuzhangzhi/vit-cuda-as-usual-ensemble-inference.
\nFor image size, I am using 384 while inference. However, I trained the effnet under a slightly bigger size.

", "rawMarkdown": "Just crop, resize and normalisation while TTA and a bit more flip & cutout for training. I did not start to test more augs yet.\n0.906 is the result of a ViT model and a self-ensemble effnet. You could see all you want at this great notebook https://www.kaggle.com/szuzhangzhi/vit-cuda-as-usual-ensemble-inference.\nFor image size, I am using 384 while inference. However, I trained the effnet under a slightly bigger size.", "votes": null }, { "id": "1127647", "postDate": "12/26/2020 17:58:01", "content": "

thanks for the info my friend! Just did an experiment and removed some augmentations and it improbed my CV. Now im curious how many epoch do you train since you have not so many augs it must be lower than before to not overfit?

", "rawMarkdown": "thanks for the info my friend! Just did an experiment and removed some augmentations and it improbed my CV. Now im curious how many epoch do you train since you have not so many augs it must be lower than before to not overfit?", "votes": null }, { "id": "1127656", "postDate": "12/26/2020 18:04:23", "content": "

What's your CV score?

", "rawMarkdown": "What's your CV score?", "votes": null }, { "id": "1127679", "postDate": "12/26/2020 18:28:37", "content": "

I think RandomBrightnessContrast Augmentation, it is right??

", "rawMarkdown": "I think `RandomBrightnessContrast Augmentation`, it is right??", "votes": null }, { "id": "1127684", "postDate": "12/26/2020 18:32:49", "content": "

\"Light\" augmentations means the augmentations which do not change the images much. For example H/V Flips, RandomCrop, Cutouts etc. are \"lighter\" then Mixup, CutMix, GridMask etc.

", "rawMarkdown": "\"Light\" augmentations means the augmentations which do not change the images much. For example H/V Flips, RandomCrop, Cutouts etc. are \"lighter\" then Mixup, CutMix, GridMask etc.", "votes": null }, { "id": "1127696", "postDate": "12/26/2020 19:01:05", "content": "

CV acc 0.919 for the best one.
\nMost of them between 0.902-0.905 have CV acc around 0.92.
\nThe 0.906 submission had a lower CV acc than other 0.903 submissions.
\nWe'd better pay more attention to the CV score, the shake-up on the private LB might be a critical problem.
\nI would recommend more experiments on the ViT, it weights a lot in my recent submissions.

", "rawMarkdown": "CV acc 0.919 for the best one. \nMost of them between 0.902-0.905 have CV acc around 0.92.\nThe 0.906 submission had a lower CV acc than other 0.903 submissions. \nWe'd better pay more attention to the CV score, the shake-up on the private LB might be a critical problem.\nI would recommend more experiments on the ViT, it weights a lot in my recent submissions.", "votes": null }, { "id": "1127700", "postDate": "12/26/2020 19:04:05", "content": "

Oh great. Thanks!!

", "rawMarkdown": "Oh great. Thanks!!", "votes": null }, { "id": "1127703", "postDate": "12/26/2020 19:07:20", "content": "

Currently no more than 10 epochs

", "rawMarkdown": "Currently no more than 10 epochs", "votes": null }, { "id": "1127755", "postDate": "12/26/2020 20:27:30", "content": "

Hi again,

\n

After reading your post, I just removed some of the augmentions from the TTA pipeline (brightness etc.), we still got .0901 but advanced 20 more rank in the LB. Wow!

", "rawMarkdown": "Hi again, \n\nAfter reading your post, I just removed some of the augmentions from the TTA pipeline (brightness etc.), we still got .0901 but advanced 20 more rank in the LB. Wow!", "votes": null }, { "id": "1127885", "postDate": "12/27/2020 01:35:51", "content": "

@kaushal2896 Oh, thank you so much for letting me know.!!

", "rawMarkdown": "kaushal2896 Oh, thank you so much for letting me know.!!", "votes": null }, { "id": "1127960", "postDate": "12/27/2020 04:22:41", "content": "

@zlanan Oh my, when I tried to train my model under 512 size and tried to ensemble with the VIT (which is 384), it seems to have a pretty bad validation score. How did you manage to ensemble a different image size with VIT? Thanks :)

", "rawMarkdown": "zlanan Oh my, when I tried to train my model under 512 size and tried to ensemble with the VIT (which is 384), it seems to have a pretty bad validation score. How did you manage to ensemble a different image size with VIT? Thanks :)", "votes": null }, { "id": "1128151", "postDate": "12/27/2020 08:04:09", "content": "

Thanks for the information, I really appreciate it. (Upvoted)

\n

I hope it doesn't cost you your \"In The Money\" position though 😅.

", "rawMarkdown": "Thanks for the information, I really appreciate it. (Upvoted)\n\nI hope it doesn't cost you your \"In The Money\" position though 😅.", "votes": null }, { "id": "1128248", "postDate": "12/27/2020 09:23:25", "content": "

Since the default input size of effnet B5 is 456, I trained it with this size.
\nYou can directly feed size 384 of the image, or you can load data separately then sum predictions with weights.

", "rawMarkdown": "Since the default input size of effnet B5 is 456, I trained it with this size.\nYou can directly feed size 384 of the image, or you can load data separately then sum predictions with weights.", "votes": null }, { "id": "1128328", "postDate": "12/27/2020 10:39:47", "content": "

Thanks! Got it! I guess I will just load separately and sum for different ensembles since different image size.

", "rawMarkdown": "Thanks! Got it! I guess I will just load separately and sum for different ensembles since different image size.", "votes": null }, { "id": "1128407", "postDate": "12/27/2020 12:22:04", "content": "

Do you use the light augmentation also in TTA?

", "rawMarkdown": "Do you use the light augmentation also in TTA?", "votes": null }, { "id": "1129785", "postDate": "12/28/2020 14:50:56", "content": "

Thanks for the insight.

", "rawMarkdown": "Thanks for the insight.", "votes": null }, { "id": "1130641", "postDate": "12/29/2020 07:40:41", "content": "

Hello, what is the LB accuracy of your effnet? Didn't you use something like flipping in the test? Thank you very much.

", "rawMarkdown": "Hello, what is the LB accuracy of your effnet? Didn't you use something like flipping in the test? Thank you very much.", "votes": null }, { "id": "1133783", "postDate": "12/31/2020 14:24:10", "content": "

Wow, sounds really interesting! I thought heavy augmentations like snapmix or cutmix are really helpful, but it might be don't. Thanks for sharing your experience and great work!

", "rawMarkdown": "Wow, sounds really interesting! I thought heavy augmentations like snapmix or cutmix are really helpful, but it might be don't. Thanks for sharing your experience and great work!", "votes": null }, { "id": "1140771", "postDate": "01/06/2021 08:54:01", "content": "

Are you kidding :)

", "rawMarkdown": "Are you kidding :)", "votes": null }, { "id": "1140799", "postDate": "01/06/2021 09:24:38", "content": "

Do you use the rwightman's code to train EfficientNet, or some notebook like https://www.kaggle.com/khyeh0719/pytorch-efficientnet-baseline-train-amp-aug ?

", "rawMarkdown": "Do you use the rwightman's code to train EfficientNet, or some notebook like https://www.kaggle.com/khyeh0719/pytorch-efficientnet-baseline-train-amp-aug ?", "votes": null }, { "id": "2500384", "postDate": "10/26/2023 16:14:03", "content": "

thanks for your advice on light augs，which is very important to me

", "rawMarkdown": "thanks for your advice on light augs，which is very important to me", "votes": null } ], "comments": [ { "id": 1127368, "author_name": "qq1623620766", "author_url": "", "post_date": "12/26/2020 13:32:14", "content": "

Hello,What means \"Light\" ?

", "votes": null, "replies": [ { "id": 1127679, "author_name": "chocozzz", "author_url": "", "post_date": "12/26/2020 18:28:37", "content": "

I think RandomBrightnessContrast Augmentation, it is right??

", "votes": null, "replies": [] }, { "id": 1127684, "author_name": "kaushal2896", "author_url": "", "post_date": "12/26/2020 18:32:49", "content": "

\"Light\" augmentations means the augmentations which do not change the images much. For example H/V Flips, RandomCrop, Cutouts etc. are \"lighter\" then Mixup, CutMix, GridMask etc.

", "votes": null, "replies": [] }, { "id": 1127885, "author_name": "chocozzz", "author_url": "", "post_date": "12/27/2020 01:35:51", "content": "

@kaushal2896 Oh, thank you so much for letting me know.!!

", "votes": null, "replies": [] }, { "id": 1140771, "author_name": "clwwlc", "author_url": "", "post_date": "01/06/2021 08:54:01", "content": "

Are you kidding :)

", "votes": null, "replies": [] } ] }, { "id": 1127380, "author_name": "richardoshaw", "author_url": "", "post_date": "12/26/2020 13:49:25", "content": "

So are you doing just crop, resize and normalisation? No TTA? Is 0.906 the result of an ensemble or a single model (5 folds)? What image size are you cropping?

", "votes": null, "replies": [ { "id": 1127534, "author_name": "zlanan", "author_url": "", "post_date": "12/26/2020 16:05:39", "content": "

Just crop, resize and normalisation while TTA and a bit more flip & cutout for training. I did not start to test more augs yet.
\n0.906 is the result of a ViT model and a self-ensemble effnet. You could see all you want at this great notebook https://www.kaggle.com/szuzhangzhi/vit-cuda-as-usual-ensemble-inference.
\nFor image size, I am using 384 while inference. However, I trained the effnet under a slightly bigger size.

", "votes": null, "replies": [] }, { "id": 1127960, "author_name": "reighns", "author_url": "", "post_date": "12/27/2020 04:22:41", "content": "

@zlanan Oh my, when I tried to train my model under 512 size and tried to ensemble with the VIT (which is 384), it seems to have a pretty bad validation score. How did you manage to ensemble a different image size with VIT? Thanks :)

", "votes": null, "replies": [] }, { "id": 1128248, "author_name": "zlanan", "author_url": "", "post_date": "12/27/2020 09:23:25", "content": "

Since the default input size of effnet B5 is 456, I trained it with this size.
\nYou can directly feed size 384 of the image, or you can load data separately then sum predictions with weights.

", "votes": null, "replies": [] }, { "id": 1128328, "author_name": "reighns", "author_url": "", "post_date": "12/27/2020 10:39:47", "content": "

Thanks! Got it! I guess I will just load separately and sum for different ensembles since different image size.

", "votes": null, "replies": [] } ] }, { "id": 1127416, "author_name": "hks520110", "author_url": "", "post_date": "12/26/2020 14:25:38", "content": "

less is more!

", "votes": null, "replies": [] }, { "id": 1127466, "author_name": "snnclsr", "author_url": "", "post_date": "12/26/2020 15:05:37", "content": "

Details of the light seems interesting here :)

", "votes": null, "replies": [] }, { "id": 1127477, "author_name": "snnclsr", "author_url": "", "post_date": "12/26/2020 15:12:06", "content": "

Btw, can you share how many TTA steps do you apply? It's perfectly fine if you don't want to share :)

", "votes": null, "replies": [ { "id": 1127531, "author_name": "zlanan", "author_url": "", "post_date": "12/26/2020 16:00:30", "content": "

4 for the best score currently

", "votes": null, "replies": [] }, { "id": 1127700, "author_name": "snnclsr", "author_url": "", "post_date": "12/26/2020 19:04:05", "content": "

Oh great. Thanks!!

", "votes": null, "replies": [] } ] }, { "id": 1127647, "author_name": "yannmajewski", "author_url": "", "post_date": "12/26/2020 17:58:01", "content": "

thanks for the info my friend! Just did an experiment and removed some augmentations and it improbed my CV. Now im curious how many epoch do you train since you have not so many augs it must be lower than before to not overfit?

", "votes": null, "replies": [ { "id": 1127703, "author_name": "zlanan", "author_url": "", "post_date": "12/26/2020 19:07:20", "content": "

Currently no more than 10 epochs

", "votes": null, "replies": [] } ] }, { "id": 1127656, "author_name": "kaushal2896", "author_url": "", "post_date": "12/26/2020 18:04:23", "content": "

What's your CV score?

", "votes": null, "replies": [ { "id": 1127696, "author_name": "zlanan", "author_url": "", "post_date": "12/26/2020 19:01:05", "content": "

CV acc 0.919 for the best one.
\nMost of them between 0.902-0.905 have CV acc around 0.92.
\nThe 0.906 submission had a lower CV acc than other 0.903 submissions.
\nWe'd better pay more attention to the CV score, the shake-up on the private LB might be a critical problem.
\nI would recommend more experiments on the ViT, it weights a lot in my recent submissions.

", "votes": null, "replies": [] } ] }, { "id": 1127755, "author_name": "snnclsr", "author_url": "", "post_date": "12/26/2020 20:27:30", "content": "

Hi again,

\n

After reading your post, I just removed some of the augmentions from the TTA pipeline (brightness etc.), we still got .0901 but advanced 20 more rank in the LB. Wow!

", "votes": null, "replies": [] }, { "id": 1128151, "author_name": "dumbluck", "author_url": "", "post_date": "12/27/2020 08:04:09", "content": "

Thanks for the information, I really appreciate it. (Upvoted)

\n

I hope it doesn't cost you your \"In The Money\" position though 😅.

", "votes": null, "replies": [] }, { "id": 1128407, "author_name": "aliabdin1", "author_url": "", "post_date": "12/27/2020 12:22:04", "content": "

Do you use the light augmentation also in TTA?

", "votes": null, "replies": [] }, { "id": 1129785, "author_name": "discoveringrealty", "author_url": "", "post_date": "12/28/2020 14:50:56", "content": "

Thanks for the insight.

", "votes": null, "replies": [] }, { "id": 1130641, "author_name": "sjtuyxc", "author_url": "", "post_date": "12/29/2020 07:40:41", "content": "

Hello, what is the LB accuracy of your effnet? Didn't you use something like flipping in the test? Thank you very much.

", "votes": null, "replies": [] }, { "id": 1133783, "author_name": "vkehfdl1", "author_url": "", "post_date": "12/31/2020 14:24:10", "content": "

Wow, sounds really interesting! I thought heavy augmentations like snapmix or cutmix are really helpful, but it might be don't. Thanks for sharing your experience and great work!

", "votes": null, "replies": [] }, { "id": 1140799, "author_name": "clwwlc", "author_url": "", "post_date": "01/06/2021 09:24:38", "content": "

Do you use the rwightman's code to train EfficientNet, or some notebook like https://www.kaggle.com/khyeh0719/pytorch-efficientnet-baseline-train-amp-aug ?

", "votes": null, "replies": [] }, { "id": 2500384, "author_name": "fredrickunderwood", "author_url": "", "post_date": "10/26/2023 16:14:03", "content": "

thanks for your advice on light augs，which is very important to me

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1127092": "Yesterday, I shared the opinion here [https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/206489](url) that we might need carefully select TTA augs or even augs while training.\n\nI rarely train a model with almost no augs in CV task because regular augs always help.\n\nHowever, I noticed light augs in this task leads to an obvious boost on LB (did not boost that much on CV).\n\nTherefore, I trained a light aug version model today, and it boosts my LB to the 1st.\n\nNever been such a high rank before. Although the CV score indicates that **it is likely to be overfitting on public LB**. I think it still worth sharing for those who wanna climb a higher place.\n\nEdit: date 1.27\n**I shall highlight again, it boosts a bit on public LB, but not with a good CV. For unseen data, enough augs are still needed to be robust. Do not pay too much attention on climbing public LB, trust your CV.**", "1127368": "Hello,What means \"Light\" ?", "1127380": "So are you doing just crop, resize and normalisation? No TTA? Is 0.906 the result of an ensemble or a single model (5 folds)? What image size are you cropping?", "1127416": "less is more!", "1127466": "Details of the light seems interesting here :)", "1127477": "Btw, can you share how many TTA steps do you apply? It's perfectly fine if you don't want to share :)", "1127531": "4 for the best score currently", "1127534": "Just crop, resize and normalisation while TTA and a bit more flip & cutout for training. I did not start to test more augs yet.\n0.906 is the result of a ViT model and a self-ensemble effnet. You could see all you want at this great notebook https://www.kaggle.com/szuzhangzhi/vit-cuda-as-usual-ensemble-inference.\nFor image size, I am using 384 while inference. However, I trained the effnet under a slightly bigger size.", "1127647": "thanks for the info my friend! Just did an experiment and removed some augmentations and it improbed my CV. Now im curious how many epoch do you train since you have not so many augs it must be lower than before to not overfit?", "1127656": "What's your CV score?", "1127679": "I think `RandomBrightnessContrast Augmentation`, it is right??", "1127684": "\"Light\" augmentations means the augmentations which do not change the images much. For example H/V Flips, RandomCrop, Cutouts etc. are \"lighter\" then Mixup, CutMix, GridMask etc.", "1127696": "CV acc 0.919 for the best one. \nMost of them between 0.902-0.905 have CV acc around 0.92.\nThe 0.906 submission had a lower CV acc than other 0.903 submissions. \nWe'd better pay more attention to the CV score, the shake-up on the private LB might be a critical problem.\nI would recommend more experiments on the ViT, it weights a lot in my recent submissions.", "1127700": "Oh great. Thanks!!", "1127703": "Currently no more than 10 epochs", "1127755": "Hi again, \n\nAfter reading your post, I just removed some of the augmentions from the TTA pipeline (brightness etc.), we still got .0901 but advanced 20 more rank in the LB. Wow!", "1127885": "kaushal2896 Oh, thank you so much for letting me know.!!", "1127960": "zlanan Oh my, when I tried to train my model under 512 size and tried to ensemble with the VIT (which is 384), it seems to have a pretty bad validation score. How did you manage to ensemble a different image size with VIT? Thanks :)", "1128151": "Thanks for the information, I really appreciate it. (Upvoted)\n\nI hope it doesn't cost you your \"In The Money\" position though 😅.", "1128248": "Since the default input size of effnet B5 is 456, I trained it with this size.\nYou can directly feed size 384 of the image, or you can load data separately then sum predictions with weights.", "1128328": "Thanks! Got it! I guess I will just load separately and sum for different ensembles since different image size.", "1128407": "Do you use the light augmentation also in TTA?", "1129785": "Thanks for the insight.", "1130641": "Hello, what is the LB accuracy of your effnet? Didn't you use something like flipping in the test? Thank you very much.", "1133783": "Wow, sounds really interesting! I thought heavy augmentations like snapmix or cutmix are really helpful, but it might be don't. Thanks for sharing your experience and great work!", "1140771": "Are you kidding :)", "1140799": "Do you use the rwightman's code to train EfficientNet, or some notebook like https://www.kaggle.com/khyeh0719/pytorch-efficientnet-baseline-train-amp-aug ?", "2500384": "thanks for your advice on light augs，which is very important to me" }, "source": "meta" }