{ "id": 209065, "title": "labelsmooth and cutmix", "url": "/competitions/cassava-leaf-disease-classification/discussion/209065", "author_name": "simple", "post_date": "2021-01-06T06:00:31.655000", "votes": 23, "comment_count": 20, "views": 0, "content": "

In my experiment, half the probability is labelsmooth, half the probability is cutmix, the effect is best.

", "messages": [ { "id": 1140603, "postDate": "2021-01-06T06:00:31.657Z", "content": "

In my experiment, half the probability is labelsmooth, half the probability is cutmix, the effect is best.

", "rawMarkdown": "In my experiment, half the probability is labelsmooth, half the probability is cutmix, the effect is best.", "votes": 23 }, { "id": 1153905, "postDate": "2021-01-15T08:27:35.020Z", "content": "

how do you train VIT? with size 224*224? some augs like flip are useful?

", "rawMarkdown": "how do you train VIT? with size 224*224? some augs like flip are useful?" }, { "id": 1151443, "postDate": "2021-01-13T10:29:32.163Z", "content": "

Thanks for sharing. Which ViT version and pre-train weight did you use? and What framework and hardware (GPU/TPU) you used?

", "rawMarkdown": "Thanks for sharing. Which ViT version and pre-train weight did you use? and What framework and hardware (GPU/TPU) you used? " }, { "id": 1151124, "postDate": "2021-01-13T06:38:53.970Z", "content": "

Hi, I've tried the same method(cutmix's prob = 0.5 ，labelsmooth's prob =0.5 and labelsmooth=0.2), but the CV is lower than that using cutout and labelsmooth. What's your CV? Thanks!

", "rawMarkdown": "Hi, I've tried the same method(cutmix's prob = 0.5 ，labelsmooth's prob =0.5 and labelsmooth=0.2), but the CV is lower than that using cutout and labelsmooth. What's your CV? Thanks!", "replies": [ { "id": 1151199, "postDate": "2021-01-13T07:25:46.487Z", "content": "

我5fold平均cv大概在90.0%,最好的一个fold为90.3%.

", "rawMarkdown": "我5fold平均cv大概在90.0%,最好的一个fold为90.3%.", "votes": 1 } ] }, { "id": 1146333, "postDate": "2021-01-09T17:42:35.813Z", "content": "

Hi, what model do you use for your best model, efficientNet or Vit or ResNet? Thanks!

", "rawMarkdown": "Hi, what model do you use for your best model, efficientNet or Vit or ResNet? Thanks!", "replies": [ { "id": 1147056, "postDate": "2021-01-10T09:01:34.263Z", "content": "

vit…….

", "rawMarkdown": "vit......." } ] }, { "id": 1140791, "postDate": "2021-01-06T09:10:09.570Z", "content": "

Hi,how many epochs do you train in one fold?Thanks!

", "rawMarkdown": "Hi,how many epochs do you train in one fold?Thanks!", "replies": [ { "id": 1140867, "postDate": "2021-01-06T10:32:58.913Z", "content": "

10 epoch,
\nI try more than 10 epochs, but nothing is better effect

", "rawMarkdown": "10 epoch,\nI try more than 10 epochs, but nothing is better effect" }, { "id": 1140912, "postDate": "2021-01-06T11:07:44.233Z", "content": "

Thanks for your reply!By the way,what scheduler do you use?cosinscheduler?Thanks!

", "rawMarkdown": "Thanks for your reply!By the way,what scheduler do you use?cosinscheduler?Thanks!" }, { "id": 1140914, "postDate": "2021-01-06T11:10:30.483Z", "content": "

yes，I also use warmup scheduler ,it improve the cv but not lb

", "rawMarkdown": "yes，I also use warmup scheduler ,it improve the cv but not lb" }, { "id": 1142513, "postDate": "2021-01-07T12:42:21.837Z", "content": "

I use the learning rate is 1e-4,and CosineAnnealingWarmRestarts scheduler,but I only get cv:0.8944 when I use your method.I don't know there is why. @hks520110

", "rawMarkdown": "I use the learning rate is 1e-4,and CosineAnnealingWarmRestarts scheduler,but I only get cv:0.8944 when I use your method.I don't know there is why. @hks520110" }, { "id": 1142735, "postDate": "2021-01-07T15:09:13.730Z", "content": "

try more models ,such as vit or resneXt,and use tta

", "rawMarkdown": "try more models ,such as vit or resneXt,and use tta" } ] }, { "id": 1140661, "postDate": "2021-01-06T07:10:40.207Z", "content": "

Thanks for sharing! What's the difference of scores when using cutmix's prob = 1 and cutmix's prob = 0.5? (both cv and lb) I'm testing cutmix now.

", "rawMarkdown": "Thanks for sharing! What's the difference of scores when using cutmix's prob = 1 and cutmix's prob = 0.5? (both cv and lb) I'm testing cutmix now. ", "replies": [ { "id": 1140712, "postDate": "2021-01-06T08:04:52.540Z", "content": "

when cutmix's prob = 1，lb=0.901, when cutmix's prob = 0.5 ，labelsmooth's prob =0.5 and labelsmooth=0.2,lb=0.903

", "rawMarkdown": "when cutmix's prob = 1，lb=0.901, when cutmix's prob = 0.5 ，labelsmooth's prob =0.5 and labelsmooth=0.2,lb=0.903 ", "votes": 1 }, { "id": 1140773, "postDate": "2021-01-06T08:54:21.067Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1140781, "postDate": "2021-01-06T08:58:25.207Z", "content": "

I can see you everywhere :) Is that the desire for gold medals

", "rawMarkdown": "I can see you everywhere :) Is that the desire for gold medals" }, { "id": 1140849, "postDate": "2021-01-06T10:10:57.543Z", "content": "

Hello,is it just one fold?

", "rawMarkdown": "Hello,is it just one fold?" }, { "id": 1140873, "postDate": "2021-01-06T10:35:36.247Z", "content": "

5 StratifiedKFold, In my experiment,cv and lb is the best when half the probability is labelsmooth, half the probability is cutmix

", "rawMarkdown": "5 StratifiedKFold, In my experiment,cv and lb is the best when half the probability is labelsmooth, half the probability is cutmix", "replies": [ { "id": 1146987, "postDate": "2021-01-10T08:29:39.600Z", "content": "
\n

half the probability is labelsmooth

\n
\n

I did not understand, this means we apply LabelSmoothingCrossEntropy with 0.5 probability, other time we just use softmax cross entropy?

", "rawMarkdown": "> half the probability is labelsmooth\n\nI did not understand, this means we apply LabelSmoothingCrossEntropy with 0.5 probability, other time we just use softmax cross entropy?" } ] }, { "id": 1147055, "postDate": "2021-01-10T09:00:24.520Z", "content": "

yes,other time we just use softmax cross entropy with cutmix

", "rawMarkdown": "yes,other time we just use softmax cross entropy with cutmix", "votes": 1 } ] } ], "comments": [ { "id": 1153905, "author_name": "Qixin Yan", "author_url": "", "post_date": "2021-01-15T08:27:35.020000", "content": "

how do you train VIT? with size 224*224? some augs like flip are useful?

", "votes": 0, "replies": [] }, { "id": 1151443, "author_name": "Md. Masud Rana", "author_url": "", "post_date": "2021-01-13T10:29:32.163000", "content": "

Thanks for sharing. Which ViT version and pre-train weight did you use? and What framework and hardware (GPU/TPU) you used?

", "votes": 0, "replies": [] }, { "id": 1151124, "author_name": "Lin Deng", "author_url": "", "post_date": "2021-01-13T06:38:53.970000", "content": "

Hi, I've tried the same method(cutmix's prob = 0.5 ，labelsmooth's prob =0.5 and labelsmooth=0.2), but the CV is lower than that using cutout and labelsmooth. What's your CV? Thanks!

", "votes": 0, "replies": [ { "id": 1151199, "author_name": "simple", "author_url": "", "post_date": "2021-01-13T07:25:46.487000", "content": "

我5fold平均cv大概在90.0%,最好的一个fold为90.3%.

", "votes": 1, "replies": [] } ] }, { "id": 1146333, "author_name": "Wenkang Wei", "author_url": "", "post_date": "2021-01-09T17:42:35.813000", "content": "

Hi, what model do you use for your best model, efficientNet or Vit or ResNet? Thanks!

", "votes": 0, "replies": [ { "id": 1147056, "author_name": "simple", "author_url": "", "post_date": "2021-01-10T09:01:34.263000", "content": "

vit…….

", "votes": 0, "replies": [] } ] }, { "id": 1140791, "author_name": "Bcw93", "author_url": "", "post_date": "2021-01-06T09:10:09.570000", "content": "

Hi,how many epochs do you train in one fold?Thanks!

", "votes": 0, "replies": [ { "id": 1140867, "author_name": "simple", "author_url": "", "post_date": "2021-01-06T10:32:58.913000", "content": "

10 epoch,
\nI try more than 10 epochs, but nothing is better effect

", "votes": 0, "replies": [] }, { "id": 1140912, "author_name": "Bcw93", "author_url": "", "post_date": "2021-01-06T11:07:44.233000", "content": "

Thanks for your reply!By the way,what scheduler do you use?cosinscheduler?Thanks!

", "votes": 0, "replies": [] }, { "id": 1140914, "author_name": "simple", "author_url": "", "post_date": "2021-01-06T11:10:30.483000", "content": "

yes，I also use warmup scheduler ,it improve the cv but not lb

", "votes": 0, "replies": [] }, { "id": 1142513, "author_name": "Bcw93", "author_url": "", "post_date": "2021-01-07T12:42:21.837000", "content": "

I use the learning rate is 1e-4,and CosineAnnealingWarmRestarts scheduler,but I only get cv:0.8944 when I use your method.I don't know there is why. @hks520110

", "votes": 0, "replies": [] }, { "id": 1142735, "author_name": "simple", "author_url": "", "post_date": "2021-01-07T15:09:13.730000", "content": "

try more models ,such as vit or resneXt,and use tta

", "votes": 0, "replies": [] } ] }, { "id": 1140661, "author_name": "clwclw", "author_url": "", "post_date": "2021-01-06T07:10:40.207000", "content": "

Thanks for sharing! What's the difference of scores when using cutmix's prob = 1 and cutmix's prob = 0.5? (both cv and lb) I'm testing cutmix now.

", "votes": 0, "replies": [ { "id": 1140712, "author_name": "simple", "author_url": "", "post_date": "2021-01-06T08:04:52.540000", "content": "

when cutmix's prob = 1，lb=0.901, when cutmix's prob = 0.5 ，labelsmooth's prob =0.5 and labelsmooth=0.2,lb=0.903

", "votes": 1, "replies": [] }, { "id": 1140773, "author_name": "", "author_url": "", "post_date": "2021-01-06T08:54:21.067000", "content": "", "votes": 0, "replies": [] }, { "id": 1140781, "author_name": "clwclw", "author_url": "", "post_date": "2021-01-06T08:58:25.207000", "content": "

I can see you everywhere :) Is that the desire for gold medals

", "votes": 0, "replies": [] }, { "id": 1140849, "author_name": "Zekun", "author_url": "", "post_date": "2021-01-06T10:10:57.543000", "content": "

Hello,is it just one fold?

", "votes": 0, "replies": [] }, { "id": 1140873, "author_name": "simple", "author_url": "", "post_date": "2021-01-06T10:35:36.247000", "content": "

5 StratifiedKFold, In my experiment,cv and lb is the best when half the probability is labelsmooth, half the probability is cutmix

", "votes": 0, "replies": [ { "id": 1146987, "author_name": "Kevin Mathew T", "author_url": "", "post_date": "2021-01-10T08:29:39.600000", "content": "
\n

half the probability is labelsmooth

\n
\n

I did not understand, this means we apply LabelSmoothingCrossEntropy with 0.5 probability, other time we just use softmax cross entropy?

", "votes": 0, "replies": [] } ] }, { "id": 1147055, "author_name": "simple", "author_url": "", "post_date": "2021-01-10T09:00:24.520000", "content": "

yes,other time we just use softmax cross entropy with cutmix

", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1140603": "In my experiment, half the probability is labelsmooth, half the probability is cutmix, the effect is best.", "1153905": "how do you train VIT? with size 224*224? some augs like flip are useful?", "1151443": "Thanks for sharing. Which ViT version and pre-train weight did you use? and What framework and hardware (GPU/TPU) you used? ", "1151124": "Hi, I've tried the same method(cutmix's prob = 0.5 ，labelsmooth's prob =0.5 and labelsmooth=0.2), but the CV is lower than that using cutout and labelsmooth. What's your CV? Thanks!", "1146333": "Hi, what model do you use for your best model, efficientNet or Vit or ResNet? Thanks!", "1140791": "Hi,how many epochs do you train in one fold?Thanks!", "1140661": "Thanks for sharing! What's the difference of scores when using cutmix's prob = 1 and cutmix's prob = 0.5? (both cv and lb) I'm testing cutmix now. " } }