![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1461289%2F10caf6a0aad75e18bdcdae6d0b1e1478%2Ffold_1.png?generation=1608141530630976&alt=media\")
Good old best single model score thread :)
\nWe have 0.898 on LB.
\nWhat about yours?
", "messages": [ { "id": 1111515, "postDate": "2020-12-13T19:09:34.120Z", "content": "Good old best single model score thread :)
\nWe have 0.898 on LB.
\nWhat about yours?
", "rawMarkdown": "Good old best single model score thread :) \n\nWe have 0.898 on LB.\n\nWhat about yours?", "votes": 67 }, { "id": 1111578, "postDate": "2020-12-13T20:20:51.377Z", "content": "0.904 single EfficientNet B4
", "rawMarkdown": "0.904 single EfficientNet B4", "votes": 18, "replies": [ { "id": 1112116, "postDate": "2020-12-14T09:43:11.010Z", "content": "@serigne what is your CV ? Also how many folds ?
", "rawMarkdown": "@serigne what is your CV ? Also how many folds ?" }, { "id": 1112130, "postDate": "2020-12-14T10:06:36.783Z", "content": "CV = 0.902 and 5 folds , mixed precision training.
", "rawMarkdown": "CV = 0.902 and 5 folds , mixed precision training. ", "votes": 7 }, { "id": 1112142, "postDate": "2020-12-14T10:23:02.220Z", "content": "Thanks. That helps a lot !
", "rawMarkdown": "Thanks. That helps a lot !" }, { "id": 1112308, "postDate": "2020-12-14T13:13:23.747Z", "content": "@serigne - The cv is OOF or avg across folds?
", "rawMarkdown": "@serigne - The cv is OOF or avg across folds?", "votes": 1 }, { "id": 1112426, "postDate": "2020-12-14T15:29:26.167Z", "content": "Average across folds.
\nAll logs and best Chekpoints saved for every successful training experiment. I will compute OOF if ever I plan to blend with other models.
", "rawMarkdown": "Average across folds. \n\nAll logs and best Chekpoints saved for every successful training experiment. I will compute OOF if ever I plan to blend with other models. ", "votes": 3 }, { "id": 1112729, "postDate": "2020-12-14T20:49:44.023Z", "content": "what is the image size for B4? 512x512 or larger size?
", "rawMarkdown": "what is the image size for B4? 512x512 or larger size?" }, { "id": 1112989, "postDate": "2020-12-15T04:54:54.263Z", "content": "Can u share any trick you used for this challange ?
", "rawMarkdown": "Can u share any trick you used for this challange ?" }, { "id": 1113382, "postDate": "2020-12-15T12:06:44.243Z", "content": "Nice score! If you like to answer - do you use 2019 or additional data as well ?
", "rawMarkdown": "Nice score! If you like to answer - do you use 2019 or additional data as well ? " }, { "id": 1115471, "postDate": "2020-12-16T10:09:34.987Z", "content": "I use 512x512 and 2019 dataset too, duplicates identified with DBSCAN Clustering and removed
", "rawMarkdown": "I use 512x512 and 2019 dataset too, duplicates identified with DBSCAN Clustering and removed", "votes": 8 }, { "id": 1115497, "postDate": "2020-12-16T10:21:37.017Z", "content": "@serigne Good to know. For my score, i.e. 0.898, I have not used 2019 data. What's your score without the 2019 data?
", "rawMarkdown": "@serigne Good to know. For my score, i.e. 0.898, I have not used 2019 data. What's your score without the 2019 data?" }, { "id": 1115506, "postDate": "2020-12-16T10:33:17.833Z", "content": "@abhishek I don't know. I've been using it for all my expemiments. , at least before the LB re-score. And I have modified my training process meanwhile. Thus I need to re-run only the 2020 dataset with the same training process in order to compare.
", "rawMarkdown": "@abhishek I don't know. I've been using it for all my expemiments. , at least before the LB re-score. And I have modified my training process meanwhile. Thus I need to re-run only the 2020 dataset with the same training process in order to compare. " }, { "id": 1115829, "postDate": "2020-12-16T15:52:15.210Z", "content": "Hi,what batch_size do you use?And what scheduler do you use?Thanks!
", "rawMarkdown": "Hi,what batch_size do you use?And what scheduler do you use?Thanks!" }, { "id": 1115841, "postDate": "2020-12-16T16:06:48.607Z", "content": "Hi, can you share the merged data?Thanks!
", "rawMarkdown": "Hi, can you share the merged data?Thanks!" }, { "id": 1121815, "postDate": "2020-12-21T23:45:55.693Z", "content": "Hi @serigne,
\nDoes this score comes from a single instance of your model or from an ensemble of all 5 folds?
", "rawMarkdown": "Hi @serigne,\n\nDoes this score comes from a single instance of your model or from an ensemble of all 5 folds?" }, { "id": 1149551, "postDate": "2021-01-11T23:57:51.390Z", "content": "Hi @serigne,
\nAre you using TTA or any augmentations such as Cutmix or Mixup?
", "rawMarkdown": "Hi @serigne,\n\nAre you using TTA or any augmentations such as Cutmix or Mixup?" }, { "id": 1164037, "postDate": "2021-01-22T05:14:01.997Z", "content": "Batch size? did you train on kaggle?
", "rawMarkdown": "Batch size? did you train on kaggle?\n" }, { "id": 1167333, "postDate": "2021-01-24T07:31:59.157Z", "content": "@serigne is it single model 5 folds ensemble or single model single fold ?
", "rawMarkdown": "@serigne is it single model 5 folds ensemble or single model single fold ?" } ] }, { "id": 1165230, "postDate": "2021-01-22T19:40:15.937Z", "content": "My best model is now ViT base 16x16 patches.
\nLB 0.904 (slightly better than my effnet B4)
\nBut I do not trust it at all : Huge gap between CV and LB and the LB result is highly sensitive to the number of TTA.
\nMy Effnet is much more stable across many experiments, and has much higher CV.
", "rawMarkdown": "My best model is now ViT base 16x16 patches. \n\nLB 0.904 (slightly better than my effnet B4)\n\nBut I do not trust it at all : Huge gap between CV and LB and the LB result is highly sensitive to the number of TTA. \n\nMy Effnet is much more stable across many experiments, and has much higher CV. \n\n", "votes": 9, "replies": [ { "id": 1165256, "postDate": "2021-01-22T20:04:01.877Z", "content": "Even this thread mentions low CV with ViT but high LB
", "rawMarkdown": "Even this [thread](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/212411#1162195) mentions low CV with ViT but high LB" }, { "id": 1165329, "postDate": "2021-01-22T21:59:55.090Z", "content": "I will give a try to ViT Large and others training process but it needs TPU or higher quality GPU. It's been a while I haven't used Torch/XLA ^^. I will try to configure it later.
", "rawMarkdown": "I will give a try to ViT Large and others training process but it needs TPU or higher quality GPU. It's been a while I haven't used Torch/XLA ^^. I will try to configure it later. " } ] }, { "id": 1149675, "postDate": "2021-01-12T04:00:53.080Z", "content": "CV 0.883 LB 0.890 (same model) I'm having trouble getting my CV scores over 0.885 during training. And don't know what i could be doing wrong at the moment. What i'm doing:
\nImage Sizes : 384
\nModels: EffNetB3,B4,ResNext50 (all train to around 88.0 ~ 88.5 CV) +Dropout(0.3) at last layer
\nEarlyStopping: 2 Patience with Minimum Loss and save best weights at minimum
\nStratifiedKFolds = 5
\nAdded 2019 Data (was careful with what @serigne told me to not include them at the validation set) : improved a tiny bit
\nOptimizer: Adam,Ranger Adam actually worked a bit better than Ranger with schedulers
\nScheduler: OneCycleLR >= CosineAnnealingWarmRestarts > ReduceLROnPlateau >> LambdaLR -> OneCycle depends on the LossFunction to improve CV score
\nLosses: TaylorCrossEntropy > BiTemperedLogistic(>20 permutations of T1,T2 mostly low T1 and T2~= 1.0) > SymmetricCrossEntropy
\nAugmentations: Tried Only Flips/Rotates and Normalization and that + ColorJitter and CoarseDropout and AutoAugment from imagenet
\nI'm really not sure what strategy you guys are using to reach over 88.5 on CV, i tried almost everything that was dicussed here on the forums.
I think (Se)Resnext50 is a bit tiny for the data.
\nI managed to improve its CV and LB by implementing some tricks (CV 0.900 and LB 0.901). But I can't disclose before the end of the competition
Otherwise bigger models like EFFB4 and image sizes 512 are the way to go for \"more standard\" stuffs
", "rawMarkdown": "I think (Se)Resnext50 is a bit tiny for the data. \nI managed to improve its CV and LB by implementing some tricks (CV 0.900 and LB 0.901). But I can't disclose before the end of the competition\n\nOtherwise bigger models like EFFB4 and image sizes 512 are the way to go for \"more standard\" stuffs", "votes": 4 }, { "id": 1150354, "postDate": "2021-01-12T14:27:58.577Z", "content": "If you wouldn't mind, after the competition i'm sure a lot of people would be interesting in you sharing your notebook if possible :)
", "rawMarkdown": "If you wouldn't mind, after the competition i'm sure a lot of people would be interesting in you sharing your notebook if possible :)", "votes": 1 }, { "id": 1150443, "postDate": "2021-01-12T15:40:01.463Z", "content": "Maybe you should try Cutmix, for me it does improve CV by a a good margin.
\n@serigne My best single model is a SeResnext50 with CV 0.90068 LB 0.899 but i cant seem to make efficientnets work.. Do you have any tips for me?
", "rawMarkdown": "Maybe you should try Cutmix, for me it does improve CV by a a good margin.\n\n@serigne My best single model is a SeResnext50 with CV 0.90068 LB 0.899 but i cant seem to make efficientnets work.. Do you have any tips for me?", "votes": 3 }, { "id": 1150509, "postDate": "2021-01-12T16:25:44.930Z", "content": "@serigne - Is you are comfortable sharing - Can you tell us if those results are using the 2019+2020 data or just 2020?
", "rawMarkdown": "@serigne - Is you are comfortable sharing - Can you tell us if those results are using the 2019+2020 data or just 2020?" }, { "id": 1151568, "postDate": "2021-01-13T12:19:25.827Z", "content": "In my experiments efficientnet series perform well over than ResNets or its variants
", "rawMarkdown": "In my experiments efficientnet series perform well over than ResNets or its variants", "votes": 2 }, { "id": 1151669, "postDate": "2021-01-13T13:37:18.463Z", "content": "@yannmajewski I use Effb4, standard but heavy augmentations (no cutmix, mixup, fmix etc..) and a custom training process.
\n@pheadrus Yes I use all data as I said earlier in this thread. But I don't know if it's still necessary.
", "rawMarkdown": "@yannmajewski I use Effb4, standard but heavy augmentations (no cutmix, mixup, fmix etc..) and a custom training process. \n\n@pheadrus Yes I use all data as I said earlier in this thread. But I don't know if it's still necessary.\n\n", "votes": 2 } ] }, { "id": 1121264, "postDate": "2020-12-21T13:59:34.373Z", "content": "0.901 on LB - ResNeXt50_32x4d, 5 folds ensemble, no external data, partially relabeled
\n(updated)
", "rawMarkdown": "0.901 on LB - ResNeXt50_32x4d, 5 folds ensemble, no external data, partially relabeled\n\n(updated)", "votes": 7, "replies": [ { "id": 1121279, "postDate": "2020-12-21T14:12:52.047Z", "content": "hi, how do you use relabeled?Thanks!
", "rawMarkdown": "hi, how do you use relabeled?Thanks!", "votes": 3 } ] }, { "id": 1112721, "postDate": "2020-12-14T20:43:01.817Z", "content": "i have a strange feeling that we may all be moving to 0.905 and stop there
", "rawMarkdown": "i have a strange feeling that we may all be moving to 0.905 and stop there", "votes": 7, "replies": [ { "id": 1112735, "postDate": "2020-12-14T21:05:53.410Z", "content": "If lb score goes up more than that, it seems to be similar to MoA competition.
\nMaybe Shake-up…
that is what i concluded in the previous iteration of this challenge.
\nequake is inevitable :)
@piantic, that could be the case. The shake-up is pretty probable, due to the presence of many incorrectly labeled images or as we are thinking them to… Let's see what happens… All the best and cheers everyone!👍
", "rawMarkdown": "@piantic, that could be the case. The shake-up is pretty probable, due to the presence of many incorrectly labeled images or as we are thinking them to... Let's see what happens... All the best and cheers everyone!👍 ", "votes": 3 }, { "id": 1115092, "postDate": "2020-12-16T01:29:29.367Z", "content": "@hengck23 Dont worry, your strange feeling is wrong, that is what the maths says. ;-)
\nAlready getting .894 single fold and can get it way higher, my target is to get ~.903 single fold. Totally possible, but near the limit of a single fold performance.
", "rawMarkdown": "@hengck23 Dont worry, your strange feeling is wrong, that is what the maths says. ;-)\n\nAlready getting .894 single fold and can get it way higher, my target is to get ~.903 single fold. Totally possible, but near the limit of a single fold performance." }, { "id": 1127951, "postDate": "2020-12-27T04:16:38.900Z", "content": "i am busy at other competitions … and today I take a peek at the cassava leader board.
\ni think I am right that we will be stuck at 0.905 or nearby.
performance is limited by noise
", "rawMarkdown": "i am busy at other competitions ... and today I take a peek at the cassava leader board.\ni think I am right that we will be stuck at 0.905 or nearby.\n\nperformance is limited by noise", "votes": 2 }, { "id": 1127966, "postDate": "2020-12-27T04:30:54.997Z", "content": "Hello!Will the orgnization correct the noisy in test set?
", "rawMarkdown": "Hello!Will the orgnization correct the noisy in test set?" } ] }, { "id": 1115980, "postDate": "2020-12-16T18:02:40.247Z", "content": "I have not tried submitting my predictions yet, so I do not know how high (or low :-) ) my model will score on the leaderboard. The 5-fold CV is 0.90140 (average across folds). But the agreement between the training and validation accuracies seems to be very good (see an example of my first training fold below).
\n
which model did you used also did you tried to focus on the class imbalance problem ? and did you used augmentations for this experiment ?
\nI am also trying the train a resnet50 without any augmentation but the training is not smooth it is kinda noisy(the validation loss spikes up and down a lot)
", "rawMarkdown": "which model did you used also did you tried to focus on the class imbalance problem ? and did you used augmentations for this experiment ?\n\nI am also trying the train a resnet50 without any augmentation but the training is not smooth it is kinda noisy(the validation loss spikes up and down a lot)" } ] }, { "id": 1133922, "postDate": "2020-12-31T16:45:26.690Z", "content": "Beside my effnet B4, I have 3 others (lighter) models scoring 0.901, 0.900 , 0.900 and trained with another strategy than the Effnet b4. However, it's very difficult to improve them further.
\nMay be I should start ensembling ^^
", "rawMarkdown": "Beside my effnet B4, I have 3 others (lighter) models scoring 0.901, 0.900 , 0.900 and trained with another strategy than the Effnet b4. However, it's very difficult to improve them further. \n\nMay be I should start ensembling ^^", "votes": 5, "replies": [ { "id": 1134240, "postDate": "2021-01-01T03:47:32.903Z", "content": "Hi,can you share some details about the lr and lr_scheduler?Thanks!Now,I stoped in 0.898,I dont know how to improve the score.I also try some loss: like label smooth or bi_tempelet_loss,there are no improve.Thanks!
", "rawMarkdown": "Hi,can you share some details about the lr and lr_scheduler?Thanks!Now,I stoped in 0.898,I dont know how to improve the score.I also try some loss: like label smooth or bi_tempelet_loss,there are no improve.Thanks!" }, { "id": 1134569, "postDate": "2021-01-01T11:54:36.920Z", "content": "I use gradual warmup with cosine annealing. But I don't think it improves compared to others lr schedulers
\nFor loss function , I built my own by combining different approaches read from papers. And it surely improved my CV and LB.
", "rawMarkdown": "I use gradual warmup with cosine annealing. But I don't think it improves compared to others lr schedulers \n\nFor loss function , I built my own by combining different approaches read from papers. And it surely improved my CV and LB.\n\n", "votes": 2 }, { "id": 1134611, "postDate": "2021-01-01T12:31:47.797Z", "content": "Thanks!how many learning rate do you use?1e-4 or others?Thanks!
", "rawMarkdown": "Thanks!how many learning rate do you use?1e-4 or others?Thanks!" } ] }, { "id": 1120584, "postDate": "2020-12-21T00:05:22.423Z", "content": "Hi, can I just ask - when people say 'single model' here, does this refer to a single design (e.g. EfficientNetB3) or to a single copy / set of model weights?
\nFor example I've trained an EffNetB3 with 5-fold CV so have 5 sets of weights, all of which are used in inference notebook - is that what others would report as a 'single model' score?
\nLeaderboard score for the above for me is 0.893.
\nModel is in tensorflow and I used the very helpful notebooks provided by @dimitreoliveira as a starting point, with TTA during prediction of test set.
", "rawMarkdown": "Hi, can I just ask - when people say 'single model' here, does this refer to a single design (e.g. EfficientNetB3) or to a single copy / set of model weights?\n\nFor example I've trained an EffNetB3 with 5-fold CV so have 5 sets of weights, all of which are used in inference notebook - is that what others would report as a 'single model' score?\n\nLeaderboard score for the above for me is 0.893.\n\nModel is in tensorflow and I used the very helpful notebooks provided by @dimitreoliveira as a starting point, with TTA during prediction of test set. ", "votes": 5 }, { "id": 1111612, "postDate": "2020-12-13T21:13:01.857Z", "content": "0.901 single EfficientNet B3_ns
\n0.901 single EfficientNet B3 imgnet
resnet200d CV 0.9028 LB 0.901(w/o tta)
", "rawMarkdown": "resnet200d CV 0.9028 LB 0.901(w/o tta)", "votes": 3 }, { "id": 1159230, "postDate": "2021-01-19T05:43:36.233Z", "content": "Efficientnet B0, CV fold5: 0.898, LB: 0.896 with image size 512*512
", "rawMarkdown": "Efficientnet B0, CV fold5: 0.898, LB: 0.896 with image size 512*512", "votes": 3 }, { "id": 1152612, "postDate": "2021-01-14T10:17:40.243Z", "content": "Small and standard start, to have a baseline.
\nSingle B0ns dim 512 1 fold, extra data, many aug features, p.5 of mixup&cutout , AdamP, CosAnnWarm, LabelSm.
\nVal Accuracy: 0.8936977980258163
\nLB 0.892
A reflection, seems be shadows and different quality in the images, both can be implemented in the Albumentations tool, maybe can be worth a try to this training.
", "rawMarkdown": "Small and standard start, to have a baseline.\nSingle B0ns dim 512 1 fold, extra data, many aug features, p.5 of mixup&cutout , AdamP, CosAnnWarm, LabelSm.\nVal Accuracy: 0.8936977980258163\nLB 0.892\n\nA reflection, seems be shadows and different quality in the images, both can be implemented in the Albumentations tool, maybe can be worth a try to this training.", "votes": 3, "replies": [ { "id": 1165273, "postDate": "2021-01-22T20:31:16.130Z", "content": "That's actually impressive for a single fold, single model EffNetB0 or is that over 5 folds? Have you tried higher effnets, specially B4_ns or B5_ns?
", "rawMarkdown": "That's actually impressive for a single fold, single model EffNetB0 or is that over 5 folds? Have you tried higher effnets, specially B4_ns or B5_ns?" }, { "id": 1166885, "postDate": "2021-01-23T21:44:08.567Z", "content": "Yes single fold, update now with B0ns 2 folds are CV 0.8964 LB 0.897. The plan is to train larger, different and ensembles models.
", "rawMarkdown": "Yes single fold, update now with B0ns 2 folds are CV 0.8964 LB 0.897. The plan is to train larger, different and ensembles models." } ] }, { "id": 1130214, "postDate": "2020-12-28T20:31:54.033Z", "content": "Efficientnet B5 noisy student with 512 image size.
\n5folds CV is 0.892 and LB is 0.898. No TTA has been performed.
\nNothing compared to the monstrous results of @serigne 0.904 score… maybe I’ll give 2019 a try
", "rawMarkdown": "Efficientnet B5 noisy student with 512 image size. \n\n5folds CV is 0.892 and LB is 0.898. No TTA has been performed. \n\nNothing compared to the monstrous results of @serigne 0.904 score... maybe I’ll give 2019 a try ", "votes": 3, "replies": [ { "id": 1130234, "postDate": "2020-12-28T21:31:07.183Z", "content": "@reighns Im curious what is your CV strategy, i currently have 0.89751 CV but 0.895 lb with single model. Im using StratKfold
", "rawMarkdown": "@reighns Im curious what is your CV strategy, i currently have 0.89751 CV but 0.895 lb with single model. Im using StratKfold", "votes": 1 }, { "id": 1130236, "postDate": "2020-12-28T21:35:44.077Z", "content": "I believe most of us use the same cv strategy as yours, me included. And on the plus side, your CVLB relation is pretty decent.
", "rawMarkdown": "I believe most of us use the same cv strategy as yours, me included. And on the plus side, your CVLB relation is pretty decent. ", "votes": 1 } ] }, { "id": 1128483, "postDate": "2020-12-27T13:29:35.130Z", "content": "0.891
\nseresnext50, 1 fold, tta, 512x512
@artgor Nice to see you on this competition. Good luck !
", "rawMarkdown": "@artgor Nice to see you on this competition. Good luck !", "votes": 1 }, { "id": 1130438, "postDate": "2020-12-29T03:57:02.240Z", "content": "@vladvdv thanks!
", "rawMarkdown": "@vladvdv thanks!" } ] }, { "id": 1128169, "postDate": "2020-12-27T08:20:38.580Z", "content": "0.901 single efficientnet b3 with tta and 2019 dataset
", "rawMarkdown": "0.901 single efficientnet b3 with tta and 2019 dataset", "votes": 3, "replies": [ { "id": 1129170, "postDate": "2020-12-28T06:08:16.057Z", "content": "can you share how do you use scheduler or optimizer?Thanks!
", "rawMarkdown": "can you share how do you use scheduler or optimizer?Thanks!" }, { "id": 1129173, "postDate": "2020-12-28T06:09:17.763Z", "content": "I'm stopping in 0.898,and there is no improve whatever I do!
", "rawMarkdown": "I'm stopping in 0.898,and there is no improve whatever I do!", "votes": 1 }, { "id": 1129383, "postDate": "2020-12-28T09:53:44.340Z", "content": "i used ranger optimizer that mentioned here
", "rawMarkdown": "i used ranger optimizer that mentioned [here](https://lessw.medium.com/new-deep-learning-optimizer-ranger-synergistic-combination-of-radam-lookahead-for-the-best-of-2dc83f79a48d)" }, { "id": 1131943, "postDate": "2020-12-30T04:59:30.217Z", "content": "Thanks for your reply！En…did you use 'cossin' scheduler?Thanks!
", "rawMarkdown": "Thanks for your reply！En...did you use 'cossin' scheduler?Thanks!" } ] }, { "id": 1115982, "postDate": "2020-12-16T18:08:17.137Z", "content": "Resnet50 (no TTA, no external data) - 0.873
", "rawMarkdown": "Resnet50 (no TTA, no external data) - 0.873", "votes": 3, "replies": [ { "id": 1117953, "postDate": "2020-12-18T16:02:27.797Z", "content": "would you mind to share what augmentation you use for that result?
", "rawMarkdown": "would you mind to share what augmentation you use for that result?" }, { "id": 1118663, "postDate": "2020-12-19T09:33:07.093Z", "content": "Sure :) I used vertical and horizontal flip, default albumentations rotate, random contrast and brightness. I also lightly blurred 10% of the images.
\n80% of the time, I randomly cropped 300x400 part from the input images.
\nI didn't use oversampling but class weights seemed to work.
\nAlthough I resized the images to 512x512, I tried not to destroy the original aspect ratio, so I padded top and bottom parts of the Image equally to convert image into a square before resizing.
\nHope it helps.
", "rawMarkdown": "Sure :) I used vertical and horizontal flip, default albumentations rotate, random contrast and brightness. I also lightly blurred 10% of the images.\n\n80% of the time, I randomly cropped 300x400 part from the input images.\n\nI didn't use oversampling but class weights seemed to work. \n\nAlthough I resized the images to 512x512, I tried not to destroy the original aspect ratio, so I padded top and bottom parts of the Image equally to convert image into a square before resizing.\n\nHope it helps.", "votes": 3 }, { "id": 1118753, "postDate": "2020-12-19T11:16:47.393Z", "content": "Thanks for your response, I used default ImageDataGenerator from keras and stuck in 0.85. I'll try albumentation then :)
", "rawMarkdown": "Thanks for your response, I used default ImageDataGenerator from keras and stuck in 0.85. I'll try albumentation then :)", "votes": 1 } ] }, { "id": 1112348, "postDate": "2020-12-14T13:59:56.783Z", "content": "Is using different sizes improving anything? I have been using 224x224 and stuck to my score since the start; (not using 512 x512 as it takes time and I am using kaggle notebooks.)
", "rawMarkdown": "Is using different sizes improving anything? I have been using 224x224 and stuck to my score since the start; (not using 512 x512 as it takes time and I am using kaggle notebooks.)", "votes": 3, "replies": [ { "id": 1112429, "postDate": "2020-12-14T15:30:41.470Z", "content": "in my experiment's, the same model with a smaller image size than 512 gives me poor results than 512.
", "rawMarkdown": "in my experiment's, the same model with a smaller image size than 512 gives me poor results than 512.", "votes": 3 }, { "id": 1112549, "postDate": "2020-12-14T17:47:57.550Z", "content": "yes it do improve my score , but it also depend upon the model architecture you are using like how big your model is such as for bigger models you probably going to need higher resolution images and for smaller model it is vice versa .
", "rawMarkdown": "yes it do improve my score , but it also depend upon the model architecture you are using like how big your model is such as for bigger models you probably going to need higher resolution images and for smaller model it is vice versa .", "votes": 2 }, { "id": 1115220, "postDate": "2020-12-16T05:12:09.690Z", "content": "448 is better for me.👀😄
", "rawMarkdown": "448 is better for me.👀😄" }, { "id": 1149481, "postDate": "2021-01-11T21:37:02.487Z", "content": "Increasing the image size from my initial 224 x 224 gave me a greater gain in accuracy for CV and LB than any other factor. I'm using 512 x 512 right now.
", "rawMarkdown": "Increasing the image size from my initial 224 x 224 gave me a greater gain in accuracy for CV and LB than any other factor. I'm using 512 x 512 right now." } ] }, { "id": 1120591, "postDate": "2020-12-21T00:31:08.953Z", "content": "Resnet50 with SnapMix (single fold, no TTA, no external data) - LB: 0.891
", "rawMarkdown": "Resnet50 with SnapMix (single fold, no TTA, no external data) - LB: 0.891", "votes": 4, "replies": [ { "id": 1120613, "postDate": "2020-12-21T01:30:50.840Z", "content": "Intresting augmentation!
", "rawMarkdown": "Intresting augmentation! " }, { "id": 1120641, "postDate": "2020-12-21T02:13:06.323Z", "content": "but this score cannot achieve that high rank
", "rawMarkdown": "but this score cannot achieve that high rank" }, { "id": 1120652, "postDate": "2020-12-21T02:36:19.937Z", "content": "using multi folds and model ensemble will further increase the socre
", "rawMarkdown": "using multi folds and model ensemble will further increase the socre", "votes": 1 }, { "id": 1120664, "postDate": "2020-12-21T03:13:06.937Z", "content": "I already used 5fold and average ensemble, improments are scrace.
", "rawMarkdown": "I already used 5fold and average ensemble, improments are scrace." }, { "id": 1120671, "postDate": "2020-12-21T03:32:27.103Z", "content": "In my case, single Resnet50 with 5 folds ensemble can get 0.897 on LB.
", "rawMarkdown": "In my case, single Resnet50 with 5 folds ensemble can get 0.897 on LB. ", "votes": 1 }, { "id": 1120699, "postDate": "2020-12-21T04:08:53.803Z", "content": "I am getting 895 with a seresnext50 single fold in my case.
", "rawMarkdown": "I am getting 895 with a seresnext50 single fold in my case.", "votes": 1 }, { "id": 1120762, "postDate": "2020-12-21T05:11:12.093Z", "content": "what can we do to furtherly push the score higher when training seems approaching the limit, say decreasing lr, but I got overfitting then.
", "rawMarkdown": "what can we do to furtherly push the score higher when training seems approaching the limit, say decreasing lr, but I got overfitting then.", "votes": 1 }, { "id": 1121986, "postDate": "2020-12-22T05:03:12.260Z", "content": "are you using keras or pytorch?
", "rawMarkdown": "are you using keras or pytorch?" }, { "id": 1122323, "postDate": "2020-12-22T10:59:49.720Z", "content": "I am using pytorch
", "rawMarkdown": "I am using pytorch", "votes": 1 } ] }, { "id": 1113151, "postDate": "2020-12-15T08:37:57.703Z", "content": "Best model is a Effnet B3 with imagenet weights which lead to a 0.900 to leaderboard.
\nAnother interesting question which anybody can answer beside the model and score will be if the model was trained only on data provided on this competition or using also the data from the past similar one.
are you using the data from the 2019 competition?
", "rawMarkdown": "are you using the data from the 2019 competition?" } ] }, { "id": 1111861, "postDate": "2020-12-14T04:59:06.320Z", "content": "0.901 Effnet B4 NS with TTA, no external data yet.
", "rawMarkdown": "0.901 Effnet B4 NS with TTA, no external data yet.", "votes": 4, "replies": [ { "id": 1117786, "postDate": "2020-12-18T12:45:49.013Z", "content": "Hello,How many TTA times did u use?
", "rawMarkdown": "Hello,How many TTA times did u use?" }, { "id": 1117947, "postDate": "2020-12-18T15:58:54.823Z", "content": "I'm experimenting with either 5 or 8. This one was from TTA = 5.
", "rawMarkdown": "I'm experimenting with either 5 or 8. This one was from TTA = 5." }, { "id": 1119241, "postDate": "2020-12-19T20:52:54.017Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1196278, "postDate": "2021-02-11T11:00:48.900Z", "content": "5-fold CV 0.9019, LB 0.901 (no tta).
", "rawMarkdown": "5-fold CV 0.9019, LB 0.901 (no tta).", "votes": 1 }, { "id": 1196168, "postDate": "2021-02-11T09:48:32.627Z", "content": "Single model 0.901, model ensemble 0.905, no tta for stability. TTA might overfit in 31% test data given
", "rawMarkdown": "Single model 0.901, model ensemble 0.905, no tta for stability. TTA might overfit in 31% test data given", "votes": 1, "replies": [ { "id": 1196878, "postDate": "2021-02-11T18:21:43.217Z", "content": "why do you think that TTA might overfit to the public test data given?
", "rawMarkdown": "why do you think that TTA might overfit to the public test data given?" } ] }, { "id": 1196099, "postDate": "2021-02-11T08:49:14.683Z", "content": "It is only me with a 0.894 5-fold CV and 0.901 LB? 😂
\nI am very confused and worried at the same time…
0.9018 single model CV/0.9 lb,
\n5 model ensemble 0.9045 CV/0.90 lb..
\nweird
Hello, which model did you use if you don't mind?
", "rawMarkdown": "Hello, which model did you use if you don't mind?" } ] }, { "id": 1173718, "postDate": "2021-01-28T04:45:29.017Z", "content": "my single efficient net 5 CV 0.8997 LB 0.901
", "rawMarkdown": "my single efficient net 5 CV 0.8997 LB 0.901", "votes": 1, "replies": [ { "id": 1191339, "postDate": "2021-02-08T12:15:17.263Z", "content": "did you use any cut mix, f-mix or any other methods
", "rawMarkdown": "did you use any cut mix, f-mix or any other methods", "votes": 1 }, { "id": 1196874, "postDate": "2021-02-11T18:19:26.747Z", "content": "coarsedropout, mixup, cutmix used
", "rawMarkdown": "coarsedropout, mixup, cutmix used" } ] }, { "id": 1167434, "postDate": "2021-01-24T08:52:44.627Z", "content": "single model 0.901,model essemble 0.904
", "rawMarkdown": "single model 0.901,model essemble 0.904", "votes": 1 }, { "id": 1165886, "postDate": "2021-01-23T09:39:51.123Z", "content": "I've tried training on Efficientnet-b4 but my validation accuracy seems to peak at 77%. I am using 2019 data, removing the duplicates, have done mixed precision training, 512x512 images, Adam. My transforms may be a little bit different but i don't think it justifies the huge gap in the score. Can you suggest something that i may be missing?
", "rawMarkdown": "I've tried training on Efficientnet-b4 but my validation accuracy seems to peak at 77%. I am using 2019 data, removing the duplicates, have done mixed precision training, 512x512 images, Adam. My transforms may be a little bit different but i don't think it justifies the huge gap in the score. Can you suggest something that i may be missing?\n", "votes": 1, "replies": [ { "id": 1174494, "postDate": "2021-01-28T14:11:53.630Z", "content": "Try training it without the transforms to make sure that is not the problem.
", "rawMarkdown": "Try training it without the transforms to make sure that is not the problem.", "votes": 1 }, { "id": 1195476, "postDate": "2021-02-10T20:36:57.240Z", "content": "Sorry for late reply, but make sure that you are using the pretrained weights for EfficientNet-B4, it should solve your problem of low validation accuracy.
", "rawMarkdown": "Sorry for late reply, but make sure that you are using the pretrained weights for EfficientNet-B4, it should solve your problem of low validation accuracy." } ] }, { "id": 1164076, "postDate": "2021-01-22T06:12:56.267Z", "content": "| Name (5-fold) | \nCV | \nLB | \n
|---|---|---|
| hrnet exp019 | \n0.8947 | \n0.901 | \n
| hrnet exp020 | \n0.8940 | \n0.899 | \n
| hrnet exp021 | \n0.8969 | \n0.898 | \n
| resnext exp001 | \n0.8967 | \n0.901 | \n
| resnext exp001 + hrnet_exp019 | \n\n | 0.900 | \n
| resnext exp001 + hrnet_exp021 | \n\n | 0.901 | \n
A simple ensemble did not improve the accuracy.
\nWould it be better to adjust for each label?
Well did you test the ensemble with your CV? it might have increased it
", "rawMarkdown": "Well did you test the ensemble with your CV? it might have increased it", "votes": 1 }, { "id": 1165133, "postDate": "2021-01-22T18:06:27.357Z", "content": "I tried, but these models did not improve lb.
", "rawMarkdown": "I tried, but these models did not improve lb.", "votes": 1 } ] }, { "id": 1134955, "postDate": "2021-01-01T18:20:24.857Z", "content": "87.7 …. I dont want to add previous year data and ensemble model yet.
\nWant to improve this
\nAny suggestion ?
I have tried
\nres50,34,101
\neffnet b3,b4,b5,b6
\nVisionTransformers
Cutmix, Labelsmooth, TTA
\nWhat i am missing ? Have no clue how to push.
\n( Everything was learnt from kaggle contributes )
", "rawMarkdown": "87.7 .... I dont want to add previous year data and ensemble model yet. \nWant to improve this \nAny suggestion ?\n\nI have tried \n\nres50,34,101\neffnet b3,b4,b5,b6\nVisionTransformers \n\nCutmix, Labelsmooth, TTA\n\nWhat i am missing ? Have no clue how to push.\n\n( Everything was learnt from kaggle contributes )", "votes": 1, "replies": [ { "id": 1134986, "postDate": "2021-01-01T19:16:13.793Z", "content": "bs is around 6 to 3
", "rawMarkdown": "bs is around 6 to 3 " }, { "id": 1149550, "postDate": "2021-01-11T23:53:29.373Z", "content": "Are you using augmentations such as flip and rotate? Maybe you could also try a different loss function such as Bi-tempered logistic loss. I think some people found that it gave better results.
", "rawMarkdown": "Are you using augmentations such as flip and rotate? Maybe you could also try a different loss function such as Bi-tempered logistic loss. I think some people found that it gave better results." } ] }, { "id": 1128353, "postDate": "2020-12-27T11:18:20.607Z", "content": "Thank you for your thread and thank you for all comments! It helps me a lot.
\nI just start this competition and I got 0.822 (really poor score…) with efficientnetb0 and random selected 1000 images in dataset.
0.900 single resnext50
", "rawMarkdown": "0.900 single resnext50", "votes": 1, "replies": [ { "id": 1119427, "postDate": "2020-12-20T04:05:29.603Z", "content": "awesome, so hard to achieve
", "rawMarkdown": "awesome, so hard to achieve" } ] }, { "id": 1111694, "postDate": "2020-12-13T23:26:48.853Z", "content": "0.899 EfficientNet B4_ns with TTA
", "rawMarkdown": "0.899 EfficientNet B4_ns with TTA", "votes": 1 }, { "id": 1155173, "postDate": "2021-01-16T09:08:01.737Z", "content": "Resnet18, 512, basic augmentations, Adam, CrossEntropy, ReduceLROnPlateau, Early stopping--> LB 0.875/random_split 0.87336 (may not be reliable, better to try with cv)
", "rawMarkdown": " Resnet18, 512, basic augmentations, Adam, CrossEntropy, ReduceLROnPlateau, Early stopping--> LB 0.875/random_split 0.87336 (may not be reliable, better to try with cv)", "votes": 2 }, { "id": 1148321, "postDate": "2021-01-11T04:35:34.433Z", "content": "0.900 SEResNet152, but this local CV is not significuntly better than other model.
\nso, I start to doubt whether public LB can be reliable or not….
LB on its own is not (necessarily) reliable.
\nI think there will likely be huge shake up here, just like PANDA
", "rawMarkdown": "LB on its own is not (necessarily) reliable.\n\nI think there will likely be huge shake up here, just like PANDA", "votes": 1 }, { "id": 1149549, "postDate": "2021-01-11T23:51:20.460Z", "content": "Is there a big difference between this competition and the 2019 one because I don't think there was a big shake-up in that one?
", "rawMarkdown": "Is there a big difference between this competition and the 2019 one because I don't think there was a big shake-up in that one?", "votes": 1 }, { "id": 1150316, "postDate": "2021-01-12T14:07:59.867Z", "content": "Yes ! More dataset and the labelling has been improved according to kaggle admins
", "rawMarkdown": "Yes ! More dataset and the labelling has been improved according to kaggle admins", "votes": 1 }, { "id": 1150479, "postDate": "2021-01-12T16:08:17.460Z", "content": "Thank you for your reply!
\nI don't know previous competition, but noised label may be removed compared to previous one.
\nright?
The best (for now) is efficientnet_b4 with 0.894 on public.
\nI think this model could achieve much better results.
seresnext50
\n5 folds
\n384x384
\n8 epochs
\nCutmix
\n0.896 lb
Hi all,
\nScores reported in this discussion refers to a single model, but I cannot clearly understand if they are actually scores of a single instance of the model or if they refers to the ensemble over all folds.
In my case I have a single EfficientNetB4 trained on images 512x512 (smaller sizes gives me lower scores)
CV: 0.8958LB: 0.889CutMix + MixUpNo External DataNo TTAWhile with an ensemble of all folds i get LB score of 0.896
How can you use such a big image size? Every time that I use something above 380x380 my notebook says that I'm trying to allocate more memory than is available. Do you have suggestions on how to solve this issue? I'm using EfficientNetB4 too.
\nThanks!
Are you using gpu or tpu?
\nActually i didnt trained bigger resolution than 300x300 on gpu. Maybe you can try to train your model on tpu and see if you can increase resolution.
\nP.s. im using tensoflow with tf.data
", "rawMarkdown": "Are you using gpu or tpu? \n\nActually i didnt trained bigger resolution than 300x300 on gpu. Maybe you can try to train your model on tpu and see if you can increase resolution.\n\nP.s. im using tensoflow with tf.data", "votes": 1 }, { "id": 1129516, "postDate": "2020-12-28T12:00:29.263Z", "content": "I started from tutorial notebooks by @dimitreoliveira
", "rawMarkdown": "I started from tutorial notebooks by @dimitreoliveira " }, { "id": 1130729, "postDate": "2020-12-29T09:21:33.117Z", "content": "Thank you for your answer! Right now I'm using GPU with an image resolution of 380x380. Nevertheless, to use such a big resolution I had to decrease my batch size to 15 and this led to huge training times. Just to understand, I cannot run the model more than a couple of times with the free 30 hours. I'll try to use the TPU then, thank you!
", "rawMarkdown": "Thank you for your answer! Right now I'm using GPU with an image resolution of 380x380. Nevertheless, to use such a big resolution I had to decrease my batch size to 15 and this led to huge training times. Just to understand, I cannot run the model more than a couple of times with the free 30 hours. I'll try to use the TPU then, thank you!", "votes": 1 }, { "id": 1138161, "postDate": "2021-01-04T13:07:46.270Z", "content": "@marto24 It's possible to train bigger images (such as 512x512) on gpus. Of course it depends on size of your model. But for this competition is model as EfficientNetB4 more than enought.
\nTry to implement this into your training loop:
from torch.cuda.amp import autocast, GradScaler\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nscaler = GradScaler()\n\ndef trainining_loop(model, train_loader):\n model.to(device)\n\n for img, label in train_loader:\n img = img.to(device)\n label = label.to(device)\n\n optimizer.zero_grad()\n with autocast():\n outputs = model(inputs)\n loss = criterion(outputs, labels)\n\n scaler.scale(loss).backward()\n scaler.step(optimizer)\n scaler.update()\n\n etc...\nThis works for my approach. By this solution you can have images of size 512x512 and batch size = 16
", "rawMarkdown": "@marto24 It's possible to train bigger images (such as 512x512) on gpus. Of course it depends on size of your model. But for this competition is model as EfficientNetB4 more than enought.\nTry to implement this into your training loop:\n```\nfrom torch.cuda.amp import autocast, GradScaler\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nscaler = GradScaler()\n\ndef trainining_loop(model, train_loader):\n model.to(device)\n\n for img, label in train_loader:\n img = img.to(device)\n label = label.to(device)\n\n optimizer.zero_grad()\n with autocast():\n outputs = model(inputs)\n loss = criterion(outputs, labels)\n \n scaler.scale(loss).backward()\n scaler.step(optimizer)\n scaler.update()\n\n etc...\n```\nThis works for my approach. By this solution you can have images of size 512x512 and batch size = 16", "votes": 3 } ] }, { "id": 1120766, "postDate": "2020-12-21T05:17:52.707Z", "content": "Is anyone freezing the layers of their transfer learning model; or training the whole model?
", "rawMarkdown": "Is anyone freezing the layers of their transfer learning model; or training the whole model?", "votes": 2, "replies": [ { "id": 1122412, "postDate": "2020-12-22T12:28:13.583Z", "content": "I tried finetuning only the last layer by freezing all other layers, but the accuracy was too low.
\nFor example, 1st epoch accuracy was 80%, but with fine-tuning only the last layer, it fell to 68% in the 1st epoch
@debarshichanda Isn't the 1st epoch accuracy supposed to be low? When tuning a pre-trained model, we usually start with a very low learning rate to avoid creating large changes to the weights in the beginning of the training process.
", "rawMarkdown": "@debarshichanda Isn't the 1st epoch accuracy supposed to be low? When tuning a pre-trained model, we usually start with a very low learning rate to avoid creating large changes to the weights in the beginning of the training process. ", "votes": 2 }, { "id": 1123476, "postDate": "2020-12-23T08:53:35.387Z", "content": "@debarshichanda 80 percent in 1st epoch!! thats amazing; do you mind to tell what model are you using?
", "rawMarkdown": "@debarshichanda 80 percent in 1st epoch!! thats amazing; do you mind to tell what model are you using?", "votes": 1 }, { "id": 1123521, "postDate": "2020-12-23T09:46:09.043Z", "content": "@graf10a I am using initial lr of 1e-4 and then using Cosine scheduling to reduce it to 1e-6. Till now its working good for me
", "rawMarkdown": "@graf10a I am using initial lr of 1e-4 and then using Cosine scheduling to reduce it to 1e-6. Till now its working good for me", "votes": 1 }, { "id": 1123524, "postDate": "2020-12-23T09:46:55.213Z", "content": "@mrinath EfficientNet B4 NS
", "rawMarkdown": "@mrinath EfficientNet B4 NS", "votes": 1 }, { "id": 1149476, "postDate": "2021-01-11T21:26:26.470Z", "content": "@debarshichanda did you try freezing up to the last layer(s) and training for just one epoch, and then unfreezing for the rest of the training? This first epoch should let the head that wasn't trained on imagenet get up to speed quickly with the rest of the layers before resuming training in earnest.
", "rawMarkdown": "@debarshichanda did you try freezing up to the last layer(s) and training for just one epoch, and then unfreezing for the rest of the training? This first epoch should let the head that wasn't trained on imagenet get up to speed quickly with the rest of the layers before resuming training in earnest. ", "votes": 1 }, { "id": 1149484, "postDate": "2021-01-11T21:38:37.463Z", "content": "@vedder I haven't tried this yet but will definitely try
\nThank you!
0.894 effb3 single fold seed 42 stratified fold 0th, NO external data
", "rawMarkdown": "0.894 effb3 single fold seed 42 stratified fold 0th, NO external data", "votes": 2 }, { "id": 1112315, "postDate": "2020-12-14T13:23:46.720Z", "content": "0.842 with Xception
", "rawMarkdown": "0.842 with Xception", "votes": 2 }, { "id": 1111670, "postDate": "2020-12-13T22:52:31.407Z", "content": "0.900 - single seresnext50
", "rawMarkdown": "0.900 - single seresnext50", "votes": 2 }, { "id": 1111656, "postDate": "2020-12-13T22:15:36.843Z", "content": "I believe the important question is not only how the model eventually scores on the LB but also how much time does it take to achieve the reported score. Heavy augmentation can slow inference down.
", "rawMarkdown": "I believe the important question is not only how the model eventually scores on the LB but also how much time does it take to achieve the reported score. Heavy augmentation can slow inference down.", "votes": 2 }, { "id": 1111557, "postDate": "2020-12-13T19:44:40.153Z", "content": "I think there's already a discussion thread for this, here.
", "rawMarkdown": "I think there's already a discussion thread for this, [here](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198805). ", "votes": 2, "replies": [ { "id": 1111561, "postDate": "2020-12-13T19:49:27.680Z", "content": "oopsie….
", "rawMarkdown": "oopsie....", "votes": 2 }, { "id": 1111564, "postDate": "2020-12-13T19:50:39.257Z", "content": "seems like many are old scores there ;)
", "rawMarkdown": "seems like many are old scores there ;)", "votes": 3 }, { "id": 1112259, "postDate": "2020-12-14T12:40:36.507Z", "content": "@abhishek Yeah, think we needed a post-rescore one, anyway! :)
", "rawMarkdown": "@abhishek Yeah, think we needed a post-rescore one, anyway! :)", "votes": 1 } ] }, { "id": 1111556, "postDate": "2020-12-13T19:43:31.583Z", "content": "0.897 on the LB - all my scores are single model, so far. And thanks for your tez starter notebook, @abhishek, that played a large role in getting me there.
", "rawMarkdown": "0.897 on the LB - all my scores are single model, so far. And thanks for your tez starter notebook, @abhishek, that played a large role in getting me there.", "votes": 2, "replies": [ { "id": 1111580, "postDate": "2020-12-13T20:25:47.063Z", "content": "tez is clearly the best module for those who want to make a smooth transition from keras to Pytorch ;)
", "rawMarkdown": "tez is clearly the best module for those who want to make a smooth transition from keras to Pytorch ;)", "votes": 1 }, { "id": 1131907, "postDate": "2020-12-30T04:17:06.567Z", "content": "or,
\ntez is clearly the best module for those who want to make a smooth transition from pytorch to Keras ;)
EffNetB4 0.884 LB and 0.902 CV, i'm having trouble dealing with overfitting and proper data augmentations.
", "rawMarkdown": "EffNetB4 0.884 LB and 0.902 CV, i'm having trouble dealing with overfitting and proper data augmentations. ", "replies": [ { "id": 1115476, "postDate": "2020-12-16T10:12:54.673Z", "content": "Beware of the data in your validation set. There might be some leaks.
\nA good rule of thumb is to never use external data in the validation set, particularly when duplicated are not identified correctly.
", "rawMarkdown": "Beware of the data in your validation set. There might be some leaks. \n\nA good rule of thumb is to never use external data in the validation set, particularly when duplicated are not identified correctly. ", "votes": 4 } ] }, { "id": 1174215, "postDate": "2021-01-28T10:51:34.947Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1130979, "postDate": "2020-12-29T13:23:05.527Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 1113427, "postDate": "2020-12-15T12:35:59.580Z", "rawMarkdown": "", "votes": -3, "isDeleted": true }, { "id": 1113321, "postDate": "2020-12-15T11:21:50.417Z", "rawMarkdown": "", "votes": -3, "isDeleted": true } ], "comments": [ { "id": 1111578, "author_name": "Serigne ", "author_url": "", "post_date": "2020-12-13T20:20:51.377000", "content": "0.904 single EfficientNet B4
", "votes": 18, "replies": [ { "id": 1112116, "author_name": "Atharva Phatak", "author_url": "", "post_date": "2020-12-14T09:43:11.010000", "content": "@serigne what is your CV ? Also how many folds ?
", "votes": 0, "replies": [] }, { "id": 1112130, "author_name": "Serigne ", "author_url": "", "post_date": "2020-12-14T10:06:36.783000", "content": "CV = 0.902 and 5 folds , mixed precision training.
", "votes": 7, "replies": [] }, { "id": 1112142, "author_name": "Atharva Phatak", "author_url": "", "post_date": "2020-12-14T10:23:02.220000", "content": "Thanks. That helps a lot !
", "votes": 0, "replies": [] }, { "id": 1112308, "author_name": "Phaedrus", "author_url": "", "post_date": "2020-12-14T13:13:23.747000", "content": "@serigne - The cv is OOF or avg across folds?
", "votes": 1, "replies": [] }, { "id": 1112426, "author_name": "Serigne ", "author_url": "", "post_date": "2020-12-14T15:29:26.167000", "content": "Average across folds.
\nAll logs and best Chekpoints saved for every successful training experiment. I will compute OOF if ever I plan to blend with other models.
", "votes": 3, "replies": [] }, { "id": 1112729, "author_name": "Yawei Li", "author_url": "", "post_date": "2020-12-14T20:49:44.023000", "content": "what is the image size for B4? 512x512 or larger size?
", "votes": 0, "replies": [] }, { "id": 1112989, "author_name": "Pay Money Man", "author_url": "", "post_date": "2020-12-15T04:54:54.263000", "content": "Can u share any trick you used for this challange ?
", "votes": 0, "replies": [] }, { "id": 1113382, "author_name": "Ioannis M", "author_url": "", "post_date": "2020-12-15T12:06:44.243000", "content": "Nice score! If you like to answer - do you use 2019 or additional data as well ?
", "votes": 0, "replies": [] }, { "id": 1115471, "author_name": "Serigne ", "author_url": "", "post_date": "2020-12-16T10:09:34.987000", "content": "I use 512x512 and 2019 dataset too, duplicates identified with DBSCAN Clustering and removed
", "votes": 8, "replies": [] }, { "id": 1115497, "author_name": "Abhishek Thakur", "author_url": "", "post_date": "2020-12-16T10:21:37.017000", "content": "@serigne Good to know. For my score, i.e. 0.898, I have not used 2019 data. What's your score without the 2019 data?
", "votes": 0, "replies": [] }, { "id": 1115506, "author_name": "Serigne ", "author_url": "", "post_date": "2020-12-16T10:33:17.833000", "content": "@abhishek I don't know. I've been using it for all my expemiments. , at least before the LB re-score. And I have modified my training process meanwhile. Thus I need to re-run only the 2020 dataset with the same training process in order to compare.
", "votes": 0, "replies": [] }, { "id": 1115829, "author_name": "Bcw93", "author_url": "", "post_date": "2020-12-16T15:52:15.210000", "content": "Hi,what batch_size do you use?And what scheduler do you use?Thanks!
", "votes": 0, "replies": [] }, { "id": 1115841, "author_name": "Bcw93", "author_url": "", "post_date": "2020-12-16T16:06:48.607000", "content": "Hi, can you share the merged data?Thanks!
", "votes": 0, "replies": [] }, { "id": 1121815, "author_name": "LAZCoder", "author_url": "", "post_date": "2020-12-21T23:45:55.693000", "content": "Hi @serigne,
\nDoes this score comes from a single instance of your model or from an ensemble of all 5 folds?
", "votes": 0, "replies": [] }, { "id": 1149551, "author_name": "ayu055", "author_url": "", "post_date": "2021-01-11T23:57:51.390000", "content": "Hi @serigne,
\nAre you using TTA or any augmentations such as Cutmix or Mixup?
", "votes": 0, "replies": [] }, { "id": 1164037, "author_name": "Prathamesh Sonawane", "author_url": "", "post_date": "2021-01-22T05:14:01.997000", "content": "Batch size? did you train on kaggle?
", "votes": 0, "replies": [] }, { "id": 1167333, "author_name": "Jaideep", "author_url": "", "post_date": "2021-01-24T07:31:59.157000", "content": "@serigne is it single model 5 folds ensemble or single model single fold ?
", "votes": 0, "replies": [] } ] }, { "id": 1165230, "author_name": "Serigne ", "author_url": "", "post_date": "2021-01-22T19:40:15.937000", "content": "My best model is now ViT base 16x16 patches.
\nLB 0.904 (slightly better than my effnet B4)
\nBut I do not trust it at all : Huge gap between CV and LB and the LB result is highly sensitive to the number of TTA.
\nMy Effnet is much more stable across many experiments, and has much higher CV.
", "votes": 9, "replies": [ { "id": 1165256, "author_name": "Debarshi Chanda", "author_url": "", "post_date": "2021-01-22T20:04:01.877000", "content": "Even this thread mentions low CV with ViT but high LB
", "votes": 0, "replies": [] }, { "id": 1165329, "author_name": "Serigne ", "author_url": "", "post_date": "2021-01-22T21:59:55.090000", "content": "I will give a try to ViT Large and others training process but it needs TPU or higher quality GPU. It's been a while I haven't used Torch/XLA ^^. I will try to configure it later.
", "votes": 0, "replies": [] } ] }, { "id": 1149675, "author_name": "Gabriel Prado", "author_url": "", "post_date": "2021-01-12T04:00:53.080000", "content": "CV 0.883 LB 0.890 (same model) I'm having trouble getting my CV scores over 0.885 during training. And don't know what i could be doing wrong at the moment. What i'm doing:
\nImage Sizes : 384
\nModels: EffNetB3,B4,ResNext50 (all train to around 88.0 ~ 88.5 CV) +Dropout(0.3) at last layer
\nEarlyStopping: 2 Patience with Minimum Loss and save best weights at minimum
\nStratifiedKFolds = 5
\nAdded 2019 Data (was careful with what @serigne told me to not include them at the validation set) : improved a tiny bit
\nOptimizer: Adam,Ranger Adam actually worked a bit better than Ranger with schedulers
\nScheduler: OneCycleLR >= CosineAnnealingWarmRestarts > ReduceLROnPlateau >> LambdaLR -> OneCycle depends on the LossFunction to improve CV score
\nLosses: TaylorCrossEntropy > BiTemperedLogistic(>20 permutations of T1,T2 mostly low T1 and T2~= 1.0) > SymmetricCrossEntropy
\nAugmentations: Tried Only Flips/Rotates and Normalization and that + ColorJitter and CoarseDropout and AutoAugment from imagenet
\nI'm really not sure what strategy you guys are using to reach over 88.5 on CV, i tried almost everything that was dicussed here on the forums.
I think (Se)Resnext50 is a bit tiny for the data.
\nI managed to improve its CV and LB by implementing some tricks (CV 0.900 and LB 0.901). But I can't disclose before the end of the competition
Otherwise bigger models like EFFB4 and image sizes 512 are the way to go for \"more standard\" stuffs
", "votes": 4, "replies": [] }, { "id": 1150354, "author_name": "Gabriel Prado", "author_url": "", "post_date": "2021-01-12T14:27:58.577000", "content": "If you wouldn't mind, after the competition i'm sure a lot of people would be interesting in you sharing your notebook if possible :)
", "votes": 1, "replies": [] }, { "id": 1150443, "author_name": "Yann Majewski", "author_url": "", "post_date": "2021-01-12T15:40:01.463000", "content": "Maybe you should try Cutmix, for me it does improve CV by a a good margin.
\n@serigne My best single model is a SeResnext50 with CV 0.90068 LB 0.899 but i cant seem to make efficientnets work.. Do you have any tips for me?
", "votes": 3, "replies": [] }, { "id": 1150509, "author_name": "Phaedrus", "author_url": "", "post_date": "2021-01-12T16:25:44.930000", "content": "@serigne - Is you are comfortable sharing - Can you tell us if those results are using the 2019+2020 data or just 2020?
", "votes": 0, "replies": [] }, { "id": 1151568, "author_name": "Matrix", "author_url": "", "post_date": "2021-01-13T12:19:25.827000", "content": "In my experiments efficientnet series perform well over than ResNets or its variants
", "votes": 2, "replies": [] }, { "id": 1151669, "author_name": "Serigne ", "author_url": "", "post_date": "2021-01-13T13:37:18.463000", "content": "@yannmajewski I use Effb4, standard but heavy augmentations (no cutmix, mixup, fmix etc..) and a custom training process.
\n@pheadrus Yes I use all data as I said earlier in this thread. But I don't know if it's still necessary.
", "votes": 2, "replies": [] } ] }, { "id": 1121264, "author_name": "S. Tomizawa", "author_url": "", "post_date": "2020-12-21T13:59:34.373000", "content": "0.901 on LB - ResNeXt50_32x4d, 5 folds ensemble, no external data, partially relabeled
\n(updated)
", "votes": 7, "replies": [ { "id": 1121279, "author_name": "Bcw93", "author_url": "", "post_date": "2020-12-21T14:12:52.047000", "content": "hi, how do you use relabeled?Thanks!
", "votes": 3, "replies": [] } ] }, { "id": 1112721, "author_name": "hengck23", "author_url": "", "post_date": "2020-12-14T20:43:01.817000", "content": "i have a strange feeling that we may all be moving to 0.905 and stop there
", "votes": 7, "replies": [ { "id": 1112735, "author_name": "Heroseo", "author_url": "", "post_date": "2020-12-14T21:05:53.410000", "content": "If lb score goes up more than that, it seems to be similar to MoA competition.
\nMaybe Shake-up…
that is what i concluded in the previous iteration of this challenge.
\nequake is inevitable :)
@piantic, that could be the case. The shake-up is pretty probable, due to the presence of many incorrectly labeled images or as we are thinking them to… Let's see what happens… All the best and cheers everyone!👍
", "votes": 3, "replies": [] }, { "id": 1115092, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-12-16T01:29:29.367000", "content": "@hengck23 Dont worry, your strange feeling is wrong, that is what the maths says. ;-)
\nAlready getting .894 single fold and can get it way higher, my target is to get ~.903 single fold. Totally possible, but near the limit of a single fold performance.
", "votes": 0, "replies": [] }, { "id": 1127951, "author_name": "hengck23", "author_url": "", "post_date": "2020-12-27T04:16:38.900000", "content": "i am busy at other competitions … and today I take a peek at the cassava leader board.
\ni think I am right that we will be stuck at 0.905 or nearby.
performance is limited by noise
", "votes": 2, "replies": [] }, { "id": 1127966, "author_name": "HAaHAa", "author_url": "", "post_date": "2020-12-27T04:30:54.997000", "content": "Hello!Will the orgnization correct the noisy in test set?
", "votes": 0, "replies": [] } ] }, { "id": 1115980, "author_name": "Alexey Pronin", "author_url": "", "post_date": "2020-12-16T18:02:40.247000", "content": "I have not tried submitting my predictions yet, so I do not know how high (or low :-) ) my model will score on the leaderboard. The 5-fold CV is 0.90140 (average across folds). But the agreement between the training and validation accuracies seems to be very good (see an example of my first training fold below).
\n
which model did you used also did you tried to focus on the class imbalance problem ? and did you used augmentations for this experiment ?
\nI am also trying the train a resnet50 without any augmentation but the training is not smooth it is kinda noisy(the validation loss spikes up and down a lot)
", "votes": 0, "replies": [] } ] }, { "id": 1133922, "author_name": "Serigne ", "author_url": "", "post_date": "2020-12-31T16:45:26.690000", "content": "Beside my effnet B4, I have 3 others (lighter) models scoring 0.901, 0.900 , 0.900 and trained with another strategy than the Effnet b4. However, it's very difficult to improve them further.
\nMay be I should start ensembling ^^
", "votes": 5, "replies": [ { "id": 1134240, "author_name": "Bcw93", "author_url": "", "post_date": "2021-01-01T03:47:32.903000", "content": "Hi,can you share some details about the lr and lr_scheduler?Thanks!Now,I stoped in 0.898,I dont know how to improve the score.I also try some loss: like label smooth or bi_tempelet_loss,there are no improve.Thanks!
", "votes": 0, "replies": [] }, { "id": 1134569, "author_name": "Serigne ", "author_url": "", "post_date": "2021-01-01T11:54:36.920000", "content": "I use gradual warmup with cosine annealing. But I don't think it improves compared to others lr schedulers
\nFor loss function , I built my own by combining different approaches read from papers. And it surely improved my CV and LB.
", "votes": 2, "replies": [] }, { "id": 1134611, "author_name": "Bcw93", "author_url": "", "post_date": "2021-01-01T12:31:47.797000", "content": "Thanks!how many learning rate do you use?1e-4 or others?Thanks!
", "votes": 0, "replies": [] } ] }, { "id": 1120584, "author_name": "Dave E", "author_url": "", "post_date": "2020-12-21T00:05:22.423000", "content": "Hi, can I just ask - when people say 'single model' here, does this refer to a single design (e.g. EfficientNetB3) or to a single copy / set of model weights?
\nFor example I've trained an EffNetB3 with 5-fold CV so have 5 sets of weights, all of which are used in inference notebook - is that what others would report as a 'single model' score?
\nLeaderboard score for the above for me is 0.893.
\nModel is in tensorflow and I used the very helpful notebooks provided by @dimitreoliveira as a starting point, with TTA during prediction of test set.
", "votes": 5, "replies": [] }, { "id": 1111612, "author_name": "", "author_url": "", "post_date": "2020-12-13T21:13:01.857000", "content": "0.901 single EfficientNet B3_ns
\n0.901 single EfficientNet B3 imgnet
resnet200d CV 0.9028 LB 0.901(w/o tta)
", "votes": 3, "replies": [] }, { "id": 1159230, "author_name": "wangli_coder", "author_url": "", "post_date": "2021-01-19T05:43:36.233000", "content": "Efficientnet B0, CV fold5: 0.898, LB: 0.896 with image size 512*512
", "votes": 3, "replies": [] }, { "id": 1152612, "author_name": "Kirderf", "author_url": "", "post_date": "2021-01-14T10:17:40.243000", "content": "Small and standard start, to have a baseline.
\nSingle B0ns dim 512 1 fold, extra data, many aug features, p.5 of mixup&cutout , AdamP, CosAnnWarm, LabelSm.
\nVal Accuracy: 0.8936977980258163
\nLB 0.892
A reflection, seems be shadows and different quality in the images, both can be implemented in the Albumentations tool, maybe can be worth a try to this training.
", "votes": 3, "replies": [ { "id": 1165273, "author_name": "Gabriel Prado", "author_url": "", "post_date": "2021-01-22T20:31:16.130000", "content": "That's actually impressive for a single fold, single model EffNetB0 or is that over 5 folds? Have you tried higher effnets, specially B4_ns or B5_ns?
", "votes": 0, "replies": [] }, { "id": 1166885, "author_name": "Kirderf", "author_url": "", "post_date": "2021-01-23T21:44:08.567000", "content": "Yes single fold, update now with B0ns 2 folds are CV 0.8964 LB 0.897. The plan is to train larger, different and ensembles models.
", "votes": 0, "replies": [] } ] }, { "id": 1130214, "author_name": "gao-hongnan", "author_url": "", "post_date": "2020-12-28T20:31:54.033000", "content": "Efficientnet B5 noisy student with 512 image size.
\n5folds CV is 0.892 and LB is 0.898. No TTA has been performed.
\nNothing compared to the monstrous results of @serigne 0.904 score… maybe I’ll give 2019 a try
", "votes": 3, "replies": [ { "id": 1130234, "author_name": "Yann Majewski", "author_url": "", "post_date": "2020-12-28T21:31:07.183000", "content": "@reighns Im curious what is your CV strategy, i currently have 0.89751 CV but 0.895 lb with single model. Im using StratKfold
", "votes": 1, "replies": [] }, { "id": 1130236, "author_name": "gao-hongnan", "author_url": "", "post_date": "2020-12-28T21:35:44.077000", "content": "I believe most of us use the same cv strategy as yours, me included. And on the plus side, your CVLB relation is pretty decent.
", "votes": 1, "replies": [] } ] }, { "id": 1128483, "author_name": "Andrey Lukyanenko", "author_url": "", "post_date": "2020-12-27T13:29:35.130000", "content": "0.891
\nseresnext50, 1 fold, tta, 512x512
@artgor Nice to see you on this competition. Good luck !
", "votes": 1, "replies": [] }, { "id": 1130438, "author_name": "Andrey Lukyanenko", "author_url": "", "post_date": "2020-12-29T03:57:02.240000", "content": "@vladvdv thanks!
", "votes": 0, "replies": [] } ] }, { "id": 1128169, "author_name": "Eren Tekin", "author_url": "", "post_date": "2020-12-27T08:20:38.580000", "content": "0.901 single efficientnet b3 with tta and 2019 dataset
", "votes": 3, "replies": [ { "id": 1129170, "author_name": "Bcw93", "author_url": "", "post_date": "2020-12-28T06:08:16.057000", "content": "can you share how do you use scheduler or optimizer?Thanks!
", "votes": 0, "replies": [] }, { "id": 1129173, "author_name": "Bcw93", "author_url": "", "post_date": "2020-12-28T06:09:17.763000", "content": "I'm stopping in 0.898,and there is no improve whatever I do!
", "votes": 1, "replies": [] }, { "id": 1129383, "author_name": "Eren Tekin", "author_url": "", "post_date": "2020-12-28T09:53:44.340000", "content": "i used ranger optimizer that mentioned here
", "votes": 0, "replies": [] }, { "id": 1131943, "author_name": "Bcw93", "author_url": "", "post_date": "2020-12-30T04:59:30.217000", "content": "Thanks for your reply！En…did you use 'cossin' scheduler?Thanks!
", "votes": 0, "replies": [] } ] }, { "id": 1115982, "author_name": "danieliusk", "author_url": "", "post_date": "2020-12-16T18:08:17.137000", "content": "Resnet50 (no TTA, no external data) - 0.873
", "votes": 3, "replies": [ { "id": 1117953, "author_name": "Yanfei SIMP", "author_url": "", "post_date": "2020-12-18T16:02:27.797000", "content": "would you mind to share what augmentation you use for that result?
", "votes": 0, "replies": [] }, { "id": 1118663, "author_name": "danieliusk", "author_url": "", "post_date": "2020-12-19T09:33:07.093000", "content": "Sure :) I used vertical and horizontal flip, default albumentations rotate, random contrast and brightness. I also lightly blurred 10% of the images.
\n80% of the time, I randomly cropped 300x400 part from the input images.
\nI didn't use oversampling but class weights seemed to work.
\nAlthough I resized the images to 512x512, I tried not to destroy the original aspect ratio, so I padded top and bottom parts of the Image equally to convert image into a square before resizing.
\nHope it helps.
", "votes": 3, "replies": [] }, { "id": 1118753, "author_name": "Yanfei SIMP", "author_url": "", "post_date": "2020-12-19T11:16:47.393000", "content": "Thanks for your response, I used default ImageDataGenerator from keras and stuck in 0.85. I'll try albumentation then :)
", "votes": 1, "replies": [] } ] }, { "id": 1112348, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2020-12-14T13:59:56.783000", "content": "Is using different sizes improving anything? I have been using 224x224 and stuck to my score since the start; (not using 512 x512 as it takes time and I am using kaggle notebooks.)
", "votes": 3, "replies": [ { "id": 1112429, "author_name": "", "author_url": "", "post_date": "2020-12-14T15:30:41.470000", "content": "in my experiment's, the same model with a smaller image size than 512 gives me poor results than 512.
", "votes": 3, "replies": [] }, { "id": 1112549, "author_name": "Shubham Thapa", "author_url": "", "post_date": "2020-12-14T17:47:57.550000", "content": "yes it do improve my score , but it also depend upon the model architecture you are using like how big your model is such as for bigger models you probably going to need higher resolution images and for smaller model it is vice versa .
", "votes": 2, "replies": [] }, { "id": 1115220, "author_name": "Wang Xinliang", "author_url": "", "post_date": "2020-12-16T05:12:09.690000", "content": "448 is better for me.👀😄
", "votes": 0, "replies": [] }, { "id": 1149481, "author_name": "Vedder", "author_url": "", "post_date": "2021-01-11T21:37:02.487000", "content": "Increasing the image size from my initial 224 x 224 gave me a greater gain in accuracy for CV and LB than any other factor. I'm using 512 x 512 right now.
", "votes": 0, "replies": [] } ] }, { "id": 1120591, "author_name": "David", "author_url": "", "post_date": "2020-12-21T00:31:08.953000", "content": "Resnet50 with SnapMix (single fold, no TTA, no external data) - LB: 0.891
", "votes": 4, "replies": [ { "id": 1120613, "author_name": "Yann Majewski", "author_url": "", "post_date": "2020-12-21T01:30:50.840000", "content": "Intresting augmentation!
", "votes": 0, "replies": [] }, { "id": 1120641, "author_name": "Yi Wu", "author_url": "", "post_date": "2020-12-21T02:13:06.323000", "content": "but this score cannot achieve that high rank
", "votes": 0, "replies": [] }, { "id": 1120652, "author_name": "David", "author_url": "", "post_date": "2020-12-21T02:36:19.937000", "content": "using multi folds and model ensemble will further increase the socre
", "votes": 1, "replies": [] }, { "id": 1120664, "author_name": "Yi Wu", "author_url": "", "post_date": "2020-12-21T03:13:06.937000", "content": "I already used 5fold and average ensemble, improments are scrace.
", "votes": 0, "replies": [] }, { "id": 1120671, "author_name": "David", "author_url": "", "post_date": "2020-12-21T03:32:27.103000", "content": "In my case, single Resnet50 with 5 folds ensemble can get 0.897 on LB.
", "votes": 1, "replies": [] }, { "id": 1120699, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2020-12-21T04:08:53.803000", "content": "I am getting 895 with a seresnext50 single fold in my case.
", "votes": 1, "replies": [] }, { "id": 1120762, "author_name": "Yi Wu", "author_url": "", "post_date": "2020-12-21T05:11:12.093000", "content": "what can we do to furtherly push the score higher when training seems approaching the limit, say decreasing lr, but I got overfitting then.
", "votes": 1, "replies": [] }, { "id": 1121986, "author_name": "kmostafavi3", "author_url": "", "post_date": "2020-12-22T05:03:12.260000", "content": "are you using keras or pytorch?
", "votes": 0, "replies": [] }, { "id": 1122323, "author_name": "David", "author_url": "", "post_date": "2020-12-22T10:59:49.720000", "content": "I am using pytorch
", "votes": 1, "replies": [] } ] }, { "id": 1113151, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-12-15T08:37:57.703000", "content": "Best model is a Effnet B3 with imagenet weights which lead to a 0.900 to leaderboard.
\nAnother interesting question which anybody can answer beside the model and score will be if the model was trained only on data provided on this competition or using also the data from the past similar one.
are you using the data from the 2019 competition?
", "votes": 0, "replies": [] } ] }, { "id": 1111861, "author_name": "Phaedrus", "author_url": "", "post_date": "2020-12-14T04:59:06.320000", "content": "0.901 Effnet B4 NS with TTA, no external data yet.
", "votes": 4, "replies": [ { "id": 1117786, "author_name": "HAaHAa", "author_url": "", "post_date": "2020-12-18T12:45:49.013000", "content": "Hello,How many TTA times did u use?
", "votes": 0, "replies": [] }, { "id": 1117947, "author_name": "Phaedrus", "author_url": "", "post_date": "2020-12-18T15:58:54.823000", "content": "I'm experimenting with either 5 or 8. This one was from TTA = 5.
", "votes": 0, "replies": [] }, { "id": 1119241, "author_name": "", "author_url": "", "post_date": "2020-12-19T20:52:54.017000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1196278, "author_name": "yabea", "author_url": "", "post_date": "2021-02-11T11:00:48.900000", "content": "5-fold CV 0.9019, LB 0.901 (no tta).
", "votes": 1, "replies": [] }, { "id": 1196168, "author_name": "Vicky Goyal", "author_url": "", "post_date": "2021-02-11T09:48:32.627000", "content": "Single model 0.901, model ensemble 0.905, no tta for stability. TTA might overfit in 31% test data given
", "votes": 1, "replies": [ { "id": 1196878, "author_name": "kaggler", "author_url": "", "post_date": "2021-02-11T18:21:43.217000", "content": "why do you think that TTA might overfit to the public test data given?
", "votes": 0, "replies": [] } ] }, { "id": 1196099, "author_name": "Massimiliano Viola", "author_url": "", "post_date": "2021-02-11T08:49:14.683000", "content": "It is only me with a 0.894 5-fold CV and 0.901 LB? 😂
\nI am very confused and worried at the same time…
0.9018 single model CV/0.9 lb,
\n5 model ensemble 0.9045 CV/0.90 lb..
\nweird
Hello, which model did you use if you don't mind?
", "votes": 0, "replies": [] } ] }, { "id": 1173718, "author_name": "kaggler", "author_url": "", "post_date": "2021-01-28T04:45:29.017000", "content": "my single efficient net 5 CV 0.8997 LB 0.901
", "votes": 1, "replies": [ { "id": 1191339, "author_name": "V.Prasanna Kumar", "author_url": "", "post_date": "2021-02-08T12:15:17.263000", "content": "did you use any cut mix, f-mix or any other methods
", "votes": 1, "replies": [] }, { "id": 1196874, "author_name": "kaggler", "author_url": "", "post_date": "2021-02-11T18:19:26.747000", "content": "coarsedropout, mixup, cutmix used
", "votes": 0, "replies": [] } ] }, { "id": 1167434, "author_name": "DarknessZX", "author_url": "", "post_date": "2021-01-24T08:52:44.627000", "content": "single model 0.901,model essemble 0.904
", "votes": 1, "replies": [] }, { "id": 1165886, "author_name": "Prathamesh Sonawane", "author_url": "", "post_date": "2021-01-23T09:39:51.123000", "content": "I've tried training on Efficientnet-b4 but my validation accuracy seems to peak at 77%. I am using 2019 data, removing the duplicates, have done mixed precision training, 512x512 images, Adam. My transforms may be a little bit different but i don't think it justifies the huge gap in the score. Can you suggest something that i may be missing?
", "votes": 1, "replies": [ { "id": 1174494, "author_name": "impulsecorp", "author_url": "", "post_date": "2021-01-28T14:11:53.630000", "content": "Try training it without the transforms to make sure that is not the problem.
", "votes": 1, "replies": [] }, { "id": 1195476, "author_name": "Aiusha Sangadiev", "author_url": "", "post_date": "2021-02-10T20:36:57.240000", "content": "Sorry for late reply, but make sure that you are using the pretrained weights for EfficientNet-B4, it should solve your problem of low validation accuracy.
", "votes": 0, "replies": [] } ] }, { "id": 1164076, "author_name": "ikki1111", "author_url": "", "post_date": "2021-01-22T06:12:56.267000", "content": "| Name (5-fold) | \nCV | \nLB | \n
|---|---|---|
| hrnet exp019 | \n0.8947 | \n0.901 | \n
| hrnet exp020 | \n0.8940 | \n0.899 | \n
| hrnet exp021 | \n0.8969 | \n0.898 | \n
| resnext exp001 | \n0.8967 | \n0.901 | \n
| resnext exp001 + hrnet_exp019 | \n\n | 0.900 | \n
| resnext exp001 + hrnet_exp021 | \n\n | 0.901 | \n
A simple ensemble did not improve the accuracy.
\nWould it be better to adjust for each label?
Well did you test the ensemble with your CV? it might have increased it
", "votes": 1, "replies": [] }, { "id": 1165133, "author_name": "ikki1111", "author_url": "", "post_date": "2021-01-22T18:06:27.357000", "content": "I tried, but these models did not improve lb.
", "votes": 1, "replies": [] } ] }, { "id": 1134955, "author_name": "rajan", "author_url": "", "post_date": "2021-01-01T18:20:24.857000", "content": "87.7 …. I dont want to add previous year data and ensemble model yet.
\nWant to improve this
\nAny suggestion ?
I have tried
\nres50,34,101
\neffnet b3,b4,b5,b6
\nVisionTransformers
Cutmix, Labelsmooth, TTA
\nWhat i am missing ? Have no clue how to push.
\n( Everything was learnt from kaggle contributes )
", "votes": 1, "replies": [ { "id": 1134986, "author_name": "rajan", "author_url": "", "post_date": "2021-01-01T19:16:13.793000", "content": "bs is around 6 to 3
", "votes": 0, "replies": [] }, { "id": 1149550, "author_name": "ayu055", "author_url": "", "post_date": "2021-01-11T23:53:29.373000", "content": "Are you using augmentations such as flip and rotate? Maybe you could also try a different loss function such as Bi-tempered logistic loss. I think some people found that it gave better results.
", "votes": 0, "replies": [] } ] }, { "id": 1128353, "author_name": "", "author_url": "", "post_date": "2020-12-27T11:18:20.607000", "content": "", "votes": 1, "replies": [] }, { "id": 1111748, "author_name": "", "author_url": "", "post_date": "2020-12-14T02:34:06.703000", "content": "", "votes": 1, "replies": [ { "id": 1119427, "author_name": "", "author_url": "", "post_date": "2020-12-20T04:05:29.603000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1111694, "author_name": "", "author_url": "", "post_date": "2020-12-13T23:26:48.853000", "content": "", "votes": 1, "replies": [] }, { "id": 1155173, "author_name": "", "author_url": "", "post_date": "2021-01-16T09:08:01.737000", "content": "", "votes": 2, "replies": [] }, { "id": 1148321, "author_name": "", "author_url": "", "post_date": "2021-01-11T04:35:34.433000", "content": "", "votes": 2, "replies": [ { "id": 1148600, "author_name": "", "author_url": "", "post_date": "2021-01-11T08:47:28.447000", "content": "", "votes": 1, "replies": [] }, { "id": 1149549, "author_name": "", "author_url": "", "post_date": "2021-01-11T23:51:20.460000", "content": "", "votes": 1, "replies": [] }, { "id": 1150316, "author_name": "", "author_url": "", "post_date": "2021-01-12T14:07:59.867000", "content": "", "votes": 1, "replies": [] }, { "id": 1150479, "author_name": "", "author_url": "", "post_date": "2021-01-12T16:08:17.460000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 1129511, "author_name": "", "author_url": "", "post_date": "2020-12-28T11:53:08.530000", "content": "", "votes": 2, "replies": [] }, { "id": 1123861, "author_name": "", "author_url": "", "post_date": "2020-12-23T14:48:00.103000", "content": "", "votes": 2, "replies": [] }, { "id": 1121821, "author_name": "", "author_url": "", "post_date": "2020-12-21T23:55:10.933000", "content": "", "votes": 2, "replies": [ { "id": 1129365, "author_name": "", "author_url": "", "post_date": "2020-12-28T09:39:13.230000", "content": "", "votes": 0, "replies": [] }, { "id": 1129514, "author_name": "", "author_url": "", "post_date": "2020-12-28T11:59:10.700000", "content": "", "votes": 1, "replies": [] }, { "id": 1129516, "author_name": "", "author_url": "", "post_date": "2020-12-28T12:00:29.263000", "content": "", "votes": 0, "replies": [] }, { "id": 1130729, "author_name": "", "author_url": "", "post_date": "2020-12-29T09:21:33.117000", "content": "", "votes": 1, "replies": [] }, { "id": 1138161, "author_name": "", "author_url": "", "post_date": "2021-01-04T13:07:46.270000", "content": "", "votes": 3, "replies": [] } ] }, { "id": 1120766, "author_name": "", "author_url": "", "post_date": "2020-12-21T05:17:52.707000", "content": "", "votes": 2, "replies": [ { "id": 1122412, "author_name": "", "author_url": "", "post_date": "2020-12-22T12:28:13.583000", "content": "", "votes": 2, "replies": [] }, { "id": 1122475, "author_name": "", "author_url": "", "post_date": "2020-12-22T13:17:12.840000", "content": "", "votes": 2, "replies": [] }, { "id": 1123476, "author_name": "", "author_url": "", "post_date": "2020-12-23T08:53:35.387000", "content": "", "votes": 1, "replies": [] }, { "id": 1123521, "author_name": "", "author_url": "", "post_date": "2020-12-23T09:46:09.043000", "content": "", "votes": 1, "replies": [] }, { "id": 1123524, "author_name": "", "author_url": "", "post_date": "2020-12-23T09:46:55.213000", "content": "", "votes": 1, "replies": [] }, { "id": 1149476, "author_name": "", "author_url": "", "post_date": "2021-01-11T21:26:26.470000", "content": "", "votes": 1, "replies": [] }, { "id": 1149484, "author_name": "", "author_url": "", "post_date": "2021-01-11T21:38:37.463000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1113921, "author_name": "", "author_url": "", "post_date": "2020-12-15T20:03:46.987000", "content": "", "votes": 2, "replies": [] }, { "id": 1112315, "author_name": "", "author_url": "", "post_date": "2020-12-14T13:23:46.720000", "content": "", "votes": 2, "replies": [] }, { "id": 1111670, "author_name": "", "author_url": "", "post_date": "2020-12-13T22:52:31.407000", "content": "", "votes": 2, "replies": [] }, { "id": 1111656, "author_name": "", "author_url": "", "post_date": "2020-12-13T22:15:36.843000", "content": "", "votes": 2, "replies": [] }, { "id": 1111557, "author_name": "", "author_url": "", "post_date": "2020-12-13T19:44:40.153000", "content": "", "votes": 2, "replies": [ { "id": 1111561, "author_name": "", "author_url": "", "post_date": "2020-12-13T19:49:27.680000", "content": "", "votes": 2, "replies": [] }, { "id": 1111564, "author_name": "", "author_url": "", "post_date": "2020-12-13T19:50:39.257000", "content": "", "votes": 3, "replies": [] }, { "id": 1112259, "author_name": "", "author_url": "", "post_date": "2020-12-14T12:40:36.507000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 1111556, "author_name": "", "author_url": "", "post_date": "2020-12-13T19:43:31.583000", "content": "", "votes": 2, "replies": [ { "id": 1111580, "author_name": "", "author_url": "", "post_date": "2020-12-13T20:25:47.063000", "content": "", "votes": 1, "replies": [] }, { "id": 1131907, "author_name": "", "author_url": "", "post_date": "2020-12-30T04:17:06.567000", "content": "", "votes": 2, "replies": [] } ] }, { "id": 1115077, "author_name": "", "author_url": "", "post_date": "2020-12-16T00:59:06.077000", "content": "", "votes": 0, "replies": [ { "id": 1115476, "author_name": "", "author_url": "", "post_date": "2020-12-16T10:12:54.673000", "content": "", "votes": 4, "replies": [] } ] }, { "id": 1174215, "author_name": "", "author_url": "", "post_date": "2021-01-28T10:51:34.947000", "content": "", "votes": 0, "replies": [] }, { "id": 1130979, "author_name": "", "author_url": "", "post_date": "2020-12-29T13:23:05.527000", "content": "", "votes": 1, "replies": [] }, { "id": 1113427, "author_name": "", "author_url": "", "post_date": "2020-12-15T12:35:59.580000", "content": "", "votes": -3, "replies": [] }, { "id": 1113321, "author_name": "", "author_url": "", "post_date": "2020-12-15T11:21:50.417000", "content": "", "votes": -3, "replies": [] } ], "raw_markdown_by_id": { "1111515": "Good old best single model score thread :) \n\nWe have 0.898 on LB.\n\nWhat about yours?", "1111578": "0.904 single EfficientNet B4", "1165230": "My best model is now ViT base 16x16 patches. \n\nLB 0.904 (slightly better than my effnet B4)\n\nBut I do not trust it at all : Huge gap between CV and LB and the LB result is highly sensitive to the number of TTA. \n\nMy Effnet is much more stable across many experiments, and has much higher CV. \n\n", "1149675": "CV 0.883 LB 0.890 (same model) I'm having trouble getting my CV scores over 0.885 during training. And don't know what i could be doing wrong at the moment. What i'm doing:\nImage Sizes : 384\nModels: EffNetB3,B4,ResNext50 (all train to around 88.0 ~ 88.5 CV) +Dropout(0.3) at last layer\nEarlyStopping: 2 Patience with Minimum Loss and save best weights at minimum\nStratifiedKFolds = 5\nAdded 2019 Data (was careful with what @serigne told me to not include them at the validation set) : improved a tiny bit\nOptimizer: Adam,Ranger Adam actually worked a bit better than Ranger with schedulers\nScheduler: OneCycleLR >= CosineAnnealingWarmRestarts > ReduceLROnPlateau >> LambdaLR -> OneCycle depends on the LossFunction to improve CV score\nLosses: TaylorCrossEntropy > BiTemperedLogistic(>20 permutations of T1,T2 mostly low T1 and T2~= 1.0) > SymmetricCrossEntropy\nAugmentations: Tried Only Flips/Rotates and Normalization and that + ColorJitter and CoarseDropout and AutoAugment from imagenet\nI'm really not sure what strategy you guys are using to reach over 88.5 on CV, i tried almost everything that was dicussed here on the forums.", "1121264": "0.901 on LB - ResNeXt50_32x4d, 5 folds ensemble, no external data, partially relabeled\n\n(updated)", "1112721": "i have a strange feeling that we may all be moving to 0.905 and stop there", "1115980": "I have not tried submitting my predictions yet, so I do not know how high (or low :-) ) my model will score on the leaderboard. The 5-fold CV is 0.90140 (average across folds). But the agreement between the training and validation accuracies seems to be very good (see an example of my first training fold below). \n\n", "1133922": "Beside my effnet B4, I have 3 others (lighter) models scoring 0.901, 0.900 , 0.900 and trained with another strategy than the Effnet b4. However, it's very difficult to improve them further. \n\nMay be I should start ensembling ^^", "1120584": "Hi, can I just ask - when people say 'single model' here, does this refer to a single design (e.g. EfficientNetB3) or to a single copy / set of model weights?\n\nFor example I've trained an EffNetB3 with 5-fold CV so have 5 sets of weights, all of which are used in inference notebook - is that what others would report as a 'single model' score?\n\nLeaderboard score for the above for me is 0.893.\n\nModel is in tensorflow and I used the very helpful notebooks provided by @dimitreoliveira as a starting point, with TTA during prediction of test set. ", "1111612": "0.901 single EfficientNet B3_ns\n0.901 single EfficientNet B3 imgnet", "1196357": "resnet200d CV 0.9028 LB 0.901(w/o tta)", "1159230": "Efficientnet B0, CV fold5: 0.898, LB: 0.896 with image size 512*512", "1152612": "Small and standard start, to have a baseline.\nSingle B0ns dim 512 1 fold, extra data, many aug features, p.5 of mixup&cutout , AdamP, CosAnnWarm, LabelSm.\nVal Accuracy: 0.8936977980258163\nLB 0.892\n\nA reflection, seems be shadows and different quality in the images, both can be implemented in the Albumentations tool, maybe can be worth a try to this training.", "1130214": "Efficientnet B5 noisy student with 512 image size. \n\n5folds CV is 0.892 and LB is 0.898. No TTA has been performed. \n\nNothing compared to the monstrous results of @serigne 0.904 score... maybe I’ll give 2019 a try ", "1128483": "0.891\nseresnext50, 1 fold, tta, 512x512", "1128169": "0.901 single efficientnet b3 with tta and 2019 dataset", "1115982": "Resnet50 (no TTA, no external data) - 0.873", "1112348": "Is using different sizes improving anything? I have been using 224x224 and stuck to my score since the start; (not using 512 x512 as it takes time and I am using kaggle notebooks.)", "1120591": "Resnet50 with SnapMix (single fold, no TTA, no external data) - LB: 0.891", "1113151": "Best model is a Effnet B3 with imagenet weights which lead to a 0.900 to leaderboard.\nAnother interesting question which anybody can answer beside the model and score will be if the model was trained only on data provided on this competition or using also the data from the past similar one.", "1111861": "0.901 Effnet B4 NS with TTA, no external data yet.", "1196278": "5-fold CV 0.9019, LB 0.901 (no tta).", "1196168": "Single model 0.901, model ensemble 0.905, no tta for stability. TTA might overfit in 31% test data given", "1196099": "It is only me with a 0.894 5-fold CV and 0.901 LB? 😂\nI am very confused and worried at the same time...", "1195765": "0.9018 single model CV/0.9 lb, \n5 model ensemble 0.9045 CV/0.90 lb..\nweird", "1173718": "my single efficient net 5 CV 0.8997 LB 0.901", "1167434": "single model 0.901,model essemble 0.904", "1165886": "I've tried training on Efficientnet-b4 but my validation accuracy seems to peak at 77%. I am using 2019 data, removing the duplicates, have done mixed precision training, 512x512 images, Adam. My transforms may be a little bit different but i don't think it justifies the huge gap in the score. Can you suggest something that i may be missing?\n", "1164076": "| Name (5-fold) | CV | LB |\n|-------------------------------|-----------|----------|\n| hrnet exp019 | 0.8947 | 0.901 |\n| hrnet exp020 | 0.8940 | 0.899 |\n| hrnet exp021 | 0.8969 | 0.898 |\n| resnext exp001 | 0.8967 | 0.901 |\n| resnext exp001 + hrnet_exp019 | | 0.900 |\n| resnext exp001 + hrnet_exp021 | | 0.901 |\n\nA simple ensemble did not improve the accuracy.\nWould it be better to adjust for each label?\n", "1134955": "87.7 .... I dont want to add previous year data and ensemble model yet. \nWant to improve this \nAny suggestion ?\n\nI have tried \n\nres50,34,101\neffnet b3,b4,b5,b6\nVisionTransformers \n\nCutmix, Labelsmooth, TTA\n\nWhat i am missing ? Have no clue how to push.\n\n( Everything was learnt from kaggle contributes )", "1128353": "Thank you for your thread and thank you for all comments! It helps me a lot. \nI just start this competition and I got 0.822 (really poor score...) with efficientnetb0 and random selected 1000 images in dataset. ", "1111748": "0.900 single resnext50", "1111694": "0.899 EfficientNet B4_ns with TTA", "1155173": " Resnet18, 512, basic augmentations, Adam, CrossEntropy, ReduceLROnPlateau, Early stopping--> LB 0.875/random_split 0.87336 (may not be reliable, better to try with cv)", "1148321": "0.900 SEResNet152, but this local CV is not significuntly better than other model.\nso, I start to doubt whether public LB can be reliable or not....", "1129511": "The best (for now) is efficientnet_b4 with 0.894 on public.\nI think this model could achieve much better results.", "1123861": "seresnext50\n5 folds\n384x384\n8 epochs\nCutmix\n0.896 lb ", "1121821": "Hi all,\n\nScores reported in this discussion refers to a `single model`, but I cannot clearly understand if they are actually scores of a single instance of the model or if they refers to the ensemble over all folds.\n\nIn my case I have a single EfficientNetB4 trained on images `512x512` (smaller sizes gives me lower scores)\n- `CV: 0.8958`\n- `LB: 0.889`\n- `CutMix + MixUp`\n- `No External Data`\n- `No TTA`\n\nWhile with an ensemble of all folds i get LB score of `0.896`", "1120766": "Is anyone freezing the layers of their transfer learning model; or training the whole model?", "1113921": "0.894 effb3 single fold seed 42 stratified fold 0th, NO external data", "1112315": "0.842 with Xception", "1111670": "0.900 - single seresnext50", "1111656": "I believe the important question is not only how the model eventually scores on the LB but also how much time does it take to achieve the reported score. Heavy augmentation can slow inference down.", "1111557": "I think there's already a discussion thread for this, [here](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198805). ", "1111556": "0.897 on the LB - all my scores are single model, so far. And thanks for your tez starter notebook, @abhishek, that played a large role in getting me there.", "1115077": "EffNetB4 0.884 LB and 0.902 CV, i'm having trouble dealing with overfitting and proper data augmentations. ", "1174215": "", "1130979": "", "1113427": "", "1113321": "" } }