{
  "id": 198805,
  "title": "Best single model",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/198805",
  "author_name": "",
  "post_date": "2020-11-23T05:12:42.093714900Z",
  "votes": 32,
  "comment_count": 24,
  "views": 0,
  "content": "<p>Starting a new thread. :) </p>\n<p>I'll go first.. </p>\n<pre><code>EfficientNet-B4 - single fold\nNo TTA\nBasic Data Augmentation - FLIP, Rotate, Random Brightness Contrast.. \nImage Size - 512x512\nCV: 0.895, LB: 0.890 \n</code></pre>",
  "messages": [
    {
      "id": "1087823",
      "postDate": "11/23/2020 05:12:42",
      "content": "<p>Starting a new thread. :) </p>\n<p>I'll go first.. </p>\n<pre><code>EfficientNet-B4 - single fold\nNo TTA\nBasic Data Augmentation - FLIP, Rotate, Random Brightness Contrast.. \nImage Size - 512x512\nCV: 0.895, LB: 0.890 \n</code></pre>",
      "rawMarkdown": "Starting a new thread. :) \n\nI'll go first.. \n\n```\nEfficientNet-B4 - single fold\nNo TTA\nBasic Data Augmentation - FLIP, Rotate, Random Brightness Contrast.. \nImage Size - 512x512\nCV: 0.895, LB: 0.890 \n```",
      "votes": null
    },
    {
      "id": "1087838",
      "postDate": "11/23/2020 05:22:35",
      "content": "<p>EfficientNet-B1 - Single Fold<br>\nNo TTA<br>\nData Augmentation - Hflip, Vflip, Transpose, RandomCrop, ShiftScaleRotate<br>\nImage Size - 384x384<br>\nCV: 0.89, LB: 0.885</p>\n<p>EfficientNet-B0 - Single Fold<br>\nNo TTA<br>\nData Augmentation - Hflip, Vflip, Transpose, RandomCrop, ShiftScaleRotate<br>\nImage Size - 512x512<br>\nCV: 0.891, LB: ?</p>",
      "rawMarkdown": "EfficientNet-B1 - Single Fold\nNo TTA\nData Augmentation - Hflip, Vflip, Transpose, RandomCrop, ShiftScaleRotate\nImage Size - 384x384\nCV: 0.89, LB: 0.885\n\nEfficientNet-B0 - Single Fold\nNo TTA\nData Augmentation - Hflip, Vflip, Transpose, RandomCrop, ShiftScaleRotate\nImage Size - 512x512\nCV: 0.891, LB: ?",
      "votes": null
    },
    {
      "id": "1087873",
      "postDate": "11/23/2020 06:00:24",
      "content": "<p>resnext50<br>\n8TTA<br>\n448*448<br>\nCV 0.892<br>\nLB 0.895</p>\n<p>update1:<br>\nstrong aug<br>\nCV 0.889<br>\nLB 0.896</p>\n<p>update2:<br>\nCV 0.896<br>\nLB 0.899</p>",
      "rawMarkdown": "resnext50\n8TTA\n448*448\nCV 0.892\nLB 0.895\n\nupdate1:\nstrong aug\nCV 0.889\nLB 0.896\n\nupdate2:\nCV 0.896\nLB 0.899",
      "votes": null
    },
    {
      "id": "1087922",
      "postDate": "11/23/2020 06:56:14",
      "content": "<p>Nice! BTW what is the meaning of 8TTA, 4TTA etc.?</p>",
      "rawMarkdown": "Nice! BTW what is the meaning of 8TTA, 4TTA etc.?",
      "votes": null
    },
    {
      "id": "1087926",
      "postDate": "11/23/2020 07:01:00",
      "content": "<p><a href=\"https://www.kaggle.com/kaushal2896\" target=\"_blank\">@kaushal2896</a> </p>\n<p>Basically - TTA stands for <strong>Test Time Augmentation</strong>. Let's say you are trying to predict the masks for a single image. In a simple case you would center crop the test image and make a prediction and that's pretty much it. This is your final prediction. </p>\n<p>But with TTA - you add augmentation to the test image and create let's say <code>N</code> number of augmented copies that are similar to the image but yet different. The augmentations could be rotate, flip, etc.. </p>\n<p>Next, you make a prediction for all these N number of images and take the average as your final prediction for the test image. </p>\n<p><code>N</code> is what refers to 4,8, etc.. 8 means someone is creating 8 copies of the test image using random augmentations and then taking the average as final prediction, 4 means creating 4 copies and so on..</p>",
      "rawMarkdown": "kaushal2896 \n\nBasically - TTA stands for **Test Time Augmentation**. Let's say you are trying to predict the masks for a single image. In a simple case you would center crop the test image and make a prediction and that's pretty much it. This is your final prediction. \n\nBut with TTA - you add augmentation to the test image and create let's say `N` number of augmented copies that are similar to the image but yet different. The augmentations could be rotate, flip, etc.. \n\nNext, you make a prediction for all these N number of images and take the average as your final prediction for the test image. \n\n`N` is what refers to 4,8, etc.. 8 means someone is creating 8 copies of the test image using random augmentations and then taking the average as final prediction, 4 means creating 4 copies and so on..",
      "votes": null
    },
    {
      "id": "1087931",
      "postDate": "11/23/2020 07:06:43",
      "content": "<p><a href=\"https://www.kaggle.com/aroraaman\" target=\"_blank\">@aroraaman</a> Thanks for detailed explanation. Actually I'm using TTA but wasn't aware of this terminology!</p>",
      "rawMarkdown": "aroraaman Thanks for detailed explanation. Actually I'm using TTA but wasn't aware of this terminology!",
      "votes": null
    },
    {
      "id": "1088049",
      "postDate": "11/23/2020 09:17:24",
      "content": "<p>5 Fold CV<br>\nResnext 50 <br>\nData Augmentation - Yes<br>\nImage Size 512*512<br>\nCV 0.887<br>\nLB 0.899 </p>\n<p>Update:<br>\nCV 0.889<br>\nLB 0.901</p>\n<p>Notebook is public</p>",
      "rawMarkdown": "5 Fold CV\nResnext 50 \nData Augmentation - Yes\nImage Size 512*512\nCV 0.887\nLB 0.899 \n\nUpdate:\nCV 0.889\nLB 0.901\n\nNotebook is public",
      "votes": null
    },
    {
      "id": "1088053",
      "postDate": "11/23/2020 09:19:28",
      "content": "<p>whoa! No TTA or advanced augmentations?</p>",
      "rawMarkdown": "whoa! No TTA or advanced augmentations?",
      "votes": null
    },
    {
      "id": "1088064",
      "postDate": "11/23/2020 09:33:30",
      "content": "<p>I haven't tried TTA till now… Just focusing on right image size/architecture for now…</p>",
      "rawMarkdown": "I haven't tried TTA till now... Just focusing on right image size/architecture for now...",
      "votes": null
    },
    {
      "id": "1088245",
      "postDate": "11/23/2020 13:16:25",
      "content": "<p>Resnest50<br>\n5 fold<br>\n384x288<br>\nCutMix + basic augs<br>\nNo TTA<br>\nCV: 0.88948<br>\nLB: 0.896</p>",
      "rawMarkdown": "Resnest50\n5 fold\n384x288\nCutMix + basic augs\nNo TTA\nCV: 0.88948\nLB: 0.896",
      "votes": null
    },
    {
      "id": "1088484",
      "postDate": "11/23/2020 16:41:14",
      "content": "<blockquote>\n  <p>SE_ResNeXt50_32x4d<br>\n  16TTA<br>\n  Basic Augmentations - HV flips, Rotate, Random Brightness Contrast, Cutout, Cutmix, Mixup<br>\n  Image Size - 512x512<br>\n  CV: 0.8750, LB: 0.8930</p>\n  <p>ResNet-18<br>\n  16TTA<br>\n  Basic Augmentations - HV flips, Rotate, Random Brightness Contrast, Cutout, Cutmix, Mixup<br>\n  Image Size - 512x512<br>\n  CV: 0.8304, LB: 0.882</p>\n</blockquote>",
      "rawMarkdown": "> SE_ResNeXt50_32x4d\n16TTA\nBasic Augmentations - HV flips, Rotate, Random Brightness Contrast, Cutout, Cutmix, Mixup\nImage Size - 512x512\nCV: 0.8750, LB: 0.8930\n\n> ResNet-18\n16TTA\nBasic Augmentations - HV flips, Rotate, Random Brightness Contrast, Cutout, Cutmix, Mixup\nImage Size - 512x512\nCV: 0.8304, LB: 0.882",
      "votes": null
    },
    {
      "id": "1088499",
      "postDate": "11/23/2020 17:00:40",
      "content": "<p>Nice! I can not cross 0.893 even with TTA!</p>",
      "rawMarkdown": "Nice! I can not cross 0.893 even with TTA!",
      "votes": null
    },
    {
      "id": "1089218",
      "postDate": "11/24/2020 09:54:17",
      "content": "<p>Normal Train-Test-Split:<br>\nEffnet_b3_ns<br>\nImg_size: 512<br>\nLB 0.889<br>\nNo TTA</p>",
      "rawMarkdown": "Normal Train-Test-Split:\nEffnet_b3_ns\nImg_size: 512\nLB 0.889\nNo TTA",
      "votes": null
    },
    {
      "id": "1111629",
      "postDate": "12/13/2020 21:27:24",
      "content": "<p>512<br>\nTTA<br>\nAUG<br>\n0.901 single EfficientNet B3_ns<br>\n0.901 single EfficientNet B3 imgnet</p>",
      "rawMarkdown": "512\nTTA\nAUG\n0.901 single EfficientNet B3_ns\n0.901 single EfficientNet B3 imgnet",
      "votes": null
    },
    {
      "id": "1111743",
      "postDate": "12/14/2020 02:26:13",
      "content": "<p>Sorry for my ignorance, just started on ML.     I can infer that the LB stands for leader board score.  I know in certain context, CV stands for computer vision.  I know many people talk about CV here.   What does CV stands for?  is it the validation accuracy? why not called acc?</p>",
      "rawMarkdown": "Sorry for my ignorance, just started on ML.     I can infer that the LB stands for leader board score.  I know in certain context, CV stands for computer vision.  I know many people talk about CV here.   What does CV stands for?  is it the validation accuracy? why not called acc?",
      "votes": null
    },
    {
      "id": "1111746",
      "postDate": "12/14/2020 02:29:11",
      "content": "<p>Here usually CV refers to local cross validation score, but probably if you report only single fold score it's not cross validation anymore but just local validation score. It's important to have a local validation schema which aligns with public leaderboard in order to understand what really works and generalizes.</p>",
      "rawMarkdown": "Here usually CV refers to local cross validation score, but probably if you report only single fold score it's not cross validation anymore but just local validation score. It's important to have a local validation schema which aligns with public leaderboard in order to understand what really works and generalizes.",
      "votes": null
    },
    {
      "id": "1111855",
      "postDate": "12/14/2020 04:48:49",
      "content": "<p>Thanks for the reply.  I use TF/Keras data_generator. There does not seems a easy way to implement multiple fold cross validation.   I got very high single fold scores, but the LB score is many points lower.   If anyone has suggestion or examples on implementting multi fold cross validation using Keras data generator,  I would be appreciated. </p>",
      "rawMarkdown": "Thanks for the reply.  I use TF/Keras data_generator. There does not seems a easy way to implement multiple fold cross validation.   I got very high single fold scores, but the LB score is many points lower.   If anyone has suggestion or examples on implementting multi fold cross validation using Keras data generator,  I would be appreciated.",
      "votes": null
    },
    {
      "id": "1113726",
      "postDate": "12/15/2020 16:40:58",
      "content": "<p>Small doubt, when you guys report a single model score, you mean a k-fold mean score or just a 1-fold mode score? and when you submit you would perform K-fold? I can understand running traditional ML models like Random forests with K-Fold, but training a Deep learning model for 5 fold seems too time-consuming.</p>",
      "rawMarkdown": "Small doubt, when you guys report a single model score, you mean a k-fold mean score or just a 1-fold mode score? and when you submit you would perform K-fold? I can understand running traditional ML models like Random forests with K-Fold, but training a Deep learning model for 5 fold seems too time-consuming.",
      "votes": null
    },
    {
      "id": "1136938",
      "postDate": "01/03/2021 14:51:47",
      "content": "<p>resnet50, 512x512, 10epoch, Basic Data Augmentation - FLIP, Rotate, Random Brightness Contrast..  <br>\nsingle fold, no TTA<br>\nCV: 0.893  LB:0.891</p>\n<p>eff-b4<br>\nCV:0.896 LB:0.891</p>\n<p>eff-b2<br>\nCV:0.895 LB:0.886</p>\n<p>I can't get score higher than 0.891 when single fold, no TTA……could anybody give me some suggestions?</p>",
      "rawMarkdown": "resnet50, 512x512, 10epoch, Basic Data Augmentation - FLIP, Rotate, Random Brightness Contrast..  \nsingle fold, no TTA\nCV: 0.893  LB:0.891\n\neff-b4\nCV:0.896 LB:0.891\n\neff-b2\nCV:0.895 LB:0.886\n\nI can't get score higher than 0.891 when single fold, no TTA......could anybody give me some suggestions?",
      "votes": null
    },
    {
      "id": "1139053",
      "postDate": "01/05/2021 06:07:42",
      "content": "<p>how you get higher score than 0.893?</p>",
      "rawMarkdown": "how you get higher score than 0.893?",
      "votes": null
    },
    {
      "id": "1139661",
      "postDate": "01/05/2021 14:53:12",
      "content": "<p>You must train all the folds to get your real CV, i had the same thing at the begining of the comp. Some folds will score much lower than others</p>",
      "rawMarkdown": "You must train all the folds to get your real CV, i had the same thing at the begining of the comp. Some folds will score much lower than others",
      "votes": null
    },
    {
      "id": "1140575",
      "postDate": "01/06/2021 05:44:10",
      "content": "<p>but 5 fold cost too much time, and the highest of them, or  5fold ensemble, or top3 ensemble, is still not good.</p>",
      "rawMarkdown": "but 5 fold cost too much time, and the highest of them, or  5fold ensemble, or top3 ensemble, is still not good.",
      "votes": null
    },
    {
      "id": "1140608",
      "postDate": "01/06/2021 06:03:21",
      "content": "<p>Can you guys also mention how long you train each fold. If you are using early stopping, which epoch gives the best accuracy or eval loss in each fold and are you guys using Early stopping on eval accuracy eval loss ?<br>\nThe main reason I'm asking this is I'm not getting better eval loss/accuracy beyond the first epoch even with early stopping of 15. Would be glad to know what others are doing. </p>",
      "rawMarkdown": "Can you guys also mention how long you train each fold. If you are using early stopping, which epoch gives the best accuracy or eval loss in each fold and are you guys using Early stopping on eval accuracy eval loss ?\nThe main reason I'm asking this is I'm not getting better eval loss/accuracy beyond the first epoch even with early stopping of 15. Would be glad to know what others are doing.",
      "votes": null
    },
    {
      "id": "1155840",
      "postDate": "01/16/2021 17:50:02",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/aroraaman\" target=\"_blank\">@aroraaman</a> ! Do you mean that the copies are created with different augmentations or with the same augmentation but with different parameters?</p>",
      "rawMarkdown": "Hi, @aroraaman ! Do you mean that the copies are created with different augmentations or with the same augmentation but with different parameters?",
      "votes": null
    },
    {
      "id": "1170289",
      "postDate": "01/26/2021 05:42:05",
      "content": "<p>resnet34</p>\n<p>cv: 0.888<br>\nlb: 0.879</p>\n<p>512*512, single fold.<br>\nno tta, basic data augmentation (flip, shift scale rotate)</p>",
      "rawMarkdown": "resnet34\n\ncv: 0.888\nlb: 0.879\n\n512*512, single fold.\nno tta, basic data augmentation (flip, shift scale rotate)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1087838,
      "author_name": "harshitsheoran",
      "author_url": "",
      "post_date": "11/23/2020 05:22:35",
      "content": "<p>EfficientNet-B1 - Single Fold<br>\nNo TTA<br>\nData Augmentation - Hflip, Vflip, Transpose, RandomCrop, ShiftScaleRotate<br>\nImage Size - 384x384<br>\nCV: 0.89, LB: 0.885</p>\n<p>EfficientNet-B0 - Single Fold<br>\nNo TTA<br>\nData Augmentation - Hflip, Vflip, Transpose, RandomCrop, ShiftScaleRotate<br>\nImage Size - 512x512<br>\nCV: 0.891, LB: ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1087873,
      "author_name": "dldmw579",
      "author_url": "",
      "post_date": "11/23/2020 06:00:24",
      "content": "<p>resnext50<br>\n8TTA<br>\n448*448<br>\nCV 0.892<br>\nLB 0.895</p>\n<p>update1:<br>\nstrong aug<br>\nCV 0.889<br>\nLB 0.896</p>\n<p>update2:<br>\nCV 0.896<br>\nLB 0.899</p>",
      "votes": null,
      "replies": [
        {
          "id": 1087922,
          "author_name": "kaushal2896",
          "author_url": "",
          "post_date": "11/23/2020 06:56:14",
          "content": "<p>Nice! BTW what is the meaning of 8TTA, 4TTA etc.?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1087926,
          "author_name": "aroraaman",
          "author_url": "",
          "post_date": "11/23/2020 07:01:00",
          "content": "<p><a href=\"https://www.kaggle.com/kaushal2896\" target=\"_blank\">@kaushal2896</a> </p>\n<p>Basically - TTA stands for <strong>Test Time Augmentation</strong>. Let's say you are trying to predict the masks for a single image. In a simple case you would center crop the test image and make a prediction and that's pretty much it. This is your final prediction. </p>\n<p>But with TTA - you add augmentation to the test image and create let's say <code>N</code> number of augmented copies that are similar to the image but yet different. The augmentations could be rotate, flip, etc.. </p>\n<p>Next, you make a prediction for all these N number of images and take the average as your final prediction for the test image. </p>\n<p><code>N</code> is what refers to 4,8, etc.. 8 means someone is creating 8 copies of the test image using random augmentations and then taking the average as final prediction, 4 means creating 4 copies and so on..</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1087931,
          "author_name": "kaushal2896",
          "author_url": "",
          "post_date": "11/23/2020 07:06:43",
          "content": "<p><a href=\"https://www.kaggle.com/aroraaman\" target=\"_blank\">@aroraaman</a> Thanks for detailed explanation. Actually I'm using TTA but wasn't aware of this terminology!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1155840,
          "author_name": "etagiev",
          "author_url": "",
          "post_date": "01/16/2021 17:50:02",
          "content": "<p>Hi, <a href=\"https://www.kaggle.com/aroraaman\" target=\"_blank\">@aroraaman</a> ! Do you mean that the copies are created with different augmentations or with the same augmentation but with different parameters?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1088049,
      "author_name": "manojprabhaakr",
      "author_url": "",
      "post_date": "11/23/2020 09:17:24",
      "content": "<p>5 Fold CV<br>\nResnext 50 <br>\nData Augmentation - Yes<br>\nImage Size 512*512<br>\nCV 0.887<br>\nLB 0.899 </p>\n<p>Update:<br>\nCV 0.889<br>\nLB 0.901</p>\n<p>Notebook is public</p>",
      "votes": null,
      "replies": [
        {
          "id": 1088053,
          "author_name": "kaushal2896",
          "author_url": "",
          "post_date": "11/23/2020 09:19:28",
          "content": "<p>whoa! No TTA or advanced augmentations?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1088064,
          "author_name": "manojprabhaakr",
          "author_url": "",
          "post_date": "11/23/2020 09:33:30",
          "content": "<p>I haven't tried TTA till now… Just focusing on right image size/architecture for now…</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1088499,
          "author_name": "kaushal2896",
          "author_url": "",
          "post_date": "11/23/2020 17:00:40",
          "content": "<p>Nice! I can not cross 0.893 even with TTA!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1088245,
      "author_name": "yannmajewski",
      "author_url": "",
      "post_date": "11/23/2020 13:16:25",
      "content": "<p>Resnest50<br>\n5 fold<br>\n384x288<br>\nCutMix + basic augs<br>\nNo TTA<br>\nCV: 0.88948<br>\nLB: 0.896</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1088484,
      "author_name": "kaushal2896",
      "author_url": "",
      "post_date": "11/23/2020 16:41:14",
      "content": "<blockquote>\n  <p>SE_ResNeXt50_32x4d<br>\n  16TTA<br>\n  Basic Augmentations - HV flips, Rotate, Random Brightness Contrast, Cutout, Cutmix, Mixup<br>\n  Image Size - 512x512<br>\n  CV: 0.8750, LB: 0.8930</p>\n  <p>ResNet-18<br>\n  16TTA<br>\n  Basic Augmentations - HV flips, Rotate, Random Brightness Contrast, Cutout, Cutmix, Mixup<br>\n  Image Size - 512x512<br>\n  CV: 0.8304, LB: 0.882</p>\n</blockquote>",
      "votes": null,
      "replies": [
        {
          "id": 1139053,
          "author_name": "clwwlc",
          "author_url": "",
          "post_date": "01/05/2021 06:07:42",
          "content": "<p>how you get higher score than 0.893?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1089218,
      "author_name": "reighns",
      "author_url": "",
      "post_date": "11/24/2020 09:54:17",
      "content": "<p>Normal Train-Test-Split:<br>\nEffnet_b3_ns<br>\nImg_size: 512<br>\nLB 0.889<br>\nNo TTA</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1111629,
      "author_name": "",
      "author_url": "",
      "post_date": "12/13/2020 21:27:24",
      "content": "<p>512<br>\nTTA<br>\nAUG<br>\n0.901 single EfficientNet B3_ns<br>\n0.901 single EfficientNet B3 imgnet</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1111743,
      "author_name": "luqing2",
      "author_url": "",
      "post_date": "12/14/2020 02:26:13",
      "content": "<p>Sorry for my ignorance, just started on ML.     I can infer that the LB stands for leader board score.  I know in certain context, CV stands for computer vision.  I know many people talk about CV here.   What does CV stands for?  is it the validation accuracy? why not called acc?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1111746,
          "author_name": "keremt",
          "author_url": "",
          "post_date": "12/14/2020 02:29:11",
          "content": "<p>Here usually CV refers to local cross validation score, but probably if you report only single fold score it's not cross validation anymore but just local validation score. It's important to have a local validation schema which aligns with public leaderboard in order to understand what really works and generalizes.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1111855,
          "author_name": "luqing2",
          "author_url": "",
          "post_date": "12/14/2020 04:48:49",
          "content": "<p>Thanks for the reply.  I use TF/Keras data_generator. There does not seems a easy way to implement multiple fold cross validation.   I got very high single fold scores, but the LB score is many points lower.   If anyone has suggestion or examples on implementting multi fold cross validation using Keras data generator,  I would be appreciated. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1113726,
      "author_name": "vijayabhaskar96",
      "author_url": "",
      "post_date": "12/15/2020 16:40:58",
      "content": "<p>Small doubt, when you guys report a single model score, you mean a k-fold mean score or just a 1-fold mode score? and when you submit you would perform K-fold? I can understand running traditional ML models like Random forests with K-Fold, but training a Deep learning model for 5 fold seems too time-consuming.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1136938,
      "author_name": "clwwlc",
      "author_url": "",
      "post_date": "01/03/2021 14:51:47",
      "content": "<p>resnet50, 512x512, 10epoch, Basic Data Augmentation - FLIP, Rotate, Random Brightness Contrast..  <br>\nsingle fold, no TTA<br>\nCV: 0.893  LB:0.891</p>\n<p>eff-b4<br>\nCV:0.896 LB:0.891</p>\n<p>eff-b2<br>\nCV:0.895 LB:0.886</p>\n<p>I can't get score higher than 0.891 when single fold, no TTA……could anybody give me some suggestions?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1139661,
          "author_name": "yannmajewski",
          "author_url": "",
          "post_date": "01/05/2021 14:53:12",
          "content": "<p>You must train all the folds to get your real CV, i had the same thing at the begining of the comp. Some folds will score much lower than others</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1140575,
          "author_name": "clwwlc",
          "author_url": "",
          "post_date": "01/06/2021 05:44:10",
          "content": "<p>but 5 fold cost too much time, and the highest of them, or  5fold ensemble, or top3 ensemble, is still not good.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1140608,
      "author_name": "thanish",
      "author_url": "",
      "post_date": "01/06/2021 06:03:21",
      "content": "<p>Can you guys also mention how long you train each fold. If you are using early stopping, which epoch gives the best accuracy or eval loss in each fold and are you guys using Early stopping on eval accuracy eval loss ?<br>\nThe main reason I'm asking this is I'm not getting better eval loss/accuracy beyond the first epoch even with early stopping of 15. Would be glad to know what others are doing. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1170289,
      "author_name": "leonshangguan",
      "author_url": "",
      "post_date": "01/26/2021 05:42:05",
      "content": "<p>resnet34</p>\n<p>cv: 0.888<br>\nlb: 0.879</p>\n<p>512*512, single fold.<br>\nno tta, basic data augmentation (flip, shift scale rotate)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1087823": "Starting a new thread. :) \n\nI'll go first.. \n\n```\nEfficientNet-B4 - single fold\nNo TTA\nBasic Data Augmentation - FLIP, Rotate, Random Brightness Contrast.. \nImage Size - 512x512\nCV: 0.895, LB: 0.890 \n```",
    "1087838": "EfficientNet-B1 - Single Fold\nNo TTA\nData Augmentation - Hflip, Vflip, Transpose, RandomCrop, ShiftScaleRotate\nImage Size - 384x384\nCV: 0.89, LB: 0.885\n\nEfficientNet-B0 - Single Fold\nNo TTA\nData Augmentation - Hflip, Vflip, Transpose, RandomCrop, ShiftScaleRotate\nImage Size - 512x512\nCV: 0.891, LB: ?",
    "1087873": "resnext50\n8TTA\n448*448\nCV 0.892\nLB 0.895\n\nupdate1:\nstrong aug\nCV 0.889\nLB 0.896\n\nupdate2:\nCV 0.896\nLB 0.899",
    "1087922": "Nice! BTW what is the meaning of 8TTA, 4TTA etc.?",
    "1087926": "kaushal2896 \n\nBasically - TTA stands for **Test Time Augmentation**. Let's say you are trying to predict the masks for a single image. In a simple case you would center crop the test image and make a prediction and that's pretty much it. This is your final prediction. \n\nBut with TTA - you add augmentation to the test image and create let's say `N` number of augmented copies that are similar to the image but yet different. The augmentations could be rotate, flip, etc.. \n\nNext, you make a prediction for all these N number of images and take the average as your final prediction for the test image. \n\n`N` is what refers to 4,8, etc.. 8 means someone is creating 8 copies of the test image using random augmentations and then taking the average as final prediction, 4 means creating 4 copies and so on..",
    "1087931": "aroraaman Thanks for detailed explanation. Actually I'm using TTA but wasn't aware of this terminology!",
    "1088049": "5 Fold CV\nResnext 50 \nData Augmentation - Yes\nImage Size 512*512\nCV 0.887\nLB 0.899 \n\nUpdate:\nCV 0.889\nLB 0.901\n\nNotebook is public",
    "1088053": "whoa! No TTA or advanced augmentations?",
    "1088064": "I haven't tried TTA till now... Just focusing on right image size/architecture for now...",
    "1088245": "Resnest50\n5 fold\n384x288\nCutMix + basic augs\nNo TTA\nCV: 0.88948\nLB: 0.896",
    "1088484": "> SE_ResNeXt50_32x4d\n16TTA\nBasic Augmentations - HV flips, Rotate, Random Brightness Contrast, Cutout, Cutmix, Mixup\nImage Size - 512x512\nCV: 0.8750, LB: 0.8930\n\n> ResNet-18\n16TTA\nBasic Augmentations - HV flips, Rotate, Random Brightness Contrast, Cutout, Cutmix, Mixup\nImage Size - 512x512\nCV: 0.8304, LB: 0.882",
    "1088499": "Nice! I can not cross 0.893 even with TTA!",
    "1089218": "Normal Train-Test-Split:\nEffnet_b3_ns\nImg_size: 512\nLB 0.889\nNo TTA",
    "1111629": "512\nTTA\nAUG\n0.901 single EfficientNet B3_ns\n0.901 single EfficientNet B3 imgnet",
    "1111743": "Sorry for my ignorance, just started on ML.     I can infer that the LB stands for leader board score.  I know in certain context, CV stands for computer vision.  I know many people talk about CV here.   What does CV stands for?  is it the validation accuracy? why not called acc?",
    "1111746": "Here usually CV refers to local cross validation score, but probably if you report only single fold score it's not cross validation anymore but just local validation score. It's important to have a local validation schema which aligns with public leaderboard in order to understand what really works and generalizes.",
    "1111855": "Thanks for the reply.  I use TF/Keras data_generator. There does not seems a easy way to implement multiple fold cross validation.   I got very high single fold scores, but the LB score is many points lower.   If anyone has suggestion or examples on implementting multi fold cross validation using Keras data generator,  I would be appreciated.",
    "1113726": "Small doubt, when you guys report a single model score, you mean a k-fold mean score or just a 1-fold mode score? and when you submit you would perform K-fold? I can understand running traditional ML models like Random forests with K-Fold, but training a Deep learning model for 5 fold seems too time-consuming.",
    "1136938": "resnet50, 512x512, 10epoch, Basic Data Augmentation - FLIP, Rotate, Random Brightness Contrast..  \nsingle fold, no TTA\nCV: 0.893  LB:0.891\n\neff-b4\nCV:0.896 LB:0.891\n\neff-b2\nCV:0.895 LB:0.886\n\nI can't get score higher than 0.891 when single fold, no TTA......could anybody give me some suggestions?",
    "1139053": "how you get higher score than 0.893?",
    "1139661": "You must train all the folds to get your real CV, i had the same thing at the begining of the comp. Some folds will score much lower than others",
    "1140575": "but 5 fold cost too much time, and the highest of them, or  5fold ensemble, or top3 ensemble, is still not good.",
    "1140608": "Can you guys also mention how long you train each fold. If you are using early stopping, which epoch gives the best accuracy or eval loss in each fold and are you guys using Early stopping on eval accuracy eval loss ?\nThe main reason I'm asking this is I'm not getting better eval loss/accuracy beyond the first epoch even with early stopping of 15. Would be glad to know what others are doing.",
    "1155840": "Hi, @aroraaman ! Do you mean that the copies are created with different augmentations or with the same augmentation but with different parameters?",
    "1170289": "resnet34\n\ncv: 0.888\nlb: 0.879\n\n512*512, single fold.\nno tta, basic data augmentation (flip, shift scale rotate)"
  },
  "source": "meta"
}