{
  "id": 298514,
  "title": "No ensemble No knowledge distillation, what's your LB score?",
  "url": "/competitions/tensorflow-great-barrier-reef/discussion/298514",
  "author_name": "Chenglu",
  "post_date": "2022-01-03T09:29:03.745000",
  "votes": 29,
  "comment_count": 49,
  "views": 0,
  "content": "<p>I wander how long can we go with just one model without any ensemble method.</p>\n<p>for me:</p>\n<table>\n<thead>\n<tr>\n<th>method</th>\n<th>Local</th>\n<th>LB</th>\n<th>TTA</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>MMDetection Swin Transformer</td>\n<td>0.4747</td>\n<td>0.502</td>\n<td>No</td>\n</tr>\n<tr>\n<td>MMDetection HRNet</td>\n<td>0.5013</td>\n<td>0.555</td>\n<td>No</td>\n</tr>\n<tr>\n<td>PyTorch FasterRCNN</td>\n<td>-</td>\n<td>0.469</td>\n<td>No</td>\n</tr>\n<tr>\n<td>YOLOv5</td>\n<td>-</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>YOLOx</td>\n<td>-</td>\n<td>-</td>\n<td>-</td>\n</tr>\n</tbody>\n</table>\n<p>The local score is obtained by one model trained on video_0 and video_2, evaluate on video_1.<br>\nThe LB score is obtained by one model that trained on the whole datasets.</p>",
  "messages": [
    {
      "id": 1636801,
      "postDate": "2022-01-03T09:29:03.747Z",
      "content": "<p>I wander how long can we go with just one model without any ensemble method.</p>\n<p>for me:</p>\n<table>\n<thead>\n<tr>\n<th>method</th>\n<th>Local</th>\n<th>LB</th>\n<th>TTA</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>MMDetection Swin Transformer</td>\n<td>0.4747</td>\n<td>0.502</td>\n<td>No</td>\n</tr>\n<tr>\n<td>MMDetection HRNet</td>\n<td>0.5013</td>\n<td>0.555</td>\n<td>No</td>\n</tr>\n<tr>\n<td>PyTorch FasterRCNN</td>\n<td>-</td>\n<td>0.469</td>\n<td>No</td>\n</tr>\n<tr>\n<td>YOLOv5</td>\n<td>-</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>YOLOx</td>\n<td>-</td>\n<td>-</td>\n<td>-</td>\n</tr>\n</tbody>\n</table>\n<p>The local score is obtained by one model trained on video_0 and video_2, evaluate on video_1.<br>\nThe LB score is obtained by one model that trained on the whole datasets.</p>",
      "rawMarkdown": "I wander how long can we go with just one model without any ensemble method.\n\nfor me:\n\n|  method | Local |  LB  | TTA |\n| --- | --- | --- | --- |\n| MMDetection Swin Transformer | 0.4747 | 0.502 | No |\n| MMDetection HRNet |  0.5013 | 0.555 | No |\n| PyTorch FasterRCNN | - | 0.469 | No |\n| YOLOv5 |  -| - | - | \n| YOLOx | - | - | - |\n\nThe local score is obtained by one model trained on video_0 and video_2, evaluate on video_1.\nThe LB score is obtained by one model that trained on the whole datasets.\n\n",
      "votes": 29
    },
    {
      "id": 1638333,
      "postDate": "2022-01-04T16:16:52.753Z",
      "content": "<p>Yolov5 1 fold :<br>\nCV: 0.62<br>\nLB: 0.61</p>",
      "rawMarkdown": "Yolov5 1 fold :\nCV: 0.62\nLB: 0.61",
      "votes": 7,
      "replies": [
        {
          "id": 1638468,
          "postDate": "2022-01-04T18:45:28.077Z",
          "content": "<p>That's honestly amazing. YoloX is said to perform better than Yolov5 on COCO, and yet I can't get YoloX to even come close to your score. </p>",
          "rawMarkdown": "That's honestly amazing. YoloX is said to perform better than Yolov5 on COCO, and yet I can't get YoloX to even come close to your score. ",
          "votes": 2
        },
        {
          "id": 1638637,
          "postDate": "2022-01-04T22:18:36.927Z",
          "content": "<p>Thanks for sharing, that is absolutely inspiring</p>",
          "rawMarkdown": "Thanks for sharing, that is absolutely inspiring",
          "votes": 1
        },
        {
          "id": 1639131,
          "postDate": "2022-01-05T12:04:38.830Z",
          "content": "<p><a href=\"https://www.kaggle.com/matthieuplante\" target=\"_blank\">@matthieuplante</a> which fold and split are you using?</p>",
          "rawMarkdown": "@matthieuplante which fold and split are you using?",
          "votes": 5
        },
        {
          "id": 1642378,
          "postDate": "2022-01-08T10:05:25.320Z",
          "content": "<p>Are you sure there was no leak while cross validation? YOLO is typically giving lower CV than Public LB </p>",
          "rawMarkdown": "Are you sure there was no leak while cross validation? YOLO is typically giving lower CV than Public LB "
        },
        {
          "id": 1645141,
          "postDate": "2022-01-10T18:06:06.970Z",
          "content": "<p><a href=\"https://www.kaggle.com/matthieuplante\" target=\"_blank\">@matthieuplante</a> amazing job! is the CV score calculated with all images (labeled + unlabeled) or just unlabeled ?  </p>",
          "rawMarkdown": "@matthieuplante amazing job! is the CV score calculated with all images (labeled + unlabeled) or just unlabeled ?  "
        }
      ]
    },
    {
      "id": 1642788,
      "postDate": "2022-01-08T17:05:25.547Z",
      "content": "<p>YoloX 1 fold (5 fold split)<br>\nCV - 0.520<br>\nLB - 0.556<br>\nLB with <a href=\"https://www.kaggle.com/parapapapam/yolox-inference-tracking-on-cots-lb-0-539\" target=\"_blank\">tracking</a> - 0.563</p>",
      "rawMarkdown": "YoloX 1 fold (5 fold split)\nCV - 0.520\nLB - 0.556\nLB with [tracking](https://www.kaggle.com/parapapapam/yolox-inference-tracking-on-cots-lb-0-539) - 0.563",
      "votes": 3,
      "replies": [
        {
          "id": 1642798,
          "postDate": "2022-01-08T17:24:37.557Z",
          "content": "<p>This is with yolox default Hyperparmeter? <br>\nWhat img size you used while model training? </p>",
          "rawMarkdown": "This is with yolox default Hyperparmeter? \nWhat img size you used while model training? "
        },
        {
          "id": 1642805,
          "postDate": "2022-01-08T17:31:26.043Z",
          "content": "<p><a href=\"https://www.kaggle.com/anshulkhadse\" target=\"_blank\">@anshulkhadse</a> No, this is with modified hyperparameters, optimizer and lr scheduler. <br>\nImage size is 960x1280.</p>",
          "rawMarkdown": "@anshulkhadse No, this is with modified hyperparameters, optimizer and lr scheduler. \nImage size is 960x1280.\n\n"
        },
        {
          "id": 1643082,
          "postDate": "2022-01-09T03:52:14.700Z",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> would u like to team up?</p>",
          "rawMarkdown": "@yovinyahathugoda would u like to team up?"
        },
        {
          "id": 1643107,
          "postDate": "2022-01-09T04:36:36.407Z",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> </p>\n<p>1) Which validation strategy do you use - Subseq, Sequence Group fold, video split? <br>\n2) Do you use background images in your training and validation set? </p>",
          "rawMarkdown": "@yovinyahathugoda \n\n1) Which validation strategy do you use - Subseq, Sequence Group fold, video split? \n2) Do you use background images in your training and validation set? "
        },
        {
          "id": 1643403,
          "postDate": "2022-01-09T11:16:39.243Z",
          "content": "<p><a href=\"https://www.kaggle.com/anshulkhadse\" target=\"_blank\">@anshulkhadse</a> replied to your team up request email</p>",
          "rawMarkdown": "@anshulkhadse replied to your team up request email",
          "votes": 1
        },
        {
          "id": 1643405,
          "postDate": "2022-01-09T11:18:02.410Z",
          "content": "<p><a href=\"https://www.kaggle.com/garvitgarg\" target=\"_blank\">@garvitgarg</a> train/validation split is based on subsequences. I have not used background images yet</p>",
          "rawMarkdown": "@garvitgarg train/validation split is based on subsequences. I have not used background images yet"
        },
        {
          "id": 1643469,
          "postDate": "2022-01-09T12:39:46.450Z",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> which fold did u use? and splitting by sequence_id did u do u from urself or use from this link <a href=\"url\" target=\"_blank\">https://www.kaggle.com/julian3833/reef-a-cv-strategy-subsequences?scriptVersionId=80623179</a> CV fold?</p>",
          "rawMarkdown": "@yovinyahathugoda which fold did u use? and splitting by sequence_id did u do u from urself or use from this link [https://www.kaggle.com/julian3833/reef-a-cv-strategy-subsequences?scriptVersionId=80623179](url) CV fold?"
        },
        {
          "id": 1643481,
          "postDate": "2022-01-09T12:55:39.460Z",
          "content": "<p><a href=\"https://www.kaggle.com/anshulkhadse\" target=\"_blank\">@anshulkhadse</a> the subsequence ids were from the notebook in the link you posted, but I split the data later in the training notebook separately. I only used fold 0 for validation for now.</p>",
          "rawMarkdown": "@anshulkhadse the subsequence ids were from the notebook in the link you posted, but I split the data later in the training notebook separately. I only used fold 0 for validation for now.",
          "votes": 1
        },
        {
          "id": 1643560,
          "postDate": "2022-01-09T14:30:39.877Z",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> Understand ,thanks for sharing</p>",
          "rawMarkdown": "@yovinyahathugoda Understand ,thanks for sharing"
        },
        {
          "id": 1644195,
          "postDate": "2022-01-10T06:02:06.840Z",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> In CV, What metrics do you use f2score? If you don't mind, can you share code snippet for f2 score? My CV/LB gap is so high. 😄</p>",
          "rawMarkdown": "@yovinyahathugoda In CV, What metrics do you use f2score? If you don't mind, can you share code snippet for f2 score? My CV/LB gap is so high. 😄"
        },
        {
          "id": 1645210,
          "postDate": "2022-01-10T18:51:24.280Z",
          "content": "<p><a href=\"https://www.kaggle.com/tanjirooo\" target=\"_blank\">@tanjirooo</a> I use f2 metric for validation at the end of training. I used the code in the discussion <a href=\"https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/290757#1595988\" target=\"_blank\">here</a></p>",
          "rawMarkdown": "@tanjirooo I use f2 metric for validation at the end of training. I used the code in the discussion [here](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/290757#1595988)",
          "votes": 1
        }
      ]
    },
    {
      "id": 1638002,
      "postDate": "2022-01-04T11:12:34.303Z",
      "content": "<p>The maximum LB for my single model is 0.547 (Yolov5, No ensemble No knowledge distillation, No TTA).<br>\nBut there is no certainty that this is compatible with Private LB.</p>",
      "rawMarkdown": "The maximum LB for my single model is 0.547 (Yolov5, No ensemble No knowledge distillation, No TTA).\nBut there is no certainty that this is compatible with Private LB.",
      "votes": 1,
      "replies": [
        {
          "id": 1638287,
          "postDate": "2022-01-04T15:42:33.793Z",
          "content": "<p>use TTA actually given lower LB score</p>",
          "rawMarkdown": "use TTA actually given lower LB score",
          "votes": 1
        },
        {
          "id": 1638344,
          "postDate": "2022-01-04T16:27:50.510Z",
          "content": "<p>me too.<br>\nI'm wondering why TTA doesn't work.</p>",
          "rawMarkdown": "me too.\nI'm wondering why TTA doesn't work."
        },
        {
          "id": 1639165,
          "postDate": "2022-01-05T12:45:34.740Z",
          "content": "<p><a href=\"https://www.kaggle.com/syurenuko\" target=\"_blank\">@syurenuko</a>, which folding method and split are you using?</p>",
          "rawMarkdown": "@syurenuko, which folding method and split are you using?",
          "votes": 1
        },
        {
          "id": 1639425,
          "postDate": "2022-01-05T16:12:24.463Z",
          "content": "<p><a href=\"https://www.kaggle.com/garvitgarg\" target=\"_blank\">@garvitgarg</a> <br>\nI used GroupKFold (n = 5, groups = sequence).</p>",
          "rawMarkdown": "@garvitgarg \nI used GroupKFold (n = 5, groups = sequence).",
          "votes": 1
        },
        {
          "id": 1642564,
          "postDate": "2022-01-08T13:57:14.707Z",
          "content": "<p>which split did you used to get that</p>",
          "rawMarkdown": "which split did you used to get that",
          "votes": 1
        },
        {
          "id": 1642593,
          "postDate": "2022-01-08T14:13:28.307Z",
          "content": "<p><a href=\"https://www.kaggle.com/dineshmanikanta\" target=\"_blank\">@dineshmanikanta</a> <br>\nThere are 5 folds from 0 to 4.<br>\nUse fold = 4 for validation.</p>",
          "rawMarkdown": "@dineshmanikanta \nThere are 5 folds from 0 to 4.\nUse fold = 4 for validation."
        },
        {
          "id": 1642617,
          "postDate": "2022-01-08T14:34:00.843Z",
          "rawMarkdown": "",
          "votes": 1,
          "isDeleted": true
        },
        {
          "id": 1642622,
          "postDate": "2022-01-08T14:38:37.687Z",
          "content": "<p><a href=\"https://www.kaggle.com/shivanigarg32\" target=\"_blank\">@shivanigarg32</a><br>\nbackground images are not used.</p>",
          "rawMarkdown": "@shivanigarg32\nbackground images are not used."
        },
        {
          "id": 1642635,
          "postDate": "2022-01-08T14:42:57.297Z",
          "content": "<p><a href=\"https://www.kaggle.com/syurenuko\" target=\"_blank\">@syurenuko</a>, can tell which hyperparameter file u used from the available yolov5 repo in the data folder?</p>",
          "rawMarkdown": "@syurenuko, can tell which hyperparameter file u used from the available yolov5 repo in the data folder?"
        }
      ]
    },
    {
      "id": 1659336,
      "postDate": "2022-01-21T18:02:15.103Z",
      "content": "<p>FYI, there is a transformer version of HRNet: <a href=\"https://github.com/HRNet/HRFormer\" target=\"_blank\">https://github.com/HRNet/HRFormer</a></p>",
      "rawMarkdown": "FYI, there is a transformer version of HRNet: https://github.com/HRNet/HRFormer"
    },
    {
      "id": 1645776,
      "postDate": "2022-01-11T08:05:32.190Z",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/snaker\" target=\"_blank\">@snaker</a> . What do you mean for the \"knowledge distillation\" in this problem?</p>",
      "rawMarkdown": "Hi, @snaker . What do you mean for the \"knowledge distillation\" in this problem?",
      "replies": [
        {
          "id": 1645795,
          "postDate": "2022-01-11T08:19:23.263Z",
          "content": "<p>Not much we can do about this cause we are lack of test set, i just mention it because it's still a ensemling method.</p>",
          "rawMarkdown": "Not much we can do about this cause we are lack of test set, i just mention it because it's still a ensemling method."
        }
      ]
    },
    {
      "id": 1644022,
      "postDate": "2022-01-10T01:12:05.210Z",
      "content": "<p>cas r50 1 fold(5 fold split)<br>\nCV - 66.0<br>\nLB - 0.579<br>\nno tracking no tta</p>",
      "rawMarkdown": "cas r50 1 fold(5 fold split)\nCV - 66.0\nLB - 0.579\nno tracking no tta",
      "replies": [
        {
          "id": 1647797,
          "postDate": "2022-01-12T23:26:47.917Z",
          "content": "<p>is this using mmdet cascade or detectron2?</p>",
          "rawMarkdown": "is this using mmdet cascade or detectron2?\n"
        }
      ]
    },
    {
      "id": 1641440,
      "postDate": "2022-01-07T12:51:15.330Z",
      "content": "<p>model: yolov5-l<br>\nsplit: video_id<br>\nCV: 0.642<br>\nLB: 0.522</p>",
      "rawMarkdown": "model: yolov5-l\nsplit: video_id\nCV: 0.642\nLB: 0.522"
    },
    {
      "id": 1639914,
      "postDate": "2022-01-06T03:30:17.367Z",
      "content": "<p>Amazing results for hrnet, which backbone did you use, HR-W18, HR-W32 or W40? <br>\nFor mmdetection, I only get 0.47 in LB with Faster R-CNN and 0.46 with Cascade R-CNN. </p>",
      "rawMarkdown": "Amazing results for hrnet, which backbone did you use, HR-W18, HR-W32 or W40? \nFor mmdetection, I only get 0.47 in LB with Faster R-CNN and 0.46 with Cascade R-CNN. ",
      "replies": [
        {
          "id": 1640030,
          "postDate": "2022-01-06T06:05:36.397Z",
          "content": "<p>HR-W18 ，the smallest one</p>",
          "rawMarkdown": "HR-W18 ，the smallest one"
        },
        {
          "id": 1641689,
          "postDate": "2022-01-07T16:28:54.900Z",
          "content": "<p>Do you use single model cascade HR-W18 to get highest score? Thanks! Now I use yolov5, but the result is unstable in both CV and LB.</p>",
          "rawMarkdown": "Do you use single model cascade HR-W18 to get highest score? Thanks! Now I use yolov5, but the result is unstable in both CV and LB."
        },
        {
          "id": 1642089,
          "postDate": "2022-01-08T03:29:05.903Z",
          "content": "<p>And do you use F2 metric for Local ?</p>",
          "rawMarkdown": "And do you use F2 metric for Local ?"
        },
        {
          "id": 1643138,
          "postDate": "2022-01-09T05:13:31.713Z",
          "content": "<p>yes, all the metric is F2, not mAP</p>",
          "rawMarkdown": "yes, all the metric is F2, not mAP"
        },
        {
          "id": 1643985,
          "postDate": "2022-01-09T23:15:06.247Z",
          "content": "<p>did you ever encounter <code>loss_rpn_cls: nan, loss_rpn_bbox: nan, loss_cls: nan, acc: 100.0000, loss_bbox: nan, loss: nan</code> while setting up your training?</p>",
          "rawMarkdown": "did you ever encounter `loss_rpn_cls: nan, loss_rpn_bbox: nan, loss_cls: nan, acc: 100.0000, loss_bbox: nan, loss: nan` while setting up your training?"
        },
        {
          "id": 1644013,
          "postDate": "2022-01-10T00:48:12.797Z",
          "content": "<p>figured it out, torch 1.9.0 is REQUIRED anything higher will give issues during training ig</p>",
          "rawMarkdown": "figured it out, torch 1.9.0 is REQUIRED anything higher will give issues during training ig"
        }
      ]
    },
    {
      "id": 1639893,
      "postDate": "2022-01-06T03:10:17.090Z",
      "content": "<p>Hi snaker, As you said you finally trained your model on the whole datasets, I wonder how you select the conf_threshold and nms_threshold. Thankyou.</p>",
      "rawMarkdown": "Hi snaker, As you said you finally trained your model on the whole datasets, I wonder how you select the conf_threshold and nms_threshold. Thankyou.",
      "replies": [
        {
          "id": 1640037,
          "postDate": "2022-01-06T06:11:50.700Z",
          "content": "<p>The dataset is hard to split, so i just split it by video_id, video_1 for validation and video0, video2 for training, but this is <em>just for</em> experiments.<br>\nAfter I get a higher score in local i will use the same training set and strategy for the whole dataset, the thresholds is get from the experiment, which is video1 actually.</p>",
          "rawMarkdown": "The dataset is hard to split, so i just split it by video_id, video_1 for validation and video0, video2 for training, but this is *just for* experiments.\nAfter I get a higher score in local i will use the same training set and strategy for the whole dataset, the thresholds is get from the experiment, which is video1 actually."
        },
        {
          "id": 1641039,
          "postDate": "2022-01-07T04:49:14.493Z",
          "content": "<p>\" i just split it by video_id, video_1 for validation and video0, video2 for training,\"</p>\n<p>These results may be biased.<br>\nif you use video_0 for validation, your LB score will be different</p>",
          "rawMarkdown": "\" i just split it by video_id, video_1 for validation and video0, video2 for training,\"\n\nThese results may be biased.\nif you use video\\_0 for validation, your LB score will be different",
          "votes": 1
        },
        {
          "id": 1641056,
          "postDate": "2022-01-07T05:08:56.803Z",
          "content": "<p>yes, i have tested on eighter the three of them as validation set, the gap is large(so is the LB), and the video_1 is the lowest so i chose it finally</p>",
          "rawMarkdown": "yes, i have tested on eighter the three of them as validation set, the gap is large(so is the LB), and the video_1 is the lowest so i chose it finally"
        },
        {
          "id": 1642344,
          "postDate": "2022-01-08T09:37:07.903Z",
          "content": "<p>\"the gap is large(so is the LB),\"</p>\n<p>if i apply different augmentation to different dataset, I get better results. but I am not sure if it is because of the difference in the augmented raining set or the difference in the validation set.</p>",
          "rawMarkdown": "\"the gap is large(so is the LB),\"\n\nif i apply different augmentation to different dataset, I get better results. but I am not sure if it is because of the difference in the augmented raining set or the difference in the validation set.",
          "votes": 1
        },
        {
          "id": 1642457,
          "postDate": "2022-01-08T12:01:52.837Z",
          "content": "<blockquote>\n  <p>and the video_1 is the lowest so i chose it finally</p>\n</blockquote>\n<p>I don’t think this is correct strategy. It might cause over fitting to public LB.</p>",
          "rawMarkdown": "> and the video_1 is the lowest so i chose it finally\n\nI don’t think this is correct strategy. It might cause over fitting to public LB."
        },
        {
          "id": 1643139,
          "postDate": "2022-01-09T05:14:47.627Z",
          "content": "<p>we need a solid method to evaluate model in local, this is i think <strong>the best</strong> way to do it. cause the LB and private LB could be a totally another video sequence.</p>\n<p>I'm not using the split to do model selection BTW, just for local experiment evaluation.</p>",
          "rawMarkdown": "we need a solid method to evaluate model in local, this is i think **the best** way to do it. cause the LB and private LB could be a totally another video sequence.\n\nI'm not using the split to do model selection BTW, just for local experiment evaluation."
        },
        {
          "id": 1644258,
          "postDate": "2022-01-10T07:07:57.813Z",
          "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> yes, same here. I saw your comments in another thread. And I think what you suggest is correct and we are doing it now: \"to find the augmentation that minimize the gap between the 3 videos\".</p>",
          "rawMarkdown": "@hengck23 yes, same here. I saw your comments in another thread. And I think what you suggest is correct and we are doing it now: \"to find the augmentation that minimize the gap between the 3 videos\"."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1638333,
      "author_name": "Matthieu Planté",
      "author_url": "",
      "post_date": "2022-01-04T16:16:52.753000",
      "content": "<p>Yolov5 1 fold :<br>\nCV: 0.62<br>\nLB: 0.61</p>",
      "votes": 7,
      "replies": [
        {
          "id": 1638468,
          "author_name": "Lime-Cake",
          "author_url": "",
          "post_date": "2022-01-04T18:45:28.077000",
          "content": "<p>That's honestly amazing. YoloX is said to perform better than Yolov5 on COCO, and yet I can't get YoloX to even come close to your score. </p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1638637,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2022-01-04T22:18:36.927000",
          "content": "<p>Thanks for sharing, that is absolutely inspiring</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1639131,
          "author_name": "Awsaf",
          "author_url": "",
          "post_date": "2022-01-05T12:04:38.830000",
          "content": "<p><a href=\"https://www.kaggle.com/matthieuplante\" target=\"_blank\">@matthieuplante</a> which fold and split are you using?</p>",
          "votes": 5,
          "replies": []
        },
        {
          "id": 1642378,
          "author_name": "Hannah B",
          "author_url": "",
          "post_date": "2022-01-08T10:05:25.320000",
          "content": "<p>Are you sure there was no leak while cross validation? YOLO is typically giving lower CV than Public LB </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1645141,
          "author_name": "Ioannis M",
          "author_url": "",
          "post_date": "2022-01-10T18:06:06.970000",
          "content": "<p><a href=\"https://www.kaggle.com/matthieuplante\" target=\"_blank\">@matthieuplante</a> amazing job! is the CV score calculated with all images (labeled + unlabeled) or just unlabeled ?  </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1642788,
      "author_name": "Yovin Yahathugoda",
      "author_url": "",
      "post_date": "2022-01-08T17:05:25.547000",
      "content": "<p>YoloX 1 fold (5 fold split)<br>\nCV - 0.520<br>\nLB - 0.556<br>\nLB with <a href=\"https://www.kaggle.com/parapapapam/yolox-inference-tracking-on-cots-lb-0-539\" target=\"_blank\">tracking</a> - 0.563</p>",
      "votes": 3,
      "replies": [
        {
          "id": 1642798,
          "author_name": "Anshul Khadse",
          "author_url": "",
          "post_date": "2022-01-08T17:24:37.557000",
          "content": "<p>This is with yolox default Hyperparmeter? <br>\nWhat img size you used while model training? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1642805,
          "author_name": "Yovin Yahathugoda",
          "author_url": "",
          "post_date": "2022-01-08T17:31:26.043000",
          "content": "<p><a href=\"https://www.kaggle.com/anshulkhadse\" target=\"_blank\">@anshulkhadse</a> No, this is with modified hyperparameters, optimizer and lr scheduler. <br>\nImage size is 960x1280.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1643082,
          "author_name": "Anshul Khadse",
          "author_url": "",
          "post_date": "2022-01-09T03:52:14.700000",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> would u like to team up?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1643107,
          "author_name": "Garvit Garg",
          "author_url": "",
          "post_date": "2022-01-09T04:36:36.407000",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> </p>\n<p>1) Which validation strategy do you use - Subseq, Sequence Group fold, video split? <br>\n2) Do you use background images in your training and validation set? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1643403,
          "author_name": "Yovin Yahathugoda",
          "author_url": "",
          "post_date": "2022-01-09T11:16:39.243000",
          "content": "<p><a href=\"https://www.kaggle.com/anshulkhadse\" target=\"_blank\">@anshulkhadse</a> replied to your team up request email</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1643405,
          "author_name": "Yovin Yahathugoda",
          "author_url": "",
          "post_date": "2022-01-09T11:18:02.410000",
          "content": "<p><a href=\"https://www.kaggle.com/garvitgarg\" target=\"_blank\">@garvitgarg</a> train/validation split is based on subsequences. I have not used background images yet</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1643469,
          "author_name": "Anshul Khadse",
          "author_url": "",
          "post_date": "2022-01-09T12:39:46.450000",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> which fold did u use? and splitting by sequence_id did u do u from urself or use from this link <a href=\"url\" target=\"_blank\">https://www.kaggle.com/julian3833/reef-a-cv-strategy-subsequences?scriptVersionId=80623179</a> CV fold?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1643481,
          "author_name": "Yovin Yahathugoda",
          "author_url": "",
          "post_date": "2022-01-09T12:55:39.460000",
          "content": "<p><a href=\"https://www.kaggle.com/anshulkhadse\" target=\"_blank\">@anshulkhadse</a> the subsequence ids were from the notebook in the link you posted, but I split the data later in the training notebook separately. I only used fold 0 for validation for now.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1643560,
          "author_name": "Anshul Khadse",
          "author_url": "",
          "post_date": "2022-01-09T14:30:39.877000",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> Understand ,thanks for sharing</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1644195,
          "author_name": "Tanjirooo",
          "author_url": "",
          "post_date": "2022-01-10T06:02:06.840000",
          "content": "<p><a href=\"https://www.kaggle.com/yovinyahathugoda\" target=\"_blank\">@yovinyahathugoda</a> In CV, What metrics do you use f2score? If you don't mind, can you share code snippet for f2 score? My CV/LB gap is so high. 😄</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1645210,
          "author_name": "Yovin Yahathugoda",
          "author_url": "",
          "post_date": "2022-01-10T18:51:24.280000",
          "content": "<p><a href=\"https://www.kaggle.com/tanjirooo\" target=\"_blank\">@tanjirooo</a> I use f2 metric for validation at the end of training. I used the code in the discussion <a href=\"https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/290757#1595988\" target=\"_blank\">here</a></p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1638002,
      "author_name": "syurenuko",
      "author_url": "",
      "post_date": "2022-01-04T11:12:34.303000",
      "content": "<p>The maximum LB for my single model is 0.547 (Yolov5, No ensemble No knowledge distillation, No TTA).<br>\nBut there is no certainty that this is compatible with Private LB.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1638287,
          "author_name": "Phat Tran",
          "author_url": "",
          "post_date": "2022-01-04T15:42:33.793000",
          "content": "<p>use TTA actually given lower LB score</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1638344,
          "author_name": "syurenuko",
          "author_url": "",
          "post_date": "2022-01-04T16:27:50.510000",
          "content": "<p>me too.<br>\nI'm wondering why TTA doesn't work.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1639165,
          "author_name": "Garvit Garg",
          "author_url": "",
          "post_date": "2022-01-05T12:45:34.740000",
          "content": "<p><a href=\"https://www.kaggle.com/syurenuko\" target=\"_blank\">@syurenuko</a>, which folding method and split are you using?</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1639425,
          "author_name": "syurenuko",
          "author_url": "",
          "post_date": "2022-01-05T16:12:24.463000",
          "content": "<p><a href=\"https://www.kaggle.com/garvitgarg\" target=\"_blank\">@garvitgarg</a> <br>\nI used GroupKFold (n = 5, groups = sequence).</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1642564,
          "author_name": "DineshManikanta",
          "author_url": "",
          "post_date": "2022-01-08T13:57:14.707000",
          "content": "<p>which split did you used to get that</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1642593,
          "author_name": "syurenuko",
          "author_url": "",
          "post_date": "2022-01-08T14:13:28.307000",
          "content": "<p><a href=\"https://www.kaggle.com/dineshmanikanta\" target=\"_blank\">@dineshmanikanta</a> <br>\nThere are 5 folds from 0 to 4.<br>\nUse fold = 4 for validation.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1642617,
          "author_name": "",
          "author_url": "",
          "post_date": "2022-01-08T14:34:00.843000",
          "content": "",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1642622,
          "author_name": "syurenuko",
          "author_url": "",
          "post_date": "2022-01-08T14:38:37.687000",
          "content": "<p><a href=\"https://www.kaggle.com/shivanigarg32\" target=\"_blank\">@shivanigarg32</a><br>\nbackground images are not used.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1642635,
          "author_name": "Anshul Khadse",
          "author_url": "",
          "post_date": "2022-01-08T14:42:57.297000",
          "content": "<p><a href=\"https://www.kaggle.com/syurenuko\" target=\"_blank\">@syurenuko</a>, can tell which hyperparameter file u used from the available yolov5 repo in the data folder?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1659336,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2022-01-21T18:02:15.103000",
      "content": "<p>FYI, there is a transformer version of HRNet: <a href=\"https://github.com/HRNet/HRFormer\" target=\"_blank\">https://github.com/HRNet/HRFormer</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1645776,
      "author_name": "Dapeng",
      "author_url": "",
      "post_date": "2022-01-11T08:05:32.190000",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/snaker\" target=\"_blank\">@snaker</a> . What do you mean for the \"knowledge distillation\" in this problem?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1645795,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2022-01-11T08:19:23.263000",
          "content": "<p>Not much we can do about this cause we are lack of test set, i just mention it because it's still a ensemling method.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1644022,
      "author_name": "CarryHJR",
      "author_url": "",
      "post_date": "2022-01-10T01:12:05.210000",
      "content": "<p>cas r50 1 fold(5 fold split)<br>\nCV - 66.0<br>\nLB - 0.579<br>\nno tracking no tta</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1647797,
          "author_name": "Hannah B",
          "author_url": "",
          "post_date": "2022-01-12T23:26:47.917000",
          "content": "<p>is this using mmdet cascade or detectron2?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1641440,
      "author_name": "Garvit Garg",
      "author_url": "",
      "post_date": "2022-01-07T12:51:15.330000",
      "content": "<p>model: yolov5-l<br>\nsplit: video_id<br>\nCV: 0.642<br>\nLB: 0.522</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1639914,
      "author_name": "zhengye",
      "author_url": "",
      "post_date": "2022-01-06T03:30:17.367000",
      "content": "<p>Amazing results for hrnet, which backbone did you use, HR-W18, HR-W32 or W40? <br>\nFor mmdetection, I only get 0.47 in LB with Faster R-CNN and 0.46 with Cascade R-CNN. </p>",
      "votes": 0,
      "replies": [
        {
          "id": 1640030,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2022-01-06T06:05:36.397000",
          "content": "<p>HR-W18 ，the smallest one</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1641689,
          "author_name": "clwclw",
          "author_url": "",
          "post_date": "2022-01-07T16:28:54.900000",
          "content": "<p>Do you use single model cascade HR-W18 to get highest score? Thanks! Now I use yolov5, but the result is unstable in both CV and LB.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1642089,
          "author_name": "clwclw",
          "author_url": "",
          "post_date": "2022-01-08T03:29:05.903000",
          "content": "<p>And do you use F2 metric for Local ?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1643138,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2022-01-09T05:13:31.713000",
          "content": "<p>yes, all the metric is F2, not mAP</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1643985,
          "author_name": "Hannah B",
          "author_url": "",
          "post_date": "2022-01-09T23:15:06.247000",
          "content": "<p>did you ever encounter <code>loss_rpn_cls: nan, loss_rpn_bbox: nan, loss_cls: nan, acc: 100.0000, loss_bbox: nan, loss: nan</code> while setting up your training?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1644013,
          "author_name": "Hannah B",
          "author_url": "",
          "post_date": "2022-01-10T00:48:12.797000",
          "content": "<p>figured it out, torch 1.9.0 is REQUIRED anything higher will give issues during training ig</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1639893,
      "author_name": "Kevin",
      "author_url": "",
      "post_date": "2022-01-06T03:10:17.090000",
      "content": "<p>Hi snaker, As you said you finally trained your model on the whole datasets, I wonder how you select the conf_threshold and nms_threshold. Thankyou.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1640037,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2022-01-06T06:11:50.700000",
          "content": "<p>The dataset is hard to split, so i just split it by video_id, video_1 for validation and video0, video2 for training, but this is <em>just for</em> experiments.<br>\nAfter I get a higher score in local i will use the same training set and strategy for the whole dataset, the thresholds is get from the experiment, which is video1 actually.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1641039,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2022-01-07T04:49:14.493000",
          "content": "<p>\" i just split it by video_id, video_1 for validation and video0, video2 for training,\"</p>\n<p>These results may be biased.<br>\nif you use video_0 for validation, your LB score will be different</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1641056,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2022-01-07T05:08:56.803000",
          "content": "<p>yes, i have tested on eighter the three of them as validation set, the gap is large(so is the LB), and the video_1 is the lowest so i chose it finally</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1642344,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2022-01-08T09:37:07.903000",
          "content": "<p>\"the gap is large(so is the LB),\"</p>\n<p>if i apply different augmentation to different dataset, I get better results. but I am not sure if it is because of the difference in the augmented raining set or the difference in the validation set.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1642457,
          "author_name": "Bilzard",
          "author_url": "",
          "post_date": "2022-01-08T12:01:52.837000",
          "content": "<blockquote>\n  <p>and the video_1 is the lowest so i chose it finally</p>\n</blockquote>\n<p>I don’t think this is correct strategy. It might cause over fitting to public LB.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1643139,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2022-01-09T05:14:47.627000",
          "content": "<p>we need a solid method to evaluate model in local, this is i think <strong>the best</strong> way to do it. cause the LB and private LB could be a totally another video sequence.</p>\n<p>I'm not using the split to do model selection BTW, just for local experiment evaluation.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1644258,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2022-01-10T07:07:57.813000",
          "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> yes, same here. I saw your comments in another thread. And I think what you suggest is correct and we are doing it now: \"to find the augmentation that minimize the gap between the 3 videos\".</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1636801": "I wander how long can we go with just one model without any ensemble method.\n\nfor me:\n\n|  method | Local |  LB  | TTA |\n| --- | --- | --- | --- |\n| MMDetection Swin Transformer | 0.4747 | 0.502 | No |\n| MMDetection HRNet |  0.5013 | 0.555 | No |\n| PyTorch FasterRCNN | - | 0.469 | No |\n| YOLOv5 |  -| - | - | \n| YOLOx | - | - | - |\n\nThe local score is obtained by one model trained on video_0 and video_2, evaluate on video_1.\nThe LB score is obtained by one model that trained on the whole datasets.\n\n",
    "1638333": "Yolov5 1 fold :\nCV: 0.62\nLB: 0.61",
    "1642788": "YoloX 1 fold (5 fold split)\nCV - 0.520\nLB - 0.556\nLB with [tracking](https://www.kaggle.com/parapapapam/yolox-inference-tracking-on-cots-lb-0-539) - 0.563",
    "1638002": "The maximum LB for my single model is 0.547 (Yolov5, No ensemble No knowledge distillation, No TTA).\nBut there is no certainty that this is compatible with Private LB.",
    "1659336": "FYI, there is a transformer version of HRNet: https://github.com/HRNet/HRFormer",
    "1645776": "Hi, @snaker . What do you mean for the \"knowledge distillation\" in this problem?",
    "1644022": "cas r50 1 fold(5 fold split)\nCV - 66.0\nLB - 0.579\nno tracking no tta",
    "1641440": "model: yolov5-l\nsplit: video_id\nCV: 0.642\nLB: 0.522",
    "1639914": "Amazing results for hrnet, which backbone did you use, HR-W18, HR-W32 or W40? \nFor mmdetection, I only get 0.47 in LB with Faster R-CNN and 0.46 with Cascade R-CNN. ",
    "1639893": "Hi snaker, As you said you finally trained your model on the whole datasets, I wonder how you select the conf_threshold and nms_threshold. Thankyou."
  }
}