{
  "id": 307878,
  "title": "Trust CV -- 1st Place Solution",
  "url": "/competitions/tensorflow-great-barrier-reef/discussion/307878",
  "author_name": "Qishen Ha",
  "post_date": "2022-02-16T02:53:25.713000",
  "votes": 192,
  "comment_count": 77,
  "views": 0,
  "content": "<p>Thanks to the organizers and congrats to all the winners and my wonderful teammates <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> and <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> </p>\n<p>This is really very unexpected for us. Because we don't have any NEW THING, we just keep optimizing cross validation F2 of our pipeline locally from the beginning till the end.</p>\n<h1>Summary</h1>\n<p>We designed a 2-stage pipeline. object detection -&gt; classification re-score. Then a post-processing method follows.</p>\n<p>Validation strategy: 3-fold cross validation split by video_id.</p>\n<h2>Object detection</h2>\n<ul>\n<li>6 yolov5 models, 3 trained on 3648 images and 3 trained on 1536 image patches (described below)</li>\n<li>image patches: we cut original image (1280x720) into many patches (512x320), removed boxes near boundary, then only train yolov5 on those patches with cots.</li>\n<li>modified some yolo hyper-parameters based on default: <code>box=0.2</code>, <code>iou_t=0.3</code></li>\n<li>augmentations: based on yolov5 default augmentations, we added: rotation, mixup and albumentations.Transpose, then removed HSV.</li>\n<li>after the optimization was completed by cross validation, we trained the final models with all the data.</li>\n<li>all models are inferred using the same image size as trained.</li>\n<li>ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676.</li>\n</ul>\n<h2>Classification re-score:</h2>\n<ul>\n<li>crop out all predicted boxes (3-fold OOF) into squares wich conf &gt; 0.01. The side length of the square is <code>max(length, width)</code> of the predicted boxes, then extended by 20%.</li>\n<li>we calculate the iou as the maximum of the iou values of each predicted box and GT boxes of this image.</li>\n<li>classification target of each cropped box: iou&gt;0.3, iou&gt;0.4, iou&gt;0.5, iou&gt;0.6, iou&gt;0.7, iou&gt;0.8 and iou&gt;0.9 Simply put, the iou is divided into 7 bins. e.g.: <code>[1,1,1,0,0,0,0]</code> indicates the iou is between 0.5 and 0.6.</li>\n<li>during inference we average 7 bin outputs as classification score.</li>\n<li>then we use BCELoss to train those cropped boxes by size 256x256 or 224x224.</li>\n<li>a very high dropout_rate or drop_path_rate can help a lot to improve the performance of the classification model. We use <code>dropout_rate=0.7</code> and <code>drop_path_rate=0.5</code></li>\n<li>augmentations: hflip, vflip, transpose, 45° rotation and cutout.<br>\nThe best classification model can boost out CV to 0.727</li>\n<li>after ensemble some classification models, our CV comes to 0.73+</li>\n</ul>\n<h2>Post-processing</h2>\n<p>Finally, we use a simple post-processing to further boost our CV to 0.74+.<br>\nFor example, the model has predicted some boxes B at #N frame, select the boxes from B which has a high confidence, these boxes are marked as \"attention area\".<br>\nin the #N+1, #N+2, #N+3 frame, for the predicted boxes with conf &gt; 0.01,  if it has an IoU with the \"attention area\" larger than 0, boost the score of these boxes with <code>score += confidence * IOU</code></p>\n<p>We also tried the tracking method, which gives us a CV of +0.002. However, it introduces two additional hyperparameters. We therefore chose not to use it.</p>\n<h1>Little story</h1>\n<p>At the beginning of the competition, we used different F2 algorithms for each of the three members of our team, and later we found that for the same oof, we did not calculate the same score.<br>\nFor example, nvnn shared an OOF file with <code>F2=0.62</code>, and sheep calculated <code>F2=0.66</code>, while I calculated <code>F2=0.68</code>.<br>\nWe finally chose to use the F2 algorithm with the lowest score from nvnn to evaluate all our models.</p>\n<p><a href=\"https://www.kaggle.com/haqishen/f2-evaluation/script\" target=\"_blank\">https://www.kaggle.com/haqishen/f2-evaluation/script</a></p>\n<p>Here's our final F2 algorithm, if you are interested you can use this algorithm to compare your CV with ours!</p>\n<h1>Acknowledge</h1>\n<p>As usual, I trained many models in this competition using Z by HP Z8G4 Workstation with dual A6000 GPU. The large memory of 48G for a single GPU allowed me to train large resolution images with ease. Thanks to Z by HP for sponsoring!</p>",
  "messages": [
    {
      "id": 1692417,
      "postDate": "2022-02-16T02:53:25.713Z",
      "content": "<p>Thanks to the organizers and congrats to all the winners and my wonderful teammates <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> and <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> </p>\n<p>This is really very unexpected for us. Because we don't have any NEW THING, we just keep optimizing cross validation F2 of our pipeline locally from the beginning till the end.</p>\n<h1>Summary</h1>\n<p>We designed a 2-stage pipeline. object detection -&gt; classification re-score. Then a post-processing method follows.</p>\n<p>Validation strategy: 3-fold cross validation split by video_id.</p>\n<h2>Object detection</h2>\n<ul>\n<li>6 yolov5 models, 3 trained on 3648 images and 3 trained on 1536 image patches (described below)</li>\n<li>image patches: we cut original image (1280x720) into many patches (512x320), removed boxes near boundary, then only train yolov5 on those patches with cots.</li>\n<li>modified some yolo hyper-parameters based on default: <code>box=0.2</code>, <code>iou_t=0.3</code></li>\n<li>augmentations: based on yolov5 default augmentations, we added: rotation, mixup and albumentations.Transpose, then removed HSV.</li>\n<li>after the optimization was completed by cross validation, we trained the final models with all the data.</li>\n<li>all models are inferred using the same image size as trained.</li>\n<li>ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676.</li>\n</ul>\n<h2>Classification re-score:</h2>\n<ul>\n<li>crop out all predicted boxes (3-fold OOF) into squares wich conf &gt; 0.01. The side length of the square is <code>max(length, width)</code> of the predicted boxes, then extended by 20%.</li>\n<li>we calculate the iou as the maximum of the iou values of each predicted box and GT boxes of this image.</li>\n<li>classification target of each cropped box: iou&gt;0.3, iou&gt;0.4, iou&gt;0.5, iou&gt;0.6, iou&gt;0.7, iou&gt;0.8 and iou&gt;0.9 Simply put, the iou is divided into 7 bins. e.g.: <code>[1,1,1,0,0,0,0]</code> indicates the iou is between 0.5 and 0.6.</li>\n<li>during inference we average 7 bin outputs as classification score.</li>\n<li>then we use BCELoss to train those cropped boxes by size 256x256 or 224x224.</li>\n<li>a very high dropout_rate or drop_path_rate can help a lot to improve the performance of the classification model. We use <code>dropout_rate=0.7</code> and <code>drop_path_rate=0.5</code></li>\n<li>augmentations: hflip, vflip, transpose, 45° rotation and cutout.<br>\nThe best classification model can boost out CV to 0.727</li>\n<li>after ensemble some classification models, our CV comes to 0.73+</li>\n</ul>\n<h2>Post-processing</h2>\n<p>Finally, we use a simple post-processing to further boost our CV to 0.74+.<br>\nFor example, the model has predicted some boxes B at #N frame, select the boxes from B which has a high confidence, these boxes are marked as \"attention area\".<br>\nin the #N+1, #N+2, #N+3 frame, for the predicted boxes with conf &gt; 0.01,  if it has an IoU with the \"attention area\" larger than 0, boost the score of these boxes with <code>score += confidence * IOU</code></p>\n<p>We also tried the tracking method, which gives us a CV of +0.002. However, it introduces two additional hyperparameters. We therefore chose not to use it.</p>\n<h1>Little story</h1>\n<p>At the beginning of the competition, we used different F2 algorithms for each of the three members of our team, and later we found that for the same oof, we did not calculate the same score.<br>\nFor example, nvnn shared an OOF file with <code>F2=0.62</code>, and sheep calculated <code>F2=0.66</code>, while I calculated <code>F2=0.68</code>.<br>\nWe finally chose to use the F2 algorithm with the lowest score from nvnn to evaluate all our models.</p>\n<p><a href=\"https://www.kaggle.com/haqishen/f2-evaluation/script\" target=\"_blank\">https://www.kaggle.com/haqishen/f2-evaluation/script</a></p>\n<p>Here's our final F2 algorithm, if you are interested you can use this algorithm to compare your CV with ours!</p>\n<h1>Acknowledge</h1>\n<p>As usual, I trained many models in this competition using Z by HP Z8G4 Workstation with dual A6000 GPU. The large memory of 48G for a single GPU allowed me to train large resolution images with ease. Thanks to Z by HP for sponsoring!</p>",
      "rawMarkdown": "Thanks to the organizers and congrats to all the winners and my wonderful teammates @nvnnghia and @steamedsheep \n\nThis is really very unexpected for us. Because we don't have any NEW THING, we just keep optimizing cross validation F2 of our pipeline locally from the beginning till the end.\n\n# Summary\n\nWe designed a 2-stage pipeline. object detection -> classification re-score. Then a post-processing method follows.\n\nValidation strategy: 3-fold cross validation split by video_id.\n\n## Object detection\n\n* 6 yolov5 models, 3 trained on 3648 images and 3 trained on 1536 image patches (described below)\n* image patches: we cut original image (1280x720) into many patches (512x320), removed boxes near boundary, then only train yolov5 on those patches with cots.\n* modified some yolo hyper-parameters based on default: `box=0.2`, `iou_t=0.3`\n* augmentations: based on yolov5 default augmentations, we added: rotation, mixup and albumentations.Transpose, then removed HSV.\n* after the optimization was completed by cross validation, we trained the final models with all the data.\n* all models are inferred using the same image size as trained.\n* ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676.\n\n## Classification re-score:\n* crop out all predicted boxes (3-fold OOF) into squares wich conf > 0.01. The side length of the square is `max(length, width)` of the predicted boxes, then extended by 20%.\n* we calculate the iou as the maximum of the iou values of each predicted box and GT boxes of this image.\n* classification target of each cropped box: iou>0.3, iou>0.4, iou>0.5, iou>0.6, iou>0.7, iou>0.8 and iou>0.9 Simply put, the iou is divided into 7 bins. e.g.: `[1,1,1,0,0,0,0]` indicates the iou is between 0.5 and 0.6.\n* during inference we average 7 bin outputs as classification score.\n* then we use BCELoss to train those cropped boxes by size 256x256 or 224x224.\n* a very high dropout_rate or drop_path_rate can help a lot to improve the performance of the classification model. We use `dropout_rate=0.7` and `drop_path_rate=0.5`\n* augmentations: hflip, vflip, transpose, 45° rotation and cutout.\nThe best classification model can boost out CV to 0.727\n* after ensemble some classification models, our CV comes to 0.73+\n\n## Post-processing\n\nFinally, we use a simple post-processing to further boost our CV to 0.74+.\nFor example, the model has predicted some boxes B at #N frame, select the boxes from B which has a high confidence, these boxes are marked as \"attention area\".\nin the #N+1, #N+2, #N+3 frame, for the predicted boxes with conf > 0.01,  if it has an IoU with the \"attention area\" larger than 0, boost the score of these boxes with `score += confidence * IOU`\n\nWe also tried the tracking method, which gives us a CV of +0.002. However, it introduces two additional hyperparameters. We therefore chose not to use it.\n\n# Little story\n\nAt the beginning of the competition, we used different F2 algorithms for each of the three members of our team, and later we found that for the same oof, we did not calculate the same score.\nFor example, nvnn shared an OOF file with `F2=0.62`, and sheep calculated `F2=0.66`, while I calculated `F2=0.68`.\nWe finally chose to use the F2 algorithm with the lowest score from nvnn to evaluate all our models.\n\nhttps://www.kaggle.com/haqishen/f2-evaluation/script\n\nHere's our final F2 algorithm, if you are interested you can use this algorithm to compare your CV with ours!\n\n# Acknowledge\n\nAs usual, I trained many models in this competition using Z by HP Z8G4 Workstation with dual A6000 GPU. The large memory of 48G for a single GPU allowed me to train large resolution images with ease. Thanks to Z by HP for sponsoring!",
      "votes": 190
    },
    {
      "id": 1692710,
      "postDate": "2022-02-16T07:41:47.787Z",
      "content": "<p>First of all congratulations! Great work and great solution description! 👋👋👋 Attention is really great tip and your way to decreading number of hyperparameters cool!</p>\n<ul>\n<li>As I can see you did not inference on mega resolution ;) Just as you trained model. How did you come to the conclution that it would lead to overfitt priv LB?</li>\n<li>What NN architecture did you use for classifier?</li>\n<li>What Albumentations pipeline looks like?</li>\n<li>Are final model trained on all videos or each model on 3/1 split?</li>\n</ul>\n<p>Sorry for long list of question but I have an opportunity to learn from the best 👍👍👍</p>",
      "rawMarkdown": "First of all congratulations! Great work and great solution description! 👋👋👋 Attention is really great tip and your way to decreading number of hyperparameters cool!\n\n- As I can see you did not inference on mega resolution ;) Just as you trained model. How did you come to the conclution that it would lead to overfitt priv LB?\n- What NN architecture did you use for classifier?\n- What Albumentations pipeline looks like?\n- Are final model trained on all videos or each model on 3/1 split?\n\nSorry for long list of question but I have an opportunity to learn from the best 👍👍👍",
      "votes": 8,
      "replies": [
        {
          "id": 1692825,
          "postDate": "2022-02-16T09:06:40.467Z",
          "content": "<p>In fact, compared to the #1 question, the other questions are less important because they are only parameter tuning works or try and error things right?</p>\n<p>So for the #1 question, it should be clarified that it's not a conclution, but how things should be done.<br>\nImagine you are doing a realistic machine learning task. You only have a training data, so what would you optimize your model based on? Only the cv score. I'm not willing to do something that will lower the cv score. That's it. If we're still at 150th when the pvt LB comes out (we had been prepared for this), I'm perfectly fine with that because I did what I thought was right. </p>\n<p>Of course I have no problem with many people willing to probe public LB, as long as the rules still allow them to do so.</p>",
          "rawMarkdown": "In fact, compared to the #1 question, the other questions are less important because they are only parameter tuning works or try and error things right?\n\nSo for the #1 question, it should be clarified that it's not a conclution, but how things should be done.\nImagine you are doing a realistic machine learning task. You only have a training data, so what would you optimize your model based on? Only the cv score. I'm not willing to do something that will lower the cv score. That's it. If we're still at 150th when the pvt LB comes out (we had been prepared for this), I'm perfectly fine with that because I did what I thought was right. \n\nOf course I have no problem with many people willing to probe public LB, as long as the rules still allow them to do so.",
          "votes": 11
        },
        {
          "id": 1692845,
          "postDate": "2022-02-16T09:28:06.607Z",
          "content": "<p>I do not understand your answer :( </p>\n<p>I'm just wondering which of the solutions you used had a real impact on the final result. … We tested most of them (instead of having 2xGPU) and … had no such big influence on score (I analyzed local CV / private and public LB score). Just trying to understand where I sould learn more and … develop better solution in future. </p>\n<p>Topic I asked:</p>\n<ul>\n<li>infrerence resizing - impact?</li>\n<li>classifier - architeture and impact?</li>\n<li>albumentations -  pipeline steps - what wored and what no?</li>\n<li>training procedure - whole dataset or part of them?</li>\n</ul>",
          "rawMarkdown": "I do not understand your answer :( \n\nI'm just wondering which of the solutions you used had a real impact on the final result. ... We tested most of them (instead of having 2xGPU) and ... had no such big influence on score (I analyzed local CV / private and public LB score). Just trying to understand where I sould learn more and ... develop better solution in future. \n\nTopic I asked:\n- infrerence resizing - impact?\n- classifier - architeture and impact?\n- albumentations -  pipeline steps - what wored and what no?\n- training procedure - whole dataset or part of them?"
        },
        {
          "id": 1692862,
          "postDate": "2022-02-16T09:46:34.477Z",
          "content": "<p>OK, simply put,</p>\n<ul>\n<li>infrerence resizing - same size is <strong>best on local cv</strong>.</li>\n<li>classifier - convnext work <strong>best on local cv</strong>.</li>\n<li>albumentations - hflip/vflip/transpose, 45° random rotation, all p=0.5 <strong>work best for us on local cv</strong>.</li>\n<li>training procedure - for yolo, all data. for cls, 3-fold models. </li>\n</ul>\n<p>Is it helpful to you?</p>\n<p>Anyway, if you guys finished 34th without a GPU, that's an amazing achievement. Congratulations to you guys.</p>\n<p>PS: Your score is less than 0.02 from the gold zone, please don't think your solution is bad. It is good enough, maybe you guys just lack some luck.</p>",
          "rawMarkdown": "OK, simply put,\n\n* infrerence resizing - same size is **best on local cv**.\n* classifier - convnext work **best on local cv**.\n* albumentations - hflip/vflip/transpose, 45° random rotation, all p=0.5 **work best for us on local cv**.\n* training procedure - for yolo, all data. for cls, 3-fold models. \n\nIs it helpful to you?\n\nAnyway, if you guys finished 34th without a GPU, that's an amazing achievement. Congratulations to you guys.\n\nPS: Your score is less than 0.02 from the gold zone, please don't think your solution is bad. It is good enough, maybe you guys just lack some luck.",
          "votes": 7
        },
        {
          "id": 1692865,
          "postDate": "2022-02-16T09:53:25.073Z",
          "content": "<p>QiShen's idea is that question #1 is different compare to rest of questions and the question #1 is more important. Question #1 is related to How we evaluate our model yet question #234 are related to how we optimize our model. <br>\nIn real world situation, we do not have a Leaderboard, splited to public and private, what we can depend on is just the CV score, so we do not really care about the Public Leaderboard.<br>\nSpecificly in this competition, I released that yolo5 notebook in order to show the advantage of training large size while handling small object, however, when guys start to probe the LB by trying different inference size, I know the overfitting began. <br>\nQuestion #234 could be simplely answered by experiment and it is different according to the dataset you are working with. As for Question #1, according to our experiment, inference at the training size get best CV.<br>\nThere had been a chance that public and private have a same distribution, then we will end with the rank like 100+ place, but we will still evaluate our model base on CV both in competition and work.</p>",
          "rawMarkdown": "QiShen's idea is that question #1 is different compare to rest of questions and the question #1 is more important. Question #1 is related to How we evaluate our model yet question #234 are related to how we optimize our model. \nIn real world situation, we do not have a Leaderboard, splited to public and private, what we can depend on is just the CV score, so we do not really care about the Public Leaderboard.\nSpecificly in this competition, I released that yolo5 notebook in order to show the advantage of training large size while handling small object, however, when guys start to probe the LB by trying different inference size, I know the overfitting began. \nQuestion #234 could be simplely answered by experiment and it is different according to the dataset you are working with. As for Question #1, according to our experiment, inference at the training size get best CV.\nThere had been a chance that public and private have a same distribution, then we will end with the rank like 100+ place, but we will still evaluate our model base on CV both in competition and work.",
          "votes": 10
        },
        {
          "id": 1693012,
          "postDate": "2022-02-16T11:52:09.983Z",
          "content": "<p>Thank you <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> and <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> for your answers. Great source of knowledge and experience. You know <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> that I was against talking about mega size inference (we dicovered this in December but quickly realized that it was good for LB and not exactly good for final score) - I even criticized you for this talking about this. Sorry if this was too offensive.  You know competiiton … emotions … Now I would like to thank you guys for sharing. I have larned a lot from you. Respect! 👍👍🙏🙏</p>",
          "rawMarkdown": "Thank you @steamedsheep and @haqishen for your answers. Great source of knowledge and experience. You know @steamedsheep that I was against talking about mega size inference (we dicovered this in December but quickly realized that it was good for LB and not exactly good for final score) - I even criticized you for this talking about this. Sorry if this was too offensive.  You know competiiton ... emotions ... Now I would like to thank you guys for sharing. I have larned a lot from you. Respect! 👍👍🙏🙏",
          "votes": 2
        },
        {
          "id": 1693081,
          "postDate": "2022-02-16T12:22:09.727Z",
          "content": "<p>我英文不好，避免誤會我講中文。<br>\n我覺得這沒有對錯。今天我們確實面對兩種狀況，因此有些人押一邊贏了，有些人押另一邊輸了，而我因為被kaggle耍太多次所以押兩邊。😅<br>\n關於真實世界，我們永遠會有客戶的回饋(leaderboard)。例如我們幫客戶弄好一個廠(train)的AI方案，客戶移到另一廠(public test)發現好像有點問題，這時我們絕對不會完全不介入處理。<br>\n我相信這場比賽一定有一些參賽者，努力試圖想要弭平兩者(train/public test)的差異，結果兩頭空。這絕對不是一句\"trust cv\"就可以輕鬆帶過的。<br>\n最後還是恭喜啦，完全沒有批評的意思，希望你們不要誤會，因為這就是kaggle。😄</p>",
          "rawMarkdown": "我英文不好，避免誤會我講中文。\n我覺得這沒有對錯。今天我們確實面對兩種狀況，因此有些人押一邊贏了，有些人押另一邊輸了，而我因為被kaggle耍太多次所以押兩邊。😅\n關於真實世界，我們永遠會有客戶的回饋(leaderboard)。例如我們幫客戶弄好一個廠(train)的AI方案，客戶移到另一廠(public test)發現好像有點問題，這時我們絕對不會完全不介入處理。\n我相信這場比賽一定有一些參賽者，努力試圖想要弭平兩者(train/public test)的差異，結果兩頭空。這絕對不是一句\"trust cv\"就可以輕鬆帶過的。\n最後還是恭喜啦，完全沒有批評的意思，希望你們不要誤會，因為這就是kaggle。😄",
          "votes": 5
        },
        {
          "id": 1693086,
          "postDate": "2022-02-16T12:26:00.063Z",
          "content": "<p><a href=\"https://www.kaggle.com/outrunner\" target=\"_blank\">@outrunner</a><br>\nThanks for sharing your experience! <br>\n台北之光</p>",
          "rawMarkdown": "@outrunner\nThanks for sharing your experience! \n台北之光",
          "votes": -1
        },
        {
          "id": 1693100,
          "postDate": "2022-02-16T12:38:48.483Z",
          "content": "<p><a href=\"https://www.kaggle.com/outrunner\" target=\"_blank\">@outrunner</a> 恭喜喜提solo奖金。我们的平时经常处理客户反馈，但我们看得到数据和标签，我们会理解为另外批次的数据。Kaggle的比赛有很多probe learderboard的tricks，有的比赛有很好的效果，有的比赛会过拟合很严重。不过按照我们的经验，相信CV胜算大一点😃。另外我们其实没找到publiclearder board的trick(除了更大的尺寸)。</p>",
          "rawMarkdown": "@outrunner 恭喜喜提solo奖金。我们的平时经常处理客户反馈，但我们看得到数据和标签，我们会理解为另外批次的数据。Kaggle的比赛有很多probe learderboard的tricks，有的比赛有很好的效果，有的比赛会过拟合很严重。不过按照我们的经验，相信CV胜算大一点😃。另外我们其实没找到publiclearder board的trick(除了更大的尺寸)。"
        },
        {
          "id": 1693102,
          "postDate": "2022-02-16T12:38:53.073Z",
          "content": "<p><a href=\"https://www.kaggle.com/outrunner\" target=\"_blank\">@outrunner</a> <br>\n哈哈，也恭喜你的another solo gold！<br>\n从你的这个角度的话，我一般只压CV这边。因为如果客户想要迁移能力的话他应该从一开始就提出来，比如 Google Landmark 以及 Bengali 这样的，那我也会有特殊的处理办法。<br>\n另外我有一条消息想与你分享，请查收一下邮件。</p>",
          "rawMarkdown": "@outrunner \n哈哈，也恭喜你的another solo gold！\n从你的这个角度的话，我一般只压CV这边。因为如果客户想要迁移能力的话他应该从一开始就提出来，比如 Google Landmark 以及 Bengali 这样的，那我也会有特殊的处理办法。\n另外我有一条消息想与你分享，请查收一下邮件。\n\n"
        },
        {
          "id": 1693120,
          "postDate": "2022-02-16T12:55:16.747Z",
          "content": "<p>哈，我也想提Bengali，原本想說難保這場不是這種，結果是我想多了。<br>\n<a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> 回信了，感謝。</p>",
          "rawMarkdown": "哈，我也想提Bengali，原本想說難保這場不是這種，結果是我想多了。\n@haqishen 回信了，感謝。",
          "votes": -1
        }
      ]
    },
    {
      "id": 1692456,
      "postDate": "2022-02-16T03:49:40.840Z",
      "content": "<p>Good job. Congrats <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a>, <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> and <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> on results!</p>",
      "rawMarkdown": "Good job. Congrats @nvnnghia, @haqishen and @steamedsheep on results!",
      "votes": 3
    },
    {
      "id": 1692835,
      "postDate": "2022-02-16T09:19:22.500Z",
      "content": "<p>You guys are my heros I will learn many things from you!!!<br>\n<a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> And there is maybe a stupid question I confuse it from long time ago.<br>\nHope someone could correct me.</p>\n<p><strong>What's exactly \"CV\" result?</strong><br>\nIn my thougth, CV means cross validation, so we could get different score at each fold experiment. <br>\nFor example, in the comepetion we pursuit F2 score, so if could get</p>\n<table>\n<thead>\n<tr>\n<th>fold</th>\n<th>f2 score</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>0.6XX</td>\n</tr>\n<tr>\n<td>2</td>\n<td>0.5XX</td>\n</tr>\n<tr>\n<td>3</td>\n<td>0.7XX</td>\n</tr>\n</tbody>\n</table>\n<p>So I call fold1 model get CV0.6XX, fold2 model get CV0.5XX, and fold3 model get CV0.7XX.</p>\n<p><em>\"after the optimization was completed by cross validation, we trained the final models with all the data.\"</em><br>\nHowever, how to evaluate \"CV\" after we trained the all data? they are all be trained.</p>\n<p><em>\"ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676.\"</em><br>\nWe ensemble all fold, there are all be trained, too.</p>\n<p>It should be a stupid question, thanks guys who read it for me~</p>",
      "rawMarkdown": "You guys are my heros I will learn many things from you!!!\n@haqishen And there is maybe a stupid question I confuse it from long time ago.\nHope someone could correct me.\n\n**What's exactly \"CV\" result?**\nIn my thougth, CV means cross validation, so we could get different score at each fold experiment. \nFor example, in the comepetion we pursuit F2 score, so if could get\n| fold | f2 score   |\n| --- | --- |\n|  1| 0.6XX  |\n|  2| 0.5XX  |\n|  3| 0.7XX  |\n\nSo I call fold1 model get CV0.6XX, fold2 model get CV0.5XX, and fold3 model get CV0.7XX.\n\n*\"after the optimization was completed by cross validation, we trained the final models with all the data.\"*\nHowever, how to evaluate \"CV\" after we trained the all data? they are all be trained.\n\n*\"ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676.\"*\nWe ensemble all fold, there are all be trained, too.\n\nIt should be a stupid question, thanks guys who read it for me~",
      "votes": 4,
      "replies": [
        {
          "id": 1693016,
          "postDate": "2022-02-16T11:53:30.897Z",
          "content": "<p>Hi, actually the CV stands for cross validation, and the key word is <strong>cross</strong> so the local score of 1 fold we don't call it CV.</p>\n<p>when you combine your predicted result from all the folds and calculate validation score on it, it can be called CV score.</p>\n<p>When we train model on all data, we don't do evaluation. Just set a proper number of epoch and run it.</p>",
          "rawMarkdown": "Hi, actually the CV stands for cross validation, and the key word is **cross** so the local score of 1 fold we don't call it CV.\n\nwhen you combine your predicted result from all the folds and calculate validation score on it, it can be called CV score.\n\nWhen we train model on all data, we don't do evaluation. Just set a proper number of epoch and run it.",
          "votes": 3
        },
        {
          "id": 1693065,
          "postDate": "2022-02-16T12:17:01.290Z",
          "content": "<p>Thanks your response <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>!<br>\nSorry, but I still not understand when everyone says their best CV score, how to calculate?</p>\n<blockquote>\n  <p>when you combine your predicted result from all the folds and calculate validation score on it, it can be called CV score.</p>\n</blockquote>\n<p>Does it mean we calculate fold1 score to foldN score, and average them?</p>",
          "rawMarkdown": "Thanks your response @haqishen!\nSorry, but I still not understand when everyone says their best CV score, how to calculate?\n> when you combine your predicted result from all the folds and calculate validation score on it, it can be called CV score.\n\nDoes it mean we calculate fold1 score to foldN score, and average them?",
          "votes": 1
        },
        {
          "id": 1693118,
          "postDate": "2022-02-16T12:52:41.793Z",
          "content": "<p>Near but not the same. combine all labels and predictions from all folds, treat it as a single output, then calculate metric on it.</p>\n<p><a href=\"https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614\" target=\"_blank\">https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614</a></p>\n<p>maybe this post can help you.</p>",
          "rawMarkdown": "Near but not the same. combine all labels and predictions from all folds, treat it as a single output, then calculate metric on it.\n\nhttps://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614\n\nmaybe this post can help you.",
          "votes": 1
        },
        {
          "id": 1693143,
          "postDate": "2022-02-16T13:20:44.893Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> I am reading the post and still find it a bit confusing:</p>\n<p>Training I understand. You have N folds. You create N independent models, train them on every fold but one each and then combine the test fold of each to create the oof file. You then compare the oof file to the groundtruth targets.</p>\n<p>What do you do in inference? Do you infere each image on each of the N models and then ensemble?</p>\n<p>Thank you in advance!</p>",
          "rawMarkdown": "Hi @haqishen I am reading the post and still find it a bit confusing:\n\nTraining I understand. You have N folds. You create N independent models, train them on every fold but one each and then combine the test fold of each to create the oof file. You then compare the oof file to the groundtruth targets.\n\nWhat do you do in inference? Do you infere each image on each of the N models and then ensemble?\n\nThank you in advance!"
        },
        {
          "id": 1693161,
          "postDate": "2022-02-16T13:36:11.640Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>!<br>\nThank you so much. I think I got the point~</p>",
          "rawMarkdown": "Hi @haqishen!\nThank you so much. I think I got the point~"
        }
      ]
    },
    {
      "id": 1738298,
      "postDate": "2022-03-29T07:36:13.697Z",
      "content": "<p>This work of your team is amazing. Thank for your sharing <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>  !<br>\nThere is some part of your solution that I don't understand, please help me to understand it:</p>\n<p>1) How you tuned each parameters of model yolo v5, I assume that it's like you track the performance of the parameter at a range of value with keep all other parameters at 0 (during n epochs) ? If yes, what is the n that you usually choose ?<br>\n2) Why do you choose the image patch with the size (512x320), how you choose this size ?<br>\n3) I don't understand well the part of \"Classification re-score\". As I understand, you crop out all predicted boxes and use the average of 7 bin outputs as classification score, and you use classification score as a label (target) to train a model classification with input -  all predicted boxes already preprocessed.</p>\n<p>I would be glad to hear from you, the champion !</p>",
      "rawMarkdown": "This work of your team is amazing. Thank for your sharing @haqishen  !\nThere is some part of your solution that I don't understand, please help me to understand it:\n\n1) How you tuned each parameters of model yolo v5, I assume that it's like you track the performance of the parameter at a range of value with keep all other parameters at 0 (during n epochs) ? If yes, what is the n that you usually choose ?\n2) Why do you choose the image patch with the size (512x320), how you choose this size ?\n3) I don't understand well the part of \"Classification re-score\". As I understand, you crop out all predicted boxes and use the average of 7 bin outputs as classification score, and you use classification score as a label (target) to train a model classification with input -  all predicted boxes already preprocessed.\n\nI would be glad to hear from you, the champion !",
      "votes": 1
    },
    {
      "id": 1734664,
      "postDate": "2022-03-25T14:38:57.287Z",
      "content": "<p>抱歉,現在再用此數據集做研究<br>\n想請問數據集中的標註excel檔我跟怎麼轉換yolov5的txt格式</p>",
      "rawMarkdown": "抱歉,現在再用此數據集做研究\n想請問數據集中的標註excel檔我跟怎麼轉換yolov5的txt格式",
      "votes": 1
    },
    {
      "id": 1700915,
      "postDate": "2022-02-22T11:29:16.257Z",
      "content": "<p><a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> Could you please explain what is the meaning of \"averaging out the output of all the 7 bins to get the classification score\". For example, in the example list that you had given [1, 1, 1, 0, 0, 0, 0], what is the output of the average of the 7 bins in this case. I know this is a stupid question but it would be really nice if someone could give the correct meaning behind that line. Thanks in advance!! Also, great work on securing the first place in the competiition.</p>",
      "rawMarkdown": "@haqishen Could you please explain what is the meaning of \"averaging out the output of all the 7 bins to get the classification score\". For example, in the example list that you had given [1, 1, 1, 0, 0, 0, 0], what is the output of the average of the 7 bins in this case. I know this is a stupid question but it would be really nice if someone could give the correct meaning behind that line. Thanks in advance!! Also, great work on securing the first place in the competiition.",
      "votes": 1
    },
    {
      "id": 1698424,
      "postDate": "2022-02-20T11:28:33.350Z",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>. Thanks for sharing your solution and congrats to 1 place! Could you please answer some questions?</p>\n<ul>\n<li>What yolo models did you use? These were yolov5l6 or yolov5s6 models or some other ones. </li>\n<li>Did you train detection models only on images with labels or also add some background images without labels?</li>\n</ul>",
      "rawMarkdown": "Hi, @haqishen. Thanks for sharing your solution and congrats to 1 place! Could you please answer some questions?\n \n- What yolo models did you use? These were yolov5l6 or yolov5s6 models or some other ones. \n- Did you train detection models only on images with labels or also add some background images without labels?",
      "votes": 1
    },
    {
      "id": 1696237,
      "postDate": "2022-02-18T17:21:43.657Z",
      "content": "<p>Good job. Nice</p>",
      "rawMarkdown": "Good job. Nice",
      "votes": 1
    },
    {
      "id": 1695440,
      "postDate": "2022-02-18T06:37:40.147Z",
      "content": "<p>Congrats ! and thanks for sharing the approach..</p>",
      "rawMarkdown": "Congrats ! and thanks for sharing the approach..",
      "votes": 1
    },
    {
      "id": 1695332,
      "postDate": "2022-02-18T04:50:29.690Z",
      "content": "<p>Good job. Congrats on results</p>",
      "rawMarkdown": "Good job. Congrats on results",
      "votes": 1
    },
    {
      "id": 1694811,
      "postDate": "2022-02-17T18:18:11.953Z",
      "content": "<p>congratulations Qishen Ha , nvnnghia and steamedsheep</p>",
      "rawMarkdown": "congratulations Qishen Ha , nvnnghia and steamedsheep",
      "votes": 1
    },
    {
      "id": 1694118,
      "postDate": "2022-02-17T07:00:05.460Z",
      "content": "<p>Congratulations on 1st place and thanks for sharing - so much to learn from this team 🙌</p>",
      "rawMarkdown": "Congratulations on 1st place and thanks for sharing - so much to learn from this team 🙌",
      "votes": 1
    },
    {
      "id": 1693070,
      "postDate": "2022-02-16T12:18:32.620Z",
      "content": "<p>Congrats for your decisive win and your wise focus on staying aligned to CV only. It really looks like an incredible work of optimization. Can you elaborate a little bit on how you tuned your parameters (conf threshold, nms or wbf iou) to maximize F2 for your <strong>ensemble</strong> of models and on which subset of train data the CV 0.716 (average of OOF?) was calculated ? Or in other words, were the 6 models below trained on the same folds for your experiments but just with different hyperparameters or seeds ?</p>\n<blockquote>\n  <p>ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676. </p>\n</blockquote>",
      "rawMarkdown": "Congrats for your decisive win and your wise focus on staying aligned to CV only. It really looks like an incredible work of optimization. Can you elaborate a little bit on how you tuned your parameters (conf threshold, nms or wbf iou) to maximize F2 for your **ensemble** of models and on which subset of train data the CV 0.716 (average of OOF?) was calculated ? Or in other words, were the 6 models below trained on the same folds for your experiments but just with different hyperparameters or seeds ?\n> ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676. ",
      "votes": 1,
      "replies": [
        {
          "id": 1693156,
          "postDate": "2022-02-16T13:32:33.870Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/alexandrecc\" target=\"_blank\">@alexandrecc</a> <br>\nthe parameters are tuned using oof files. A model has an oof file, this oof file is created by combining raw predictions of all its folds.  so, an oof file contains the predictions and labels of the entire training data. The parameters are then tuned by trial and error to achieve the best F2 score.<br>\nThe 6 models are trained on all 3 folds (18 checkpoint in total), and the CV is calculate from the OOF (it covers entire training set)</p>",
          "rawMarkdown": "Hi @alexandrecc \nthe parameters are tuned using oof files. A model has an oof file, this oof file is created by combining raw predictions of all its folds.  so, an oof file contains the predictions and labels of the entire training data. The parameters are then tuned by trial and error to achieve the best F2 score.\nThe 6 models are trained on all 3 folds (18 checkpoint in total), and the CV is calculate from the OOF (it covers entire training set)",
          "votes": 3
        },
        {
          "id": 1695392,
          "postDate": "2022-02-18T05:45:36.387Z",
          "content": "<p>Can I ask if you guys used any special method to tune your hparams? Did you guys use something like YOLOv5's wandb sweeps or did you guys search params manually.</p>",
          "rawMarkdown": "Can I ask if you guys used any special method to tune your hparams? Did you guys use something like YOLOv5's wandb sweeps or did you guys search params manually."
        },
        {
          "id": 1695751,
          "postDate": "2022-02-18T10:44:16.387Z",
          "content": "<p>For me, manually tune by intuition.</p>",
          "rawMarkdown": "For me, manually tune by intuition."
        }
      ]
    },
    {
      "id": 1692841,
      "postDate": "2022-02-16T09:24:26.920Z",
      "content": "<p>Congrats! I have learned a lot from <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> notebook and what you have described here was awesome too.  </p>",
      "rawMarkdown": "Congrats! I have learned a lot from @steamedsheep notebook and what you have described here was awesome too.  ",
      "votes": 1
    },
    {
      "id": 1692453,
      "postDate": "2022-02-16T03:46:23.460Z",
      "content": "<p>congrats!  I have similar classification idea etc. without good hardware support, no patience to try.</p>",
      "rawMarkdown": "congrats!  I have similar classification idea etc. without good hardware support, no patience to try.",
      "votes": 1
    },
    {
      "id": 1692428,
      "postDate": "2022-02-16T03:21:23.127Z",
      "content": "<p>Thanks for sharing your code. i have some question about classification part. </p>\n<ul>\n<li>why using not binary but 7 bins? (we are trying &lt; 0.3 label 0 &gt; 0.8 label 1 but failed :( ) </li>\n<li>How do you use it in reference? You used 7 different results? Or did you use max or something else?</li>\n<li>Our model didn't learn well because of few label noise. Did your team deal with label noise?</li>\n</ul>\n<p>Thank you for a very interesting approach. Congratulations on winning first place.</p>",
      "rawMarkdown": "Thanks for sharing your code. i have some question about classification part. \n- why using not binary but 7 bins? (we are trying < 0.3 label 0 > 0.8 label 1 but failed :( ) \n- How do you use it in reference? You used 7 different results? Or did you use max or something else?\n- Our model didn't learn well because of few label noise. Did your team deal with label noise?\n\nThank you for a very interesting approach. Congratulations on winning first place.\n",
      "votes": 1,
      "replies": [
        {
          "id": 1692447,
          "postDate": "2022-02-16T03:43:51.063Z",
          "content": "<p>Thank you and congratulations to you, too!</p>\n<ol>\n<li>It was a gut feeling, I used this method very directly when writing the code and didn't try any other method else.</li>\n<li>avg the output of 7 bins.</li>\n<li>compared to most of the competitions, this dataset label noise is not that serious, so we chose not to deal with it.</li>\n</ol>",
          "rawMarkdown": "Thank you and congratulations to you, too!\n\n1. It was a gut feeling, I used this method very directly when writing the code and didn't try any other method else.\n2. avg the output of 7 bins.\n3. compared to most of the competitions, this dataset label noise is not that serious, so we chose not to deal with it.",
          "votes": 1
        },
        {
          "id": 1692482,
          "postDate": "2022-02-16T04:15:29.497Z",
          "content": "<p>Thanks. Clearly understand !! 👍👍</p>",
          "rawMarkdown": "Thanks. Clearly understand !! 👍👍"
        }
      ]
    },
    {
      "id": 1692776,
      "postDate": "2022-02-16T08:16:15.180Z",
      "content": "<p>Congratulations again to the team! </p>\n<p>I wanted to mention, I'll be interviewing Grandmaster <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> next weekend, if anyone has any questions/topics that you'd like to be discussed in the chai interview, please let me know! </p>\n<p>TIA! </p>",
      "rawMarkdown": "Congratulations again to the team! \n\nI wanted to mention, I'll be interviewing Grandmaster @haqishen next weekend, if anyone has any questions/topics that you'd like to be discussed in the chai interview, please let me know! \n\nTIA! ",
      "votes": 2
    },
    {
      "id": 1692464,
      "postDate": "2022-02-16T03:57:56.783Z",
      "content": "<p>great job, Congrats. </p>\n<p>can you please provide your training notebooks? </p>",
      "rawMarkdown": "great job, Congrats. \n\ncan you please provide your training notebooks? ",
      "votes": 2
    },
    {
      "id": 1696460,
      "postDate": "2022-02-18T21:14:45.277Z",
      "content": "<p>abcdefghij</p>",
      "rawMarkdown": "abcdefghij\n"
    },
    {
      "id": 1695676,
      "postDate": "2022-02-18T10:01:52.503Z",
      "content": "<p>Congraatulations🤩🤩Thanks for sharing your code <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> , new follower 🙋‍♀️😊</p>",
      "rawMarkdown": "Congraatulations🤩🤩Thanks for sharing your code @haqishen , new follower 🙋‍♀️😊\n",
      "votes": -1
    },
    {
      "id": 2075482,
      "postDate": "2022-12-25T13:41:09.993Z",
      "content": "<p>New to kaggle, where I can find the code?</p>",
      "rawMarkdown": "New to kaggle, where I can find the code?"
    },
    {
      "id": 1961408,
      "postDate": "2022-09-29T06:42:18.540Z",
      "content": "<p>good job!!</p>",
      "rawMarkdown": "good job!!"
    },
    {
      "id": 1778633,
      "postDate": "2022-05-05T14:30:34.817Z",
      "content": "<p>Congratulations! Your solution is helpful for me!</p>",
      "rawMarkdown": "Congratulations! Your solution is helpful for me!"
    },
    {
      "id": 1734661,
      "postDate": "2022-03-25T14:36:24.603Z",
      "content": "<p>hello.<br>\nInquiry data (excel) conversion yolov5 (txt)<br>\nHow is the conversion?</p>",
      "rawMarkdown": "hello.\nInquiry data (excel) conversion yolov5 (txt)\nHow is the conversion?"
    },
    {
      "id": 1734636,
      "postDate": "2022-03-25T14:16:41.380Z",
      "content": "<p>Inquiry data (excel) conversion yolov5 (txt)<br>\nHow is the conversion?</p>",
      "rawMarkdown": "Inquiry data (excel) conversion yolov5 (txt)\nHow is the conversion?"
    },
    {
      "id": 1708849,
      "postDate": "2022-03-01T18:35:44.553Z",
      "content": "<p>Congratulations🎉</p>",
      "rawMarkdown": "Congratulations🎉"
    },
    {
      "id": 1706956,
      "postDate": "2022-02-28T02:36:37.117Z",
      "content": "<p>this is a nice work!</p>",
      "rawMarkdown": "this is a nice work!"
    },
    {
      "id": 1703548,
      "postDate": "2022-02-24T16:03:33.493Z",
      "content": "<p>nice job bro</p>",
      "rawMarkdown": "nice job bro"
    },
    {
      "id": 1702468,
      "postDate": "2022-02-23T16:22:05.607Z",
      "content": "<p>Well done!</p>",
      "rawMarkdown": "Well done!"
    },
    {
      "id": 1702357,
      "postDate": "2022-02-23T14:41:57.493Z",
      "content": "<p>It's an interesting cross-validation method</p>",
      "rawMarkdown": "It's an interesting cross-validation method"
    },
    {
      "id": 1701970,
      "postDate": "2022-02-23T07:58:17.170Z",
      "content": "<p>Congrats!  Thanks for sharing this approach toward a solution.</p>",
      "rawMarkdown": "Congrats!  Thanks for sharing this approach toward a solution.\n\n\n"
    },
    {
      "id": 1699741,
      "postDate": "2022-02-21T11:58:31.517Z",
      "content": "<p>I also follow the motto of always trusting my cv, but sometimes especially when I am dealing with imbalanced datasets, for some reason I tend to doubt my CVs, is my doubt justified? Also, Nice work and congratulations.</p>",
      "rawMarkdown": "I also follow the motto of always trusting my cv, but sometimes especially when I am dealing with imbalanced datasets, for some reason I tend to doubt my CVs, is my doubt justified? Also, Nice work and congratulations."
    },
    {
      "id": 1699355,
      "postDate": "2022-02-21T06:04:54.237Z",
      "content": "<p>Congrats ! Thx for sharing ! </p>",
      "rawMarkdown": "Congrats ! Thx for sharing ! "
    },
    {
      "id": 1698734,
      "postDate": "2022-02-20T16:11:43.300Z",
      "content": "<p>Congratulations and thank you for sharing!</p>",
      "rawMarkdown": "Congratulations and thank you for sharing!"
    },
    {
      "id": 1697799,
      "postDate": "2022-02-19T21:04:41.447Z",
      "content": "<p>Congrats. Good job.</p>",
      "rawMarkdown": "Congrats. Good job."
    },
    {
      "id": 1697489,
      "postDate": "2022-02-19T16:51:17.890Z",
      "content": "<p>This is great and thanks you for the brief explanation.</p>",
      "rawMarkdown": "This is great and thanks you for the brief explanation."
    },
    {
      "id": 1697290,
      "postDate": "2022-02-19T14:11:53.700Z",
      "content": "<p>WOW. You got a huge prize. congrats </p>",
      "rawMarkdown": "WOW. You got a huge prize. congrats "
    },
    {
      "id": 1697242,
      "postDate": "2022-02-19T13:26:39.497Z",
      "content": "<p>good job………</p>",
      "rawMarkdown": "good job........."
    },
    {
      "id": 1698239,
      "postDate": "2022-02-20T08:25:45.930Z",
      "content": "<p>Good job! Congrats</p>",
      "rawMarkdown": "Good job! Congrats",
      "isDeleted": true
    },
    {
      "id": 1696387,
      "postDate": "2022-02-18T19:49:16.010Z",
      "content": "<p>Thank you for sharing</p>",
      "rawMarkdown": "Thank you for sharing",
      "votes": 1
    },
    {
      "id": 1695708,
      "postDate": "2022-02-18T10:26:13.853Z",
      "content": "<p>Good job! Congrats, Thanks for sharing</p>",
      "rawMarkdown": "Good job! Congrats, Thanks for sharing",
      "votes": 1
    },
    {
      "id": 1694944,
      "postDate": "2022-02-17T21:02:48.180Z",
      "content": "<p>Thank you for sharing your solution!</p>",
      "rawMarkdown": "Thank you for sharing your solution!",
      "votes": 1
    },
    {
      "id": 1701034,
      "postDate": "2022-02-22T13:26:12.220Z",
      "content": "<p><a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>, thanks for sharing!</p>",
      "rawMarkdown": "@haqishen, thanks for sharing!"
    },
    {
      "id": 1700413,
      "postDate": "2022-02-22T00:33:01.920Z",
      "content": "<p>thank for your sharing!</p>",
      "rawMarkdown": "thank for your sharing!"
    },
    {
      "id": 1700251,
      "postDate": "2022-02-21T19:01:26.943Z",
      "content": "<p>Thank you and congrats!</p>",
      "rawMarkdown": "Thank you and congrats!"
    },
    {
      "id": 1699345,
      "postDate": "2022-02-21T05:53:52.757Z",
      "content": "<p>Great job! Thank you for sharing!</p>",
      "rawMarkdown": "Great job! Thank you for sharing!"
    },
    {
      "id": 1699240,
      "postDate": "2022-02-21T03:39:53.800Z",
      "content": "<p>Thank you for sharing the approach</p>",
      "rawMarkdown": "Thank you for sharing the approach"
    },
    {
      "id": 1699204,
      "postDate": "2022-02-21T02:26:46.600Z",
      "content": "<p>Thank you for sharing</p>",
      "rawMarkdown": "Thank you for sharing"
    },
    {
      "id": 1699133,
      "postDate": "2022-02-21T00:16:29.953Z",
      "content": "<p>Thank you for sharing</p>",
      "rawMarkdown": "Thank you for sharing"
    },
    {
      "id": 1699114,
      "postDate": "2022-02-20T23:54:41.927Z",
      "content": "<p>Great work, thanks for sharing</p>",
      "rawMarkdown": "Great work, thanks for sharing"
    },
    {
      "id": 1698712,
      "postDate": "2022-02-20T15:45:57.643Z",
      "content": "<p>Good job! Congrats, Thanks for sharing</p>",
      "rawMarkdown": "Good job! Congrats, Thanks for sharing"
    },
    {
      "id": 1698661,
      "postDate": "2022-02-20T15:04:59.873Z",
      "content": "<p>wow, that's help! thanks a lot!</p>",
      "rawMarkdown": "wow, that's help! thanks a lot!"
    },
    {
      "id": 1698439,
      "postDate": "2022-02-20T11:44:06.267Z",
      "content": "<p>Thank you for Sharing</p>",
      "rawMarkdown": "Thank you for Sharing"
    },
    {
      "id": 1697957,
      "postDate": "2022-02-20T02:26:43.327Z",
      "content": "<p>Thanks for sharing.</p>",
      "rawMarkdown": "Thanks for sharing."
    },
    {
      "id": 1697843,
      "postDate": "2022-02-19T22:06:14.070Z",
      "content": "<p>Congrats and thanks for sharing.</p>",
      "rawMarkdown": "Congrats and thanks for sharing."
    },
    {
      "id": 1697368,
      "postDate": "2022-02-19T15:13:45.413Z",
      "content": "<p>Thank you for sharing! 👍</p>",
      "rawMarkdown": "Thank you for sharing! 👍"
    },
    {
      "id": 1697187,
      "postDate": "2022-02-19T12:41:59.587Z",
      "content": "<p>thank you so much</p>",
      "rawMarkdown": "thank you so much\n"
    }
  ],
  "comments": [
    {
      "id": 1692710,
      "author_name": "Remek Kinas",
      "author_url": "",
      "post_date": "2022-02-16T07:41:47.787000",
      "content": "<p>First of all congratulations! Great work and great solution description! 👋👋👋 Attention is really great tip and your way to decreading number of hyperparameters cool!</p>\n<ul>\n<li>As I can see you did not inference on mega resolution ;) Just as you trained model. How did you come to the conclution that it would lead to overfitt priv LB?</li>\n<li>What NN architecture did you use for classifier?</li>\n<li>What Albumentations pipeline looks like?</li>\n<li>Are final model trained on all videos or each model on 3/1 split?</li>\n</ul>\n<p>Sorry for long list of question but I have an opportunity to learn from the best 👍👍👍</p>",
      "votes": 8,
      "replies": [
        {
          "id": 1692825,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2022-02-16T09:06:40.467000",
          "content": "<p>In fact, compared to the #1 question, the other questions are less important because they are only parameter tuning works or try and error things right?</p>\n<p>So for the #1 question, it should be clarified that it's not a conclution, but how things should be done.<br>\nImagine you are doing a realistic machine learning task. You only have a training data, so what would you optimize your model based on? Only the cv score. I'm not willing to do something that will lower the cv score. That's it. If we're still at 150th when the pvt LB comes out (we had been prepared for this), I'm perfectly fine with that because I did what I thought was right. </p>\n<p>Of course I have no problem with many people willing to probe public LB, as long as the rules still allow them to do so.</p>",
          "votes": 11,
          "replies": []
        },
        {
          "id": 1692845,
          "author_name": "Remek Kinas",
          "author_url": "",
          "post_date": "2022-02-16T09:28:06.607000",
          "content": "<p>I do not understand your answer :( </p>\n<p>I'm just wondering which of the solutions you used had a real impact on the final result. … We tested most of them (instead of having 2xGPU) and … had no such big influence on score (I analyzed local CV / private and public LB score). Just trying to understand where I sould learn more and … develop better solution in future. </p>\n<p>Topic I asked:</p>\n<ul>\n<li>infrerence resizing - impact?</li>\n<li>classifier - architeture and impact?</li>\n<li>albumentations -  pipeline steps - what wored and what no?</li>\n<li>training procedure - whole dataset or part of them?</li>\n</ul>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1692862,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2022-02-16T09:46:34.477000",
          "content": "<p>OK, simply put,</p>\n<ul>\n<li>infrerence resizing - same size is <strong>best on local cv</strong>.</li>\n<li>classifier - convnext work <strong>best on local cv</strong>.</li>\n<li>albumentations - hflip/vflip/transpose, 45° random rotation, all p=0.5 <strong>work best for us on local cv</strong>.</li>\n<li>training procedure - for yolo, all data. for cls, 3-fold models. </li>\n</ul>\n<p>Is it helpful to you?</p>\n<p>Anyway, if you guys finished 34th without a GPU, that's an amazing achievement. Congratulations to you guys.</p>\n<p>PS: Your score is less than 0.02 from the gold zone, please don't think your solution is bad. It is good enough, maybe you guys just lack some luck.</p>",
          "votes": 7,
          "replies": []
        },
        {
          "id": 1692865,
          "author_name": "sheep",
          "author_url": "",
          "post_date": "2022-02-16T09:53:25.073000",
          "content": "<p>QiShen's idea is that question #1 is different compare to rest of questions and the question #1 is more important. Question #1 is related to How we evaluate our model yet question #234 are related to how we optimize our model. <br>\nIn real world situation, we do not have a Leaderboard, splited to public and private, what we can depend on is just the CV score, so we do not really care about the Public Leaderboard.<br>\nSpecificly in this competition, I released that yolo5 notebook in order to show the advantage of training large size while handling small object, however, when guys start to probe the LB by trying different inference size, I know the overfitting began. <br>\nQuestion #234 could be simplely answered by experiment and it is different according to the dataset you are working with. As for Question #1, according to our experiment, inference at the training size get best CV.<br>\nThere had been a chance that public and private have a same distribution, then we will end with the rank like 100+ place, but we will still evaluate our model base on CV both in competition and work.</p>",
          "votes": 10,
          "replies": []
        },
        {
          "id": 1693012,
          "author_name": "Remek Kinas",
          "author_url": "",
          "post_date": "2022-02-16T11:52:09.983000",
          "content": "<p>Thank you <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> and <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> for your answers. Great source of knowledge and experience. You know <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> that I was against talking about mega size inference (we dicovered this in December but quickly realized that it was good for LB and not exactly good for final score) - I even criticized you for this talking about this. Sorry if this was too offensive.  You know competiiton … emotions … Now I would like to thank you guys for sharing. I have larned a lot from you. Respect! 👍👍🙏🙏</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1693081,
          "author_name": "outrunner",
          "author_url": "",
          "post_date": "2022-02-16T12:22:09.727000",
          "content": "<p>我英文不好，避免誤會我講中文。<br>\n我覺得這沒有對錯。今天我們確實面對兩種狀況，因此有些人押一邊贏了，有些人押另一邊輸了，而我因為被kaggle耍太多次所以押兩邊。😅<br>\n關於真實世界，我們永遠會有客戶的回饋(leaderboard)。例如我們幫客戶弄好一個廠(train)的AI方案，客戶移到另一廠(public test)發現好像有點問題，這時我們絕對不會完全不介入處理。<br>\n我相信這場比賽一定有一些參賽者，努力試圖想要弭平兩者(train/public test)的差異，結果兩頭空。這絕對不是一句\"trust cv\"就可以輕鬆帶過的。<br>\n最後還是恭喜啦，完全沒有批評的意思，希望你們不要誤會，因為這就是kaggle。😄</p>",
          "votes": 5,
          "replies": []
        },
        {
          "id": 1693086,
          "author_name": "Lilin Chen",
          "author_url": "",
          "post_date": "2022-02-16T12:26:00.063000",
          "content": "<p><a href=\"https://www.kaggle.com/outrunner\" target=\"_blank\">@outrunner</a><br>\nThanks for sharing your experience! <br>\n台北之光</p>",
          "votes": -1,
          "replies": []
        },
        {
          "id": 1693100,
          "author_name": "sheep",
          "author_url": "",
          "post_date": "2022-02-16T12:38:48.483000",
          "content": "<p><a href=\"https://www.kaggle.com/outrunner\" target=\"_blank\">@outrunner</a> 恭喜喜提solo奖金。我们的平时经常处理客户反馈，但我们看得到数据和标签，我们会理解为另外批次的数据。Kaggle的比赛有很多probe learderboard的tricks，有的比赛有很好的效果，有的比赛会过拟合很严重。不过按照我们的经验，相信CV胜算大一点😃。另外我们其实没找到publiclearder board的trick(除了更大的尺寸)。</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1693102,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2022-02-16T12:38:53.073000",
          "content": "<p><a href=\"https://www.kaggle.com/outrunner\" target=\"_blank\">@outrunner</a> <br>\n哈哈，也恭喜你的another solo gold！<br>\n从你的这个角度的话，我一般只压CV这边。因为如果客户想要迁移能力的话他应该从一开始就提出来，比如 Google Landmark 以及 Bengali 这样的，那我也会有特殊的处理办法。<br>\n另外我有一条消息想与你分享，请查收一下邮件。</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1693120,
          "author_name": "outrunner",
          "author_url": "",
          "post_date": "2022-02-16T12:55:16.747000",
          "content": "<p>哈，我也想提Bengali，原本想說難保這場不是這種，結果是我想多了。<br>\n<a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> 回信了，感謝。</p>",
          "votes": -1,
          "replies": []
        }
      ]
    },
    {
      "id": 1692456,
      "author_name": "KhanhVD",
      "author_url": "",
      "post_date": "2022-02-16T03:49:40.840000",
      "content": "<p>Good job. Congrats <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a>, <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> and <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> on results!</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 1692835,
      "author_name": "Lilin Chen",
      "author_url": "",
      "post_date": "2022-02-16T09:19:22.500000",
      "content": "<p>You guys are my heros I will learn many things from you!!!<br>\n<a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> And there is maybe a stupid question I confuse it from long time ago.<br>\nHope someone could correct me.</p>\n<p><strong>What's exactly \"CV\" result?</strong><br>\nIn my thougth, CV means cross validation, so we could get different score at each fold experiment. <br>\nFor example, in the comepetion we pursuit F2 score, so if could get</p>\n<table>\n<thead>\n<tr>\n<th>fold</th>\n<th>f2 score</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>0.6XX</td>\n</tr>\n<tr>\n<td>2</td>\n<td>0.5XX</td>\n</tr>\n<tr>\n<td>3</td>\n<td>0.7XX</td>\n</tr>\n</tbody>\n</table>\n<p>So I call fold1 model get CV0.6XX, fold2 model get CV0.5XX, and fold3 model get CV0.7XX.</p>\n<p><em>\"after the optimization was completed by cross validation, we trained the final models with all the data.\"</em><br>\nHowever, how to evaluate \"CV\" after we trained the all data? they are all be trained.</p>\n<p><em>\"ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676.\"</em><br>\nWe ensemble all fold, there are all be trained, too.</p>\n<p>It should be a stupid question, thanks guys who read it for me~</p>",
      "votes": 4,
      "replies": [
        {
          "id": 1693016,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2022-02-16T11:53:30.897000",
          "content": "<p>Hi, actually the CV stands for cross validation, and the key word is <strong>cross</strong> so the local score of 1 fold we don't call it CV.</p>\n<p>when you combine your predicted result from all the folds and calculate validation score on it, it can be called CV score.</p>\n<p>When we train model on all data, we don't do evaluation. Just set a proper number of epoch and run it.</p>",
          "votes": 3,
          "replies": []
        },
        {
          "id": 1693065,
          "author_name": "Lilin Chen",
          "author_url": "",
          "post_date": "2022-02-16T12:17:01.290000",
          "content": "<p>Thanks your response <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>!<br>\nSorry, but I still not understand when everyone says their best CV score, how to calculate?</p>\n<blockquote>\n  <p>when you combine your predicted result from all the folds and calculate validation score on it, it can be called CV score.</p>\n</blockquote>\n<p>Does it mean we calculate fold1 score to foldN score, and average them?</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1693118,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2022-02-16T12:52:41.793000",
          "content": "<p>Near but not the same. combine all labels and predictions from all folds, treat it as a single output, then calculate metric on it.</p>\n<p><a href=\"https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614\" target=\"_blank\">https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175614</a></p>\n<p>maybe this post can help you.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1693143,
          "author_name": "dauriel",
          "author_url": "",
          "post_date": "2022-02-16T13:20:44.893000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> I am reading the post and still find it a bit confusing:</p>\n<p>Training I understand. You have N folds. You create N independent models, train them on every fold but one each and then combine the test fold of each to create the oof file. You then compare the oof file to the groundtruth targets.</p>\n<p>What do you do in inference? Do you infere each image on each of the N models and then ensemble?</p>\n<p>Thank you in advance!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1693161,
          "author_name": "Lilin Chen",
          "author_url": "",
          "post_date": "2022-02-16T13:36:11.640000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>!<br>\nThank you so much. I think I got the point~</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1738298,
      "author_name": "Hokaggle",
      "author_url": "",
      "post_date": "2022-03-29T07:36:13.697000",
      "content": "<p>This work of your team is amazing. Thank for your sharing <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>  !<br>\nThere is some part of your solution that I don't understand, please help me to understand it:</p>\n<p>1) How you tuned each parameters of model yolo v5, I assume that it's like you track the performance of the parameter at a range of value with keep all other parameters at 0 (during n epochs) ? If yes, what is the n that you usually choose ?<br>\n2) Why do you choose the image patch with the size (512x320), how you choose this size ?<br>\n3) I don't understand well the part of \"Classification re-score\". As I understand, you crop out all predicted boxes and use the average of 7 bin outputs as classification score, and you use classification score as a label (target) to train a model classification with input -  all predicted boxes already preprocessed.</p>\n<p>I would be glad to hear from you, the champion !</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1734664,
      "author_name": "greatmickey",
      "author_url": "",
      "post_date": "2022-03-25T14:38:57.287000",
      "content": "<p>抱歉,現在再用此數據集做研究<br>\n想請問數據集中的標註excel檔我跟怎麼轉換yolov5的txt格式</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1700915,
      "author_name": "Bavesh123",
      "author_url": "",
      "post_date": "2022-02-22T11:29:16.257000",
      "content": "<p><a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> Could you please explain what is the meaning of \"averaging out the output of all the 7 bins to get the classification score\". For example, in the example list that you had given [1, 1, 1, 0, 0, 0, 0], what is the output of the average of the 7 bins in this case. I know this is a stupid question but it would be really nice if someone could give the correct meaning behind that line. Thanks in advance!! Also, great work on securing the first place in the competiition.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1698424,
      "author_name": "Anton Semenistyy",
      "author_url": "",
      "post_date": "2022-02-20T11:28:33.350000",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a>. Thanks for sharing your solution and congrats to 1 place! Could you please answer some questions?</p>\n<ul>\n<li>What yolo models did you use? These were yolov5l6 or yolov5s6 models or some other ones. </li>\n<li>Did you train detection models only on images with labels or also add some background images without labels?</li>\n</ul>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1696237,
      "author_name": "GalChris",
      "author_url": "",
      "post_date": "2022-02-18T17:21:43.657000",
      "content": "<p>Good job. Nice</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1695440,
      "author_name": "viraj kadam",
      "author_url": "",
      "post_date": "2022-02-18T06:37:40.147000",
      "content": "<p>Congrats ! and thanks for sharing the approach..</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1695332,
      "author_name": "sbdarijit",
      "author_url": "",
      "post_date": "2022-02-18T04:50:29.690000",
      "content": "<p>Good job. Congrats on results</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1694811,
      "author_name": "gökay aydoğan",
      "author_url": "",
      "post_date": "2022-02-17T18:18:11.953000",
      "content": "<p>congratulations Qishen Ha , nvnnghia and steamedsheep</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1694118,
      "author_name": "Niek van der Zwaag",
      "author_url": "",
      "post_date": "2022-02-17T07:00:05.460000",
      "content": "<p>Congratulations on 1st place and thanks for sharing - so much to learn from this team 🙌</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1693070,
      "author_name": "Alexandre Cadrin-Chênevert",
      "author_url": "",
      "post_date": "2022-02-16T12:18:32.620000",
      "content": "<p>Congrats for your decisive win and your wise focus on staying aligned to CV only. It really looks like an incredible work of optimization. Can you elaborate a little bit on how you tuned your parameters (conf threshold, nms or wbf iou) to maximize F2 for your <strong>ensemble</strong> of models and on which subset of train data the CV 0.716 (average of OOF?) was calculated ? Or in other words, were the 6 models below trained on the same folds for your experiments but just with different hyperparameters or seeds ?</p>\n<blockquote>\n  <p>ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676. </p>\n</blockquote>",
      "votes": 1,
      "replies": [
        {
          "id": 1693156,
          "author_name": "nvnn",
          "author_url": "",
          "post_date": "2022-02-16T13:32:33.870000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/alexandrecc\" target=\"_blank\">@alexandrecc</a> <br>\nthe parameters are tuned using oof files. A model has an oof file, this oof file is created by combining raw predictions of all its folds.  so, an oof file contains the predictions and labels of the entire training data. The parameters are then tuned by trial and error to achieve the best F2 score.<br>\nThe 6 models are trained on all 3 folds (18 checkpoint in total), and the CV is calculate from the OOF (it covers entire training set)</p>",
          "votes": 3,
          "replies": []
        },
        {
          "id": 1695392,
          "author_name": "aquaright",
          "author_url": "",
          "post_date": "2022-02-18T05:45:36.387000",
          "content": "<p>Can I ask if you guys used any special method to tune your hparams? Did you guys use something like YOLOv5's wandb sweeps or did you guys search params manually.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1695751,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2022-02-18T10:44:16.387000",
          "content": "<p>For me, manually tune by intuition.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1692841,
      "author_name": "Hoda",
      "author_url": "",
      "post_date": "2022-02-16T09:24:26.920000",
      "content": "<p>Congrats! I have learned a lot from <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> notebook and what you have described here was awesome too.  </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1692453,
      "author_name": "dragon zhang",
      "author_url": "",
      "post_date": "2022-02-16T03:46:23.460000",
      "content": "<p>congrats!  I have similar classification idea etc. without good hardware support, no patience to try.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1692428,
      "author_name": "choco",
      "author_url": "",
      "post_date": "2022-02-16T03:21:23.127000",
      "content": "<p>Thanks for sharing your code. i have some question about classification part. </p>\n<ul>\n<li>why using not binary but 7 bins? (we are trying &lt; 0.3 label 0 &gt; 0.8 label 1 but failed :( ) </li>\n<li>How do you use it in reference? You used 7 different results? Or did you use max or something else?</li>\n<li>Our model didn't learn well because of few label noise. Did your team deal with label noise?</li>\n</ul>\n<p>Thank you for a very interesting approach. Congratulations on winning first place.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1692447,
          "author_name": "Qishen Ha",
          "author_url": "",
          "post_date": "2022-02-16T03:43:51.063000",
          "content": "<p>Thank you and congratulations to you, too!</p>\n<ol>\n<li>It was a gut feeling, I used this method very directly when writing the code and didn't try any other method else.</li>\n<li>avg the output of 7 bins.</li>\n<li>compared to most of the competitions, this dataset label noise is not that serious, so we chose not to deal with it.</li>\n</ol>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1692482,
          "author_name": "choco",
          "author_url": "",
          "post_date": "2022-02-16T04:15:29.497000",
          "content": "<p>Thanks. Clearly understand !! 👍👍</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1692776,
      "author_name": "Sanyam Bhutani",
      "author_url": "",
      "post_date": "2022-02-16T08:16:15.180000",
      "content": "<p>Congratulations again to the team! </p>\n<p>I wanted to mention, I'll be interviewing Grandmaster <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> next weekend, if anyone has any questions/topics that you'd like to be discussed in the chai interview, please let me know! </p>\n<p>TIA! </p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1692464,
      "author_name": "Somesh88",
      "author_url": "",
      "post_date": "2022-02-16T03:57:56.783000",
      "content": "<p>great job, Congrats. </p>\n<p>can you please provide your training notebooks? </p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1696460,
      "author_name": "Fghjkgcfxx56u",
      "author_url": "",
      "post_date": "2022-02-18T21:14:45.277000",
      "content": "<p>abcdefghij</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1695676,
      "author_name": "Aruna S",
      "author_url": "",
      "post_date": "2022-02-18T10:01:52.503000",
      "content": "<p>Congraatulations🤩🤩Thanks for sharing your code <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> , new follower 🙋‍♀️😊</p>",
      "votes": -1,
      "replies": []
    },
    {
      "id": 2075482,
      "author_name": "Seeing Times",
      "author_url": "",
      "post_date": "2022-12-25T13:41:09.993000",
      "content": "<p>New to kaggle, where I can find the code?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1961408,
      "author_name": "Kangseongdeok",
      "author_url": "",
      "post_date": "2022-09-29T06:42:18.540000",
      "content": "<p>good job!!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1778633,
      "author_name": "Zexuan Yang",
      "author_url": "",
      "post_date": "2022-05-05T14:30:34.817000",
      "content": "<p>Congratulations! Your solution is helpful for me!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1734661,
      "author_name": "greatmickey",
      "author_url": "",
      "post_date": "2022-03-25T14:36:24.603000",
      "content": "<p>hello.<br>\nInquiry data (excel) conversion yolov5 (txt)<br>\nHow is the conversion?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1734636,
      "author_name": "greatmickey",
      "author_url": "",
      "post_date": "2022-03-25T14:16:41.380000",
      "content": "<p>Inquiry data (excel) conversion yolov5 (txt)<br>\nHow is the conversion?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1708849,
      "author_name": "Ishan Mehta115",
      "author_url": "",
      "post_date": "2022-03-01T18:35:44.553000",
      "content": "<p>Congratulations🎉</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1706956,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-28T02:36:37.117000",
      "content": "<p>this is a nice work!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1703548,
      "author_name": "DUNHAOTIAN",
      "author_url": "",
      "post_date": "2022-02-24T16:03:33.493000",
      "content": "<p>nice job bro</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1702468,
      "author_name": "Aditya Danayak",
      "author_url": "",
      "post_date": "2022-02-23T16:22:05.607000",
      "content": "<p>Well done!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1702357,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-23T14:41:57.493000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1701970,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-23T07:58:17.170000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1699741,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-21T11:58:31.517000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1699355,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-21T06:04:54.237000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1698734,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-20T16:11:43.300000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1697799,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-19T21:04:41.447000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1697489,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-19T16:51:17.890000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1697290,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-19T14:11:53.700000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1697242,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-19T13:26:39.497000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1698239,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-20T08:25:45.930000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1696387,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-18T19:49:16.010000",
      "content": "",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1695708,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-18T10:26:13.853000",
      "content": "",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1694944,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-17T21:02:48.180000",
      "content": "",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1701034,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-22T13:26:12.220000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1700413,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-22T00:33:01.920000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1700251,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-21T19:01:26.943000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1699345,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-21T05:53:52.757000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1699240,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-21T03:39:53.800000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1699204,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-21T02:26:46.600000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1699133,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-21T00:16:29.953000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1699114,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-20T23:54:41.927000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1698712,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-20T15:45:57.643000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1698661,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-20T15:04:59.873000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1698439,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-20T11:44:06.267000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1697957,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-20T02:26:43.327000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1697843,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-19T22:06:14.070000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1697368,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-19T15:13:45.413000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1697187,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-02-19T12:41:59.587000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1692417": "Thanks to the organizers and congrats to all the winners and my wonderful teammates @nvnnghia and @steamedsheep \n\nThis is really very unexpected for us. Because we don't have any NEW THING, we just keep optimizing cross validation F2 of our pipeline locally from the beginning till the end.\n\n# Summary\n\nWe designed a 2-stage pipeline. object detection -> classification re-score. Then a post-processing method follows.\n\nValidation strategy: 3-fold cross validation split by video_id.\n\n## Object detection\n\n* 6 yolov5 models, 3 trained on 3648 images and 3 trained on 1536 image patches (described below)\n* image patches: we cut original image (1280x720) into many patches (512x320), removed boxes near boundary, then only train yolov5 on those patches with cots.\n* modified some yolo hyper-parameters based on default: `box=0.2`, `iou_t=0.3`\n* augmentations: based on yolov5 default augmentations, we added: rotation, mixup and albumentations.Transpose, then removed HSV.\n* after the optimization was completed by cross validation, we trained the final models with all the data.\n* all models are inferred using the same image size as trained.\n* ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676.\n\n## Classification re-score:\n* crop out all predicted boxes (3-fold OOF) into squares wich conf > 0.01. The side length of the square is `max(length, width)` of the predicted boxes, then extended by 20%.\n* we calculate the iou as the maximum of the iou values of each predicted box and GT boxes of this image.\n* classification target of each cropped box: iou>0.3, iou>0.4, iou>0.5, iou>0.6, iou>0.7, iou>0.8 and iou>0.9 Simply put, the iou is divided into 7 bins. e.g.: `[1,1,1,0,0,0,0]` indicates the iou is between 0.5 and 0.6.\n* during inference we average 7 bin outputs as classification score.\n* then we use BCELoss to train those cropped boxes by size 256x256 or 224x224.\n* a very high dropout_rate or drop_path_rate can help a lot to improve the performance of the classification model. We use `dropout_rate=0.7` and `drop_path_rate=0.5`\n* augmentations: hflip, vflip, transpose, 45° rotation and cutout.\nThe best classification model can boost out CV to 0.727\n* after ensemble some classification models, our CV comes to 0.73+\n\n## Post-processing\n\nFinally, we use a simple post-processing to further boost our CV to 0.74+.\nFor example, the model has predicted some boxes B at #N frame, select the boxes from B which has a high confidence, these boxes are marked as \"attention area\".\nin the #N+1, #N+2, #N+3 frame, for the predicted boxes with conf > 0.01,  if it has an IoU with the \"attention area\" larger than 0, boost the score of these boxes with `score += confidence * IOU`\n\nWe also tried the tracking method, which gives us a CV of +0.002. However, it introduces two additional hyperparameters. We therefore chose not to use it.\n\n# Little story\n\nAt the beginning of the competition, we used different F2 algorithms for each of the three members of our team, and later we found that for the same oof, we did not calculate the same score.\nFor example, nvnn shared an OOF file with `F2=0.62`, and sheep calculated `F2=0.66`, while I calculated `F2=0.68`.\nWe finally chose to use the F2 algorithm with the lowest score from nvnn to evaluate all our models.\n\nhttps://www.kaggle.com/haqishen/f2-evaluation/script\n\nHere's our final F2 algorithm, if you are interested you can use this algorithm to compare your CV with ours!\n\n# Acknowledge\n\nAs usual, I trained many models in this competition using Z by HP Z8G4 Workstation with dual A6000 GPU. The large memory of 48G for a single GPU allowed me to train large resolution images with ease. Thanks to Z by HP for sponsoring!",
    "1692710": "First of all congratulations! Great work and great solution description! 👋👋👋 Attention is really great tip and your way to decreading number of hyperparameters cool!\n\n- As I can see you did not inference on mega resolution ;) Just as you trained model. How did you come to the conclution that it would lead to overfitt priv LB?\n- What NN architecture did you use for classifier?\n- What Albumentations pipeline looks like?\n- Are final model trained on all videos or each model on 3/1 split?\n\nSorry for long list of question but I have an opportunity to learn from the best 👍👍👍",
    "1692456": "Good job. Congrats @nvnnghia, @haqishen and @steamedsheep on results!",
    "1692835": "You guys are my heros I will learn many things from you!!!\n@haqishen And there is maybe a stupid question I confuse it from long time ago.\nHope someone could correct me.\n\n**What's exactly \"CV\" result?**\nIn my thougth, CV means cross validation, so we could get different score at each fold experiment. \nFor example, in the comepetion we pursuit F2 score, so if could get\n| fold | f2 score   |\n| --- | --- |\n|  1| 0.6XX  |\n|  2| 0.5XX  |\n|  3| 0.7XX  |\n\nSo I call fold1 model get CV0.6XX, fold2 model get CV0.5XX, and fold3 model get CV0.7XX.\n\n*\"after the optimization was completed by cross validation, we trained the final models with all the data.\"*\nHowever, how to evaluate \"CV\" after we trained the all data? they are all be trained.\n\n*\"ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676.\"*\nWe ensemble all fold, there are all be trained, too.\n\nIt should be a stupid question, thanks guys who read it for me~",
    "1738298": "This work of your team is amazing. Thank for your sharing @haqishen  !\nThere is some part of your solution that I don't understand, please help me to understand it:\n\n1) How you tuned each parameters of model yolo v5, I assume that it's like you track the performance of the parameter at a range of value with keep all other parameters at 0 (during n epochs) ? If yes, what is the n that you usually choose ?\n2) Why do you choose the image patch with the size (512x320), how you choose this size ?\n3) I don't understand well the part of \"Classification re-score\". As I understand, you crop out all predicted boxes and use the average of 7 bin outputs as classification score, and you use classification score as a label (target) to train a model classification with input -  all predicted boxes already preprocessed.\n\nI would be glad to hear from you, the champion !",
    "1734664": "抱歉,現在再用此數據集做研究\n想請問數據集中的標註excel檔我跟怎麼轉換yolov5的txt格式",
    "1700915": "@haqishen Could you please explain what is the meaning of \"averaging out the output of all the 7 bins to get the classification score\". For example, in the example list that you had given [1, 1, 1, 0, 0, 0, 0], what is the output of the average of the 7 bins in this case. I know this is a stupid question but it would be really nice if someone could give the correct meaning behind that line. Thanks in advance!! Also, great work on securing the first place in the competiition.",
    "1698424": "Hi, @haqishen. Thanks for sharing your solution and congrats to 1 place! Could you please answer some questions?\n \n- What yolo models did you use? These were yolov5l6 or yolov5s6 models or some other ones. \n- Did you train detection models only on images with labels or also add some background images without labels?",
    "1696237": "Good job. Nice",
    "1695440": "Congrats ! and thanks for sharing the approach..",
    "1695332": "Good job. Congrats on results",
    "1694811": "congratulations Qishen Ha , nvnnghia and steamedsheep",
    "1694118": "Congratulations on 1st place and thanks for sharing - so much to learn from this team 🙌",
    "1693070": "Congrats for your decisive win and your wise focus on staying aligned to CV only. It really looks like an incredible work of optimization. Can you elaborate a little bit on how you tuned your parameters (conf threshold, nms or wbf iou) to maximize F2 for your **ensemble** of models and on which subset of train data the CV 0.716 (average of OOF?) was calculated ? Or in other words, were the 6 models below trained on the same folds for your experiments but just with different hyperparameters or seeds ?\n> ensemble these 6 yolov5 models gives us CV0.716, in addition the best one is CV0.676. ",
    "1692841": "Congrats! I have learned a lot from @steamedsheep notebook and what you have described here was awesome too.  ",
    "1692453": "congrats!  I have similar classification idea etc. without good hardware support, no patience to try.",
    "1692428": "Thanks for sharing your code. i have some question about classification part. \n- why using not binary but 7 bins? (we are trying < 0.3 label 0 > 0.8 label 1 but failed :( ) \n- How do you use it in reference? You used 7 different results? Or did you use max or something else?\n- Our model didn't learn well because of few label noise. Did your team deal with label noise?\n\nThank you for a very interesting approach. Congratulations on winning first place.\n",
    "1692776": "Congratulations again to the team! \n\nI wanted to mention, I'll be interviewing Grandmaster @haqishen next weekend, if anyone has any questions/topics that you'd like to be discussed in the chai interview, please let me know! \n\nTIA! ",
    "1692464": "great job, Congrats. \n\ncan you please provide your training notebooks? ",
    "1696460": "abcdefghij\n",
    "1695676": "Congraatulations🤩🤩Thanks for sharing your code @haqishen , new follower 🙋‍♀️😊\n",
    "2075482": "New to kaggle, where I can find the code?",
    "1961408": "good job!!",
    "1778633": "Congratulations! Your solution is helpful for me!",
    "1734661": "hello.\nInquiry data (excel) conversion yolov5 (txt)\nHow is the conversion?",
    "1734636": "Inquiry data (excel) conversion yolov5 (txt)\nHow is the conversion?",
    "1708849": "Congratulations🎉",
    "1706956": "this is a nice work!",
    "1703548": "nice job bro",
    "1702468": "Well done!",
    "1702357": "It's an interesting cross-validation method",
    "1701970": "Congrats!  Thanks for sharing this approach toward a solution.\n\n\n",
    "1699741": "I also follow the motto of always trusting my cv, but sometimes especially when I am dealing with imbalanced datasets, for some reason I tend to doubt my CVs, is my doubt justified? Also, Nice work and congratulations.",
    "1699355": "Congrats ! Thx for sharing ! ",
    "1698734": "Congratulations and thank you for sharing!",
    "1697799": "Congrats. Good job.",
    "1697489": "This is great and thanks you for the brief explanation.",
    "1697290": "WOW. You got a huge prize. congrats ",
    "1697242": "good job.........",
    "1698239": "Good job! Congrats",
    "1696387": "Thank you for sharing",
    "1695708": "Good job! Congrats, Thanks for sharing",
    "1694944": "Thank you for sharing your solution!",
    "1701034": "@haqishen, thanks for sharing!",
    "1700413": "thank for your sharing!",
    "1700251": "Thank you and congrats!",
    "1699345": "Great job! Thank you for sharing!",
    "1699240": "Thank you for sharing the approach",
    "1699204": "Thank you for sharing",
    "1699133": "Thank you for sharing",
    "1699114": "Great work, thanks for sharing",
    "1698712": "Good job! Congrats, Thanks for sharing",
    "1698661": "wow, that's help! thanks a lot!",
    "1698439": "Thank you for Sharing",
    "1697957": "Thanks for sharing.",
    "1697843": "Congrats and thanks for sharing.",
    "1697368": "Thank you for sharing! 👍",
    "1697187": "thank you so much\n"
  }
}