post your best single models =)
\nframework; PyTorch\nmodel: FasterRCNN\nsplit: 80/20 (only containing label)\nepoch: 20\nCV: 0.474\nLB: 0.496\nI think some work has to be done to figure what is the best way to split ..
", "messages": [ { "id": 1595843, "postDate": "2021-11-26T03:27:58.050Z", "content": "post your best single models =)
\nframework; PyTorch\nmodel: FasterRCNN\nsplit: 80/20 (only containing label)\nepoch: 20\nCV: 0.474\nLB: 0.496\nI think some work has to be done to figure what is the best way to split ..
", "rawMarkdown": "post your best single models =) \n\n\n```\nframework; PyTorch\nmodel: FasterRCNN\nsplit: 80/20 (only containing label)\nepoch: 20\nCV: 0.474\nLB: 0.496\n```\n\nI think some work has to be done to figure what is the best way to split ..", "votes": 109 }, { "id": 1677259, "postDate": "2022-02-05T15:52:29.437Z", "content": "model: Cascade R-CNN R50 (MMDET)
\nspilt : cross validation by video id then train all
\nLB : 0.741
\nhardware: RTX3090
comment
\n1.yolov5 is not all you need.
\n2.to fit or not to fit that is the question.
\n3.I'm not going to answer any questions.
\n The post itself is the biggest hint .
nice, it's enough. I will try it and come back soon.
", "rawMarkdown": "nice, it's enough. I will try it and come back soon." }, { "id": 1677271, "postDate": "2022-02-05T16:02:36.857Z", "content": "great job @atom1231!
\nif you like to reply, what is the CV score, and
\nwhat evaluator you use for MMDET (standard coco, or modified for f2) ?
@imeintanis coco MAP
", "rawMarkdown": "@imeintanis coco MAP", "votes": 1 }, { "id": 1678057, "postDate": "2022-02-06T08:25:39.287Z", "content": "great!
\n1.Could you please tell me the img size for traing? multi-scale right？
\n2.\" cross validation by video id then train all\" What exactly does this mean？
\n3.I once thought hrnet was the best potential，Have you tested it？
Can you please give some tips on how to train on all data after validation?
\nLike what all do we have to fix?
Ohhhh … noooo????? Yolov5 is not all you need? …. Are you sure???? And size is not all you need? Ohhhhh…… oh…… 😜😜😜😜😜 I was writing about this month ago (that this is …. just not responsible to put people in this direction ….) These topics were just the gratest clickbait during this competition ….
", "rawMarkdown": "Ohhhh ... noooo????? Yolov5 is not all you need? .... Are you sure???? And size is not all you need? Ohhhhh...... oh...... 😜😜😜😜😜 I was writing about this month ago (that this is .... just not responsible to put people in this direction ....) These topics were just the gratest clickbait during this competition .... ", "votes": -6 }, { "id": 1678484, "postDate": "2022-02-06T15:05:30.370Z", "content": "@junyun1002 @shashwatraman
\n1.->ya
\n2.->make sure the model performance is improved in different folds then use the hyperparameter to train with all data.
\n3.->no .
Thank you very much 😃
", "rawMarkdown": "Thank you very much 😃" }, { "id": 1678499, "postDate": "2022-02-06T15:16:52.923Z", "content": "Just one more thing, how do we know that which epoch is the best?
", "rawMarkdown": "Just one more thing, how do we know that which epoch is the best? " }, { "id": 1678504, "postDate": "2022-02-06T15:20:39.177Z", "content": "Yeah, I almost sink into yolov5 until I see the comment. I find that I seems ignored the two stage method.
", "rawMarkdown": "Yeah, I almost sink into yolov5 until I see the comment. I find that I seems ignored the two stage method.", "votes": 1 }, { "id": 1678508, "postDate": "2022-02-06T15:22:39.497Z", "content": "Thank you very much。
", "rawMarkdown": "Thank you very much。" }, { "id": 1680712, "postDate": "2022-02-08T01:27:59.733Z", "content": "@shashwatraman
\nwe do not know but at least we can submit all checkpoints to get best LB score.
\nI choose the epoch value by the result of cross validation.
Thanks 😬😬
", "rawMarkdown": "Thanks 😬😬" }, { "id": 1682587, "postDate": "2022-02-09T09:00:06.330Z", "content": "Thank you for sharing, mmdet rocks 🤘
\nIs mAP make sense when choosing model? mAP of mmdet models better then yolo?
\nchoosing conf and resolution is just a LB probbing?
@nicksergievskiy
\nSince I found the ridiculous \"high resolution solution\" , I decided not to spend too much time in the competition and regarded it as game or lottery.
\nMy goal is limited to get best LB score.
Is mAP make sense when choosing model?
\n=> Of course F2 make sense , but I found my mmdet model mAP correlated with LB in limited cases. I
\ndo not implement F2 metric in my pipeline.
mAP of mmdet models better then yolo?
\n=>For LB , It is hard to say.
choosing conf and resolution is just a LB probbing?
\n=>resolution selected by the analysis of training data.
\n conf is simply LB probing.
Hi, Could I ask one more hint? Please!
\nDoes this single model use TTA to get LB : 0.741 or not?
@lilinchen
\nyes, but without TTA the model also got similar LB score.
@atom1231 thanks!
\nI will look forward to your final solution, and hope you win!!!
\nDefeat YOLO family!!!
@atom1231 and @lilinchen it would be very very nice to see solution based on C-RCNN - I agree. I keep my fingers crossed for success 👍🤛💪
", "rawMarkdown": "@atom1231 and @lilinchen it would be very very nice to see solution based on C-RCNN - I agree. I keep my fingers crossed for success 👍🤛💪" }, { "id": 1684115, "postDate": "2022-02-10T09:26:19.247Z", "content": "Sorry to disturb, really want to know if without the tracking or other post-processing, the 0.741 LB model will decrease how many score, maybe -0.1 ?
", "rawMarkdown": "Sorry to disturb, really want to know if without the tracking or other post-processing, the 0.741 LB model will decrease how many score, maybe -0.1 ?", "votes": -3 }, { "id": 1684591, "postDate": "2022-02-10T15:37:24.977Z", "content": "@atom1231 I spent over two weeks developing a solution with pre-training with 100+ epochs, gan-aided copy&paste, and other optimizations to LR only to submit a model that had a bad cv for fun to get 0.753 public LB. I came back to your comment here because it is true.
\n\n", "rawMarkdown": "@atom1231 I spent over two weeks developing a solution with pre-training with 100+ epochs, gan-aided copy&paste, and other optimizations to LR only to submit a model that had a bad cv for fun to get 0.753 public LB. I came back to your comment here because it is true.\n\n> Since I found the ridiculous \"high resolution solution\" , I decided not to spend too much time in the competition and regarded it as game or lottery.", "votes": 6 } ] }, { "id": 1595988, "postDate": "2021-11-26T05:51:26.217Z", "content": "Since I found the ridiculous \"high resolution solution\" , I decided not to spend too much time in the competition and regarded it as game or lottery.
\n
model: yolox
\nsplit: 80/20 (train: only containing label, val: all data)
\nval score: 0.496
\nlb: 0.484
my metric implementation
\nfrom typing import List\n\nimport numpy as np\nimport torch\nfrom torchvision.ops import box_iou\n\n\ndef calculate_score(\n preds: List[torch.Tensor],\n gts: List[torch.Tensor],\n iou_th: float\n) -> float:\n num_tp = 0\n num_fp = 0\n num_fn = 0\n for p, gt in zip(preds, gts):\n if len(p) and len(gt):\n iou_matrix = box_iou(p, gt)\n tp = len(torch.where(iou_matrix.max(0)[0] >= iou_th)[0])\n fp = len(p) - tp\n fn = len(torch.where(iou_matrix.max(0)[0] < iou_th)[0])\n num_tp += tp\n num_fp += fp\n num_fn += fn\n elif len(p) == 0 and len(gt):\n num_fn += len(gt)\n elif len(p) and len(gt) == 0:\n num_fp += len(p)\n score = 5 * num_tp / (5 * num_tp + 4 * num_fn + num_fp)\n return score\n\niou_ths = np.arange(0.3, 0.85, 0.05)\nscores = [calculate_score(predictions, gts, iou_th) for iou_th in iou_ths]\nnp.mean(scores)\n>>> 0.49588419817665447\ncan you provide the github or any link for the yolox model you are using?
", "rawMarkdown": "can you provide the github or any link for the yolox model you are using?", "votes": 2 }, { "id": 1598200, "postDate": "2021-11-28T10:03:19.477Z", "content": "https://github.com/open-mmlab/mmdetection/blob/master/configs/yolox/README.md
", "rawMarkdown": "https://github.com/open-mmlab/mmdetection/blob/master/configs/yolox/README.md", "votes": 5 }, { "id": 1598977, "postDate": "2021-11-29T03:44:48.367Z", "content": "I use this repo.
\nhttps://github.com/Megvii-BaseDetection/YOLOX
\nAbout yolox, main contributions to mAP are augmentation, label assignment and center sampling.
\nthese are general-purpose methods, so you can combine them with other mtehods such as faster r-cnn, fcos.
@phalanx could you please explain, label assignment and center sampling ?
@awsaf49 You can refer to the yolox paper.
\nhttps://arxiv.org/abs/2107.08430
What input resolution? 640px?
\nI am asking because in this competition I would like to learn something new. YoloX seems to me promising in both - speed, accuracy and … as a last it is opportunity yo learn something new (I have not use it so far - y3/y4/y5/yr yes but not x).
you can check out official repo of yolo they have multiple resolution as input
\nhttps://github.com/ultralytics/yolov5
Thank you. I just asked about yoloX
\nBTW Your score with FasterRCNN is quite good so far! 👍
", "rawMarkdown": "Thank you. I just asked about yoloX \n\nBTW Your score with FasterRCNN is quite good so far! 👍", "votes": 1 }, { "id": 1599666, "postDate": "2021-11-29T16:54:55.440Z", "content": "ohh I assumed yoloX and yolo5x are the same =(
Thanks for sharing the metric, I've been trying it but unfortunately it seems to be overestimating my LB during CV (CV >> LB). I've tried different validation strategies, Groupkfold, startified group kfold by sequence id and num annotations, … but always CV >> LB. Then I tried to modify the metric by making it less elegant but more verbose to:
\ndef calculate_score(\n preds,\n gts,\n iou_th):\n num_tp = 0\n num_fp = 0\n num_fn = 0\n for p, gt in zip(preds, gts):\n if len(p) and len(gt):\n tp = 0\n fp = 0\n iou_matrix = box_iou(p, gt)\n n = iou_matrix.shape[0]\n for i in range(n):\n row = iou_matrix[i]\n match_idxs = torch.where(row > iou_th)[0]\n if len(match_idxs)>0:\n match_idx = match_idxs[0].item()\n # remove matched gt box\n iou_matrix = torch.cat([iou_matrix[:,:match_idx], iou_matrix[:,match_idx+1:]], dim=1)\n tp += 1\n else:\n fp += 1\n fn = iou_matrix.shape[1]\n\n num_tp += tp\n num_fp += fp\n num_fn += fn\n elif len(p) == 0 and len(gt):\n num_fn += len(gt)\n elif len(p) and len(gt) == 0:\n num_fp += len(p)\n score = 5 * num_tp / (5 * num_tp + 4 * num_fn + num_fp)\n return score\n, after this change now LB and CV are closer.
", "rawMarkdown": "Thanks for sharing the metric, I've been trying it but unfortunately it seems to be overestimating my LB during CV (CV >> LB). I've tried different validation strategies, Groupkfold, startified group kfold by sequence id and num annotations, ... but always CV >> LB. Then I tried to modify the metric by making it less elegant but more verbose to: \n```\ndef calculate_score(\n preds,\n gts,\n iou_th):\n num_tp = 0\n num_fp = 0\n num_fn = 0\n for p, gt in zip(preds, gts):\n if len(p) and len(gt):\n tp = 0\n fp = 0\n iou_matrix = box_iou(p, gt)\n n = iou_matrix.shape[0]\n for i in range(n):\n row = iou_matrix[i]\n match_idxs = torch.where(row > iou_th)[0]\n if len(match_idxs)>0:\n match_idx = match_idxs[0].item()\n # remove matched gt box\n iou_matrix = torch.cat([iou_matrix[:,:match_idx], iou_matrix[:,match_idx+1:]], dim=1)\n tp += 1\n else:\n fp += 1\n fn = iou_matrix.shape[1]\n\n num_tp += tp\n num_fp += fp\n num_fn += fn\n elif len(p) == 0 and len(gt):\n num_fn += len(gt)\n elif len(p) and len(gt) == 0:\n num_fp += len(p)\n score = 5 * num_tp / (5 * num_tp + 4 * num_fn + num_fp)\n return score\n```, after this change now LB and CV are closer.", "votes": 6 }, { "id": 1600462, "postDate": "2021-11-30T12:48:57.883Z", "content": "@mrinath look here I implemented full COTS competition pipeline using YoloX -> https://www.kaggle.com/remekkinas/yolox-full-training-pipeline-for-cots-dataset
\n@phalanx - could you tell me how to make some improvements in my YoloX notebook?
", "rawMarkdown": "@mrinath look here I implemented full COTS competition pipeline using YoloX -> https://www.kaggle.com/remekkinas/yolox-full-training-pipeline-for-cots-dataset\n\n@phalanx - could you tell me how to make some improvements in my YoloX notebook?", "votes": 3 }, { "id": 1600487, "postDate": "2021-11-30T13:14:04.617Z", "content": "Thanks @remekkinas
", "rawMarkdown": "Thanks @remekkinas ", "votes": 1 }, { "id": 1606804, "postDate": "2021-12-05T09:20:04.893Z", "content": "Hi, I use the author's metric, and my local cv is also high, but lb is very low, just 0.021, did you figure out why?
", "rawMarkdown": "Hi, I use the author's metric, and my local cv is also high, but lb is very low, just 0.021, did you figure out why?" }, { "id": 1611581, "postDate": "2021-12-08T04:51:13.380Z", "content": "I still use the metric that I shared above, you can give it a try, it seems to work e.g. CV and LB correlates. We don't know the distribution of empty images in the public test set, so if your validation has a drastically different ratio of empty images that might also underestimate or overestimate depending on which side you are on.
\nMy latest model
\nTrain : Video 0,1 (Only positive images)\nValid: Video 2 (All images)\nYOLOX-L (Full res: 800,1280) TH:0.03 (Probably overfitting)- CV: 0.58 LB: 0.486\nOK, now my model's result looks like normal, before my lb is only 0.021, now my lb is 0.467, and cv is 0.79, although still have large gap, I will try your metric to see if cv can be down
", "rawMarkdown": "OK, now my model's result looks like normal, before my lb is only 0.021, now my lb is 0.467, and cv is 0.79, although still have large gap, I will try your metric to see if cv can be down" }, { "id": 1611936, "postDate": "2021-12-08T12:01:55.213Z", "content": "the metric's result don't have different with phalanx's metric, I should study how to split train/val
", "rawMarkdown": "the metric's result don't have different with phalanx's metric, I should study how to split train/val" }, { "id": 1614649, "postDate": "2021-12-11T11:15:06.353Z", "content": "@keremt Hi, I try your metric, NameError: name 'box_iou' is not defined. May I know how to calculate box_iou?
", "rawMarkdown": "@keremt Hi, I try your metric, NameError: name 'box_iou' is not defined. May I know how to calculate box_iou?" }, { "id": 1615349, "postDate": "2021-12-12T07:11:16.573Z", "content": "from torchvision.ops import box_iou
", "rawMarkdown": "from torchvision.ops import box_iou" }, { "id": 1616336, "postDate": "2021-12-13T10:55:50.193Z", "content": "I found that changing one param in coco_eval.py constructs competition metrics for precission and recall, its easy to calculate F2 score. I made topic how to make it work
\nhttps://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/294854
", "rawMarkdown": "I found that changing one param in coco_eval.py constructs competition metrics for precission and recall, its easy to calculate F2 score. I made topic how to make it work\n\nhttps://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/294854", "votes": 2 }, { "id": 1622132, "postDate": "2021-12-18T11:52:54.010Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1622136, "postDate": "2021-12-18T11:56:14.110Z", "content": "Yes.
\nBetter CV dataset. My published implementation uses simple one. You can find some notebooks which use better dataset cv strategy. Then … you will be able without problem jump over 0.5. I am sure.
Regarding to @phalanx implementation of metric, i made some changes to it. Now it correctly do metrics with empty images. Tested with only annotated and annotated+unannoted and recall is same in both metrics which means it works :) i have added confidence threshold to calculate score for given confthre. There is also script how to feed tensor to this procedure. And finally calculation of recall and precision in 0.3 and 0.5 iou threshold. Cheers and good luck in competition :)
\ndl_val is pytorch dataloader for validation set - you can feed images with empty annotation and it will still hold the recall)
\ndef calculate_score(\n preds: List[torch.Tensor],\n gts: List[torch.Tensor],\n iou_th: float\n) -> float:\n num_tp = 0\n num_fp = 0\n num_fn = 0\n for p, gt in zip(preds, gts):\n if len(p) and len(gt):\n iou_matrix = box_iou(p, gt)\n tp = len(torch.where(iou_matrix.max(0)[0] >= iou_th)[0])\n fp = len(p) - tp\n fn = len(torch.where(iou_matrix.max(0)[0] < iou_th)[0])\n num_tp += tp\n num_fp += fp\n num_fn += fn\n elif len(p) == 0 and len(gt):\n num_fn += len(gt)\n elif len(p) and len(gt) == 0:\n num_fp += len(p)\n if (5 * num_tp + 4 * num_fn + num_fp )!=0:\n score = 5 * num_tp / (5 * num_tp + 4 * num_fn + num_fp )\n else:\n score = np.nan\n if (num_tp+num_fn) != 0:\n recall = num_tp/ (num_tp+num_fn)\n else:\n recall=np.nan\n if (num_tp+num_fp)!=0:\n precission = num_tp/ (num_tp+num_fp)\n else:\n precission=np.nan\n\n\n return score, precission, recall\ndef evaluate_f2(confthre):\n scores = []\n prec05 = []\n rec05 = []\n prec03 = []\n rec03 = []\n iou_ths = np.arange(0.3, 0.85, 0.05)\n with torch.no_grad():\n for images, targets in dl_val:\n model.eval()\n images = list(image.to(device) for image in images)\n targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n preds = model(images)\n\n for i in range(len(images)):\n preds[i]['boxes']=preds[i]['boxes'].int()\n preds[i]['boxes']=preds[i]['boxes'][preds[i]['scores']>confthre]\n score = [calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), iou_th)[0] for iou_th in iou_ths]\n scores.append(np.nanmean(score))\n prec05.append(calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), 0.5)[1]) \n prec03.append(calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), 0.3)[1]) \n rec05.append(calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), 0.5)[2]) \n rec03.append(calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), 0.3)[2]) \n print(f'F2 Score for confthre , {confthre}, : {np.nanmean(scores):.3f} Precission .5: {np.nanmean(prec05):.3f} Precission .3: {np.nanmean(prec03):.3f} Recall .5: {np.nanmean(rec05):.3f} Recall .3: {np.nanmean(rec03):.3f}')\ngood work and knowledgable，i am confused how can i use it for my own model, such as yolov5. thanks ahead.
", "rawMarkdown": "good work and knowledgable，i am confused how can i use it for my own model, such as yolov5. thanks ahead." }, { "id": 1636863, "postDate": "2022-01-03T10:36:18.720Z", "content": "@remekkinas thanks or sharing ideas.
\nI just joined the competition.
\n1) Which valiation strategy do you use - Subseq, Seq Gkf, videos
\n2) does YoloX has same metric as competition.
\n3) Do you use background images, how mch improvement do you get with it.
sorry … only one answer came to my mind 😂😂😳😳🙄😭😭
", "rawMarkdown": "sorry … only one answer came to my mind 😂😂😳😳🙄😭😭" }, { "id": 1644908, "postDate": "2022-01-10T15:42:40.153Z", "content": "@xiaojiu1414 from torchvision.ops import box_iou
Hi @lukaszborecki I used your provided above code but the model which is giving 0.393 on LB, it is showing f2 = 0.132 using that code,
\nF2 Score for confthre , 0.13, : 0.132 Precission .5: 0.863 Precission .3: 0.902 Recall .5: 0.148 Recall .3: 0.153
\nAm I doing anything wrong here? Sorry for asking this qsn now, when the code has been posted over a month ago.
There are so many implementations of F2 score all around the discussion posts, which one are you guys using ? I am confused
", "rawMarkdown": "There are so many implementations of F2 score all around the discussion posts, which one are you guys using ? I am confused", "votes": 1 }, { "id": 1663614, "postDate": "2022-01-25T09:11:01.050Z", "content": "Hi @mrinath yes, there are many but these two are working fine. There was some problem with my model, But, code provided by lukasz is working fine,
\nand the comp metric implementation by camaro is also ok [although you may need to adjust some code for fasterRCNN, good for yolo]. for camaro's implementation you can add this code for replacing calc_is_correct and calc_f2_score. This modification is bcoz of this reason: f2_score has to be calculated for each IoU threshold. But in the current implementation, f2_score seems to be calculated from the sum of tp/fp/fn for all thresholds. Use the detection threshold first and filter the desired bbox first and then pass them in this code.
def calc_is_correct(gt_bboxes, pred_bboxes, iou_th=0.5):\n \"\"\"\n gt_bboxes: (N, 4) np.array in xywh format\n pred_bboxes: (N, 5) np.array in conf+xywh format\n \"\"\"\n if len(gt_bboxes) == 0 and len(pred_bboxes) == 0:\n tps, fps, fns = 0, 0, 0\n return tps, fps, fns\n\n elif len(gt_bboxes) == 0:\n tps, fps, fns = 0, len(pred_bboxes), 0\n return tps, fps, fns\n\n elif len(pred_bboxes) == 0:\n tps, fps, fns = 0, 0, len(gt_bboxes)\n return tps, fps, fns\n\n pred_bboxes = pred_bboxes[pred_bboxes[:,0].argsort()[::-1]] # sort by conf\n\n tps, fps, fns = 0, 0, 0\n tp, fp, fn = calc_is_correct_at_iou_th(gt_bboxes, pred_bboxes, iou_th)\n tps += tp\n fps += fp\n fns += fn\n return tps, fps, fns\n\ndef calc_f2_score(gt_bboxes_list, pred_bboxes_list, verbose=False):\n \"\"\"\n gt_bboxes_list: list of (N, 4) np.array in xywh format\n pred_bboxes_list: list of (N, 5) np.array in conf+xywh format\n \"\"\"\n f2s = []\n for iou_th in np.arange(0.3, 0.85, 0.05):\n tps, fps, fns = 0, 0, 0\n for gt_bboxes, pred_bboxes in zip(gt_bboxes_list, pred_bboxes_list):\n tp, fp, fn = calc_is_correct(gt_bboxes, pred_bboxes, iou_th)\n tps += tp\n fps += fp\n fns += fn\n if verbose:\n num_gt = len(gt_bboxes)\n num_pred = len(pred_bboxes)\n print(f'num_gt:{num_gt:<3} num_pred:{num_pred:<3} tp:{tp:<3} fp:{fp:<3} fn:{fn:<3}')\n f2 = f_beta(tps, fps, fns, beta=2) \n print(f'f2@{iou_th}:{f2}')\n f2s.append(f2)\n return np.mean(f2s)\nThanks @soumya9977
", "rawMarkdown": "Thanks @soumya9977 ", "votes": 1 } ] }, { "id": 1664138, "postDate": "2022-01-25T18:23:41.220Z", "content": "LB 0.698 -> 0.700
\nInfer Size: 2400
\nFold 90/10 single model
\nGPU: P100'
\nModel: yolo5
May I ask what your cv score of that model is?
", "rawMarkdown": "May I ask what your cv score of that model is?", "votes": 1 }, { "id": 1664664, "postDate": "2022-01-26T06:39:31.990Z", "content": "really crazy…, my best single one is also yolo, but just 0.627, If I got your score, everything wil goes easier!😃
", "rawMarkdown": "really crazy..., my best single one is also yolo, but just 0.627, If I got your score, everything wil goes easier!😃" }, { "id": 1664702, "postDate": "2022-01-26T07:28:16.103Z", "content": "To bo honest on IMG 2400 CV was weaker but on training size it was 0.601
", "rawMarkdown": "To bo honest on IMG 2400 CV was weaker but on training size it was 0.601", "votes": 2 }, { "id": 1664745, "postDate": "2022-01-26T08:36:50.840Z", "content": "Could you share how many epochs did it take?
", "rawMarkdown": "Could you share how many epochs did it take?" }, { "id": 1664755, "postDate": "2022-01-26T08:55:34.093Z", "content": "Training was performed on 10 epochs
", "rawMarkdown": "Training was performed on 10 epochs" }, { "id": 1664764, "postDate": "2022-01-26T09:13:19.553Z", "content": "OK, thanks, I'm giving a try.
", "rawMarkdown": "OK, thanks, I'm giving a try." }, { "id": 1664795, "postDate": "2022-01-26T09:28:58.347Z", "content": "what was the training image size?
", "rawMarkdown": "what was the training image size?" }, { "id": 1664797, "postDate": "2022-01-26T09:30:11.723Z", "content": "So you are using yolov5l6, right? I'm still using yolov5s6…
", "rawMarkdown": "So you are using yolov5l6, right? I'm still using yolov5s6...", "votes": 1 }, { "id": 1664826, "postDate": "2022-01-26T09:55:48.407Z", "content": "yes … but …. I am almost sure that you can use \"s\" as well …. We have plan to switch to S … as well.
", "rawMarkdown": "yes ... but .... I am almost sure that you can use \"s\" as well .... We have plan to switch to S ... as well.", "votes": 1 }, { "id": 1664844, "postDate": "2022-01-26T10:11:35.160Z", "content": "Yeah, I do love s6 more, so tiny. How about the time of l6 for inferencing? if it's really time consuming, I'll give up my mind…
", "rawMarkdown": "Yeah, I do love s6 more, so tiny. How about the time of l6 for inferencing? if it's really time consuming, I'll give up my mind...", "votes": 2 }, { "id": 1664900, "postDate": "2022-01-26T11:29:46.853Z", "content": "@lukaszborecki can you explain what is FOLD 90/10?
", "rawMarkdown": "@lukaszborecki can you explain what is FOLD 90/10?" }, { "id": 1664921, "postDate": "2022-01-26T11:54:18.747Z", "content": "only 1 fold out of 10
", "rawMarkdown": "only 1 fold out of 10", "votes": 1 }, { "id": 1665105, "postDate": "2022-01-26T15:19:29.103Z", "content": "Do you use any tricks to get this score or just use yolov5 to get this score?
", "rawMarkdown": "Do you use any tricks to get this score or just use yolov5 to get this score?" }, { "id": 1665170, "postDate": "2022-01-26T16:21:13.880Z", "content": "Of course we have ;)
", "rawMarkdown": "Of course we have ;)", "votes": 1 }, { "id": 1665188, "postDate": "2022-01-26T16:35:12.303Z", "content": "Oh, i see, by the way now i have no idea to get higher score, can you share some tips about the tricks you trained the yolov5. Thank you !
", "rawMarkdown": "Oh, i see, by the way now i have no idea to get higher score, can you share some tips about the tricks you trained the yolov5. Thank you !" }, { "id": 1665196, "postDate": "2022-01-26T16:38:41.940Z", "content": "@jackiemai of course not :) after competition sure
", "rawMarkdown": "@jackiemai of course not :) after competition sure", "votes": 9 }, { "id": 1665390, "postDate": "2022-01-26T19:36:38.707Z", "content": "@deepkim @w3579628328 is this a challenge :P haha we fight for 0.001 :)
", "rawMarkdown": "@deepkim @w3579628328 is this a challenge :P haha we fight for 0.001 :)", "votes": 2 }, { "id": 1665690, "postDate": "2022-01-27T04:18:05.583Z", "content": "yeah ,haha, best luck to both of our team, fight for higher rank!😋
", "rawMarkdown": "yeah ,haha, best luck to both of our team, fight for higher rank!😋" }, { "id": 1666132, "postDate": "2022-01-27T13:11:33.437Z", "content": "That's a really good score! Did you use a split based on subsequences?
", "rawMarkdown": "That's a really good score! Did you use a split based on [subsequences](https://www.kaggle.com/julian3833/reef-a-cv-strategy-subsequences)?", "votes": 1 }, { "id": 1666714, "postDate": "2022-01-28T02:27:21.340Z", "content": "Hi. Could you tell me which fold do you use to val while there are 10folds.
", "rawMarkdown": "Hi. Could you tell me which fold do you use to val while there are 10folds." }, { "id": 1669895, "postDate": "2022-01-31T03:40:33.557Z", "content": "@remakkinas @lukaszboreki were you guys able to get V100's on pro+ or just P100's
\nalso, is that lb score with post processing or without post processing?
", "rawMarkdown": "@remakkinas @lukaszboreki were you guys able to get V100's on pro+ or just P100's\n\nalso, is that lb score with post processing or without post processing?" }, { "id": 1670052, "postDate": "2022-01-31T07:23:03.797Z", "content": "Most of the time I have Tesla V100-SXM2.
", "rawMarkdown": "Most of the time I have Tesla V100-SXM2. ", "votes": 1 }, { "id": 1671697, "postDate": "2022-02-01T16:58:34.683Z", "content": "@lukaszborecki good luck!
", "rawMarkdown": "@lukaszborecki good luck!", "votes": 1 }, { "id": 1671738, "postDate": "2022-02-01T17:37:31.647Z", "content": "@w3579628328 you found 'my precious' ? :D
", "rawMarkdown": "@w3579628328 you found 'my precious' ? :D", "votes": 1 }, { "id": 1675464, "postDate": "2022-02-04T09:30:42.943Z", "content": "I guess you did some preprocessing isnt it? @lukaszborecki
", "rawMarkdown": "I guess you did some preprocessing isnt it? @lukaszborecki " }, { "id": 1676136, "postDate": "2022-02-04T17:58:15.120Z", "content": "nope we didnt , only preprocessing was increasing from 1920 size to 2400
", "rawMarkdown": "nope we didnt , only preprocessing was increasing from 1920 size to 2400" }, { "id": 1677197, "postDate": "2022-02-05T15:17:48.727Z", "content": "@lukaszborecki what was you cv split method
", "rawMarkdown": "@lukaszborecki what was you cv split method" } ] }, { "id": 1614381, "postDate": "2021-12-11T02:42:41.230Z", "content": "model: yolov5
\nsplit: 5 folds
\nepoch: 20
\nRecall: 0.81787
\nCV: 0.68264
\nLB: 0.591
can I ask how you split train data to 5 folds? many thanks.
", "rawMarkdown": "can I ask how you split train data to 5 folds? many thanks.", "votes": 2 }, { "id": 1617996, "postDate": "2021-12-14T14:57:23.873Z", "content": "Is it single fold prediction or ensemble of 5 folds
", "rawMarkdown": "Is it single fold prediction or ensemble of 5 folds", "votes": 1 }, { "id": 1619404, "postDate": "2021-12-15T22:38:54.577Z", "content": "Could you tell me what image input size did you use in your cv5 experiment? Thank you @onodera.
", "rawMarkdown": "Could you tell me what image input size did you use in your cv5 experiment? Thank you @onodera. " }, { "id": 1619942, "postDate": "2021-12-16T09:29:44.373Z", "content": "5 groupkfold using sequence. LB is blend with wbf.
\ninput size is 1280.
\nFYI, I'm not using sequence frame so far. I think 0.62~ would be easier if we can utilize this.
Thank you very much for answering. I am still trying to boost score using YoloX …. but y5 looks promising.
", "rawMarkdown": "Thank you very much for answering. I am still trying to boost score using YoloX .... but y5 looks promising. " }, { "id": 1623715, "postDate": "2021-12-20T07:02:25.513Z", "content": "@onodera Are you use only labeled data ?
", "rawMarkdown": "@onodera Are you use only labeled data ? " }, { "id": 1624687, "postDate": "2021-12-21T06:07:46.040Z", "content": "@onodera Hi, I am newbie in Object Detection. Mind I ask how to check the recall score? Get it from below?
\n\n", "rawMarkdown": "@onodera Hi, I am newbie in Object Detection. Mind I ask how to check the recall score? Get it from below?\n\n> 20313.4s\t1794\tAverage Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.124\n>20313.4s\t1795\tAverage Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.266\n>20313.4s\t1796\tAverage Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.089\n>20313.4s\t1797\tAverage Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.016\n>20313.4s\t1798\tAverage Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.155\n>20313.4s\t1799\tAverage Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000\n>20313.4s\t1800\tAverage Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.092\n>20313.4s\t1801\tAverage Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.167\n>20313.4s\t1802\tAverage Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.167\n>20313.4s\t1803\tAverage Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.033\n>20313.4s\t1804\tAverage Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.203\n>20313.4s\t1805\tAverage Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000" }, { "id": 1624733, "postDate": "2021-12-21T07:15:23.273Z", "content": "20313.4s 1794 Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.124
\n
\n 20313.4s 1795 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.266
\n 20313.4s 1796 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.089
\n 20313.4s 1797 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.016
\n 20313.4s 1798 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.155
\n 20313.4s 1799 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
\n 20313.4s 1800 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.092
\n 20313.4s 1801 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.167
\n 20313.4s 1802 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.167
\n 20313.4s 1803 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.033
\n 20313.4s 1804 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.203
\n 20313.4s 1805 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
I guess you are using the video_2 for validation, but there is no target with large area in video_2. Therefore, your AR/AP on large area is -1, which can't be calculated.
", "rawMarkdown": "I guess you are using the video_2 for validation, but there is no target with large area in video_2. Therefore, your AR/AP on large area is -1, which can't be calculated.", "votes": 1 }, { "id": 1624780, "postDate": "2021-12-21T08:19:20.840Z", "content": "@jarviskevin Hi, no, I use GroupKFold method with sequence as groups. Then could you advise which AR score to refer given different area conditions? Since ONODERA referred a single AR score.
", "rawMarkdown": "@jarviskevin Hi, no, I use GroupKFold method with sequence as groups. Then could you advise which AR score to refer given different area conditions? Since ONODERA referred a single AR score." }, { "id": 1624792, "postDate": "2021-12-21T08:31:22.903Z", "content": "20313.4s 1802 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.167
\nbut better is to implement f2 score
", "rawMarkdown": "20313.4s 1802 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.167\n\nbut better is to implement f2 score" }, { "id": 1624797, "postDate": "2021-12-21T08:41:20.160Z", "content": "@remekkinas Thanks bro! I did implement F2 score. Just learn from you guys. Really not experienced in Object Detection.
", "rawMarkdown": "@remekkinas Thanks bro! I did implement F2 score. Just learn from you guys. Really not experienced in Object Detection." }, { "id": 1644896, "postDate": "2022-01-10T15:33:26.243Z", "content": "@onodera Hi, do you mind if I ask how you guys succeed in wbf ensemble? Really struggled in this. Cannot get a reasonable score so far. Any tips that can be shared?
", "rawMarkdown": "@onodera Hi, do you mind if I ask how you guys succeed in wbf ensemble? Really struggled in this. Cannot get a reasonable score so far. Any tips that can be shared?" }, { "id": 1646745, "postDate": "2022-01-12T02:21:07.767Z", "content": "Single fold, validation score 0.63, LB 0.54. Highly correlated with your score, hope my LB will elevate when I boost my validation score.
\nAnother fold: validation score: 0.59, LB: 0.58 😣
", "rawMarkdown": "Single fold, validation score 0.63, LB 0.54. Highly correlated with your score, hope my LB will elevate when I boost my validation score.\n\nAnother fold: validation score: 0.59, LB: 0.58 😣", "votes": 1 }, { "id": 1647677, "postDate": "2022-01-12T19:31:38.923Z", "content": "How are you splitting your data? @steamedsheep
", "rawMarkdown": "How are you splitting your data? @steamedsheep " }, { "id": 1647815, "postDate": "2022-01-13T00:17:16.623Z", "content": "sheep哥一出手 真就不一样。
", "rawMarkdown": "sheep哥一出手 真就不一样。", "votes": 1 }, { "id": 1647968, "postDate": "2022-01-13T03:06:22.023Z", "content": "I split fold via video_id, I have only 3 folds.
", "rawMarkdown": "I split fold via video_id, I have only 3 folds.", "votes": 2 } ] }, { "id": 1672677, "postDate": "2022-02-02T08:28:11.393Z", "content": "single model:yolov5s6
\nLB 0.661
\nbatch = 8
\nepoch = 15
\ntrain and infer size: 3584
What is your hardware?
", "rawMarkdown": "What is your hardware?" }, { "id": 1672928, "postDate": "2022-02-02T11:44:12.640Z", "content": "The model needs very large GPU RAM.You can get more information in my discussion book. I prepare to do a series of training experiment. I will update the result in it.
", "rawMarkdown": "The model needs very large GPU RAM.You can get more information in my discussion book. I prepare to do a series of training experiment. I will update the result in it.", "votes": 2 }, { "id": 1672930, "postDate": "2022-02-02T11:47:17.483Z", "content": "Good observation. I wrote about batch_size in large image res … and was voted down :) Now you confirmed this observation. Thank you!
", "rawMarkdown": "Good observation. I wrote about batch_size in large image res ... and was voted down :) Now you confirmed this observation. Thank you!", "votes": 4 }, { "id": 1672966, "postDate": "2022-02-02T12:14:12.270Z", "content": "It's exciting for me to get your reply, I was inspired by your previous comments, so I did this. I will try more training in smaller GPU RAM to provide more valuable results. If small size , small batch and small model can make sense , that's will make the kagglers without enough GPU RAM to fairly join in the competition.
", "rawMarkdown": "It's exciting for me to get your reply, I was inspired by your previous comments, so I did this. I will try more training in smaller GPU RAM to provide more valuable results. If small size , small batch and small model can make sense , that's will make the kagglers without enough GPU RAM to fairly join in the competition.", "votes": 3 }, { "id": 1673128, "postDate": "2022-02-02T14:23:27.213Z", "content": "What is your cv of that model?
", "rawMarkdown": "What is your cv of that model?" }, { "id": 1673138, "postDate": "2022-02-02T14:31:13.787Z", "content": "10 fold, I only train the the first fold(leave the 1st fold as valdata) model and submit the single model.
", "rawMarkdown": "10 fold, I only train the the first fold(leave the 1st fold as valdata) model and submit the single model." }, { "id": 1673290, "postDate": "2022-02-02T16:21:22.143Z", "content": "what was your f2 score on the first fold?
", "rawMarkdown": "what was your f2 score on the first fold? " }, { "id": 1673376, "postDate": "2022-02-02T17:32:35.840Z", "content": "All coco indicator and f2 are approximate 0.8.
", "rawMarkdown": "All coco indicator and f2 are approximate 0.8." }, { "id": 1674811, "postDate": "2022-02-03T19:52:43.740Z", "content": "\n\n10 fold, I only train the the first fold…
\n
\n All coco indicator and f2 are approximate 0.8.
@freshair1996 thanks! Your cv split is based on sequences?
\nThat's quite good CV
no，I have no idea to train on sequence，put the shuffle off？
", "rawMarkdown": "no，I have no idea to train on sequence，put the shuffle off？", "votes": -1 }, { "id": 1679575, "postDate": "2022-02-07T09:50:33.047Z", "content": "I guess @imeintanis means splitting the data via GroupKFold on \"video sequence\" column
", "rawMarkdown": "I guess @imeintanis means splitting the data via GroupKFold on \"video sequence\" column", "votes": 1 } ] }, { "id": 1608665, "postDate": "2021-12-06T14:41:39.940Z", "content": "Model: Yolov5
\nSplit: 10 folds
\nepochs: 12
\nCV: 0.5639
\nLB: 0.539
Hi, Matthieu, can I ask how you calculate cv, as I use the cv code from other, I find that my cv score is large than LB very much, for example, lb 0.4, cv0.75
", "rawMarkdown": "Hi, Matthieu, can I ask how you calculate cv, as I use the cv code from other, I find that my cv score is large than LB very much, for example, lb 0.4, cv0.75" }, { "id": 1611765, "postDate": "2021-12-08T08:46:20.460Z", "content": "Hi, I think the gap between your CV and your LB comes from the way you split the data, not the code you used. I used the code @phalanx gave below.
\nYou should read this discussion
", "rawMarkdown": "Hi, I think the gap between your CV and your LB comes from the way you split the data, not the code you used. I used the code @phalanx gave below.\n\nYou should read [this discussion](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/293723) ", "votes": 1 }, { "id": 1611926, "postDate": "2021-12-08T11:42:04.613Z", "content": "OK, thank you.
", "rawMarkdown": "OK, thank you." }, { "id": 1612173, "postDate": "2021-12-08T16:22:10.837Z", "content": "can I ask what do you mean 10 folds, as I'm new to kaggle competition, the question may looks stupid.
", "rawMarkdown": "can I ask what do you mean 10 folds, as I'm new to kaggle competition, the question may looks stupid." }, { "id": 1612721, "postDate": "2021-12-09T08:52:42.270Z", "content": "There is no stupid question. I invite you to read some articles about cross validation k fold, you will find easily!
@mathieuplante how did you split the data?
", "rawMarkdown": "@mathieuplante how did you split the data?", "votes": 1 }, { "id": 1612788, "postDate": "2021-12-09T10:16:03.900Z", "content": "I used this notebook and for the moment I have a good LB/CV correlation
", "rawMarkdown": "I used [this notebook ](https://www.kaggle.com/julian3833/reef-a-cv-strategy-subsequences) and for the moment I have a good LB/CV correlation", "votes": 7 }, { "id": 1613093, "postDate": "2021-12-09T15:33:41.923Z", "content": "now I understand that k fold can be used to choose model or evaluate model performance, so wonder you train ten times with each fold as validation and get ten models, then fusion the model's result, or just train one time with one fold as validation.
", "rawMarkdown": "now I understand that k fold can be used to choose model or evaluate model performance, so wonder you train ten times with each fold as validation and get ten models, then fusion the model's result, or just train one time with one fold as validation." }, { "id": 1613103, "postDate": "2021-12-09T15:43:06.807Z", "content": "For the moment I only have one model but later I will train 10 models and merge yes
", "rawMarkdown": "For the moment I only have one model but later I will train 10 models and merge yes ", "votes": 1 }, { "id": 1618636, "postDate": "2021-12-15T07:24:46.567Z", "content": "Are you still only use yolov5 object detection model, without video object detection method?
", "rawMarkdown": "Are you still only use yolov5 object detection model, without video object detection method?", "votes": 1 }, { "id": 1619405, "postDate": "2021-12-15T22:39:29.770Z", "content": "@matthieuplante what image input size did you use during training? Thank you!
", "rawMarkdown": "@matthieuplante what image input size did you use during training? Thank you!" }, { "id": 1631055, "postDate": "2021-12-28T01:54:15.683Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1645534, "postDate": "2022-01-11T02:04:47.287Z", "content": "🙏 I use this split too, But still got a large gap between cv and lb.
", "rawMarkdown": "🙏 I use this split too, But still got a large gap between cv and lb." }, { "id": 1679605, "postDate": "2022-02-07T10:04:37.003Z", "content": "Same here, very curious how to fill the large GAP btw CV & LB
", "rawMarkdown": "Same here, very curious how to fill the large GAP btw CV & LB", "votes": -1 } ] }, { "id": 1596437, "postDate": "2021-11-26T13:47:30.847Z", "content": "If you don't mind me asking, how do you guys do 80/20 split in this dataset?
", "rawMarkdown": "If you don't mind me asking, how do you guys do 80/20 split in this dataset?", "votes": 10, "replies": [ { "id": 1596448, "postDate": "2021-11-26T13:55:24.543Z", "content": "in my baseline I did it pretty naively .. Just took all the images which has box labels around ~4k and split them in to 80/20. Currently my model doesn't see any background images.
", "rawMarkdown": "in my baseline I did it pretty naively .. Just took all the images which has box labels around ~4k and split them in to 80/20. Currently my model doesn't see any background images. ", "votes": 3 }, { "id": 1597125, "postDate": "2021-11-27T08:28:52.863Z", "content": "so your model is only training at frames with starfish, have you seen how does it perform when a blank image(no starfish) is given as input?
\nand another thing I have never worked with videos so don't have experience, How do we generally split video data, I was thinking about 3 folds(video 0,video1,video2) but now think it may not be correct.
Yes, on blank images its also predict some bbox but the p of being starfish is pretty low. So we can filter it… But there are some fp with high probability. This is first time I am working with object detection, perhaps I need to read up on how people deal with background images where no object is present. Regarding cv I still have to try few stuff.
model: yolov5l(single model, no TTA)
\nsplit: sequence_id
\nCV: 0.67
\nLB: 0.637
Can i ask you about image size when you infer?
", "rawMarkdown": "Can i ask you about image size when you infer?" }, { "id": 1643642, "postDate": "2022-01-09T15:47:59.713Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1644039, "postDate": "2022-01-10T01:59:41.490Z", "content": "Hi @clwwlc, can I ask you about batch_size and how to calculate CV ? And do you use groupkfold ?
", "rawMarkdown": "Hi @clwwlc, can I ask you about batch_size and how to calculate CV ? And do you use groupkfold ? " }, { "id": 1648687, "postDate": "2022-01-13T15:52:07.533Z", "content": "Hi @clwwlc, I was wondering if you train your models on a local machine/cloud computing or use a kaggle kernel. Currently I am training models on my 2080 but it does not have enough gpu memory to train a yolov5l and the kaggle kernels are very slow.
", "rawMarkdown": "Hi @clwwlc, I was wondering if you train your models on a local machine/cloud computing or use a kaggle kernel. Currently I am training models on my 2080 but it does not have enough gpu memory to train a yolov5l and the kaggle kernels are very slow. " }, { "id": 1649768, "postDate": "2022-01-14T15:17:37.253Z", "content": "I use f2 metric, and only 1 fold
", "rawMarkdown": "I use f2 metric, and only 1 fold" }, { "id": 1649769, "postDate": "2022-01-14T15:18:29.950Z", "content": "local machine, for my solution now 2080 is enough
", "rawMarkdown": "local machine, for my solution now 2080 is enough", "votes": 1 }, { "id": 1653851, "postDate": "2022-01-17T23:20:57.090Z", "content": "@gtownfoster batch size along with your image size may be too large.
", "rawMarkdown": "@gtownfoster batch size along with your image size may be too large. " } ] }, { "id": 1646731, "postDate": "2022-01-12T01:26:51.073Z", "content": "model: yolov5m(single model)
\nsplit: sequence_id, 4 folds
\nCV: 0.76
\nLB: 0.503
\nBatch size: 4
\nEpochs: 20
\nConf: 0.411
\nTo get this I did a lot of testing and running of different hyperparameters. I know that my cv and lb are far apart so that is what I will be working on next. I just wanted to post because I was able to get above a score of 0.5 with a yolov5m model.
model : yolov5
\nsplit : 5 folds
\nCV : 0.6384
\nLB : 0.598
Whay to you mean by CV result? It is recall, ap@95 or f2 score? Thank you!
\nCongratulations!!
it means f2-score, and single model (not 5 folds ensemble) !!
", "rawMarkdown": "it means f2-score, and single model (not 5 folds ensemble) !!" }, { "id": 1622891, "postDate": "2021-12-19T08:35:17.250Z", "content": "Thank you! I now understand.
", "rawMarkdown": "Thank you! I now understand." }, { "id": 1622929, "postDate": "2021-12-19T09:36:46.257Z", "content": "Good job, can you share some changes of hyp or input_size? I can only get 0.51 using yolov5-l with 1280 input-size, 20 epoch
", "rawMarkdown": "Good job, can you share some changes of hyp or input_size? I can only get 0.51 using yolov5-l with 1280 input-size, 20 epoch" }, { "id": 1624621, "postDate": "2021-12-21T04:36:43.413Z", "content": "image size is 1280, and model is yolov5-l.
\nI can't say the details, but some augmentation param changes may help.
ok, thanks
", "rawMarkdown": "ok, thanks" }, { "id": 1630771, "postDate": "2021-12-27T17:21:30.827Z", "content": "@clwwlc Could you tell me how you split the data? Thanks!
\nAny tips about how i could improve from the default yolov5-l?
Sorry for the delay in responding, I splitted it by extracting video subsequence.
", "rawMarkdown": "Sorry for the delay in responding, I splitted it by extracting video subsequence.", "votes": 2 }, { "id": 1646529, "postDate": "2022-01-11T20:07:10.987Z", "content": "thanks for sharing @tomyanabe good job! May I ask what CV method you are using and if the score reported is calculated with all images (labeled + empty)?
", "rawMarkdown": "thanks for sharing @tomyanabe good job! May I ask what CV method you are using and if the score reported is calculated with all images (labeled + empty)?" } ] }, { "id": 1603446, "postDate": "2021-12-02T14:32:57.803Z", "content": "framework; PyTorch
\nmodel: FasterRCNN-(backbone:resnet101)
\nsplit: 85/15 (only containing label)
\nepoch: 20
\nCV: no competition metric implemented
\nLB: 0.442
\nConfidence Threshold: 0.80
\n\nI think some work has to be done to figure what is the best way to split ..
\n
I haven't started yet, but I was thinking GroupKFold on sequence. What do you think?
I guess we have to try and see what works =)
", "rawMarkdown": "I guess we have to try and see what works =) ", "votes": 1 }, { "id": 1597476, "postDate": "2021-11-27T14:38:11.640Z", "content": "A quick single fold experiment with GroupKFold and I'm seeing validation F2 of 0.426 (4K images only). Changing to KFold and it increases to 0.70+, so there's a lot of leakage between the sequences
", "rawMarkdown": "A quick single fold experiment with GroupKFold and I'm seeing validation F2 of 0.426 (4K images only). Changing to KFold and it increases to 0.70+, so there's a lot of leakage between the sequences", "votes": 4 } ] }, { "id": 1658382, "postDate": "2022-01-21T01:24:45.390Z", "content": "Faster Cascade RCNN ResNeSt200
\nsplit 5 folds
\nCV (1st fold): 0.562
\nLB (1st fold): 0.579
yolov5s6
\nCV: 0.72
\nLB: 0.642
\nBatch size: 3
\nEpochs: 9 with transfer learning
model: yolov5
\nsplit: 4folds
\nCV: 0.64159
\nLB: 0.545
Hi @kevin1742064161 How do you evaluate your F2 score? in training or after? Thanks
", "rawMarkdown": "Hi @kevin1742064161 How do you evaluate your F2 score? in training or after? Thanks" }, { "id": 1642802, "postDate": "2022-01-08T17:27:21.633Z", "content": "@kevin1742064161 I think F2 score computed by public Model isnt looking True.
", "rawMarkdown": "@kevin1742064161 I think F2 score computed by public Model isnt looking True. ", "votes": -1 }, { "id": 1642841, "postDate": "2022-01-08T18:25:45.293Z", "content": "Interesting. What does it mean for you?
", "rawMarkdown": "Interesting. What does it mean for you?" }, { "id": 1643198, "postDate": "2022-01-09T06:38:24.933Z", "content": "0.66 vs in 0.5s at lb for quite a many
", "rawMarkdown": "0.66 vs in 0.5s at lb for quite a many " }, { "id": 1645740, "postDate": "2022-01-11T07:16:51.810Z", "content": "https://www.kaggle.com/kevin1742064161/yolov5-cal-f2score/notebook
\n I modified some code in yolov5. You can get it from this notebook
@kevin1742064161 good job! may I ask what CV strategy you are using (subseq, gkf seq, etc) and if the score you report is from single fold or ens?
", "rawMarkdown": "@kevin1742064161 good job! may I ask what CV strategy you are using (subseq, gkf seq, etc) and if the score you report is from single fold or ens? ", "votes": 1 }, { "id": 1653730, "postDate": "2022-01-17T20:21:56.850Z", "content": "Hi @imeintanis can you please tell me what is \"gkf seq\"? is it group kfold on sequence?
", "rawMarkdown": "Hi @imeintanis can you please tell me what is \"gkf seq\"? is it group kfold on sequence?", "votes": 1 }, { "id": 1653758, "postDate": "2022-01-17T20:51:32.990Z", "content": "\n\nis it group kfold on sequence?
\n
exactly
", "rawMarkdown": "> is it group kfold on sequence?\n\nexactly", "votes": 1 } ] }, { "id": 1604968, "postDate": "2021-12-03T22:22:18.387Z", "content": "@drhabib
\nHave you resize the input image from (720, 1280) to (1280, 1280)? If so, such resizing is making disturbance on the spatial information of the input samples. What do you think?
No I did not, I think for CNNs you don't have to resize from 720 x 1280 to 1280 x 1280. They should work fine on rectangular images as long as all images have same dimensions..
good experiments
", "rawMarkdown": "good experiments", "votes": 1 }, { "id": 1600002, "postDate": "2021-11-30T03:16:41.617Z", "content": "good work and knowledgable
", "rawMarkdown": "good work and knowledgable", "votes": 1 }, { "id": 1597221, "postDate": "2021-11-27T10:28:01.740Z", "content": "@drhabib what img-size did you use?
", "rawMarkdown": "@drhabib what img-size did you use?", "votes": 1, "replies": [ { "id": 1597390, "postDate": "2021-11-27T13:10:28.290Z", "content": "image size 1280
", "rawMarkdown": "image size 1280", "votes": 2 } ] }, { "id": 1649696, "postDate": "2022-01-14T14:19:40.787Z", "content": "EfficientDet d5
\nsplit 5 folds
\nCV: 0.55
\nLB: 0.586
\n1 fold of 5 folds
That's nice for EffDet. Did you train it using TF or Pytorch ? I have trained Pytorch EffDet and it had a good validation score (0.62x). However it did not work on LB (Only 0.36x). When doing analysis I noticed it predicted less bbox than other models -> Less True Positive. Maybe that was the reason why it failed on LB.
", "rawMarkdown": "That's nice for EffDet. Did you train it using TF or Pytorch ? I have trained Pytorch EffDet and it had a good validation score (0.62x). However it did not work on LB (Only 0.36x). When doing analysis I noticed it predicted less bbox than other models -> Less True Positive. Maybe that was the reason why it failed on LB.", "votes": 1 }, { "id": 1658995, "postDate": "2022-01-21T13:23:58.630Z", "content": "Hi @namgalielei This is pytorch effdet and i used this repo https://github.com/rwightman/efficientdet-pytorch
\nAbout your LB score I think you should check your dataset. Hmm maybe have leak?
yeah, maybe. Which image size did you use to train and infer? I used 1024x1024.
", "rawMarkdown": "yeah, maybe. Which image size did you use to train and infer? I used 1024x1024." }, { "id": 1662647, "postDate": "2022-01-24T12:55:07.120Z", "content": "I used 1280x1280 for both train and inference
", "rawMarkdown": "I used 1280x1280 for both train and inference", "votes": 2 }, { "id": 1664784, "postDate": "2022-01-26T09:23:48.123Z", "content": "hey I am also trying with effdet, but the max size I can train is img size = 876, how are you raining your model on such a large image size?
", "rawMarkdown": "hey I am also trying with effdet, but the max size I can train is img size = 876, how are you raining your model on such a large image size?" }, { "id": 1665011, "postDate": "2022-01-26T13:45:40.170Z", "content": "Sorry, I did not train on kaggle or colab. I trained on local machine
", "rawMarkdown": "Sorry, I did not train on kaggle or colab. I trained on local machine" }, { "id": 1665114, "postDate": "2022-01-26T15:29:12.287Z", "content": "I see ,but can you still say like how much GPU memory does it take to train 1280 sized images, I may rent some GPUs so need some idea, how big GPU should I rent.
\n@gigggggge
I used 2 3090 with sync batchnorm but I think single 3090 is enough
", "rawMarkdown": "I used 2 3090 with sync batchnorm but I think single 3090 is enough", "votes": 1 }, { "id": 1665256, "postDate": "2022-01-26T17:30:40.533Z", "content": "sorry for asking you again and again, where do you change this sync batchnorm in the model? argument for the model class?
", "rawMarkdown": "sorry for asking you again and again, where do you change this sync batchnorm in the model? argument for the model class?" }, { "id": 1665298, "postDate": "2022-01-26T17:50:26.143Z", "content": "I modified some code in the repo and add it as argument
", "rawMarkdown": "I modified some code in the repo and add it as argument", "votes": 1 } ] }, { "id": 1608671, "postDate": "2021-12-06T14:47:20.957Z", "content": "good work PyTorch
", "rawMarkdown": "good work PyTorch" }, { "id": 1604864, "postDate": "2021-12-03T19:37:22.030Z", "content": "Do You use NMS in your prediction or you are leaving it for threshold filtering ? Why i am asking because if i revise predictions there are some spams on COTS with bounding boxes but they are less than 0.2 and one is 0.9. So is the threshold enough in your opinion ?
", "rawMarkdown": "Do You use NMS in your prediction or you are leaving it for threshold filtering ? Why i am asking because if i revise predictions there are some spams on COTS with bounding boxes but they are less than 0.2 and one is 0.9. So is the threshold enough in your opinion ?" }, { "id": 1642800, "postDate": "2022-01-08T17:25:56.590Z", "content": "LB 55.5 one fold , CV 55 YoloX
", "rawMarkdown": "LB 55.5 one fold , CV 55 YoloX", "votes": -1 }, { "id": 1685078, "postDate": "2022-02-11T02:59:36.240Z", "content": "@outwrest Can you provide your ideas about your solution(100+ epochs, gan-aided copy&paste),Is this unsupervised plus fine-tuning?
", "rawMarkdown": "@outwrest Can you provide your ideas about your solution(100+ epochs, gan-aided copy&paste),Is this unsupervised plus fine-tuning?", "replies": [ { "id": 1685091, "postDate": "2022-02-11T03:10:57.320Z", "content": "I'll make a post after the competition if I score well. It is too late into the competition to talk about this.
", "rawMarkdown": "I'll make a post after the competition if I score well. It is too late into the competition to talk about this.", "votes": 3 } ] }, { "id": 1674783, "postDate": "2022-02-03T19:07:06.103Z", "content": "Hey, can somebody please share video_x fold CV? Im getting no correlation at all with public LB, like +5 f2 at cv and -5 f2 at LB
", "rawMarkdown": "Hey, can somebody please share video_x fold CV? Im getting no correlation at all with public LB, like +5 f2 at cv and -5 f2 at LB" }, { "id": 1649787, "postDate": "2022-01-14T15:29:55.450Z", "content": "Model Yolo(single model)
\nsplit : video id
\nCV(F2 Score) : 70
\nLB : 67.5
If you dont mind which image size did you use?
", "rawMarkdown": "If you dont mind which image size did you use?", "votes": 1 }, { "id": 1649811, "postDate": "2022-01-14T15:46:13.317Z", "content": "I can not say the details, but the Kaggle Gpu is enough .
\nEstablishing a proper neural network and evaluation metrics are important.
Thanks you I understood
", "rawMarkdown": "Thanks you I understood", "votes": 1 }, { "id": 1649849, "postDate": "2022-01-14T16:15:19.363Z", "content": "\n\nCV(F2 Score) : 70
\n
@deepkim On a specific video fold or global average / oof?
", "rawMarkdown": "> CV(F2 Score) : 70\n\n@deepkim On a specific video fold or global average / oof?", "votes": 2 }, { "id": 1649992, "postDate": "2022-01-14T18:25:43.607Z", "content": "@deepkim congrats! your CV score is calculated with labeled images only or empty as well ?
\nalso by single model I guess you mean single fold (?)
@philippsinger a specific fold f2 score
", "rawMarkdown": "@philippsinger a specific fold f2 score\n" }, { "id": 1650489, "postDate": "2022-01-15T04:46:07.487Z", "content": "@imeintanis background images are included, and you are right i mean single fold
", "rawMarkdown": "@imeintanis background images are included, and you are right i mean single fold\n", "votes": 1 }, { "id": 1658102, "postDate": "2022-01-20T18:30:46.547Z", "content": "@deepkim can you share your implementation of F2 score calculation?
", "rawMarkdown": "@deepkim can you share your implementation of F2 score calculation?", "votes": 1 }, { "id": 1663035, "postDate": "2022-01-24T18:16:06.180Z", "content": "it's simple. you calculate tp/fp/fn in validations sets and get a f2 score by iou 0.3,0.35,0.4,….,0.8
\nafter that, average it!
I have a question for you experts！！
\nI trained yolov5s6 with size3600 and then used the following notebook https://www.kaggle.com/kocha1/only-yolov5-tracking-lb-642 for inference. conf0.34 iou0.5, in the case where I only change the img-size of inference:
\nimg-size 3600 lb0.579
\nimg-size 9000 lb0.577
\nWhy does my self-trained model infer no change at large scale?
\nHave you encountered this situation?
I don't think LB is reliable. it's too much fluctuating. I was sometimes faced the issue you encountered
", "rawMarkdown": "I don't think LB is reliable. it's too much fluctuating. I was sometimes faced the issue you encountered" } ] }, { "id": 1642069, "postDate": "2022-01-08T02:33:49.163Z", "content": "@tomyanabe Could you tell met the Train Split size?
", "rawMarkdown": "@tomyanabe Could you tell met the Train Split size?\n" }, { "id": 1641250, "postDate": "2022-01-07T08:26:17.240Z", "content": "model: yolov5-l
\nsplit: video_id
\nCV: 0.642
\nLB: 0.522
model: yolox-l
\nsplit: 5 folds
\nCV: 0.500
\nLB: 0.561
Hi @sqqqqy If you dont mind I have a question about your CV. Did you evaluate in training? or end of the training? Thank you
", "rawMarkdown": "Hi @sqqqqy If you dont mind I have a question about your CV. Did you evaluate in training? or end of the training? Thank you" }, { "id": 1641248, "postDate": "2022-01-07T08:20:30.517Z", "content": "End of training
", "rawMarkdown": "End of training", "votes": 1 }, { "id": 1641324, "postDate": "2022-01-07T10:20:09.217Z", "content": "thanks for your reply!
", "rawMarkdown": "thanks for your reply!" } ] }, { "id": 1634622, "postDate": "2021-12-31T23:07:57.260Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1677259, "author_name": "atom1231", "author_url": "", "post_date": "2022-02-05T15:52:29.437000", "content": "model: Cascade R-CNN R50 (MMDET)
\nspilt : cross validation by video id then train all
\nLB : 0.741
\nhardware: RTX3090
comment
\n1.yolov5 is not all you need.
\n2.to fit or not to fit that is the question.
\n3.I'm not going to answer any questions.
\n The post itself is the biggest hint .
nice, it's enough. I will try it and come back soon.
", "votes": 0, "replies": [] }, { "id": 1677271, "author_name": "Ioannis M", "author_url": "", "post_date": "2022-02-05T16:02:36.857000", "content": "great job @atom1231!
\nif you like to reply, what is the CV score, and
\nwhat evaluator you use for MMDET (standard coco, or modified for f2) ?
@imeintanis coco MAP
", "votes": 1, "replies": [] }, { "id": 1678057, "author_name": "yumo", "author_url": "", "post_date": "2022-02-06T08:25:39.287000", "content": "great!
\n1.Could you please tell me the img size for traing? multi-scale right？
\n2.\" cross validation by video id then train all\" What exactly does this mean？
\n3.I once thought hrnet was the best potential，Have you tested it？
Can you please give some tips on how to train on all data after validation?
\nLike what all do we have to fix?
Ohhhh … noooo????? Yolov5 is not all you need? …. Are you sure???? And size is not all you need? Ohhhhh…… oh…… 😜😜😜😜😜 I was writing about this month ago (that this is …. just not responsible to put people in this direction ….) These topics were just the gratest clickbait during this competition ….
", "votes": -6, "replies": [] }, { "id": 1678484, "author_name": "atom1231", "author_url": "", "post_date": "2022-02-06T15:05:30.370000", "content": "@junyun1002 @shashwatraman
\n1.->ya
\n2.->make sure the model performance is improved in different folds then use the hyperparameter to train with all data.
\n3.->no .
Thank you very much 😃
", "votes": 0, "replies": [] }, { "id": 1678499, "author_name": "Shashwat Raman", "author_url": "", "post_date": "2022-02-06T15:16:52.923000", "content": "Just one more thing, how do we know that which epoch is the best?
", "votes": 0, "replies": [] }, { "id": 1678504, "author_name": "Good Moon", "author_url": "", "post_date": "2022-02-06T15:20:39.177000", "content": "Yeah, I almost sink into yolov5 until I see the comment. I find that I seems ignored the two stage method.
", "votes": 1, "replies": [] }, { "id": 1678508, "author_name": "yumo", "author_url": "", "post_date": "2022-02-06T15:22:39.497000", "content": "Thank you very much。
", "votes": 0, "replies": [] }, { "id": 1680712, "author_name": "atom1231", "author_url": "", "post_date": "2022-02-08T01:27:59.733000", "content": "@shashwatraman
\nwe do not know but at least we can submit all checkpoints to get best LB score.
\nI choose the epoch value by the result of cross validation.
Thanks 😬😬
", "votes": 0, "replies": [] }, { "id": 1682587, "author_name": "Nick Sergievskiy", "author_url": "", "post_date": "2022-02-09T09:00:06.330000", "content": "Thank you for sharing, mmdet rocks 🤘
\nIs mAP make sense when choosing model? mAP of mmdet models better then yolo?
\nchoosing conf and resolution is just a LB probbing?
@nicksergievskiy
\nSince I found the ridiculous \"high resolution solution\" , I decided not to spend too much time in the competition and regarded it as game or lottery.
\nMy goal is limited to get best LB score.
Is mAP make sense when choosing model?
\n=> Of course F2 make sense , but I found my mmdet model mAP correlated with LB in limited cases. I
\ndo not implement F2 metric in my pipeline.
mAP of mmdet models better then yolo?
\n=>For LB , It is hard to say.
choosing conf and resolution is just a LB probbing?
\n=>resolution selected by the analysis of training data.
\n conf is simply LB probing.
Hi, Could I ask one more hint? Please!
\nDoes this single model use TTA to get LB : 0.741 or not?
@lilinchen
\nyes, but without TTA the model also got similar LB score.
@atom1231 thanks!
\nI will look forward to your final solution, and hope you win!!!
\nDefeat YOLO family!!!
@atom1231 and @lilinchen it would be very very nice to see solution based on C-RCNN - I agree. I keep my fingers crossed for success 👍🤛💪
", "votes": 0, "replies": [] }, { "id": 1684115, "author_name": "clwclw", "author_url": "", "post_date": "2022-02-10T09:26:19.247000", "content": "Sorry to disturb, really want to know if without the tracking or other post-processing, the 0.741 LB model will decrease how many score, maybe -0.1 ?
", "votes": -3, "replies": [] }, { "id": 1684591, "author_name": "outwrest", "author_url": "", "post_date": "2022-02-10T15:37:24.977000", "content": "@atom1231 I spent over two weeks developing a solution with pre-training with 100+ epochs, gan-aided copy&paste, and other optimizations to LR only to submit a model that had a bad cv for fun to get 0.753 public LB. I came back to your comment here because it is true.
\n\n", "votes": 6, "replies": [] } ] }, { "id": 1595988, "author_name": "phalanx", "author_url": "", "post_date": "2021-11-26T05:51:26.217000", "content": "Since I found the ridiculous \"high resolution solution\" , I decided not to spend too much time in the competition and regarded it as game or lottery.
\n
model: yolox
\nsplit: 80/20 (train: only containing label, val: all data)
\nval score: 0.496
\nlb: 0.484
my metric implementation
\nfrom typing import List\n\nimport numpy as np\nimport torch\nfrom torchvision.ops import box_iou\n\n\ndef calculate_score(\n preds: List[torch.Tensor],\n gts: List[torch.Tensor],\n iou_th: float\n) -> float:\n num_tp = 0\n num_fp = 0\n num_fn = 0\n for p, gt in zip(preds, gts):\n if len(p) and len(gt):\n iou_matrix = box_iou(p, gt)\n tp = len(torch.where(iou_matrix.max(0)[0] >= iou_th)[0])\n fp = len(p) - tp\n fn = len(torch.where(iou_matrix.max(0)[0] < iou_th)[0])\n num_tp += tp\n num_fp += fp\n num_fn += fn\n elif len(p) == 0 and len(gt):\n num_fn += len(gt)\n elif len(p) and len(gt) == 0:\n num_fp += len(p)\n score = 5 * num_tp / (5 * num_tp + 4 * num_fn + num_fp)\n return score\n\niou_ths = np.arange(0.3, 0.85, 0.05)\nscores = [calculate_score(predictions, gts, iou_th) for iou_th in iou_ths]\nnp.mean(scores)\n>>> 0.49588419817665447\ncan you provide the github or any link for the yolox model you are using?
", "votes": 2, "replies": [] }, { "id": 1598200, "author_name": "ynhuhu", "author_url": "", "post_date": "2021-11-28T10:03:19.477000", "content": "https://github.com/open-mmlab/mmdetection/blob/master/configs/yolox/README.md
", "votes": 5, "replies": [] }, { "id": 1598977, "author_name": "phalanx", "author_url": "", "post_date": "2021-11-29T03:44:48.367000", "content": "I use this repo.
\nhttps://github.com/Megvii-BaseDetection/YOLOX
\nAbout yolox, main contributions to mAP are augmentation, label assignment and center sampling.
\nthese are general-purpose methods, so you can combine them with other mtehods such as faster r-cnn, fcos.
@phalanx could you please explain, label assignment and center sampling ?
@awsaf49 You can refer to the yolox paper.
\nhttps://arxiv.org/abs/2107.08430
What input resolution? 640px?
\nI am asking because in this competition I would like to learn something new. YoloX seems to me promising in both - speed, accuracy and … as a last it is opportunity yo learn something new (I have not use it so far - y3/y4/y5/yr yes but not x).
you can check out official repo of yolo they have multiple resolution as input
\nhttps://github.com/ultralytics/yolov5
Thank you. I just asked about yoloX
\nBTW Your score with FasterRCNN is quite good so far! 👍
", "votes": 1, "replies": [] }, { "id": 1599666, "author_name": "DrHB", "author_url": "", "post_date": "2021-11-29T16:54:55.440000", "content": "ohh I assumed yoloX and yolo5x are the same =(
Thanks for sharing the metric, I've been trying it but unfortunately it seems to be overestimating my LB during CV (CV >> LB). I've tried different validation strategies, Groupkfold, startified group kfold by sequence id and num annotations, … but always CV >> LB. Then I tried to modify the metric by making it less elegant but more verbose to:
\ndef calculate_score(\n preds,\n gts,\n iou_th):\n num_tp = 0\n num_fp = 0\n num_fn = 0\n for p, gt in zip(preds, gts):\n if len(p) and len(gt):\n tp = 0\n fp = 0\n iou_matrix = box_iou(p, gt)\n n = iou_matrix.shape[0]\n for i in range(n):\n row = iou_matrix[i]\n match_idxs = torch.where(row > iou_th)[0]\n if len(match_idxs)>0:\n match_idx = match_idxs[0].item()\n # remove matched gt box\n iou_matrix = torch.cat([iou_matrix[:,:match_idx], iou_matrix[:,match_idx+1:]], dim=1)\n tp += 1\n else:\n fp += 1\n fn = iou_matrix.shape[1]\n\n num_tp += tp\n num_fp += fp\n num_fn += fn\n elif len(p) == 0 and len(gt):\n num_fn += len(gt)\n elif len(p) and len(gt) == 0:\n num_fp += len(p)\n score = 5 * num_tp / (5 * num_tp + 4 * num_fn + num_fp)\n return score\n, after this change now LB and CV are closer.
", "votes": 6, "replies": [] }, { "id": 1600462, "author_name": "Remek Kinas", "author_url": "", "post_date": "2021-11-30T12:48:57.883000", "content": "@mrinath look here I implemented full COTS competition pipeline using YoloX -> https://www.kaggle.com/remekkinas/yolox-full-training-pipeline-for-cots-dataset
\n@phalanx - could you tell me how to make some improvements in my YoloX notebook?
", "votes": 3, "replies": [] }, { "id": 1600487, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2021-11-30T13:14:04.617000", "content": "Thanks @remekkinas
", "votes": 1, "replies": [] }, { "id": 1606804, "author_name": "AndyYuan", "author_url": "", "post_date": "2021-12-05T09:20:04.893000", "content": "Hi, I use the author's metric, and my local cv is also high, but lb is very low, just 0.021, did you figure out why?
", "votes": 0, "replies": [] }, { "id": 1611581, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2021-12-08T04:51:13.380000", "content": "I still use the metric that I shared above, you can give it a try, it seems to work e.g. CV and LB correlates. We don't know the distribution of empty images in the public test set, so if your validation has a drastically different ratio of empty images that might also underestimate or overestimate depending on which side you are on.
\nMy latest model
\nTrain : Video 0,1 (Only positive images)\nValid: Video 2 (All images)\nYOLOX-L (Full res: 800,1280) TH:0.03 (Probably overfitting)- CV: 0.58 LB: 0.486\nOK, now my model's result looks like normal, before my lb is only 0.021, now my lb is 0.467, and cv is 0.79, although still have large gap, I will try your metric to see if cv can be down
", "votes": 0, "replies": [] }, { "id": 1611936, "author_name": "AndyYuan", "author_url": "", "post_date": "2021-12-08T12:01:55.213000", "content": "the metric's result don't have different with phalanx's metric, I should study how to split train/val
", "votes": 0, "replies": [] }, { "id": 1614649, "author_name": "DeepInvolution", "author_url": "", "post_date": "2021-12-11T11:15:06.353000", "content": "@keremt Hi, I try your metric, NameError: name 'box_iou' is not defined. May I know how to calculate box_iou?
", "votes": 0, "replies": [] }, { "id": 1615349, "author_name": "ammarali32", "author_url": "", "post_date": "2021-12-12T07:11:16.573000", "content": "from torchvision.ops import box_iou
", "votes": 0, "replies": [] }, { "id": 1616336, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2021-12-13T10:55:50.193000", "content": "I found that changing one param in coco_eval.py constructs competition metrics for precission and recall, its easy to calculate F2 score. I made topic how to make it work
\nhttps://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/294854
", "votes": 2, "replies": [] }, { "id": 1622132, "author_name": "", "author_url": "", "post_date": "2021-12-18T11:52:54.010000", "content": "", "votes": 0, "replies": [] }, { "id": 1622136, "author_name": "Remek Kinas", "author_url": "", "post_date": "2021-12-18T11:56:14.110000", "content": "Yes.
\nBetter CV dataset. My published implementation uses simple one. You can find some notebooks which use better dataset cv strategy. Then … you will be able without problem jump over 0.5. I am sure.
Regarding to @phalanx implementation of metric, i made some changes to it. Now it correctly do metrics with empty images. Tested with only annotated and annotated+unannoted and recall is same in both metrics which means it works :) i have added confidence threshold to calculate score for given confthre. There is also script how to feed tensor to this procedure. And finally calculation of recall and precision in 0.3 and 0.5 iou threshold. Cheers and good luck in competition :)
\ndl_val is pytorch dataloader for validation set - you can feed images with empty annotation and it will still hold the recall)
\ndef calculate_score(\n preds: List[torch.Tensor],\n gts: List[torch.Tensor],\n iou_th: float\n) -> float:\n num_tp = 0\n num_fp = 0\n num_fn = 0\n for p, gt in zip(preds, gts):\n if len(p) and len(gt):\n iou_matrix = box_iou(p, gt)\n tp = len(torch.where(iou_matrix.max(0)[0] >= iou_th)[0])\n fp = len(p) - tp\n fn = len(torch.where(iou_matrix.max(0)[0] < iou_th)[0])\n num_tp += tp\n num_fp += fp\n num_fn += fn\n elif len(p) == 0 and len(gt):\n num_fn += len(gt)\n elif len(p) and len(gt) == 0:\n num_fp += len(p)\n if (5 * num_tp + 4 * num_fn + num_fp )!=0:\n score = 5 * num_tp / (5 * num_tp + 4 * num_fn + num_fp )\n else:\n score = np.nan\n if (num_tp+num_fn) != 0:\n recall = num_tp/ (num_tp+num_fn)\n else:\n recall=np.nan\n if (num_tp+num_fp)!=0:\n precission = num_tp/ (num_tp+num_fp)\n else:\n precission=np.nan\n\n\n return score, precission, recall\ndef evaluate_f2(confthre):\n scores = []\n prec05 = []\n rec05 = []\n prec03 = []\n rec03 = []\n iou_ths = np.arange(0.3, 0.85, 0.05)\n with torch.no_grad():\n for images, targets in dl_val:\n model.eval()\n images = list(image.to(device) for image in images)\n targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n preds = model(images)\n\n for i in range(len(images)):\n preds[i]['boxes']=preds[i]['boxes'].int()\n preds[i]['boxes']=preds[i]['boxes'][preds[i]['scores']>confthre]\n score = [calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), iou_th)[0] for iou_th in iou_ths]\n scores.append(np.nanmean(score))\n prec05.append(calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), 0.5)[1]) \n prec03.append(calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), 0.3)[1]) \n rec05.append(calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), 0.5)[2]) \n rec03.append(calculate_score(preds[i]['boxes'].unsqueeze(0), targets[i]['boxes'].unsqueeze(0), 0.3)[2]) \n print(f'F2 Score for confthre , {confthre}, : {np.nanmean(scores):.3f} Precission .5: {np.nanmean(prec05):.3f} Precission .3: {np.nanmean(prec03):.3f} Recall .5: {np.nanmean(rec05):.3f} Recall .3: {np.nanmean(rec03):.3f}')\ngood work and knowledgable，i am confused how can i use it for my own model, such as yolov5. thanks ahead.
", "votes": 0, "replies": [] }, { "id": 1636863, "author_name": "Jaideep", "author_url": "", "post_date": "2022-01-03T10:36:18.720000", "content": "@remekkinas thanks or sharing ideas.
\nI just joined the competition.
\n1) Which valiation strategy do you use - Subseq, Seq Gkf, videos
\n2) does YoloX has same metric as competition.
\n3) Do you use background images, how mch improvement do you get with it.
sorry … only one answer came to my mind 😂😂😳😳🙄😭😭
", "votes": 0, "replies": [] }, { "id": 1644908, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-01-10T15:42:40.153000", "content": "@xiaojiu1414 from torchvision.ops import box_iou
Hi @lukaszborecki I used your provided above code but the model which is giving 0.393 on LB, it is showing f2 = 0.132 using that code,
\nF2 Score for confthre , 0.13, : 0.132 Precission .5: 0.863 Precission .3: 0.902 Recall .5: 0.148 Recall .3: 0.153
\nAm I doing anything wrong here? Sorry for asking this qsn now, when the code has been posted over a month ago.
There are so many implementations of F2 score all around the discussion posts, which one are you guys using ? I am confused
", "votes": 1, "replies": [] }, { "id": 1663614, "author_name": "somuSan", "author_url": "", "post_date": "2022-01-25T09:11:01.050000", "content": "Hi @mrinath yes, there are many but these two are working fine. There was some problem with my model, But, code provided by lukasz is working fine,
\nand the comp metric implementation by camaro is also ok [although you may need to adjust some code for fasterRCNN, good for yolo]. for camaro's implementation you can add this code for replacing calc_is_correct and calc_f2_score. This modification is bcoz of this reason: f2_score has to be calculated for each IoU threshold. But in the current implementation, f2_score seems to be calculated from the sum of tp/fp/fn for all thresholds. Use the detection threshold first and filter the desired bbox first and then pass them in this code.
def calc_is_correct(gt_bboxes, pred_bboxes, iou_th=0.5):\n \"\"\"\n gt_bboxes: (N, 4) np.array in xywh format\n pred_bboxes: (N, 5) np.array in conf+xywh format\n \"\"\"\n if len(gt_bboxes) == 0 and len(pred_bboxes) == 0:\n tps, fps, fns = 0, 0, 0\n return tps, fps, fns\n\n elif len(gt_bboxes) == 0:\n tps, fps, fns = 0, len(pred_bboxes), 0\n return tps, fps, fns\n\n elif len(pred_bboxes) == 0:\n tps, fps, fns = 0, 0, len(gt_bboxes)\n return tps, fps, fns\n\n pred_bboxes = pred_bboxes[pred_bboxes[:,0].argsort()[::-1]] # sort by conf\n\n tps, fps, fns = 0, 0, 0\n tp, fp, fn = calc_is_correct_at_iou_th(gt_bboxes, pred_bboxes, iou_th)\n tps += tp\n fps += fp\n fns += fn\n return tps, fps, fns\n\ndef calc_f2_score(gt_bboxes_list, pred_bboxes_list, verbose=False):\n \"\"\"\n gt_bboxes_list: list of (N, 4) np.array in xywh format\n pred_bboxes_list: list of (N, 5) np.array in conf+xywh format\n \"\"\"\n f2s = []\n for iou_th in np.arange(0.3, 0.85, 0.05):\n tps, fps, fns = 0, 0, 0\n for gt_bboxes, pred_bboxes in zip(gt_bboxes_list, pred_bboxes_list):\n tp, fp, fn = calc_is_correct(gt_bboxes, pred_bboxes, iou_th)\n tps += tp\n fps += fp\n fns += fn\n if verbose:\n num_gt = len(gt_bboxes)\n num_pred = len(pred_bboxes)\n print(f'num_gt:{num_gt:<3} num_pred:{num_pred:<3} tp:{tp:<3} fp:{fp:<3} fn:{fn:<3}')\n f2 = f_beta(tps, fps, fns, beta=2) \n print(f'f2@{iou_th}:{f2}')\n f2s.append(f2)\n return np.mean(f2s)\nThanks @soumya9977
", "votes": 1, "replies": [] } ] }, { "id": 1664138, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-01-25T18:23:41.220000", "content": "LB 0.698 -> 0.700
\nInfer Size: 2400
\nFold 90/10 single model
\nGPU: P100'
\nModel: yolo5
May I ask what your cv score of that model is?
", "votes": 1, "replies": [] }, { "id": 1664664, "author_name": "DiamondH", "author_url": "", "post_date": "2022-01-26T06:39:31.990000", "content": "really crazy…, my best single one is also yolo, but just 0.627, If I got your score, everything wil goes easier!😃
", "votes": 0, "replies": [] }, { "id": 1664702, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-01-26T07:28:16.103000", "content": "To bo honest on IMG 2400 CV was weaker but on training size it was 0.601
", "votes": 2, "replies": [] }, { "id": 1664745, "author_name": "DiamondH", "author_url": "", "post_date": "2022-01-26T08:36:50.840000", "content": "Could you share how many epochs did it take?
", "votes": 0, "replies": [] }, { "id": 1664755, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-01-26T08:55:34.093000", "content": "Training was performed on 10 epochs
", "votes": 0, "replies": [] }, { "id": 1664764, "author_name": "DiamondH", "author_url": "", "post_date": "2022-01-26T09:13:19.553000", "content": "OK, thanks, I'm giving a try.
", "votes": 0, "replies": [] }, { "id": 1664795, "author_name": "somuSan", "author_url": "", "post_date": "2022-01-26T09:28:58.347000", "content": "what was the training image size?
", "votes": 0, "replies": [] }, { "id": 1664797, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-01-26T09:30:11.723000", "content": "So you are using yolov5l6, right? I'm still using yolov5s6…
", "votes": 1, "replies": [] }, { "id": 1664826, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-01-26T09:55:48.407000", "content": "yes … but …. I am almost sure that you can use \"s\" as well …. We have plan to switch to S … as well.
", "votes": 1, "replies": [] }, { "id": 1664844, "author_name": "DiamondH", "author_url": "", "post_date": "2022-01-26T10:11:35.160000", "content": "Yeah, I do love s6 more, so tiny. How about the time of l6 for inferencing? if it's really time consuming, I'll give up my mind…
", "votes": 2, "replies": [] }, { "id": 1664900, "author_name": "Tanish Gupta", "author_url": "", "post_date": "2022-01-26T11:29:46.853000", "content": "@lukaszborecki can you explain what is FOLD 90/10?
", "votes": 0, "replies": [] }, { "id": 1664921, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-01-26T11:54:18.747000", "content": "only 1 fold out of 10
", "votes": 1, "replies": [] }, { "id": 1665105, "author_name": "Jackie Mai", "author_url": "", "post_date": "2022-01-26T15:19:29.103000", "content": "Do you use any tricks to get this score or just use yolov5 to get this score?
", "votes": 0, "replies": [] }, { "id": 1665170, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-01-26T16:21:13.880000", "content": "Of course we have ;)
", "votes": 1, "replies": [] }, { "id": 1665188, "author_name": "Jackie Mai", "author_url": "", "post_date": "2022-01-26T16:35:12.303000", "content": "Oh, i see, by the way now i have no idea to get higher score, can you share some tips about the tricks you trained the yolov5. Thank you !
", "votes": 0, "replies": [] }, { "id": 1665196, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-01-26T16:38:41.940000", "content": "@jackiemai of course not :) after competition sure
", "votes": 9, "replies": [] }, { "id": 1665390, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-01-26T19:36:38.707000", "content": "@deepkim @w3579628328 is this a challenge :P haha we fight for 0.001 :)
", "votes": 2, "replies": [] }, { "id": 1665690, "author_name": "DiamondH", "author_url": "", "post_date": "2022-01-27T04:18:05.583000", "content": "yeah ,haha, best luck to both of our team, fight for higher rank!😋
", "votes": 0, "replies": [] }, { "id": 1666132, "author_name": "Sarvagya Malaviya", "author_url": "", "post_date": "2022-01-27T13:11:33.437000", "content": "That's a really good score! Did you use a split based on subsequences?
", "votes": 1, "replies": [] }, { "id": 1666714, "author_name": "Daniel Duda", "author_url": "", "post_date": "2022-01-28T02:27:21.340000", "content": "Hi. Could you tell me which fold do you use to val while there are 10folds.
", "votes": 0, "replies": [] }, { "id": 1669895, "author_name": "aquaright", "author_url": "", "post_date": "2022-01-31T03:40:33.557000", "content": "@remakkinas @lukaszboreki were you guys able to get V100's on pro+ or just P100's
\nalso, is that lb score with post processing or without post processing?
", "votes": 0, "replies": [] }, { "id": 1670052, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-01-31T07:23:03.797000", "content": "Most of the time I have Tesla V100-SXM2.
", "votes": 1, "replies": [] }, { "id": 1671697, "author_name": "kaggler", "author_url": "", "post_date": "2022-02-01T16:58:34.683000", "content": "@lukaszborecki good luck!
", "votes": 1, "replies": [] }, { "id": 1671738, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-02-01T17:37:31.647000", "content": "@w3579628328 you found 'my precious' ? :D
", "votes": 1, "replies": [] }, { "id": 1675464, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2022-02-04T09:30:42.943000", "content": "I guess you did some preprocessing isnt it? @lukaszborecki
", "votes": 0, "replies": [] }, { "id": 1676136, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-02-04T17:58:15.120000", "content": "nope we didnt , only preprocessing was increasing from 1920 size to 2400
", "votes": 0, "replies": [] }, { "id": 1677197, "author_name": "Jaideep", "author_url": "", "post_date": "2022-02-05T15:17:48.727000", "content": "@lukaszborecki what was you cv split method
", "votes": 0, "replies": [] } ] }, { "id": 1614381, "author_name": "ONODERA", "author_url": "", "post_date": "2021-12-11T02:42:41.230000", "content": "model: yolov5
\nsplit: 5 folds
\nepoch: 20
\nRecall: 0.81787
\nCV: 0.68264
\nLB: 0.591
can I ask how you split train data to 5 folds? many thanks.
", "votes": 2, "replies": [] }, { "id": 1617996, "author_name": "Phat Tran", "author_url": "", "post_date": "2021-12-14T14:57:23.873000", "content": "Is it single fold prediction or ensemble of 5 folds
", "votes": 1, "replies": [] }, { "id": 1619404, "author_name": "Remek Kinas", "author_url": "", "post_date": "2021-12-15T22:38:54.577000", "content": "Could you tell me what image input size did you use in your cv5 experiment? Thank you @onodera.
", "votes": 0, "replies": [] }, { "id": 1619942, "author_name": "ONODERA", "author_url": "", "post_date": "2021-12-16T09:29:44.373000", "content": "5 groupkfold using sequence. LB is blend with wbf.
\ninput size is 1280.
\nFYI, I'm not using sequence frame so far. I think 0.62~ would be easier if we can utilize this.
Thank you very much for answering. I am still trying to boost score using YoloX …. but y5 looks promising.
", "votes": 0, "replies": [] }, { "id": 1623715, "author_name": "SeongwookLee", "author_url": "", "post_date": "2021-12-20T07:02:25.513000", "content": "@onodera Are you use only labeled data ?
", "votes": 0, "replies": [] }, { "id": 1624687, "author_name": "DeepInvolution", "author_url": "", "post_date": "2021-12-21T06:07:46.040000", "content": "@onodera Hi, I am newbie in Object Detection. Mind I ask how to check the recall score? Get it from below?
\n\n", "votes": 0, "replies": [] }, { "id": 1624733, "author_name": "Kevin", "author_url": "", "post_date": "2021-12-21T07:15:23.273000", "content": "20313.4s 1794 Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.124
\n
\n 20313.4s 1795 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.266
\n 20313.4s 1796 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.089
\n 20313.4s 1797 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.016
\n 20313.4s 1798 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.155
\n 20313.4s 1799 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
\n 20313.4s 1800 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.092
\n 20313.4s 1801 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.167
\n 20313.4s 1802 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.167
\n 20313.4s 1803 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.033
\n 20313.4s 1804 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.203
\n 20313.4s 1805 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
I guess you are using the video_2 for validation, but there is no target with large area in video_2. Therefore, your AR/AP on large area is -1, which can't be calculated.
", "votes": 1, "replies": [] }, { "id": 1624780, "author_name": "DeepInvolution", "author_url": "", "post_date": "2021-12-21T08:19:20.840000", "content": "@jarviskevin Hi, no, I use GroupKFold method with sequence as groups. Then could you advise which AR score to refer given different area conditions? Since ONODERA referred a single AR score.
", "votes": 0, "replies": [] }, { "id": 1624792, "author_name": "Remek Kinas", "author_url": "", "post_date": "2021-12-21T08:31:22.903000", "content": "20313.4s 1802 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.167
\nbut better is to implement f2 score
", "votes": 0, "replies": [] }, { "id": 1624797, "author_name": "DeepInvolution", "author_url": "", "post_date": "2021-12-21T08:41:20.160000", "content": "@remekkinas Thanks bro! I did implement F2 score. Just learn from you guys. Really not experienced in Object Detection.
", "votes": 0, "replies": [] }, { "id": 1644896, "author_name": "DeepInvolution", "author_url": "", "post_date": "2022-01-10T15:33:26.243000", "content": "@onodera Hi, do you mind if I ask how you guys succeed in wbf ensemble? Really struggled in this. Cannot get a reasonable score so far. Any tips that can be shared?
", "votes": 0, "replies": [] }, { "id": 1646745, "author_name": "sheep", "author_url": "", "post_date": "2022-01-12T02:21:07.767000", "content": "Single fold, validation score 0.63, LB 0.54. Highly correlated with your score, hope my LB will elevate when I boost my validation score.
\nAnother fold: validation score: 0.59, LB: 0.58 😣
", "votes": 1, "replies": [] }, { "id": 1647677, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2022-01-12T19:31:38.923000", "content": "How are you splitting your data? @steamedsheep
", "votes": 0, "replies": [] }, { "id": 1647815, "author_name": "Tian", "author_url": "", "post_date": "2022-01-13T00:17:16.623000", "content": "sheep哥一出手 真就不一样。
", "votes": 1, "replies": [] }, { "id": 1647968, "author_name": "sheep", "author_url": "", "post_date": "2022-01-13T03:06:22.023000", "content": "I split fold via video_id, I have only 3 folds.
", "votes": 2, "replies": [] } ] }, { "id": 1672677, "author_name": "Good Moon", "author_url": "", "post_date": "2022-02-02T08:28:11.393000", "content": "single model:yolov5s6
\nLB 0.661
\nbatch = 8
\nepoch = 15
\ntrain and infer size: 3584
What is your hardware?
", "votes": 0, "replies": [] }, { "id": 1672928, "author_name": "Good Moon", "author_url": "", "post_date": "2022-02-02T11:44:12.640000", "content": "The model needs very large GPU RAM.You can get more information in my discussion book. I prepare to do a series of training experiment. I will update the result in it.
", "votes": 2, "replies": [] }, { "id": 1672930, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-02-02T11:47:17.483000", "content": "Good observation. I wrote about batch_size in large image res … and was voted down :) Now you confirmed this observation. Thank you!
", "votes": 4, "replies": [] }, { "id": 1672966, "author_name": "Good Moon", "author_url": "", "post_date": "2022-02-02T12:14:12.270000", "content": "It's exciting for me to get your reply, I was inspired by your previous comments, so I did this. I will try more training in smaller GPU RAM to provide more valuable results. If small size , small batch and small model can make sense , that's will make the kagglers without enough GPU RAM to fairly join in the competition.
", "votes": 3, "replies": [] }, { "id": 1673128, "author_name": "kaggler", "author_url": "", "post_date": "2022-02-02T14:23:27.213000", "content": "What is your cv of that model?
", "votes": 0, "replies": [] }, { "id": 1673138, "author_name": "Good Moon", "author_url": "", "post_date": "2022-02-02T14:31:13.787000", "content": "10 fold, I only train the the first fold(leave the 1st fold as valdata) model and submit the single model.
", "votes": 0, "replies": [] }, { "id": 1673290, "author_name": "somuSan", "author_url": "", "post_date": "2022-02-02T16:21:22.143000", "content": "what was your f2 score on the first fold?
", "votes": 0, "replies": [] }, { "id": 1673376, "author_name": "Good Moon", "author_url": "", "post_date": "2022-02-02T17:32:35.840000", "content": "All coco indicator and f2 are approximate 0.8.
", "votes": 0, "replies": [] }, { "id": 1674811, "author_name": "Ioannis M", "author_url": "", "post_date": "2022-02-03T19:52:43.740000", "content": "\n\n10 fold, I only train the the first fold…
\n
\n All coco indicator and f2 are approximate 0.8.
@freshair1996 thanks! Your cv split is based on sequences?
\nThat's quite good CV
no，I have no idea to train on sequence，put the shuffle off？
", "votes": -1, "replies": [] }, { "id": 1679575, "author_name": "豆柴金鯱", "author_url": "", "post_date": "2022-02-07T09:50:33.047000", "content": "I guess @imeintanis means splitting the data via GroupKFold on \"video sequence\" column
", "votes": 1, "replies": [] } ] }, { "id": 1608665, "author_name": "Matthieu Planté", "author_url": "", "post_date": "2021-12-06T14:41:39.940000", "content": "Model: Yolov5
\nSplit: 10 folds
\nepochs: 12
\nCV: 0.5639
\nLB: 0.539
Hi, Matthieu, can I ask how you calculate cv, as I use the cv code from other, I find that my cv score is large than LB very much, for example, lb 0.4, cv0.75
", "votes": 0, "replies": [] }, { "id": 1611765, "author_name": "Matthieu Planté", "author_url": "", "post_date": "2021-12-08T08:46:20.460000", "content": "Hi, I think the gap between your CV and your LB comes from the way you split the data, not the code you used. I used the code @phalanx gave below.
\nYou should read this discussion
", "votes": 1, "replies": [] }, { "id": 1611926, "author_name": "AndyYuan", "author_url": "", "post_date": "2021-12-08T11:42:04.613000", "content": "OK, thank you.
", "votes": 0, "replies": [] }, { "id": 1612173, "author_name": "AndyYuan", "author_url": "", "post_date": "2021-12-08T16:22:10.837000", "content": "can I ask what do you mean 10 folds, as I'm new to kaggle competition, the question may looks stupid.
", "votes": 0, "replies": [] }, { "id": 1612721, "author_name": "Matthieu Planté", "author_url": "", "post_date": "2021-12-09T08:52:42.270000", "content": "There is no stupid question. I invite you to read some articles about cross validation k fold, you will find easily!
@mathieuplante how did you split the data?
", "votes": 1, "replies": [] }, { "id": 1612788, "author_name": "Matthieu Planté", "author_url": "", "post_date": "2021-12-09T10:16:03.900000", "content": "I used this notebook and for the moment I have a good LB/CV correlation
", "votes": 7, "replies": [] }, { "id": 1613093, "author_name": "AndyYuan", "author_url": "", "post_date": "2021-12-09T15:33:41.923000", "content": "now I understand that k fold can be used to choose model or evaluate model performance, so wonder you train ten times with each fold as validation and get ten models, then fusion the model's result, or just train one time with one fold as validation.
", "votes": 0, "replies": [] }, { "id": 1613103, "author_name": "Matthieu Planté", "author_url": "", "post_date": "2021-12-09T15:43:06.807000", "content": "For the moment I only have one model but later I will train 10 models and merge yes
", "votes": 1, "replies": [] }, { "id": 1618636, "author_name": "clwclw", "author_url": "", "post_date": "2021-12-15T07:24:46.567000", "content": "Are you still only use yolov5 object detection model, without video object detection method?
", "votes": 1, "replies": [] }, { "id": 1619405, "author_name": "Remek Kinas", "author_url": "", "post_date": "2021-12-15T22:39:29.770000", "content": "@matthieuplante what image input size did you use during training? Thank you!
", "votes": 0, "replies": [] }, { "id": 1631055, "author_name": "", "author_url": "", "post_date": "2021-12-28T01:54:15.683000", "content": "", "votes": 0, "replies": [] }, { "id": 1645534, "author_name": "Tian", "author_url": "", "post_date": "2022-01-11T02:04:47.287000", "content": "🙏 I use this split too, But still got a large gap between cv and lb.
", "votes": 0, "replies": [] }, { "id": 1679605, "author_name": "豆柴金鯱", "author_url": "", "post_date": "2022-02-07T10:04:37.003000", "content": "Same here, very curious how to fill the large GAP btw CV & LB
", "votes": -1, "replies": [] } ] }, { "id": 1596437, "author_name": "Psi", "author_url": "", "post_date": "2021-11-26T13:47:30.847000", "content": "If you don't mind me asking, how do you guys do 80/20 split in this dataset?
", "votes": 10, "replies": [ { "id": 1596448, "author_name": "DrHB", "author_url": "", "post_date": "2021-11-26T13:55:24.543000", "content": "in my baseline I did it pretty naively .. Just took all the images which has box labels around ~4k and split them in to 80/20. Currently my model doesn't see any background images.
", "votes": 3, "replies": [] }, { "id": 1597125, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2021-11-27T08:28:52.863000", "content": "so your model is only training at frames with starfish, have you seen how does it perform when a blank image(no starfish) is given as input?
\nand another thing I have never worked with videos so don't have experience, How do we generally split video data, I was thinking about 3 folds(video 0,video1,video2) but now think it may not be correct.
Yes, on blank images its also predict some bbox but the p of being starfish is pretty low. So we can filter it… But there are some fp with high probability. This is first time I am working with object detection, perhaps I need to read up on how people deal with background images where no object is present. Regarding cv I still have to try few stuff.
model: yolov5l(single model, no TTA)
\nsplit: sequence_id
\nCV: 0.67
\nLB: 0.637
Can i ask you about image size when you infer?
", "votes": 0, "replies": [] }, { "id": 1643642, "author_name": "", "author_url": "", "post_date": "2022-01-09T15:47:59.713000", "content": "", "votes": 0, "replies": [] }, { "id": 1644039, "author_name": "SeongwookLee", "author_url": "", "post_date": "2022-01-10T01:59:41.490000", "content": "Hi @clwwlc, can I ask you about batch_size and how to calculate CV ? And do you use groupkfold ?
", "votes": 0, "replies": [] }, { "id": 1648687, "author_name": "Dwight Foster", "author_url": "", "post_date": "2022-01-13T15:52:07.533000", "content": "Hi @clwwlc, I was wondering if you train your models on a local machine/cloud computing or use a kaggle kernel. Currently I am training models on my 2080 but it does not have enough gpu memory to train a yolov5l and the kaggle kernels are very slow.
", "votes": 0, "replies": [] }, { "id": 1649768, "author_name": "clwclw", "author_url": "", "post_date": "2022-01-14T15:17:37.253000", "content": "I use f2 metric, and only 1 fold
", "votes": 0, "replies": [] }, { "id": 1649769, "author_name": "clwclw", "author_url": "", "post_date": "2022-01-14T15:18:29.950000", "content": "local machine, for my solution now 2080 is enough
", "votes": 1, "replies": [] }, { "id": 1653851, "author_name": "ayu055", "author_url": "", "post_date": "2022-01-17T23:20:57.090000", "content": "@gtownfoster batch size along with your image size may be too large.
", "votes": 0, "replies": [] } ] }, { "id": 1646731, "author_name": "Dwight Foster", "author_url": "", "post_date": "2022-01-12T01:26:51.073000", "content": "model: yolov5m(single model)
\nsplit: sequence_id, 4 folds
\nCV: 0.76
\nLB: 0.503
\nBatch size: 4
\nEpochs: 20
\nConf: 0.411
\nTo get this I did a lot of testing and running of different hyperparameters. I know that my cv and lb are far apart so that is what I will be working on next. I just wanted to post because I was able to get above a score of 0.5 with a yolov5m model.
model : yolov5
\nsplit : 5 folds
\nCV : 0.6384
\nLB : 0.598
Whay to you mean by CV result? It is recall, ap@95 or f2 score? Thank you!
\nCongratulations!!
it means f2-score, and single model (not 5 folds ensemble) !!
", "votes": 0, "replies": [] }, { "id": 1622891, "author_name": "Remek Kinas", "author_url": "", "post_date": "2021-12-19T08:35:17.250000", "content": "Thank you! I now understand.
", "votes": 0, "replies": [] }, { "id": 1622929, "author_name": "clwclw", "author_url": "", "post_date": "2021-12-19T09:36:46.257000", "content": "Good job, can you share some changes of hyp or input_size? I can only get 0.51 using yolov5-l with 1280 input-size, 20 epoch
", "votes": 0, "replies": [] }, { "id": 1624621, "author_name": "T0m", "author_url": "", "post_date": "2021-12-21T04:36:43.413000", "content": "image size is 1280, and model is yolov5-l.
\nI can't say the details, but some augmentation param changes may help.
ok, thanks
", "votes": 0, "replies": [] }, { "id": 1630771, "author_name": "LucasPimentel", "author_url": "", "post_date": "2021-12-27T17:21:30.827000", "content": "@clwwlc Could you tell me how you split the data? Thanks!
\nAny tips about how i could improve from the default yolov5-l?
Sorry for the delay in responding, I splitted it by extracting video subsequence.
", "votes": 2, "replies": [] }, { "id": 1646529, "author_name": "Ioannis M", "author_url": "", "post_date": "2022-01-11T20:07:10.987000", "content": "thanks for sharing @tomyanabe good job! May I ask what CV method you are using and if the score reported is calculated with all images (labeled + empty)?
", "votes": 0, "replies": [] } ] }, { "id": 1603446, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2021-12-02T14:32:57.803000", "content": "framework; PyTorch
\nmodel: FasterRCNN-(backbone:resnet101)
\nsplit: 85/15 (only containing label)
\nepoch: 20
\nCV: no competition metric implemented
\nLB: 0.442
\nConfidence Threshold: 0.80
\n\nI think some work has to be done to figure what is the best way to split ..
\n
I haven't started yet, but I was thinking GroupKFold on sequence. What do you think?
I guess we have to try and see what works =)
", "votes": 1, "replies": [] }, { "id": 1597476, "author_name": "datasaurus", "author_url": "", "post_date": "2021-11-27T14:38:11.640000", "content": "A quick single fold experiment with GroupKFold and I'm seeing validation F2 of 0.426 (4K images only). Changing to KFold and it increases to 0.70+, so there's a lot of leakage between the sequences
", "votes": 4, "replies": [] } ] }, { "id": 1658382, "author_name": "Liam Nguyen", "author_url": "", "post_date": "2022-01-21T01:24:45.390000", "content": "Faster Cascade RCNN ResNeSt200
\nsplit 5 folds
\nCV (1st fold): 0.562
\nLB (1st fold): 0.579
yolov5s6
\nCV: 0.72
\nLB: 0.642
\nBatch size: 3
\nEpochs: 9 with transfer learning
model: yolov5
\nsplit: 4folds
\nCV: 0.64159
\nLB: 0.545
Hi @kevin1742064161 How do you evaluate your F2 score? in training or after? Thanks
", "votes": 0, "replies": [] }, { "id": 1642802, "author_name": "Jaideep", "author_url": "", "post_date": "2022-01-08T17:27:21.633000", "content": "@kevin1742064161 I think F2 score computed by public Model isnt looking True.
", "votes": -1, "replies": [] }, { "id": 1642841, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-01-08T18:25:45.293000", "content": "Interesting. What does it mean for you?
", "votes": 0, "replies": [] }, { "id": 1643198, "author_name": "Jaideep", "author_url": "", "post_date": "2022-01-09T06:38:24.933000", "content": "0.66 vs in 0.5s at lb for quite a many
", "votes": 0, "replies": [] }, { "id": 1645740, "author_name": "MOONMOON", "author_url": "", "post_date": "2022-01-11T07:16:51.810000", "content": "https://www.kaggle.com/kevin1742064161/yolov5-cal-f2score/notebook
\n I modified some code in yolov5. You can get it from this notebook
@kevin1742064161 good job! may I ask what CV strategy you are using (subseq, gkf seq, etc) and if the score you report is from single fold or ens?
", "votes": 1, "replies": [] }, { "id": 1653730, "author_name": "somuSan", "author_url": "", "post_date": "2022-01-17T20:21:56.850000", "content": "Hi @imeintanis can you please tell me what is \"gkf seq\"? is it group kfold on sequence?
", "votes": 1, "replies": [] }, { "id": 1653758, "author_name": "Ioannis M", "author_url": "", "post_date": "2022-01-17T20:51:32.990000", "content": "\n\nis it group kfold on sequence?
\n
exactly
", "votes": 1, "replies": [] } ] }, { "id": 1604968, "author_name": "Simon Alerdic", "author_url": "", "post_date": "2021-12-03T22:22:18.387000", "content": "@drhabib
\nHave you resize the input image from (720, 1280) to (1280, 1280)? If so, such resizing is making disturbance on the spatial information of the input samples. What do you think?
No I did not, I think for CNNs you don't have to resize from 720 x 1280 to 1280 x 1280. They should work fine on rectangular images as long as all images have same dimensions..
good experiments
", "votes": 1, "replies": [] }, { "id": 1600002, "author_name": "Shubh-datascientist", "author_url": "", "post_date": "2021-11-30T03:16:41.617000", "content": "good work and knowledgable
", "votes": 1, "replies": [] }, { "id": 1597221, "author_name": "Awsaf", "author_url": "", "post_date": "2021-11-27T10:28:01.740000", "content": "@drhabib what img-size did you use?
", "votes": 1, "replies": [ { "id": 1597390, "author_name": "DrHB", "author_url": "", "post_date": "2021-11-27T13:10:28.290000", "content": "image size 1280
", "votes": 2, "replies": [] } ] }, { "id": 1649696, "author_name": "NTTT", "author_url": "", "post_date": "2022-01-14T14:19:40.787000", "content": "EfficientDet d5
\nsplit 5 folds
\nCV: 0.55
\nLB: 0.586
\n1 fold of 5 folds
That's nice for EffDet. Did you train it using TF or Pytorch ? I have trained Pytorch EffDet and it had a good validation score (0.62x). However it did not work on LB (Only 0.36x). When doing analysis I noticed it predicted less bbox than other models -> Less True Positive. Maybe that was the reason why it failed on LB.
", "votes": 1, "replies": [] }, { "id": 1658995, "author_name": "NTTT", "author_url": "", "post_date": "2022-01-21T13:23:58.630000", "content": "Hi @namgalielei This is pytorch effdet and i used this repo https://github.com/rwightman/efficientdet-pytorch
\nAbout your LB score I think you should check your dataset. Hmm maybe have leak?
yeah, maybe. Which image size did you use to train and infer? I used 1024x1024.
", "votes": 0, "replies": [] }, { "id": 1662647, "author_name": "NTTT", "author_url": "", "post_date": "2022-01-24T12:55:07.120000", "content": "I used 1280x1280 for both train and inference
", "votes": 2, "replies": [] }, { "id": 1664784, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2022-01-26T09:23:48.123000", "content": "hey I am also trying with effdet, but the max size I can train is img size = 876, how are you raining your model on such a large image size?
", "votes": 0, "replies": [] }, { "id": 1665011, "author_name": "NTTT", "author_url": "", "post_date": "2022-01-26T13:45:40.170000", "content": "Sorry, I did not train on kaggle or colab. I trained on local machine
", "votes": 0, "replies": [] }, { "id": 1665114, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2022-01-26T15:29:12.287000", "content": "I see ,but can you still say like how much GPU memory does it take to train 1280 sized images, I may rent some GPUs so need some idea, how big GPU should I rent.
\n@gigggggge
I used 2 3090 with sync batchnorm but I think single 3090 is enough
", "votes": 1, "replies": [] }, { "id": 1665256, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2022-01-26T17:30:40.533000", "content": "sorry for asking you again and again, where do you change this sync batchnorm in the model? argument for the model class?
", "votes": 0, "replies": [] }, { "id": 1665298, "author_name": "NTTT", "author_url": "", "post_date": "2022-01-26T17:50:26.143000", "content": "I modified some code in the repo and add it as argument
", "votes": 1, "replies": [] } ] }, { "id": 1608671, "author_name": "KAZI SHAMIM SHAHAREAR ISLAM", "author_url": "", "post_date": "2021-12-06T14:47:20.957000", "content": "good work PyTorch
", "votes": 0, "replies": [] }, { "id": 1604864, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2021-12-03T19:37:22.030000", "content": "Do You use NMS in your prediction or you are leaving it for threshold filtering ? Why i am asking because if i revise predictions there are some spams on COTS with bounding boxes but they are less than 0.2 and one is 0.9. So is the threshold enough in your opinion ?
", "votes": 0, "replies": [] }, { "id": 1642800, "author_name": "Jaideep", "author_url": "", "post_date": "2022-01-08T17:25:56.590000", "content": "LB 55.5 one fold , CV 55 YoloX
", "votes": -1, "replies": [] }, { "id": 1685078, "author_name": "qi0239", "author_url": "", "post_date": "2022-02-11T02:59:36.240000", "content": "@outwrest Can you provide your ideas about your solution(100+ epochs, gan-aided copy&paste),Is this unsupervised plus fine-tuning?
", "votes": 0, "replies": [ { "id": 1685091, "author_name": "outwrest", "author_url": "", "post_date": "2022-02-11T03:10:57.320000", "content": "I'll make a post after the competition if I score well. It is too late into the competition to talk about this.
", "votes": 3, "replies": [] } ] }, { "id": 1674783, "author_name": "Gleb", "author_url": "", "post_date": "2022-02-03T19:07:06.103000", "content": "Hey, can somebody please share video_x fold CV? Im getting no correlation at all with public LB, like +5 f2 at cv and -5 f2 at LB
", "votes": 0, "replies": [] }, { "id": 1649787, "author_name": "kaggler", "author_url": "", "post_date": "2022-01-14T15:29:55.450000", "content": "Model Yolo(single model)
\nsplit : video id
\nCV(F2 Score) : 70
\nLB : 67.5
If you dont mind which image size did you use?
", "votes": 1, "replies": [] }, { "id": 1649811, "author_name": "kaggler", "author_url": "", "post_date": "2022-01-14T15:46:13.317000", "content": "I can not say the details, but the Kaggle Gpu is enough .
\nEstablishing a proper neural network and evaluation metrics are important.
Thanks you I understood
", "votes": 1, "replies": [] }, { "id": 1649849, "author_name": "Psi", "author_url": "", "post_date": "2022-01-14T16:15:19.363000", "content": "\n\nCV(F2 Score) : 70
\n
@deepkim On a specific video fold or global average / oof?
", "votes": 2, "replies": [] }, { "id": 1649992, "author_name": "Ioannis M", "author_url": "", "post_date": "2022-01-14T18:25:43.607000", "content": "@deepkim congrats! your CV score is calculated with labeled images only or empty as well ?
\nalso by single model I guess you mean single fold (?)
@philippsinger a specific fold f2 score
", "votes": 0, "replies": [] }, { "id": 1650489, "author_name": "kaggler", "author_url": "", "post_date": "2022-01-15T04:46:07.487000", "content": "@imeintanis background images are included, and you are right i mean single fold
", "votes": 1, "replies": [] }, { "id": 1658102, "author_name": "Tanish Gupta", "author_url": "", "post_date": "2022-01-20T18:30:46.547000", "content": "@deepkim can you share your implementation of F2 score calculation?
", "votes": 1, "replies": [] }, { "id": 1663035, "author_name": "kaggler", "author_url": "", "post_date": "2022-01-24T18:16:06.180000", "content": "it's simple. you calculate tp/fp/fn in validations sets and get a f2 score by iou 0.3,0.35,0.4,….,0.8
\nafter that, average it!
I have a question for you experts！！
\nI trained yolov5s6 with size3600 and then used the following notebook https://www.kaggle.com/kocha1/only-yolov5-tracking-lb-642 for inference. conf0.34 iou0.5, in the case where I only change the img-size of inference:
\nimg-size 3600 lb0.579
\nimg-size 9000 lb0.577
\nWhy does my self-trained model infer no change at large scale?
\nHave you encountered this situation?
I don't think LB is reliable. it's too much fluctuating. I was sometimes faced the issue you encountered
", "votes": 0, "replies": [] } ] }, { "id": 1642069, "author_name": "Camille", "author_url": "", "post_date": "2022-01-08T02:33:49.163000", "content": "@tomyanabe Could you tell met the Train Split size?
", "votes": 0, "replies": [] }, { "id": 1641250, "author_name": "Garvit Garg", "author_url": "", "post_date": "2022-01-07T08:26:17.240000", "content": "model: yolov5-l
\nsplit: video_id
\nCV: 0.642
\nLB: 0.522
model: yolox-l
\nsplit: 5 folds
\nCV: 0.500
\nLB: 0.561
Hi @sqqqqy If you dont mind I have a question about your CV. Did you evaluate in training? or end of the training? Thank you
", "votes": 0, "replies": [] }, { "id": 1641248, "author_name": "Qingyao Shuai", "author_url": "", "post_date": "2022-01-07T08:20:30.517000", "content": "End of training
", "votes": 1, "replies": [] }, { "id": 1641324, "author_name": "Hiếu PTM", "author_url": "", "post_date": "2022-01-07T10:20:09.217000", "content": "thanks for your reply!
", "votes": 0, "replies": [] } ] }, { "id": 1634622, "author_name": "", "author_url": "", "post_date": "2021-12-31T23:07:57.260000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1595843": "post your best single models =) \n\n\n```\nframework; PyTorch\nmodel: FasterRCNN\nsplit: 80/20 (only containing label)\nepoch: 20\nCV: 0.474\nLB: 0.496\n```\n\nI think some work has to be done to figure what is the best way to split ..", "1677259": "model: Cascade R-CNN R50 (MMDET)\nspilt : cross validation by video id then train all\nLB : 0.741\nhardware: RTX3090\n\ncomment\n1.yolov5 is not all you need.\n2.to fit or not to fit that is the question.\n3.I'm not going to answer any questions.\n The post itself is the biggest hint .", "1595988": "model: yolox\nsplit: 80/20 (train: only containing label, val: all data)\nval score: 0.496\nlb: 0.484\n\nmy metric implementation\n```\nfrom typing import List\n\nimport numpy as np\nimport torch\nfrom torchvision.ops import box_iou\n\n\ndef calculate_score(\n preds: List[torch.Tensor],\n gts: List[torch.Tensor],\n iou_th: float\n) -> float:\n num_tp = 0\n num_fp = 0\n num_fn = 0\n for p, gt in zip(preds, gts):\n if len(p) and len(gt):\n iou_matrix = box_iou(p, gt)\n tp = len(torch.where(iou_matrix.max(0)[0] >= iou_th)[0])\n fp = len(p) - tp\n fn = len(torch.where(iou_matrix.max(0)[0] < iou_th)[0])\n num_tp += tp\n num_fp += fp\n num_fn += fn\n elif len(p) == 0 and len(gt):\n num_fn += len(gt)\n elif len(p) and len(gt) == 0:\n num_fp += len(p)\n score = 5 * num_tp / (5 * num_tp + 4 * num_fn + num_fp)\n return score\n\niou_ths = np.arange(0.3, 0.85, 0.05)\nscores = [calculate_score(predictions, gts, iou_th) for iou_th in iou_ths]\nnp.mean(scores)\n>>> 0.49588419817665447\n```", "1664138": "LB 0.698 -> 0.700\nInfer Size: 2400\nFold 90/10 single model\nGPU: P100'\nModel: yolo5", "1614381": "model: yolov5\nsplit: 5 folds\nepoch: 20\nRecall: 0.81787\nCV: 0.68264\nLB: 0.591", "1672677": "1. Image_size is not the key point, you can chose any size in the range (1280-3600),never use a smaller batch for getting a bigger image_size.Just chose the size suit for your GPU with a batch >=8.\n2. yolov5s/yolov5s6 is enough for the task,.do not waste time in yolov5l or more another large model. Check all setting and parameters in the model yolov5s/yolov5s6, and do some adjusting with your understanding.\n3. config=0.28 and iou=0.40 mabey a good parameter for your first inference.\n\nsingle model:yolov5s6\nLB 0.661\nbatch = 8\nepoch = 15\ntrain and infer size: 3584", "1608665": "Model: Yolov5\nSplit: 10 folds\nepochs: 12\nCV: 0.5639\nLB: 0.539", "1596437": "If you don't mind me asking, how do you guys do 80/20 split in this dataset?", "1641692": "model: yolov5l(single model, no TTA)\nsplit: sequence_id\nCV: 0.67\nLB: 0.637\n\n", "1646731": "model: yolov5m(single model)\nsplit: sequence_id, 4 folds\nCV: 0.76\nLB: 0.503\nBatch size: 4\nEpochs: 20\nConf: 0.411\nTo get this I did a lot of testing and running of different hyperparameters. I know that my cv and lb are far apart so that is what I will be working on next. I just wanted to post because I was able to get above a score of 0.5 with a yolov5m model. \n", "1622608": "model : yolov5\nsplit : 5 folds\nCV : 0.6384\nLB : 0.598", "1603446": "framework; PyTorch\nmodel: FasterRCNN-(backbone:resnet101)\nsplit: 85/15 (only containing label)\nepoch: 20\nCV: no competition metric implemented\nLB: 0.442\nConfidence Threshold: 0.80", "1596439": "> I think some work has to be done to figure what is the best way to split ..\n\nI haven't started yet, but I was thinking `GroupKFold` on `sequence`. What do you think?", "1658382": "Faster Cascade RCNN ResNeSt200\nsplit 5 folds\nCV (1st fold): 0.562\nLB (1st fold): 0.579", "1663574": "yolov5s6\nCV: 0.72\nLB: 0.642\nBatch size: 3\nEpochs: 9 with transfer learning", "1639874": "model: yolov5\nsplit: 4folds\nCV: 0.64159\nLB: 0.545", "1604968": "@drhabib \nHave you resize the input image from `(720, 1280)` to `(1280, 1280)`? If so, such resizing is making disturbance on the spatial information of the input samples. What do you think? ", "1600506": "good experiments", "1600002": "good work and knowledgable", "1597221": "@drhabib what img-size did you use?", "1649696": "EfficientDet d5\nsplit 5 folds\nCV: 0.55\nLB: 0.586\n1 fold of 5 folds", "1608671": "good work PyTorch", "1604864": "Do You use NMS in your prediction or you are leaving it for threshold filtering ? Why i am asking because if i revise predictions there are some spams on COTS with bounding boxes but they are less than 0.2 and one is 0.9. So is the threshold enough in your opinion ?", "1642800": "LB 55.5 one fold , CV 55 YoloX", "1685078": "@outwrest Can you provide your ideas about your solution(100+ epochs, gan-aided copy&paste),Is this unsupervised plus fine-tuning?", "1674783": "Hey, can somebody please share video_x fold CV? Im getting no correlation at all with public LB, like +5 f2 at cv and -5 f2 at LB", "1649787": "Model Yolo(single model)\nsplit : video id\nCV(F2 Score) : 70\nLB : 67.5", "1642069": "@tomyanabe Could you tell met the Train Split size?\n", "1641250": "model: yolov5-l\nsplit: video_id\nCV: 0.642\nLB: 0.522", "1641043": "model: yolox-l\nsplit: 5 folds\nCV: 0.500\nLB: 0.561", "1634622": "" } }