![\"\"](\"https://i.ibb.co/bQxW0wX/results-12-3.png\")
So, I took @steamedsheep's training notebook to start with, trained the model with a bit bigger image size of 3840, got a score of .588 with no inference upscaling.
\nThen tried changing a few numbers to do further experiment, and retrained model a lot of times, everytime it scored way below .588
Basically making it completely random when we get good scores and when we don't (except for Adam, we can put our hopes there).
\nThen thought that YoloV5-S6 is not stable, ran M6 one time and scored .556 with image size 3200, ran L6 one time with image size 2560 and scored .427
\nWhy is there so much randomness that the same training can go from scoring .588 to .495 by just doubling the batch size and ofc, .588 retrain is also not happening.
\nWhile this was happening, I submitted only when my CV was increasing.
\nCV for .588:
\nP R mAP@.5 mAP@
\n0.864 0.644 0.725 0.339
Same Fold Best CV but failed miserably on LB:
\nP R mAP@.5 mAP@
\n0.84 0.672 0.744 0.352
Updates:
\nGuidance for reducing this chaos of randomness would be appreciated.
\nThank You.
So, I took @steamedsheep's training notebook to start with, trained the model with a bit bigger image size of 3840, got a score of .588 with no inference upscaling.
\nThen tried changing a few numbers to do further experiment, and retrained model a lot of times, everytime it scored way below .588
Basically making it completely random when we get good scores and when we don't (except for Adam, we can put our hopes there).
\nThen thought that YoloV5-S6 is not stable, ran M6 one time and scored .556 with image size 3200, ran L6 one time with image size 2560 and scored .427
\nWhy is there so much randomness that the same training can go from scoring .588 to .495 by just doubling the batch size and ofc, .588 retrain is also not happening.
\nWhile this was happening, I submitted only when my CV was increasing.
\nCV for .588:
\nP R mAP@.5 mAP@
\n0.864 0.644 0.725 0.339
Same Fold Best CV but failed miserably on LB:
\nP R mAP@.5 mAP@
\n0.84 0.672 0.744 0.352
Updates:
\nGuidance for reducing this chaos of randomness would be appreciated.
\nThank You.
Welcome to the club 😄
\nThis is why I aked forTOP5 final model video on training data after competition :) I would like to see inference on training. It could be great research topic.
I am really waiting for solution description and …. model validations procedure. This is part I have spent most of a time … unfortunately without success. At one point, I even stated that there was something wrong with the implementation of the f2 metric on the Kaggle side, but I guess it's just my lack of knowledge in this area :) Today @lukaszborecki made interesting experiment - just submit crap model …. absolutely crap (with almost random bboxes) and it appeared that … is good (almost like first yolov5 models presented here) on LB :)
", "rawMarkdown": "Welcome to the club 😄\nThis is why I aked forTOP5 final model video on training data after competition :) I would like to see inference on training. It could be great research topic.\n\nI am really waiting for solution description and .... model validations procedure. This is part I have spent most of a time ... unfortunately without success. At one point, I even stated that there was something wrong with the implementation of the f2 metric on the Kaggle side, but I guess it's just my lack of knowledge in this area :) Today @lukaszborecki made interesting experiment - just submit crap model .... absolutely crap (with almost random bboxes) and it appeared that ... is good (almost like first yolov5 models presented here) on LB :) ", "votes": 5, "replies": [ { "id": 1653779, "postDate": "2022-01-17T21:17:45.997Z", "content": "here is the image from inference Remek said and score is 0.45 but bboxes are crap and model misses bigger starfishes, medium one also
and here how should look correct boxes
\n![\"\"](\"https://i.ibb.co/5G6b6Hk/results-12-3-1.png\")
Both photos are from valid …. and both models (first one … according to image resizing trends we just submit for fun) … got almost comparable score 😂
", "rawMarkdown": "Both photos are from valid .... and both models (first one ... according to image resizing trends we just submit for fun) ... got almost comparable score 😂" }, { "id": 1654405, "postDate": "2022-01-18T13:26:14.880Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1656355, "postDate": "2022-01-19T09:40:12.630Z", "content": "one sample doesn't prove enough, unless many more enough.
", "rawMarkdown": "one sample doesn't prove enough, unless many more enough." } ] }, { "id": 1654417, "postDate": "2022-01-18T13:43:53.130Z", "content": "What is CV?
", "rawMarkdown": "What is CV?" }, { "id": 1653882, "postDate": "2022-01-18T00:24:31.700Z", "content": "One possible reason for high CV but lower LB is that maybe you are overfitting Val data.
\nBTW, maybe I am wrong. But it seems the seed of yolov5 model will change for each training. Therefore, there are randomness.
", "rawMarkdown": "One possible reason for high CV but lower LB is that maybe you are overfitting Val data.\n\nBTW, maybe I am wrong. But it seems the seed of yolov5 model will change for each training. Therefore, there are randomness." } ], "comments": [ { "id": 1653729, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-01-17T20:19:14.863000", "content": "Welcome to the club 😄
\nThis is why I aked forTOP5 final model video on training data after competition :) I would like to see inference on training. It could be great research topic.
I am really waiting for solution description and …. model validations procedure. This is part I have spent most of a time … unfortunately without success. At one point, I even stated that there was something wrong with the implementation of the f2 metric on the Kaggle side, but I guess it's just my lack of knowledge in this area :) Today @lukaszborecki made interesting experiment - just submit crap model …. absolutely crap (with almost random bboxes) and it appeared that … is good (almost like first yolov5 models presented here) on LB :)
", "votes": 5, "replies": [ { "id": 1653779, "author_name": "Lukasz Borecki", "author_url": "", "post_date": "2022-01-17T21:17:45.997000", "content": "here is the image from inference Remek said and score is 0.45 but bboxes are crap and model misses bigger starfishes, medium one also
and here how should look correct boxes
\n
Both photos are from valid …. and both models (first one … according to image resizing trends we just submit for fun) … got almost comparable score 😂
", "votes": 0, "replies": [] }, { "id": 1654405, "author_name": "", "author_url": "", "post_date": "2022-01-18T13:26:14.880000", "content": "", "votes": 0, "replies": [] }, { "id": 1656355, "author_name": "dragon zhang", "author_url": "", "post_date": "2022-01-19T09:40:12.630000", "content": "one sample doesn't prove enough, unless many more enough.
", "votes": 0, "replies": [] } ] }, { "id": 1654417, "author_name": "Kira yang", "author_url": "", "post_date": "2022-01-18T13:43:53.130000", "content": "What is CV?
", "votes": 0, "replies": [] }, { "id": 1653882, "author_name": "DeepInvolution", "author_url": "", "post_date": "2022-01-18T00:24:31.700000", "content": "One possible reason for high CV but lower LB is that maybe you are overfitting Val data.
\nBTW, maybe I am wrong. But it seems the seed of yolov5 model will change for each training. Therefore, there are randomness.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1653703": "So, I took @steamedsheep's training notebook to start with, trained the model with a bit bigger image size of 3840, got a score of .588 with no inference upscaling.\nThen tried changing a few numbers to do further experiment, and retrained model a lot of times, everytime it scored way below .588\n1. bs increase scored .495\n2. Rerun the same code again to score .493\n3. Trying out some hyperparams on inference side .492\n3. Trying Adam optimizer scoring .554\n4. ReTrying Adam optimizer with a different batch size .552 (This was a rerun of .588 rep fail with Adam)\n5. Changing Image sizes to 3600, 3000, 3200 all scoring less.\n\nBasically making it completely random when we get good scores and when we don't (except for Adam, we can put our hopes there).\n\nThen thought that YoloV5-S6 is not stable, ran M6 one time and scored .556 with image size 3200, ran L6 one time with image size 2560 and scored .427\n\nWhy is there so much randomness that the same training can go from scoring .588 to .495 by just doubling the batch size and ofc, .588 retrain is also not happening.\n\nWhile this was happening, I submitted only when my CV was increasing.\n\nCV for .588:\nP R mAP@.5 mAP@\n0.864 0.644 0.725 0.339\n\nSame Fold Best CV but failed miserably on LB:\nP R mAP@.5 mAP@\n0.84 0.672 0.744 0.352\n\n\nUpdates:\n1. Re-M6 .581\n\nGuidance for reducing this chaos of randomness would be appreciated.\nThank You.", "1653729": "Welcome to the club 😄\nThis is why I aked forTOP5 final model video on training data after competition :) I would like to see inference on training. It could be great research topic.\n\nI am really waiting for solution description and .... model validations procedure. This is part I have spent most of a time ... unfortunately without success. At one point, I even stated that there was something wrong with the implementation of the f2 metric on the Kaggle side, but I guess it's just my lack of knowledge in this area :) Today @lukaszborecki made interesting experiment - just submit crap model .... absolutely crap (with almost random bboxes) and it appeared that ... is good (almost like first yolov5 models presented here) on LB :) ", "1654417": "What is CV?", "1653882": "One possible reason for high CV but lower LB is that maybe you are overfitting Val data.\n\nBTW, maybe I am wrong. But it seems the seed of yolov5 model will change for each training. Therefore, there are randomness." } }