I would like to thank organizers for such an intriguing competition.
\nBeing a beginner with object detection we got to understand lot of new concepts and lessons learned from our mistakes. I will share those in this post.
\nThanks to my teammates @tiquasar and @anshulkhadse
\nGPU: Tesla T4 16 GB and Tesla V100 64GB
Adam for first strategy and for second one Adam for baseline and SGD for finetuningSGD and 0.0025 for Adam.scale: 0.5, shear:0.2-0.4, degree: 0-0.3, translate: 0.1-0.2, mosaic:1, mixup:0.2-0.5, fliplr:0.5, flipud: 0-0.005, perspective: 0-0.0002, (copy_paste: we made a mistake and didn't use it for most of the competition)imgsz: 2560-4000; batch-size: 4imgsz: 3200; batch-size: 4imgsz: 2560-5200; batch-size: 2, 4, 8imgsz: 3200; batch-size: 4| Model | \nimgsz(train) | \nimgsz(infer) | \nstrategy | \nData | \nTTA | \nPrivateLB | \nPublicLB | \nFinal | \n
|---|---|---|---|---|---|---|---|---|
| m | \n2560 | \n3200 | \n1 | \noriginal | \ndefault | \n0.651 | \n0.571 | \n\n |
| m6 | \n2560 | \n3200 | \n1 | \noriginal | \ndefault | \n0.677 | \n0.552 | \n\n |
| m6 | \n4000 | \n6400 | \n2 | \ncustom | \ndefault | \n0.672 | \n0.613 | \n\n |
| m6 | \n4000 | \n8000 | \n2 | \ncustom | \ns=[1,0.9,0.8] | \n0.663 | \n0.650 | \n✔️ | \n
| l6 | \n3200 | \n6400 | \n1 | \ncustom | \ns=[1,0.85,0.72] | \n0.651 | \n0.647 | \n\n |
| s6 | \n3584 | \n6000 | \n2 | \noriginal | \ns=[1,0.7,0.6] | \n0.626 | \n0.746 | \n✔️ | \n
| s6 | \n3584 | \n3600 | \n2 | \noriginal | \ns=[1,0.8,0.6] | \n0.608 | \n0.704 | \n✔️ | \n
| w6 | \n3200 | \n6400 | \n1 | \ncustom | \ns=[1,0.85,0.72] | \n0.634 | \n0.626 | \n\n |
For evaluation we used F2-score as metric the training data and flipped it horizontally and used some augmentations like CLAHE, equlize, gaussian noise. Code for this here with name evaluation_data_gen.py. Bigger model were giving less FN and FP and but were not good on public lb with high resolution. Smaller model (s6) was performing good on public lb after finetuning but was giving more FN and FP then m6 & l6.
Our best ensemble scored 0.701 on private but didn't select it :(
\n@tiquasar will elaborate on this
We selected around 500 images from all three videos with bboxes count ranging from 1-18 such that each bbox count is there in selection with fair proportion as in the original dataset. Then used albumentations to flip them horizontally with CLAHE, equilize and gaussian noise. Code for this available here with the name augment_data_generate.py. This extra data helped bigger models in generalizing and finetuning specially.
Becuase yolor codebase was similar to yolov5 we merged the extra features of yolor into yolov5 codebase and used it train both yolov5 and yolor.
\nWeights and code used in this competition here.
I would like to thank organizers for such an intriguing competition.
\nBeing a beginner with object detection we got to understand lot of new concepts and lessons learned from our mistakes. I will share those in this post.
\nThanks to my teammates @tiquasar and @anshulkhadse
\nGPU: Tesla T4 16 GB and Tesla V100 64GB
Adam for first strategy and for second one Adam for baseline and SGD for finetuningSGD and 0.0025 for Adam.scale: 0.5, shear:0.2-0.4, degree: 0-0.3, translate: 0.1-0.2, mosaic:1, mixup:0.2-0.5, fliplr:0.5, flipud: 0-0.005, perspective: 0-0.0002, (copy_paste: we made a mistake and didn't use it for most of the competition)imgsz: 2560-4000; batch-size: 4imgsz: 3200; batch-size: 4imgsz: 2560-5200; batch-size: 2, 4, 8imgsz: 3200; batch-size: 4| Model | \nimgsz(train) | \nimgsz(infer) | \nstrategy | \nData | \nTTA | \nPrivateLB | \nPublicLB | \nFinal | \n
|---|---|---|---|---|---|---|---|---|
| m | \n2560 | \n3200 | \n1 | \noriginal | \ndefault | \n0.651 | \n0.571 | \n\n |
| m6 | \n2560 | \n3200 | \n1 | \noriginal | \ndefault | \n0.677 | \n0.552 | \n\n |
| m6 | \n4000 | \n6400 | \n2 | \ncustom | \ndefault | \n0.672 | \n0.613 | \n\n |
| m6 | \n4000 | \n8000 | \n2 | \ncustom | \ns=[1,0.9,0.8] | \n0.663 | \n0.650 | \n✔️ | \n
| l6 | \n3200 | \n6400 | \n1 | \ncustom | \ns=[1,0.85,0.72] | \n0.651 | \n0.647 | \n\n |
| s6 | \n3584 | \n6000 | \n2 | \noriginal | \ns=[1,0.7,0.6] | \n0.626 | \n0.746 | \n✔️ | \n
| s6 | \n3584 | \n3600 | \n2 | \noriginal | \ns=[1,0.8,0.6] | \n0.608 | \n0.704 | \n✔️ | \n
| w6 | \n3200 | \n6400 | \n1 | \ncustom | \ns=[1,0.85,0.72] | \n0.634 | \n0.626 | \n\n |
For evaluation we used F2-score as metric the training data and flipped it horizontally and used some augmentations like CLAHE, equlize, gaussian noise. Code for this here with name evaluation_data_gen.py. Bigger model were giving less FN and FP and but were not good on public lb with high resolution. Smaller model (s6) was performing good on public lb after finetuning but was giving more FN and FP then m6 & l6.
Our best ensemble scored 0.701 on private but didn't select it :(
\n@tiquasar will elaborate on this
We selected around 500 images from all three videos with bboxes count ranging from 1-18 such that each bbox count is there in selection with fair proportion as in the original dataset. Then used albumentations to flip them horizontally with CLAHE, equilize and gaussian noise. Code for this available here with the name augment_data_generate.py. This extra data helped bigger models in generalizing and finetuning specially.
Becuase yolor codebase was similar to yolov5 we merged the extra features of yolor into yolov5 codebase and used it train both yolov5 and yolor.
\nWeights and code used in this competition here.
Congratulations! And thank you for sharing!
\nEspecially, \"Mistakes we did\" is interesting.
\nIt will be helpful for beginners (like me).
Thanks. Mistakes must be identified :)
", "rawMarkdown": "Thanks. Mistakes must be identified :)", "votes": null }, { "id": "1694653", "postDate": "02/17/2022 16:01:49", "content": "\n\nI would like to thank organizers for such an intriguing competition.
\n
\n Being a beginner with object detection we got to understand lot of new concepts and lessons learned from our mistakes. I will share those in this post.
\n Thanks to my teammates @tiquasar and @anshulkhadse
\n GPU: Tesla T4 16 GB and Tesla V100 64GBTraining
\nFor training we followed 2 strategies
\n\n
\n- Using 0.1 and 0.2 subsequence split for cross-validation
\n- Using video-wise split strategy to train baseline and then using split 0.1 or 0.2 for finetuning it.
\nParameters
\n\n
\n- \n
Adamfor first strategy and for second oneAdamfor baseline andSGDfor finetuning- Learning rate of 0.01 for
\nSGDand 0.0025 forAdam.- \n
scale: 0.5,shear:0.2-0.4,degree: 0-0.3,translate: 0.1-0.2,mosaic:1,mixup:0.2-0.5,fliplr:0.5,flipud: 0-0.005,perspective: 0-0.0002, (copy_paste: we made a mistake and didn't use it for most of the competition)Data and Augmentations
\n\n
\n- Augmentations during training on original data with bboxes .
\n- Augmentations on customised data along with original, generated by randomly selecting images with bbox.
\nModels
\nYOLOv5
\n\n
\n- m and m6-
\nimgsz: 2560-4000;batch-size: 4- l6-
\nimgsz: 3200;batch-size: 4- s6-
\nimgsz: 2560-5200;batch-size: 2, 4, 8YOLOR
\n\n
\n- p6 and w6-
\nimgsz: 3200;batch-size: 4Analyses
\n\n \n
\n\n \n \n \nModel \nimgsz(train) \nimgsz(infer) \nstrategy \nData \nTTA \nPrivateLB \nPublicLB \nFinal \n\n \nm \n2560 \n3200 \n1 \noriginal \ndefault \n0.651 \n0.571 \n\n \n \nm6 \n2560 \n3200 \n1 \noriginal \ndefault \n0.677 \n0.552 \n\n \n \nm6 \n4000 \n6400 \n2 \ncustom \ndefault \n0.672 \n0.613 \n\n \n \nm6 \n4000 \n8000 \n2 \ncustom \ns=[1,0.9,0.8] \n0.663 \n0.650 \n✔️ \n\n \nl6 \n3200 \n6400 \n1 \ncustom \ns=[1,0.85,0.72] \n0.651 \n0.647 \n\n \n \ns6 \n3584 \n6000 \n2 \noriginal \ns=[1,0.7,0.6] \n0.626 \n0.746 \n✔️ \n\n \ns6 \n3584 \n3600 \n2 \noriginal \ns=[1,0.8,0.6] \n0.608 \n0.704 \n✔️ \n\n \n \nw6 \n3200 \n6400 \n1 \ncustom \ns=[1,0.85,0.72] \n0.634 \n0.626 \n\n For evaluation we used F2-score as metric the training data and flipped it horizontally and used some augmentations like CLAHE, equlize, gaussian noise. Code for this here with name
\nevaluation_data_gen.py. Bigger model were giving less FN and FP and but were not good on public lb with high resolution. Smaller model (s6) was performing good on public lb after finetuning but was giving more FN and FP then m6 & l6.Ensembles
\nIn this competition we tried ensemble with three yolov5 series models with individual TTA for each models as well as default TTA for all models and inference image size scaled between 1xtrain_image_size to 1.8xtrain_image_size.
\n
\n The Notebook that achieved 0.701 on private leaderboard was inferred on 3600 image size with default TTA.Generating Custom Data
\nWe selected around 500 images from all three videos with bboxes count ranging from 1-18 such that each bbox count is there in selection with fair proportion as in the original dataset. Then used albumentations to flip them horizontally with CLAHE, equilize and gaussian noise. Code for this available here with the name
\naugment_data_generate.py. This extra data helped bigger models in generalizing and finetuning specially.Mistakes we did
\n\n
\n- Didn't try training and inference with lower resolution(Bigger GPU size distracted us)
\n- Didn't trust CV and focused too much on public lb(Got emotionally attached to public lb rank)
\n- Lack of experiments with ensembles and different models like centernet, RCNN, efficientdet
\n
\nBecuase yolor codebase was similar to yolov5 we merged the extra features of yolor into yolov5 codebase and used it train both yolov5 and yolor.
\n
\n Weights and code used in this competition here.
\n", "rawMarkdown": "## Thread Update,\n> I would like to thank organizers for such an intriguing competition.\n> Being a beginner with object detection we got to understand lot of new concepts and lessons learned from our mistakes. I will share those in this post.\n> Thanks to my teammates @tiquasar and @anshulkhadse \n> **GPU: Tesla T4 16 GB and Tesla V100 64GB**\n> \n> ### Training\n> ##### For training we followed 2 strategies\n> 1. Using 0.1 and 0.2 subsequence split for cross-validation \n> 2. Using video-wise split strategy to train baseline and then using split 0.1 or 0.2 for finetuning it. \n> \n> ##### Parameters\n> 1. `Adam` for first strategy and for second one `Adam` for baseline and `SGD` for finetuning\n> 2. Learning rate of 0.01 for `SGD` and 0.0025 for `Adam`.\n> 3. `scale: 0.5`, `shear:0.2-0.4`, `degree: 0-0.3`, `translate: 0.1-0.2`, `mosaic:1`, `mixup:0.2-0.5`, `fliplr:0.5`, `flipud: 0-0.005`, `perspective: 0-0.0002`, (`copy_paste`: we made a mistake and didn't use it for most of the competition)\n> \n> ##### Data and Augmentations\n> 1. Augmentations during training on original data with bboxes .\n> 2. Augmentations on customised data along with original, generated by randomly selecting images with bbox.\n> \n> ### Models\n> ##### YOLOv5\n> 1. m and m6- `imgsz: 2560-4000`; `batch-size: 4`\n> 2. l6- `imgsz: 3200`; `batch-size: 4`\n> 3. s6- `imgsz: 2560-5200`; `batch-size: 2, 4, 8`\n> \n> ##### YOLOR\n> 1. p6 and w6- `imgsz: 3200`; `batch-size: 4`\n> \n> ### Analyses\n> | Model | imgsz(train) | imgsz(infer) | strategy | Data | TTA | PrivateLB | PublicLB | Final |\n> | --- | --- | --- | --- | --- | --- | --- | --- |\n> | m | 2560 | 3200 | 1 | original | default | 0.651 | 0.571 | |\n> | m6 | 2560 | 3200 | 1 | original |default | 0.677 | 0.552 | |\n> | m6 | 4000 | 6400 | 2 | custom |default | 0.672 | 0.613 | |\n> | m6 | 4000 | 8000 | 2 | custom |s=[1,0.9,0.8] | 0.663 | 0.650 | ✔️ |\n> | l6 | 3200 | 6400 | 1 | custom |s=[1,0.85,0.72] | 0.651 | 0.647 | |\n> | s6 | 3584 | 6000 | 2 | original |s=[1,0.7,0.6] | 0.626 | 0.746 |✔️ |\n> | s6 | 3584 | 3600 | 2 | original |s=[1,0.8,0.6] | 0.608 | 0.704 | ✔️ |\n> | w6 | 3200 | 6400 | 1 | custom |s=[1,0.85,0.72] | 0.634 | 0.626 | |\n> \n> For evaluation we used [F2-score](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/302241#1659077) as metric the training data and flipped it horizontally and used some augmentations like CLAHE, equlize, gaussian noise. Code for this [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition?select=yolo_repo) with name `evaluation_data_gen.py`. Bigger model were giving less FN and FP and but were not good on public lb with high resolution. Smaller model (s6) was performing good on public lb after finetuning but was giving more FN and FP then m6 & l6.\n> \n> ### Ensembles\n> In this competition we tried ensemble with three yolov5 series models with individual TTA for each models as well as default TTA for all models and inference image size scaled between 1xtrain_image_size to 1.8xtrain_image_size.\n> The Notebook that achieved 0.701 on private leaderboard was inferred on 3600 image size with default TTA.\n>\n> #### Generating Custom Data\n> We selected around 500 images from all three videos with bboxes count ranging from 1-18 such that each bbox count is there in selection with fair proportion as in the original dataset. Then used albumentations to flip them horizontally with CLAHE, equilize and gaussian noise. Code for this available [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition?select=yolo_repo) with the name `augment_data_generate.py`. This extra data helped bigger models in generalizing and finetuning specially.\n> \n> ### Mistakes we did\n> - Didn't try training and inference with lower resolution(Bigger GPU size distracted us)\n> - Didn't trust CV and focused too much on public lb(Got emotionally attached to public lb rank)\n> - Lack of experiments with ensembles and different models like centernet, RCNN, efficientdet\n> \n> ------------------------------------------------------------------------------------\n> Becuase [yolor](https://github.com/WongKinYiu/yolor/tree/paper) codebase was similar to [yolov5](https://github.com/ultralytics/yolov5) we merged the extra features of yolor into yolov5 codebase and used it train both yolov5 and yolor.\n> Weights and code used in this competition [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition).\n\n#### And some additional points observed from this competition :\n> - Adding score threshold before the boxes are averaged during ensemble helped in score increment as the non detected boxes from individual models were bringing down the average detection probability of the final boxes. \n> - Experimented with adding ASFF(Adaptively spatial feature fusion) with yolov5, although I didn't get enough time on experimenting more with this however in initial runs it was giving a more than decent precision but poor recall.\n> - Adding tracking with optimal parameters gave a stable boost of 0.01-0.05 on Public/Private Leaderboards.\n> - In my observation Slicing Aided Hyper Inference (SAHI) was one algorithm that gave a huge difference between private and public leaderboards( 0.431 public LB to 0.591 private LB in one of the notebooks ), definitely will be researching and working on this algorithm later.", "votes": null } ], "comments": [ { "id": 1692470, "author_name": "takasukohei", "author_url": "", "post_date": "02/16/2022 04:03:04", "content": "\n
\n- Adding score threshold before the boxes are averaged during ensemble helped in score increment as the non detected boxes from individual models were bringing down the average detection probability of the final boxes.
\n- Experimented with adding ASFF(Adaptively spatial feature fusion) with yolov5, although I didn't get enough time on experimenting more with this however in initial runs it was giving a more than decent precision but poor recall.
\n- Adding tracking with optimal parameters gave a stable boost of 0.01-0.05 on Public/Private Leaderboards.
\n- In my observation Slicing Aided Hyper Inference (SAHI) was one algorithm that gave a huge difference between private and public leaderboards( 0.431 public LB to 0.591 private LB in one of the notebooks ), definitely will be researching and working on this algorithm later.
\n
Congratulations! And thank you for sharing!
\nEspecially, \"Mistakes we did\" is interesting.
\nIt will be helpful for beginners (like me).
Thanks. Mistakes must be identified :)
", "votes": null, "replies": [] } ] }, { "id": 1694653, "author_name": "tiquasar", "author_url": "", "post_date": "02/17/2022 16:01:49", "content": "\n\nI would like to thank organizers for such an intriguing competition.
\n
\n Being a beginner with object detection we got to understand lot of new concepts and lessons learned from our mistakes. I will share those in this post.
\n Thanks to my teammates @tiquasar and @anshulkhadse
\n GPU: Tesla T4 16 GB and Tesla V100 64GBTraining
\nFor training we followed 2 strategies
\n\n
\n- Using 0.1 and 0.2 subsequence split for cross-validation
\n- Using video-wise split strategy to train baseline and then using split 0.1 or 0.2 for finetuning it.
\nParameters
\n\n
\n- \n
Adamfor first strategy and for second oneAdamfor baseline andSGDfor finetuning- Learning rate of 0.01 for
\nSGDand 0.0025 forAdam.- \n
scale: 0.5,shear:0.2-0.4,degree: 0-0.3,translate: 0.1-0.2,mosaic:1,mixup:0.2-0.5,fliplr:0.5,flipud: 0-0.005,perspective: 0-0.0002, (copy_paste: we made a mistake and didn't use it for most of the competition)Data and Augmentations
\n\n
\n- Augmentations during training on original data with bboxes .
\n- Augmentations on customised data along with original, generated by randomly selecting images with bbox.
\nModels
\nYOLOv5
\n\n
\n- m and m6-
\nimgsz: 2560-4000;batch-size: 4- l6-
\nimgsz: 3200;batch-size: 4- s6-
\nimgsz: 2560-5200;batch-size: 2, 4, 8YOLOR
\n\n
\n- p6 and w6-
\nimgsz: 3200;batch-size: 4Analyses
\n\n \n
\n\n \n \n \nModel \nimgsz(train) \nimgsz(infer) \nstrategy \nData \nTTA \nPrivateLB \nPublicLB \nFinal \n\n \nm \n2560 \n3200 \n1 \noriginal \ndefault \n0.651 \n0.571 \n\n \n \nm6 \n2560 \n3200 \n1 \noriginal \ndefault \n0.677 \n0.552 \n\n \n \nm6 \n4000 \n6400 \n2 \ncustom \ndefault \n0.672 \n0.613 \n\n \n \nm6 \n4000 \n8000 \n2 \ncustom \ns=[1,0.9,0.8] \n0.663 \n0.650 \n✔️ \n\n \nl6 \n3200 \n6400 \n1 \ncustom \ns=[1,0.85,0.72] \n0.651 \n0.647 \n\n \n \ns6 \n3584 \n6000 \n2 \noriginal \ns=[1,0.7,0.6] \n0.626 \n0.746 \n✔️ \n\n \ns6 \n3584 \n3600 \n2 \noriginal \ns=[1,0.8,0.6] \n0.608 \n0.704 \n✔️ \n\n \n \nw6 \n3200 \n6400 \n1 \ncustom \ns=[1,0.85,0.72] \n0.634 \n0.626 \n\n For evaluation we used F2-score as metric the training data and flipped it horizontally and used some augmentations like CLAHE, equlize, gaussian noise. Code for this here with name
\nevaluation_data_gen.py. Bigger model were giving less FN and FP and but were not good on public lb with high resolution. Smaller model (s6) was performing good on public lb after finetuning but was giving more FN and FP then m6 & l6.Ensembles
\nIn this competition we tried ensemble with three yolov5 series models with individual TTA for each models as well as default TTA for all models and inference image size scaled between 1xtrain_image_size to 1.8xtrain_image_size.
\n
\n The Notebook that achieved 0.701 on private leaderboard was inferred on 3600 image size with default TTA.Generating Custom Data
\nWe selected around 500 images from all three videos with bboxes count ranging from 1-18 such that each bbox count is there in selection with fair proportion as in the original dataset. Then used albumentations to flip them horizontally with CLAHE, equilize and gaussian noise. Code for this available here with the name
\naugment_data_generate.py. This extra data helped bigger models in generalizing and finetuning specially.Mistakes we did
\n\n
\n- Didn't try training and inference with lower resolution(Bigger GPU size distracted us)
\n- Didn't trust CV and focused too much on public lb(Got emotionally attached to public lb rank)
\n- Lack of experiments with ensembles and different models like centernet, RCNN, efficientdet
\n
\nBecuase yolor codebase was similar to yolov5 we merged the extra features of yolor into yolov5 codebase and used it train both yolov5 and yolor.
\n
\n Weights and code used in this competition here.
\n", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1692068": "I would like to thank organizers for such an intriguing competition.\nBeing a beginner with object detection we got to understand lot of new concepts and lessons learned from our mistakes. I will share those in this post.\nThanks to my teammates @tiquasar and @anshulkhadse \n**GPU: Tesla T4 16 GB and Tesla V100 64GB**\n\n### Training\n##### For training we followed 2 strategies\n1. Using 0.1 and 0.2 subsequence split for cross-validation \n2. Using video-wise split strategy to train baseline and then using split 0.1 or 0.2 for finetuning it. \n\n##### Parameters\n1. `Adam` for first strategy and for second one `Adam` for baseline and `SGD` for finetuning\n2. Learning rate of 0.01 for `SGD` and 0.0025 for `Adam`.\n3. `scale: 0.5`, `shear:0.2-0.4`, `degree: 0-0.3`, `translate: 0.1-0.2`, `mosaic:1`, `mixup:0.2-0.5`, `fliplr:0.5`, `flipud: 0-0.005`, `perspective: 0-0.0002`, (`copy_paste`: we made a mistake and didn't use it for most of the competition)\n\n##### Data and Augmentations\n1. Augmentations during training on original data with bboxes .\n2. Augmentations on customised data along with original, generated by randomly selecting images with bbox.\n\n### Models\n##### YOLOv5\n1. m and m6- `imgsz: 2560-4000`; `batch-size: 4`\n2. l6- `imgsz: 3200`; `batch-size: 4`\n3. s6- `imgsz: 2560-5200`; `batch-size: 2, 4, 8`\n\n##### YOLOR\n1. p6 and w6- `imgsz: 3200`; `batch-size: 4`\n\n### Analyses\n| Model | imgsz(train) | imgsz(infer) | strategy | Data | TTA | PrivateLB | PublicLB | Final |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| m | 2560 | 3200 | 1 | original | default | 0.651 | 0.571 | |\n| m6 | 2560 | 3200 | 1 | original |default | 0.677 | 0.552 | |\n| m6 | 4000 | 6400 | 2 | custom |default | 0.672 | 0.613 | |\n| m6 | 4000 | 8000 | 2 | custom |s=[1,0.9,0.8] | 0.663 | 0.650 | ✔️ |\n| l6 | 3200 | 6400 | 1 | custom |s=[1,0.85,0.72] | 0.651 | 0.647 | |\n| s6 | 3584 | 6000 | 2 | original |s=[1,0.7,0.6] | 0.626 | 0.746 |✔️ |\n| s6 | 3584 | 3600 | 2 | original |s=[1,0.8,0.6] | 0.608 | 0.704 | ✔️ |\n| w6 | 3200 | 6400 | 1 | custom |s=[1,0.85,0.72] | 0.634 | 0.626 | |\n\nFor evaluation we used [F2-score](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/302241#1659077) as metric the training data and flipped it horizontally and used some augmentations like CLAHE, equlize, gaussian noise. Code for this [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition?select=yolo_repo) with name `evaluation_data_gen.py`. Bigger model were giving less FN and FP and but were not good on public lb with high resolution. Smaller model (s6) was performing good on public lb after finetuning but was giving more FN and FP then m6 & l6.\n\n### Ensembles\nOur best ensemble scored 0.701 on private but didn't select it :(\n@tiquasar will elaborate on this\n\n#### Generating Custom Data\nWe selected around 500 images from all three videos with bboxes count ranging from 1-18 such that each bbox count is there in selection with fair proportion as in the original dataset. Then used albumentations to flip them horizontally with CLAHE, equilize and gaussian noise. Code for this available [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition?select=yolo_repo) with the name `augment_data_generate.py`. This extra data helped bigger models in generalizing and finetuning specially.\n\n### Mistakes we did\n- Didn't try training and inference with lower resolution(Bigger GPU size distracted us)\n- Didn't trust CV and focused too much on public lb(Got emotionally attached to public lb rank)\n- Lack of experiments with ensembles and different models like centernet, RCNN, efficientdet\n\n------------------------------------------------------------------------------------\nBecuase [yolor](https://github.com/WongKinYiu/yolor/tree/paper) codebase was similar to [yolov5](https://github.com/ultralytics/yolov5) we merged the extra features of yolor into yolov5 codebase and used it train both yolov5 and yolor.\nWeights and code used in this competition [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition).", "1692470": "Congratulations! And thank you for sharing!\n\nEspecially, \"Mistakes we did\" is interesting.\nIt will be helpful for beginners (like me).", "1692512": "Thanks. Mistakes must be identified :)", "1694653": "## Thread Update,\n> I would like to thank organizers for such an intriguing competition.\n> Being a beginner with object detection we got to understand lot of new concepts and lessons learned from our mistakes. I will share those in this post.\n> Thanks to my teammates @tiquasar and @anshulkhadse \n> **GPU: Tesla T4 16 GB and Tesla V100 64GB**\n> \n> ### Training\n> ##### For training we followed 2 strategies\n> 1. Using 0.1 and 0.2 subsequence split for cross-validation \n> 2. Using video-wise split strategy to train baseline and then using split 0.1 or 0.2 for finetuning it. \n> \n> ##### Parameters\n> 1. `Adam` for first strategy and for second one `Adam` for baseline and `SGD` for finetuning\n> 2. Learning rate of 0.01 for `SGD` and 0.0025 for `Adam`.\n> 3. `scale: 0.5`, `shear:0.2-0.4`, `degree: 0-0.3`, `translate: 0.1-0.2`, `mosaic:1`, `mixup:0.2-0.5`, `fliplr:0.5`, `flipud: 0-0.005`, `perspective: 0-0.0002`, (`copy_paste`: we made a mistake and didn't use it for most of the competition)\n> \n> ##### Data and Augmentations\n> 1. Augmentations during training on original data with bboxes .\n> 2. Augmentations on customised data along with original, generated by randomly selecting images with bbox.\n> \n> ### Models\n> ##### YOLOv5\n> 1. m and m6- `imgsz: 2560-4000`; `batch-size: 4`\n> 2. l6- `imgsz: 3200`; `batch-size: 4`\n> 3. s6- `imgsz: 2560-5200`; `batch-size: 2, 4, 8`\n> \n> ##### YOLOR\n> 1. p6 and w6- `imgsz: 3200`; `batch-size: 4`\n> \n> ### Analyses\n> | Model | imgsz(train) | imgsz(infer) | strategy | Data | TTA | PrivateLB | PublicLB | Final |\n> | --- | --- | --- | --- | --- | --- | --- | --- |\n> | m | 2560 | 3200 | 1 | original | default | 0.651 | 0.571 | |\n> | m6 | 2560 | 3200 | 1 | original |default | 0.677 | 0.552 | |\n> | m6 | 4000 | 6400 | 2 | custom |default | 0.672 | 0.613 | |\n> | m6 | 4000 | 8000 | 2 | custom |s=[1,0.9,0.8] | 0.663 | 0.650 | ✔️ |\n> | l6 | 3200 | 6400 | 1 | custom |s=[1,0.85,0.72] | 0.651 | 0.647 | |\n> | s6 | 3584 | 6000 | 2 | original |s=[1,0.7,0.6] | 0.626 | 0.746 |✔️ |\n> | s6 | 3584 | 3600 | 2 | original |s=[1,0.8,0.6] | 0.608 | 0.704 | ✔️ |\n> | w6 | 3200 | 6400 | 1 | custom |s=[1,0.85,0.72] | 0.634 | 0.626 | |\n> \n> For evaluation we used [F2-score](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/302241#1659077) as metric the training data and flipped it horizontally and used some augmentations like CLAHE, equlize, gaussian noise. Code for this [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition?select=yolo_repo) with name `evaluation_data_gen.py`. Bigger model were giving less FN and FP and but were not good on public lb with high resolution. Smaller model (s6) was performing good on public lb after finetuning but was giving more FN and FP then m6 & l6.\n> \n> ### Ensembles\n> In this competition we tried ensemble with three yolov5 series models with individual TTA for each models as well as default TTA for all models and inference image size scaled between 1xtrain_image_size to 1.8xtrain_image_size.\n> The Notebook that achieved 0.701 on private leaderboard was inferred on 3600 image size with default TTA.\n>\n> #### Generating Custom Data\n> We selected around 500 images from all three videos with bboxes count ranging from 1-18 such that each bbox count is there in selection with fair proportion as in the original dataset. Then used albumentations to flip them horizontally with CLAHE, equilize and gaussian noise. Code for this available [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition?select=yolo_repo) with the name `augment_data_generate.py`. This extra data helped bigger models in generalizing and finetuning specially.\n> \n> ### Mistakes we did\n> - Didn't try training and inference with lower resolution(Bigger GPU size distracted us)\n> - Didn't trust CV and focused too much on public lb(Got emotionally attached to public lb rank)\n> - Lack of experiments with ensembles and different models like centernet, RCNN, efficientdet\n> \n> ------------------------------------------------------------------------------------\n> Becuase [yolor](https://github.com/WongKinYiu/yolor/tree/paper) codebase was similar to [yolov5](https://github.com/ultralytics/yolov5) we merged the extra features of yolor into yolov5 codebase and used it train both yolov5 and yolor.\n> Weights and code used in this competition [here](https://www.kaggle.com/sanchitvj/barrier-reef-competition).\n\n#### And some additional points observed from this competition :\n> - Adding score threshold before the boxes are averaged during ensemble helped in score increment as the non detected boxes from individual models were bringing down the average detection probability of the final boxes. \n> - Experimented with adding ASFF(Adaptively spatial feature fusion) with yolov5, although I didn't get enough time on experimenting more with this however in initial runs it was giving a more than decent precision but poor recall.\n> - Adding tracking with optimal parameters gave a stable boost of 0.01-0.05 on Public/Private Leaderboards.\n> - In my observation Slicing Aided Hyper Inference (SAHI) was one algorithm that gave a huge difference between private and public leaderboards( 0.431 public LB to 0.591 private LB in one of the notebooks ), definitely will be researching and working on this algorithm later." }, "source": "meta" }\n
\n- Adding score threshold before the boxes are averaged during ensemble helped in score increment as the non detected boxes from individual models were bringing down the average detection probability of the final boxes.
\n- Experimented with adding ASFF(Adaptively spatial feature fusion) with yolov5, although I didn't get enough time on experimenting more with this however in initial runs it was giving a more than decent precision but poor recall.
\n- Adding tracking with optimal parameters gave a stable boost of 0.01-0.05 on Public/Private Leaderboards.
\n- In my observation Slicing Aided Hyper Inference (SAHI) was one algorithm that gave a huge difference between private and public leaderboards( 0.431 public LB to 0.591 private LB in one of the notebooks ), definitely will be researching and working on this algorithm later.
\n