![\"\"](\"https://i.imgur.com/fuUtT1i.jpg\")
Congratulations to the winners and all participants who finished this competition π
\nIn this competition, many participants were focused on increasing the score of Public LB, and I was not able to keep up with Public LB at least halfway through.
\nHowever, in my local experiments, I have found that methods that increase the image size only during inference, even when the threshold and ensemble methods are properly adjusted, result in extremely low scores for the entire training data.
\nSo, I thought there would be shake. Although I was not able to get into the gold zone, I am glad that I participated because I was able to recognize the necessity of believing in my own experiments.
Ok, my solution is a pretty simple one, but I would like to share.
\n1. Preprocessing
\nSince the video was basically switching almost every sequence, I built a validation with 4 folds based on the sequence (However, I think there were some sequences that were a little continuous).
\nOne coin I did was to make the number of CoTS almost the same in each fold. In training, I did not use images without CoTS, but in validation, I used all images without CoTS in order to check the False Positive properly.
\n The image was simply divided by 255 without any noise removal, and the size was 1952 x 3520.
\nDue to GPU limitations, I couldn't try a larger size, but at least up to this size, I was able to get a small gain with Local CV. However I got the most gain up to about twice the size of the original image.
2. Model: All I want to use is YOLO-X
\nThis time I really wanted to use anchor-free YOLO-X, which I had never used before, so I stuck to it, even though many people have had success with YOLO-v5 in Public LB.
\n It was hard to modify YOLO-X because many parts were hard-coded, but I tried to tweak the top-K selection algorithm, add augmentation, and some other dubious modifications. But in the end, only augmentation seemed to have an effect. In that sense, YOLO-X's perfection as a model may be high π
3. Augmentation and learning strategy
\nIn addition to the augmentation (mosaic, mixup, etc.) that YOLO-X has as a default function, I added RandomGamma, RGBshift, Sharpen, GaussNoise, etc.
\nInstead, the probability of applying mixup and mosaic has been slightly reduced from the default value and assigned the rest probability to the newly added augmentation path. This may have increased the diversity of the input images somewhat, and I was able to get a gain in Local CV(I don't know the specific ablation values in detail, as the experiment took some twists and turns).
\n In addition, I used the progressive learning method used in EfficientNetV2: gradually increasing the size of the image (e.g. 1280 => β¦ => 3520) as the learning progressed. At the same time, I remember that regularization (increasing the probability of application of augmentation) was also strengthened somewhat.
4. Inference
\nThe inference was very simple: I did TTA with Flip and ensembled with WBF for 4-fold (so 8 models). I think this is almost same as many of the participants.
\nNote that I set thresholds to improve the local CV. The F2 metrics were sensitive to the threshold settings to some extent because our evaluation metric was based on the confusion matrix.
5. An original point that I have worked out: Seq-NMS
\nOne point that I devised a little is the post-processing.
\n Since the task is object recognition in video, I made a post in this thread early stage in this competition, and I was reading almost all papers. Many of them reported that feature aggregation, which is a method of enriching input features by using past images in the neighborhood, is more performing than post-processing methods such as tracking, and I thought that was probably true. However, since the majority of feature aggregation methods is based on RPNs, it was too much of a hurdle for me to apply, since I was sticking to YOLO-X, which is anchor-free. So I decided to use Seq-NMS: a typical post-processing method. Specifically, the confidence is adjusted by the degree of overlap between the predicted bbox of the previous images and the predicted bbox of the current image (Ref: https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/293812#1611428).
\n(The implementation can be found here. The core processing is written in cython and is fast enough, but as far as I understand, there is a serious bug in the python script that makes it not work properly as is, so I fixed the bug to use it.)
\n As long as I experimented, there is no need to consider many frames for Seq-NMS, just the prediction of the previous image. I also tried to use Dense Optical Flow to compensate for the shift of predicted bboxes of the previous frame, but in the end I did not use it because Dense Optical Flow is expensive to compute and does not have much gain.
That's all I can remember right now. If I remember anything else, I'll add it.
\nThank you all for your hard work, and see you at another competition :)
Happy Kagglingπ
\n
Feb. 18. 2022, Model pipeline figure updated
", "messages": [ { "id": "1691165", "postDate": "02/15/2022 09:08:03", "content": "Congratulations to the winners and all participants who finished this competition π
\nIn this competition, many participants were focused on increasing the score of Public LB, and I was not able to keep up with Public LB at least halfway through.
\nHowever, in my local experiments, I have found that methods that increase the image size only during inference, even when the threshold and ensemble methods are properly adjusted, result in extremely low scores for the entire training data.
\nSo, I thought there would be shake. Although I was not able to get into the gold zone, I am glad that I participated because I was able to recognize the necessity of believing in my own experiments.
Ok, my solution is a pretty simple one, but I would like to share.
\n1. Preprocessing
\nSince the video was basically switching almost every sequence, I built a validation with 4 folds based on the sequence (However, I think there were some sequences that were a little continuous).
\nOne coin I did was to make the number of CoTS almost the same in each fold. In training, I did not use images without CoTS, but in validation, I used all images without CoTS in order to check the False Positive properly.
\n The image was simply divided by 255 without any noise removal, and the size was 1952 x 3520.
\nDue to GPU limitations, I couldn't try a larger size, but at least up to this size, I was able to get a small gain with Local CV. However I got the most gain up to about twice the size of the original image.
2. Model: All I want to use is YOLO-X
\nThis time I really wanted to use anchor-free YOLO-X, which I had never used before, so I stuck to it, even though many people have had success with YOLO-v5 in Public LB.
\n It was hard to modify YOLO-X because many parts were hard-coded, but I tried to tweak the top-K selection algorithm, add augmentation, and some other dubious modifications. But in the end, only augmentation seemed to have an effect. In that sense, YOLO-X's perfection as a model may be high π
3. Augmentation and learning strategy
\nIn addition to the augmentation (mosaic, mixup, etc.) that YOLO-X has as a default function, I added RandomGamma, RGBshift, Sharpen, GaussNoise, etc.
\nInstead, the probability of applying mixup and mosaic has been slightly reduced from the default value and assigned the rest probability to the newly added augmentation path. This may have increased the diversity of the input images somewhat, and I was able to get a gain in Local CV(I don't know the specific ablation values in detail, as the experiment took some twists and turns).
\n In addition, I used the progressive learning method used in EfficientNetV2: gradually increasing the size of the image (e.g. 1280 => β¦ => 3520) as the learning progressed. At the same time, I remember that regularization (increasing the probability of application of augmentation) was also strengthened somewhat.
4. Inference
\nThe inference was very simple: I did TTA with Flip and ensembled with WBF for 4-fold (so 8 models). I think this is almost same as many of the participants.
\nNote that I set thresholds to improve the local CV. The F2 metrics were sensitive to the threshold settings to some extent because our evaluation metric was based on the confusion matrix.
5. An original point that I have worked out: Seq-NMS
\nOne point that I devised a little is the post-processing.
\n Since the task is object recognition in video, I made a post in this thread early stage in this competition, and I was reading almost all papers. Many of them reported that feature aggregation, which is a method of enriching input features by using past images in the neighborhood, is more performing than post-processing methods such as tracking, and I thought that was probably true. However, since the majority of feature aggregation methods is based on RPNs, it was too much of a hurdle for me to apply, since I was sticking to YOLO-X, which is anchor-free. So I decided to use Seq-NMS: a typical post-processing method. Specifically, the confidence is adjusted by the degree of overlap between the predicted bbox of the previous images and the predicted bbox of the current image (Ref: https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/293812#1611428).
\n(The implementation can be found here. The core processing is written in cython and is fast enough, but as far as I understand, there is a serious bug in the python script that makes it not work properly as is, so I fixed the bug to use it.)
\n As long as I experimented, there is no need to consider many frames for Seq-NMS, just the prediction of the previous image. I also tried to use Dense Optical Flow to compensate for the shift of predicted bboxes of the previous frame, but in the end I did not use it because Dense Optical Flow is expensive to compute and does not have much gain.
That's all I can remember right now. If I remember anything else, I'll add it.
\nThank you all for your hard work, and see you at another competition :)
Happy Kagglingπ
\n
Feb. 18. 2022, Model pipeline figure updated
", "rawMarkdown": "Congratulations to the winners and all participants who finished this competition π\nIn this competition, many participants were focused on increasing the score of Public LB, and I was not able to keep up with Public LB at least halfway through.\nHowever, in my local experiments, I have found that methods that increase the image size only during inference, even when the threshold and ensemble methods are properly adjusted, result in extremely low scores for the entire training data.\nSo, I thought there would be shake. Although I was not able to get into the gold zone, I am glad that I participated because I was able to recognize the necessity of believing in my own experiments.\n\nOk, my solution is a pretty simple one, but I would like to share.\n\n\n**1. Preprocessing**\nSince the video was basically switching almost every sequence, I built a validation with 4 folds based on the sequence (However, I think there were some sequences that were a little continuous).\nOne coin I did was to make the number of CoTS almost the same in each fold. In training, I did not use images without CoTS, but in validation, I used all images without CoTS in order to check the False Positive properly.\n The image was simply divided by 255 without any noise removal, and the size was 1952 x 3520.\nDue to GPU limitations, I couldn't try a larger size, but at least up to this size, I was able to get a small gain with Local CV. However I got the most gain up to about twice the size of the original image.\n\n\n**2. Model: All I want to use is YOLO-X**\nThis time I really wanted to use `anchor-free` YOLO-X, which I had never used before, so I stuck to it, even though many people have had success with YOLO-v5 in Public LB.\n It was hard to modify YOLO-X because many parts were hard-coded, but I tried to tweak the [top-K selection algorithm](https://arxiv.org/pdf/2107.08430.pdf), add augmentation, and some other dubious modifications. But in the end, only augmentation seemed to have an effect. In that sense, YOLO-X's perfection as a model may be high π\n\n\n**3. Augmentation and learning strategy**\nIn addition to the augmentation (mosaic, mixup, etc.) that YOLO-X has as a default function, I added RandomGamma, RGBshift, Sharpen, GaussNoise, etc.\nInstead, the probability of applying mixup and mosaic has been slightly reduced from the default value and assigned the rest probability to the newly added augmentation path. This may have increased the diversity of the input images somewhat, and I was able to get a gain in Local CV(I don't know the specific ablation values in detail, as the experiment took some twists and turns).\n In addition, I used the [progressive learning](https://arxiv.org/abs/2104.00298) method used in EfficientNetV2: gradually increasing the size of the image (e.g. 1280 => ... => 3520) as the learning progressed. At the same time, I remember that regularization (increasing the probability of application of augmentation) was also strengthened somewhat.\n\n\n**4. Inference**\nThe inference was very simple: I did TTA with Flip and ensembled with WBF for 4-fold (so 8 models). I think this is almost same as many of the participants.\nNote that I set thresholds to improve the local CV. The F2 metrics were sensitive to the threshold settings to some extent because our evaluation metric was based on the confusion matrix.\n\n\n**5. An original point that I have worked out: Seq-NMS**\nOne point that I devised a little is the post-processing.\n Since the task is object recognition in video, I made a post in [this thread](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/293812) early stage in this competition, and I was reading almost all papers. Many of them reported that feature aggregation, which is a method of enriching input features by using past images in the neighborhood, is more performing than post-processing methods such as tracking, and I thought that was probably true. However, since the majority of feature aggregation methods is based on RPNs, it was too much of a hurdle for me to apply, since I was sticking to YOLO-X, which is anchor-free. So I decided to use [Seq-NMS](https://arxiv.org/abs/1602.08465): a typical post-processing method. Specifically, the confidence is adjusted by the degree of overlap between the predicted bbox of the previous images and the predicted bbox of the current image (Ref: https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/293812#1611428).\n(The implementation can be found [here](https://github.com/tmoopenn/seq-nms). The core processing is written in cython and is fast enough, but as far as I understand, there is a serious bug in the python script that makes it not work properly as is, so I fixed the bug to use it.)\n As long as I experimented, there is no need to consider many frames for Seq-NMS, just the prediction of the previous image. I also tried to use [Dense Optical Flow](https://docs.opencv.org/3.4/d4/dee/tutorial_optical_flow.html#:~:text=Dense%20Optical%20Flow%20in%20OpenCV) to compensate for the shift of predicted bboxes of the previous frame, but in the end I did not use it because Dense Optical Flow is expensive to compute and does not have much gain.\n\n---\n\nThat's all I can remember right now. If I remember anything else, I'll add it.\nThank you all for your hard work, and see you at another competition :)\n\nHappy Kagglingπ\n\n\n\n\nFeb. 18. 2022, Model pipeline figure updated", "votes": null }, { "id": "1691177", "postDate": "02/15/2022 09:16:35", "content": "Congratulations! Yolo-X πππͺππππ Great!
", "rawMarkdown": "Congratulations! Yolo-X πππͺππππ Great!", "votes": null }, { "id": "1691187", "postDate": "02/15/2022 09:27:45", "content": "Congratulations! How much improvement has Seq-NMS brought you?
", "rawMarkdown": "Congratulations! How much improvement has Seq-NMS brought you?", "votes": null }, { "id": "1691210", "postDate": "02/15/2022 09:48:53", "content": "Thank you. Congratulations, too!
\n\n\nQ. How much improvement has Seq-NMS brought you?
\n
A. about 0.015. I only have my experiments to base this on, but I think it's a little better than the tracking method that was shared on kernel.
\nBy the way, since you seem to like memes, I'll give you this image.
\n![\"\"](\"https://i.imgur.com/Tg7MjJS.jpg\")
lol, thanks for your reply
", "rawMarkdown": "lol, thanks for your reply", "votes": null }, { "id": "1691587", "postDate": "02/15/2022 13:43:08", "content": "I think the simple solution is the better way in real-world problem. Thanks for sharing and congrats on 16th!
", "rawMarkdown": "I think the simple solution is the better way in real-world problem. Thanks for sharing and congrats on 16th!", "votes": null }, { "id": "1693022", "postDate": "02/16/2022 11:54:48", "content": "Thanks for this solution mate, I'm glad that YoloX is able to make it up there. I had used YoloX myself but got hard stucked at 0.4X, I'm pretty new to kaggle so no suprise there lol. Seems high training resolution gave the largest boost? at least from the 0.4X perspective. Due to GPU limitation was only able to train on 800ish resolution.
\nAnyway, congrats on the 16th, especially with YoloX ππ.
", "rawMarkdown": "Thanks for this solution mate, I'm glad that YoloX is able to make it up there. I had used YoloX myself but got hard stucked at 0.4X, I'm pretty new to kaggle so no suprise there lol. Seems high training resolution gave the largest boost? at least from the 0.4X perspective. Due to GPU limitation was only able to train on 800ish resolution.\n\nAnyway, congrats on the 16th, especially with YoloX ππ.", "votes": null } ], "comments": [ { "id": 1691177, "author_name": "remekkinas", "author_url": "", "post_date": "02/15/2022 09:16:35", "content": "Congratulations! Yolo-X πππͺππππ Great!
", "votes": null, "replies": [] }, { "id": 1691187, "author_name": "zzy990106", "author_url": "", "post_date": "02/15/2022 09:27:45", "content": "Congratulations! How much improvement has Seq-NMS brought you?
", "votes": null, "replies": [ { "id": 1691210, "author_name": "maxwell110", "author_url": "", "post_date": "02/15/2022 09:48:53", "content": "Thank you. Congratulations, too!
\n\n\nQ. How much improvement has Seq-NMS brought you?
\n
A. about 0.015. I only have my experiments to base this on, but I think it's a little better than the tracking method that was shared on kernel.
\nBy the way, since you seem to like memes, I'll give you this image.
\n
lol, thanks for your reply
", "votes": null, "replies": [] } ] }, { "id": 1691587, "author_name": "ttkagglett", "author_url": "", "post_date": "02/15/2022 13:43:08", "content": "I think the simple solution is the better way in real-world problem. Thanks for sharing and congrats on 16th!
", "votes": null, "replies": [] }, { "id": 1693022, "author_name": "zhenhuazhang", "author_url": "", "post_date": "02/16/2022 11:54:48", "content": "Thanks for this solution mate, I'm glad that YoloX is able to make it up there. I had used YoloX myself but got hard stucked at 0.4X, I'm pretty new to kaggle so no suprise there lol. Seems high training resolution gave the largest boost? at least from the 0.4X perspective. Due to GPU limitation was only able to train on 800ish resolution.
\nAnyway, congrats on the 16th, especially with YoloX ππ.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1691165": "Congratulations to the winners and all participants who finished this competition π\nIn this competition, many participants were focused on increasing the score of Public LB, and I was not able to keep up with Public LB at least halfway through.\nHowever, in my local experiments, I have found that methods that increase the image size only during inference, even when the threshold and ensemble methods are properly adjusted, result in extremely low scores for the entire training data.\nSo, I thought there would be shake. Although I was not able to get into the gold zone, I am glad that I participated because I was able to recognize the necessity of believing in my own experiments.\n\nOk, my solution is a pretty simple one, but I would like to share.\n\n\n**1. Preprocessing**\nSince the video was basically switching almost every sequence, I built a validation with 4 folds based on the sequence (However, I think there were some sequences that were a little continuous).\nOne coin I did was to make the number of CoTS almost the same in each fold. In training, I did not use images without CoTS, but in validation, I used all images without CoTS in order to check the False Positive properly.\n The image was simply divided by 255 without any noise removal, and the size was 1952 x 3520.\nDue to GPU limitations, I couldn't try a larger size, but at least up to this size, I was able to get a small gain with Local CV. However I got the most gain up to about twice the size of the original image.\n\n\n**2. Model: All I want to use is YOLO-X**\nThis time I really wanted to use `anchor-free` YOLO-X, which I had never used before, so I stuck to it, even though many people have had success with YOLO-v5 in Public LB.\n It was hard to modify YOLO-X because many parts were hard-coded, but I tried to tweak the [top-K selection algorithm](https://arxiv.org/pdf/2107.08430.pdf), add augmentation, and some other dubious modifications. But in the end, only augmentation seemed to have an effect. In that sense, YOLO-X's perfection as a model may be high π\n\n\n**3. Augmentation and learning strategy**\nIn addition to the augmentation (mosaic, mixup, etc.) that YOLO-X has as a default function, I added RandomGamma, RGBshift, Sharpen, GaussNoise, etc.\nInstead, the probability of applying mixup and mosaic has been slightly reduced from the default value and assigned the rest probability to the newly added augmentation path. This may have increased the diversity of the input images somewhat, and I was able to get a gain in Local CV(I don't know the specific ablation values in detail, as the experiment took some twists and turns).\n In addition, I used the [progressive learning](https://arxiv.org/abs/2104.00298) method used in EfficientNetV2: gradually increasing the size of the image (e.g. 1280 => ... => 3520) as the learning progressed. At the same time, I remember that regularization (increasing the probability of application of augmentation) was also strengthened somewhat.\n\n\n**4. Inference**\nThe inference was very simple: I did TTA with Flip and ensembled with WBF for 4-fold (so 8 models). I think this is almost same as many of the participants.\nNote that I set thresholds to improve the local CV. The F2 metrics were sensitive to the threshold settings to some extent because our evaluation metric was based on the confusion matrix.\n\n\n**5. An original point that I have worked out: Seq-NMS**\nOne point that I devised a little is the post-processing.\n Since the task is object recognition in video, I made a post in [this thread](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/293812) early stage in this competition, and I was reading almost all papers. Many of them reported that feature aggregation, which is a method of enriching input features by using past images in the neighborhood, is more performing than post-processing methods such as tracking, and I thought that was probably true. However, since the majority of feature aggregation methods is based on RPNs, it was too much of a hurdle for me to apply, since I was sticking to YOLO-X, which is anchor-free. So I decided to use [Seq-NMS](https://arxiv.org/abs/1602.08465): a typical post-processing method. Specifically, the confidence is adjusted by the degree of overlap between the predicted bbox of the previous images and the predicted bbox of the current image (Ref: https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/293812#1611428).\n(The implementation can be found [here](https://github.com/tmoopenn/seq-nms). The core processing is written in cython and is fast enough, but as far as I understand, there is a serious bug in the python script that makes it not work properly as is, so I fixed the bug to use it.)\n As long as I experimented, there is no need to consider many frames for Seq-NMS, just the prediction of the previous image. I also tried to use [Dense Optical Flow](https://docs.opencv.org/3.4/d4/dee/tutorial_optical_flow.html#:~:text=Dense%20Optical%20Flow%20in%20OpenCV) to compensate for the shift of predicted bboxes of the previous frame, but in the end I did not use it because Dense Optical Flow is expensive to compute and does not have much gain.\n\n---\n\nThat's all I can remember right now. If I remember anything else, I'll add it.\nThank you all for your hard work, and see you at another competition :)\n\nHappy Kagglingπ\n\n\n\n\nFeb. 18. 2022, Model pipeline figure updated", "1691177": "Congratulations! Yolo-X πππͺππππ Great!", "1691187": "Congratulations! How much improvement has Seq-NMS brought you?", "1691210": "Thank you. Congratulations, too!\n\n> Q. How much improvement has Seq-NMS brought you?\n\nA. about 0.015. I only have my experiments to base this on, but I think it's a little better than the tracking method that was shared on kernel.\n\nBy the way, since you seem to like memes, I'll give you this image.\n", "1691241": "lol, thanks for your reply", "1691587": "I think the simple solution is the better way in real-world problem. Thanks for sharing and congrats on 16th!", "1693022": "Thanks for this solution mate, I'm glad that YoloX is able to make it up there. I had used YoloX myself but got hard stucked at 0.4X, I'm pretty new to kaggle so no suprise there lol. Seems high training resolution gave the largest boost? at least from the 0.4X perspective. Due to GPU limitation was only able to train on 800ish resolution.\n\nAnyway, congrats on the 16th, especially with YoloX ππ." }, "source": "meta" }