{ "id": 263895, "title": "26th Place Solution : Brief Writeup", "url": "/competitions/siim-covid19-detection/writeups/hungry-for-gold-26th-place-solution-brief-writeup", "author_name": "", "post_date": "2021-08-10T14:41:17.590Z", "votes": 34, "comment_count": 16, "views": 0, "content": "

Hi all,
\nIt was a very interesting and challenging competition, congratulations to all the winners and a big thank you to the organizers for the competition !

\n

Introduction

\n

We started this competition quite late, it was my first Object detection competition hence a great opportunity to learn. Teaming up with @pheadrus , @theoviel ,and @onodera was a great experience for me.

\n

Overview

\n

Our Final solution is fairly simple, even though we tried a lot of different things without success, that other top teams managed to make work. In this write-up we’ll focus a bit more on what differentiate our solution from others, as most of the stuff is really similar.

\n

Solution

\n

4-class Classification

\n

Our architecture is similar to heng’s idea, as it includes an auxiliary segmentation head on the two last layers. We trained a bunch of models (and image sizes) for diversity, in the end the following ones were used in the final ensembles, with horizontal flip tta :

\n\n

While Pheadrus used normal preprocessing that is used by most in the competition.
\nTheo’s model used a different preprocessing technique (automated padding removal & clipping to the 1st and 99th percentiles before normalizing), as well as an auxiliary binary classification head for the none class.

\n

\"proc\"
\nTheo’s preprocessing : windowing & padding removal

\n

\"model\"
\nClassification models overview. Phaedrus’ models don’t use the opacity head.

\n

2-class classification

\n

In addition to Theo’s model, I trained a b4, b5 and b6. They helped CV a bit.

\n

Detection

\n

@onodonera built a bunch of yolov5 (384) whereas @phaedrus used yolov5 (768). One of our two subs used the former and the other one the latter, along with some other modifications. We probably should’ve spent more time training a bigger variety of detection models.

\n

Final sub

\n

Our 0.619 sub uses @pheadrus models for detection, @theoviel model for both classification tasks and @pheadrus models for the study target.
\nOur other sub - which we thought would perform better - also included my models for the binary classification and uses @onodera's detection model.

\n

We used power ensembling for classification models (p=2), and calibrated predictions of the detection model by multiplying them by the binary model prediction^0.1. This gave a <0.001 boost though.

\n

What didn’t work for us

\n\n

Thanks for reading !

", "messages": [ { "id": "1464283", "postDate": "08/10/2021 14:17:10", "content": "

Hi all,
\nIt was a very interesting and challenging competition, congratulations to all the winners and a big thank you to the organizers for the competition !

\n

Introduction

\n

We started this competition quite late, it was my first Object detection competition hence a great opportunity to learn. Teaming up with @pheadrus , @theoviel ,and @onodera was a great experience for me.

\n

Overview

\n

Our Final solution is fairly simple, even though we tried a lot of different things without success, that other top teams managed to make work. In this write-up we’ll focus a bit more on what differentiate our solution from others, as most of the stuff is really similar.

\n

Solution

\n

4-class Classification

\n

Our architecture is similar to heng’s idea, as it includes an auxiliary segmentation head on the two last layers. We trained a bunch of models (and image sizes) for diversity, in the end the following ones were used in the final ensembles, with horizontal flip tta :

\n\n

While Pheadrus used normal preprocessing that is used by most in the competition.
\nTheo’s model used a different preprocessing technique (automated padding removal & clipping to the 1st and 99th percentiles before normalizing), as well as an auxiliary binary classification head for the none class.

\n

\"proc\"
\nTheo’s preprocessing : windowing & padding removal

\n

\"model\"
\nClassification models overview. Phaedrus’ models don’t use the opacity head.

\n

2-class classification

\n

In addition to Theo’s model, I trained a b4, b5 and b6. They helped CV a bit.

\n

Detection

\n

@onodonera built a bunch of yolov5 (384) whereas @phaedrus used yolov5 (768). One of our two subs used the former and the other one the latter, along with some other modifications. We probably should’ve spent more time training a bigger variety of detection models.

\n

Final sub

\n

Our 0.619 sub uses @pheadrus models for detection, @theoviel model for both classification tasks and @pheadrus models for the study target.
\nOur other sub - which we thought would perform better - also included my models for the binary classification and uses @onodera's detection model.

\n

We used power ensembling for classification models (p=2), and calibrated predictions of the detection model by multiplying them by the binary model prediction^0.1. This gave a <0.001 boost though.

\n

What didn’t work for us

\n\n

Thanks for reading !

", "rawMarkdown": "Hi all, \nIt was a very interesting and challenging competition, congratulations to all the winners and a big thank you to the organizers for the competition !\n\n\n## Introduction \n\nWe started this competition quite late, it was my first Object detection competition hence a great opportunity to learn. Teaming up with @pheadrus , @theoviel ,and @onodera was a great experience for me. \n\n## Overview\n\nOur Final solution is fairly simple, even though we tried a lot of different things without success, that other top teams managed to make work. In this write-up we’ll focus a bit more on what differentiate our solution from others, as most of the stuff is really similar. \n\n## Solution\n\n### 4-class Classification\n\nOur architecture is similar to heng’s idea, as it includes an auxiliary segmentation head on the two last layers. We trained a bunch of models (and image sizes) for diversity, in the end the following ones were used in the final ensembles, with horizontal flip tta :\n\n- @pheadrus : v2m (512), b5 (512), b5 (640), v2m (768), v2m + pl (512)\n- @theoviel : v2s (512), v2m (512)\n\nWhile Pheadrus used normal preprocessing that is used by most in the competition.\nTheo’s model used a different preprocessing technique (automated padding removal & clipping to the 1st and 99th percentiles before normalizing), as well as an auxiliary binary classification head for the none class.\n\n\"proc\"\nTheo’s preprocessing : windowing & padding removal\n\n\"model\"\nClassification models overview. Phaedrus’ models don’t use the opacity head.\n\n### 2-class classification\n\nIn addition to Theo’s model, I trained a b4, b5 and b6. They helped CV a bit.\n\n### Detection\n\n@onodonera built a bunch of yolov5 (384) whereas @phaedrus used yolov5 (768). One of our two subs used the former and the other one the latter, along with some other modifications. We probably should’ve spent more time training a bigger variety of detection models. \n\n### Final sub\n\nOur 0.619 sub uses @pheadrus models for detection, @theoviel model for both classification tasks and @pheadrus models for the study target. \nOur other sub - which we thought would perform better - also included my models for the binary classification and uses @onodera's detection model.\n\nWe used power ensembling for classification models (p=2), and calibrated predictions of the detection model by multiplying them by the binary model prediction^0.1. This gave a <0.001 boost though.\n\n\n### What didn’t work for us\n\n- Pretraining on the CovidX data for classification\n- Pseudo-labelling the public test set for classification\n- Using a lung detection model for preprocessing helped some of the models but not the overall blend\n- Separate classifiers for the atypical and indeterminate classes\n- We built and pseudo-labeled a dataset of 25k (covid positive and negative) images from the BimCV, Ricord and CovidX datasets, but couldn’t managed to get any improvement from that either\n\n\n\n*Thanks for reading !*", "votes": null }, { "id": "1464290", "postDate": "08/10/2021 14:20:34", "content": "

Congratulations team! Well done. I like your 4-class classification model design and clean image preprocessing.

", "rawMarkdown": "Congratulations team! Well done. I like your 4-class classification model design and clean image preprocessing.", "votes": null }, { "id": "1464298", "postDate": "08/10/2021 14:22:40", "content": "

Thanks a lot @tanulsingh077 @onodonera @phaedrus for the competition, it was nice working with you !

", "rawMarkdown": "Thanks a lot @tanulsingh077 @onodonera @phaedrus for the competition, it was nice working with you !", "votes": null }, { "id": "1464322", "postDate": "08/10/2021 14:31:47", "content": "

Thanks Chris ! You're too fast to reply we weren't even done fixing the post !

", "rawMarkdown": "Thanks Chris ! You're too fast to reply we weren't even done fixing the post !", "votes": null }, { "id": "1464324", "postDate": "08/10/2021 14:32:41", "content": "

It was such a nice experience for me personally to team up with you guys @theoviel @onodera @tanulsingh077

\n

Great learning experience for me!

", "rawMarkdown": "It was such a nice experience for me personally to team up with you guys @theoviel @onodera @tanulsingh077 \n\nGreat learning experience for me!", "votes": null }, { "id": "1464335", "postDate": "08/10/2021 14:36:08", "content": "

Thanks chris , congrats on a strong finish once again , looking forward to read your solution 😉

", "rawMarkdown": "Thanks chris , congrats on a strong finish once again , looking forward to read your solution 😉", "votes": null }, { "id": "1464346", "postDate": "08/10/2021 14:41:32", "content": "

haha, i see you changed your image host so that Kaggle displays the image in the post. That's better. (It's annoying how Kaggle changed their image policies. Many of my old posts lost their images 🙁 )

", "rawMarkdown": "haha, i see you changed your image host so that Kaggle displays the image in the post. That's better. (It's annoying how Kaggle changed their image policies. Many of my old posts lost their images 🙁 )", "votes": null }, { "id": "1465993", "postDate": "08/11/2021 09:00:20", "content": "

Update : My part of the solution is available on GitHub -> https://github.com/TheoViel/kaggle_siim_covid
\nAs usual, code is documented & cleaned :)

", "rawMarkdown": "**Update :** My part of the solution is available on GitHub -> https://github.com/TheoViel/kaggle_siim_covid\nAs usual, code is documented & cleaned :)", "votes": null }, { "id": "1467342", "postDate": "08/11/2021 23:58:29", "content": "

Good work team, congratulations 🙏

", "rawMarkdown": "Good work team, congratulations 🙏", "votes": null }, { "id": "1467512", "postDate": "08/12/2021 03:09:09", "content": "

Great write up ,helped me to understand the approach much better.

", "rawMarkdown": "Great write up ,helped me to understand the approach much better.", "votes": null }, { "id": "1468551", "postDate": "08/12/2021 13:21:11", "content": "

Very nice work. Thank you

", "rawMarkdown": "Very nice work. Thank you", "votes": null }, { "id": "1468805", "postDate": "08/12/2021 15:19:11", "content": "

Great works! Amazing experience , thanks.

", "rawMarkdown": "Great works! Amazing experience , thanks.", "votes": null }, { "id": "1474712", "postDate": "08/16/2021 08:39:05", "content": "

Congrats guy. Excellent effort.

", "rawMarkdown": "Congrats guy. Excellent effort.", "votes": null }, { "id": "1475277", "postDate": "08/16/2021 14:48:30", "content": "

Congratulations, Could you please tell me in which file we have done the preprocessing part as said in the post.
\nI actually didn't understand the part :\"clipping to the 1st and 99th percentiles before normalizing\"
\nThanks

", "rawMarkdown": "Congratulations, Could you please tell me in which file we have done the preprocessing part as said in the post.\nI actually didn't understand the part :\"clipping to the 1st and 99th percentiles before normalizing\"\nThanks", "votes": null }, { "id": "1475439", "postDate": "08/16/2021 16:15:12", "content": "

Hi @mrinath you can find the code here https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/data/extraction.py

\n

Please let me know if you need anything else

", "rawMarkdown": "Hi @mrinath you can find the code here https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/data/extraction.py\n\nPlease let me know if you need anything else", "votes": null }, { "id": "1488201", "postDate": "08/24/2021 06:50:23", "content": "

Hey, I am confused about one thing, after this preprocessing step, I have seen you have used Mean and std as None in your https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/data/transforms.py code.
\nSo does it mean you have given the image in the model without normalizing it?

", "rawMarkdown": "Hey, I am confused about one thing, after this preprocessing step, I have seen you have used Mean and std as None in your https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/data/transforms.py code.\nSo does it mean you have given the image in the model without normalizing it?", "votes": null }, { "id": "1488253", "postDate": "08/24/2021 07:17:01", "content": "

Mean and std are chosen according to the model :

\n

In training/main.py :

\n
transforms=get_transfos_cls(augment=True, mean=model.mean, std=model.std)\n
", "rawMarkdown": "Mean and std are chosen according to the model :\n\nIn [training/main.py](https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/training/main.py) :\n```\ntransforms=get_transfos_cls(augment=True, mean=model.mean, std=model.std)\n```", "votes": null } ], "comments": [ { "id": 1464290, "author_name": "cdeotte", "author_url": "", "post_date": "08/10/2021 14:20:34", "content": "

Congratulations team! Well done. I like your 4-class classification model design and clean image preprocessing.

", "votes": null, "replies": [ { "id": 1464322, "author_name": "theoviel", "author_url": "", "post_date": "08/10/2021 14:31:47", "content": "

Thanks Chris ! You're too fast to reply we weren't even done fixing the post !

", "votes": null, "replies": [] }, { "id": 1464335, "author_name": "tanulsingh077", "author_url": "", "post_date": "08/10/2021 14:36:08", "content": "

Thanks chris , congrats on a strong finish once again , looking forward to read your solution 😉

", "votes": null, "replies": [] }, { "id": 1464346, "author_name": "cdeotte", "author_url": "", "post_date": "08/10/2021 14:41:32", "content": "

haha, i see you changed your image host so that Kaggle displays the image in the post. That's better. (It's annoying how Kaggle changed their image policies. Many of my old posts lost their images 🙁 )

", "votes": null, "replies": [] } ] }, { "id": 1464298, "author_name": "theoviel", "author_url": "", "post_date": "08/10/2021 14:22:40", "content": "

Thanks a lot @tanulsingh077 @onodonera @phaedrus for the competition, it was nice working with you !

", "votes": null, "replies": [ { "id": 1464324, "author_name": "pheadrus", "author_url": "", "post_date": "08/10/2021 14:32:41", "content": "

It was such a nice experience for me personally to team up with you guys @theoviel @onodera @tanulsingh077

\n

Great learning experience for me!

", "votes": null, "replies": [] } ] }, { "id": 1465993, "author_name": "theoviel", "author_url": "", "post_date": "08/11/2021 09:00:20", "content": "

Update : My part of the solution is available on GitHub -> https://github.com/TheoViel/kaggle_siim_covid
\nAs usual, code is documented & cleaned :)

", "votes": null, "replies": [ { "id": 1475277, "author_name": "mrinath", "author_url": "", "post_date": "08/16/2021 14:48:30", "content": "

Congratulations, Could you please tell me in which file we have done the preprocessing part as said in the post.
\nI actually didn't understand the part :\"clipping to the 1st and 99th percentiles before normalizing\"
\nThanks

", "votes": null, "replies": [] }, { "id": 1475439, "author_name": "tanulsingh077", "author_url": "", "post_date": "08/16/2021 16:15:12", "content": "

Hi @mrinath you can find the code here https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/data/extraction.py

\n

Please let me know if you need anything else

", "votes": null, "replies": [] }, { "id": 1488201, "author_name": "mrinath", "author_url": "", "post_date": "08/24/2021 06:50:23", "content": "

Hey, I am confused about one thing, after this preprocessing step, I have seen you have used Mean and std as None in your https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/data/transforms.py code.
\nSo does it mean you have given the image in the model without normalizing it?

", "votes": null, "replies": [] }, { "id": 1488253, "author_name": "theoviel", "author_url": "", "post_date": "08/24/2021 07:17:01", "content": "

Mean and std are chosen according to the model :

\n

In training/main.py :

\n
transforms=get_transfos_cls(augment=True, mean=model.mean, std=model.std)\n
", "votes": null, "replies": [] } ] }, { "id": 1467342, "author_name": "iniestamoh", "author_url": "", "post_date": "08/11/2021 23:58:29", "content": "

Good work team, congratulations 🙏

", "votes": null, "replies": [] }, { "id": 1467512, "author_name": "shub99", "author_url": "", "post_date": "08/12/2021 03:09:09", "content": "

Great write up ,helped me to understand the approach much better.

", "votes": null, "replies": [] }, { "id": 1468551, "author_name": "vip30service", "author_url": "", "post_date": "08/12/2021 13:21:11", "content": "

Very nice work. Thank you

", "votes": null, "replies": [] }, { "id": 1468805, "author_name": "mdsumonhossain", "author_url": "", "post_date": "08/12/2021 15:19:11", "content": "

Great works! Amazing experience , thanks.

", "votes": null, "replies": [] }, { "id": 1474712, "author_name": "msg4wale", "author_url": "", "post_date": "08/16/2021 08:39:05", "content": "

Congrats guy. Excellent effort.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1464283": "Hi all, \nIt was a very interesting and challenging competition, congratulations to all the winners and a big thank you to the organizers for the competition !\n\n\n## Introduction \n\nWe started this competition quite late, it was my first Object detection competition hence a great opportunity to learn. Teaming up with @pheadrus , @theoviel ,and @onodera was a great experience for me. \n\n## Overview\n\nOur Final solution is fairly simple, even though we tried a lot of different things without success, that other top teams managed to make work. In this write-up we’ll focus a bit more on what differentiate our solution from others, as most of the stuff is really similar. \n\n## Solution\n\n### 4-class Classification\n\nOur architecture is similar to heng’s idea, as it includes an auxiliary segmentation head on the two last layers. We trained a bunch of models (and image sizes) for diversity, in the end the following ones were used in the final ensembles, with horizontal flip tta :\n\n- @pheadrus : v2m (512), b5 (512), b5 (640), v2m (768), v2m + pl (512)\n- @theoviel : v2s (512), v2m (512)\n\nWhile Pheadrus used normal preprocessing that is used by most in the competition.\nTheo’s model used a different preprocessing technique (automated padding removal & clipping to the 1st and 99th percentiles before normalizing), as well as an auxiliary binary classification head for the none class.\n\n\"proc\"\nTheo’s preprocessing : windowing & padding removal\n\n\"model\"\nClassification models overview. Phaedrus’ models don’t use the opacity head.\n\n### 2-class classification\n\nIn addition to Theo’s model, I trained a b4, b5 and b6. They helped CV a bit.\n\n### Detection\n\n@onodonera built a bunch of yolov5 (384) whereas @phaedrus used yolov5 (768). One of our two subs used the former and the other one the latter, along with some other modifications. We probably should’ve spent more time training a bigger variety of detection models. \n\n### Final sub\n\nOur 0.619 sub uses @pheadrus models for detection, @theoviel model for both classification tasks and @pheadrus models for the study target. \nOur other sub - which we thought would perform better - also included my models for the binary classification and uses @onodera's detection model.\n\nWe used power ensembling for classification models (p=2), and calibrated predictions of the detection model by multiplying them by the binary model prediction^0.1. This gave a <0.001 boost though.\n\n\n### What didn’t work for us\n\n- Pretraining on the CovidX data for classification\n- Pseudo-labelling the public test set for classification\n- Using a lung detection model for preprocessing helped some of the models but not the overall blend\n- Separate classifiers for the atypical and indeterminate classes\n- We built and pseudo-labeled a dataset of 25k (covid positive and negative) images from the BimCV, Ricord and CovidX datasets, but couldn’t managed to get any improvement from that either\n\n\n\n*Thanks for reading !*", "1464290": "Congratulations team! Well done. I like your 4-class classification model design and clean image preprocessing.", "1464298": "Thanks a lot @tanulsingh077 @onodonera @phaedrus for the competition, it was nice working with you !", "1464322": "Thanks Chris ! You're too fast to reply we weren't even done fixing the post !", "1464324": "It was such a nice experience for me personally to team up with you guys @theoviel @onodera @tanulsingh077 \n\nGreat learning experience for me!", "1464335": "Thanks chris , congrats on a strong finish once again , looking forward to read your solution 😉", "1464346": "haha, i see you changed your image host so that Kaggle displays the image in the post. That's better. (It's annoying how Kaggle changed their image policies. Many of my old posts lost their images 🙁 )", "1465993": "**Update :** My part of the solution is available on GitHub -> https://github.com/TheoViel/kaggle_siim_covid\nAs usual, code is documented & cleaned :)", "1467342": "Good work team, congratulations 🙏", "1467512": "Great write up ,helped me to understand the approach much better.", "1468551": "Very nice work. Thank you", "1468805": "Great works! Amazing experience , thanks.", "1474712": "Congrats guy. Excellent effort.", "1475277": "Congratulations, Could you please tell me in which file we have done the preprocessing part as said in the post.\nI actually didn't understand the part :\"clipping to the 1st and 99th percentiles before normalizing\"\nThanks", "1475439": "Hi @mrinath you can find the code here https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/data/extraction.py\n\nPlease let me know if you need anything else", "1488201": "Hey, I am confused about one thing, after this preprocessing step, I have seen you have used Mean and std as None in your https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/data/transforms.py code.\nSo does it mean you have given the image in the model without normalizing it?", "1488253": "Mean and std are chosen according to the model :\n\nIn [training/main.py](https://github.com/TheoViel/kaggle_siim_covid/blob/main/src/training/main.py) :\n```\ntransforms=get_transfos_cls(augment=True, mean=model.mean, std=model.std)\n```" }, "source": "meta" }