{ "id": 263945, "title": "5th place solution - brief summary", "url": "/competitions/siim-covid19-detection/writeups/ayushman-nischay-shivam-5th-place-solution-brief-s", "author_name": "", "post_date": "2021-08-18T15:26:32.467Z", "votes": 49, "comment_count": 23, "views": 0, "content": "

First of all I would like to thank my teammates @nischaydnk and @shivamcyborg without whom this achievement would not have been possible. We merged on On 12th July and from then onwards we explored and implemented a lot of new ideas.

\n

I would also like to thank Kaggle, SIIM, FISABIO, RSNA for this great competition, as well has everyone here who has shared great discussions and notebooks. Congrats to all the winners!

\n

Trusting our maximised cv score worked once again for us. Most of our time were spent on Classification models which were the reason we secured a decent position on leaderboard.

\n

We did not use any external training data. All the models were trained on the competition data after removing the duplicates as suggested here.

\n

Cross-validation: Stratified Group KFOLD

\n

Part 1: Study

\n
    \n
  1. We used an ensemble of 11 models.
    2. \n
  2. Base architectures = Efficientnet v2m, Efficientnet v2l, Efficientnet B5, Efficientnet B7
    3. \n
  3. Models were trained on different image sizes ranging from [512, 720] and different augmentations.
    4. \n
  4. Segmentation as aux loss suggested by @hengck23
    5. \n
  5. v2 models used a custom pcam pool head + attention.
    6. \n
  6. b5, b7 used multi head + concat pool + attention suggested by @ttahara .
    7. \n
  7. We also used noisy student training method suggested by @moewie94 in here.
    8. \n
  8. Hflip TTA was used during inference.
    9. \n
\n

Part 2: Binary (none class predictions)

\n
    \n
  1. We used an ensemble of 2x five fold models (efficientnetv2m , efficientnetb6)
    2. \n
  2. Models were trained on 512 image size.
    3. \n
  3. Segmentation as aux loss suggested by @hengck23
    4. \n
  4. v2m models used a custom pcam pool head + attention.
    5. \n
  5. b6 was trained using the pipeline provided by @hengck23.
    6. \n
  6. v2m model as actually trained on 4 class study data. We used the negative predictions as none and merged these with the binary predictions from the above models.
    7. \n
  7. Hflip TTA was used during inference.
    8. \n
\n

Part 3: Detection (opacity predictions)

\n
    \n
  1. We used an ensemble of 5x five fold models.
    2. \n
  2. Models used = efficientdetD3, efficientdetD5, yolov5x, yolov5l6, retinanet_x101_64x4d_fpn
    3. \n
  3. image sizes used = 896, 512, 620, 620, (1330, 800)
    4. \n
  4. we first trained d5, d3, yolov5x, yolov5l6 on only opacity predictions. We used WBF (iou=0.62) to generate pseudo lables of public test. We then used these labels to train d3, yolov5x, yolov5l6 models again.
    5. \n
  5. In the pseudo label training d3 was trained with ema and yolov5 models were trained with a few none images as well.
    6. \n
  6. D5 was trained with EMA from the beginning and pseudo labelling did not bring any improvements to both D5 and RetinanNet so we did not include them in the final ensemble.
    7. \n
  7. At inference predictions from the 5 models were merged using WBF(iou=0.625).
    8. \n
\n

Part 4: Postprocessing

\n
    \n
  1. Power blending with maximum opacity confidence from part 3 and none predictions from part 2
    2. \n
  2. We optimized both none predictions as well as opacity confidences.
    3. \n
  3. This was inspired from @cdeotte 's post.
    4. \n
\n

We have released our inference kernel : https://www.kaggle.com/nischaydnk/604e8587410a-v2m-bin-weighted and in the coming days we will release our full code as well. We didn't had time to prepare written solution during competitions, loads of work had to be done during the competition and also we need to clean our code a lot

\n

Update : Code will be available here

", "messages": [ { "id": "1464554", "postDate": "08/10/2021 16:01:04", "content": "

First of all I would like to thank my teammates @nischaydnk and @shivamcyborg without whom this achievement would not have been possible. We merged on On 12th July and from then onwards we explored and implemented a lot of new ideas.

\n

I would also like to thank Kaggle, SIIM, FISABIO, RSNA for this great competition, as well has everyone here who has shared great discussions and notebooks. Congrats to all the winners!

\n

Trusting our maximised cv score worked once again for us. Most of our time were spent on Classification models which were the reason we secured a decent position on leaderboard.

\n

We did not use any external training data. All the models were trained on the competition data after removing the duplicates as suggested here.

\n

Cross-validation: Stratified Group KFOLD

\n

Part 1: Study

\n
    \n
  1. We used an ensemble of 11 models.
    2. \n
  2. Base architectures = Efficientnet v2m, Efficientnet v2l, Efficientnet B5, Efficientnet B7
    3. \n
  3. Models were trained on different image sizes ranging from [512, 720] and different augmentations.
    4. \n
  4. Segmentation as aux loss suggested by @hengck23
    5. \n
  5. v2 models used a custom pcam pool head + attention.
    6. \n
  6. b5, b7 used multi head + concat pool + attention suggested by @ttahara .
    7. \n
  7. We also used noisy student training method suggested by @moewie94 in here.
    8. \n
  8. Hflip TTA was used during inference.
    9. \n
\n

Part 2: Binary (none class predictions)

\n
    \n
  1. We used an ensemble of 2x five fold models (efficientnetv2m , efficientnetb6)
    2. \n
  2. Models were trained on 512 image size.
    3. \n
  3. Segmentation as aux loss suggested by @hengck23
    4. \n
  4. v2m models used a custom pcam pool head + attention.
    5. \n
  5. b6 was trained using the pipeline provided by @hengck23.
    6. \n
  6. v2m model as actually trained on 4 class study data. We used the negative predictions as none and merged these with the binary predictions from the above models.
    7. \n
  7. Hflip TTA was used during inference.
    8. \n
\n

Part 3: Detection (opacity predictions)

\n
    \n
  1. We used an ensemble of 5x five fold models.
    2. \n
  2. Models used = efficientdetD3, efficientdetD5, yolov5x, yolov5l6, retinanet_x101_64x4d_fpn
    3. \n
  3. image sizes used = 896, 512, 620, 620, (1330, 800)
    4. \n
  4. we first trained d5, d3, yolov5x, yolov5l6 on only opacity predictions. We used WBF (iou=0.62) to generate pseudo lables of public test. We then used these labels to train d3, yolov5x, yolov5l6 models again.
    5. \n
  5. In the pseudo label training d3 was trained with ema and yolov5 models were trained with a few none images as well.
    6. \n
  6. D5 was trained with EMA from the beginning and pseudo labelling did not bring any improvements to both D5 and RetinanNet so we did not include them in the final ensemble.
    7. \n
  7. At inference predictions from the 5 models were merged using WBF(iou=0.625).
    8. \n
\n

Part 4: Postprocessing

\n
    \n
  1. Power blending with maximum opacity confidence from part 3 and none predictions from part 2
    2. \n
  2. We optimized both none predictions as well as opacity confidences.
    3. \n
  3. This was inspired from @cdeotte 's post.
    4. \n
\n

We have released our inference kernel : https://www.kaggle.com/nischaydnk/604e8587410a-v2m-bin-weighted and in the coming days we will release our full code as well. We didn't had time to prepare written solution during competitions, loads of work had to be done during the competition and also we need to clean our code a lot

\n

Update : Code will be available here

", "rawMarkdown": "First of all I would like to thank my teammates @nischaydnk and @shivamcyborg without whom this achievement would not have been possible. We merged on On 12th July and from then onwards we explored and implemented a lot of new ideas.\n\nI would also like to thank Kaggle, SIIM, FISABIO, RSNA for this great competition, as well has everyone here who has shared great discussions and notebooks. Congrats to all the winners!\n\nTrusting our maximised cv score worked once again for us. Most of our time were spent on Classification models which were the reason we secured a decent position on leaderboard. \n\nWe did not use any external training data. All the models were trained on the competition data after removing the duplicates as suggested [here](https://www.kaggle.com/c/siim-covid19-detection/discussion/246597).\n\n**Cross-validation:** Stratified Group KFOLD\n\n**Part 1: Study**\n\n1. We used an ensemble of 11 models.\n2. Base architectures = *Efficientnet v2m, Efficientnet v2l, Efficientnet B5, Efficientnet B7*\n3. Models were trained on different image sizes ranging from [512, 720] and different augmentations.\n4. Segmentation as aux loss suggested by @hengck23 \n5. v2 models used a custom [pcam](https://github.com/jfhealthcare/Chexpert) pool head + attention.\n6. b5, b7 used multi head + concat pool + attention suggested by @ttahara .\n7. We also used noisy student training method suggested by @moewie94 in [here](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/226616).\n8. Hflip TTA was used during inference.\n\n\n**Part 2: Binary (`none` class predictions)**\n\n1. We used an ensemble of 2x five fold models (efficientnetv2m , efficientnetb6)\n2. Models were trained on 512 image size.\n3. Segmentation as aux loss suggested by @hengck23 \n4. v2m models used a custom [pcam](https://github.com/jfhealthcare/Chexpert) pool head + attention.\n5. b6 was trained using the pipeline provided by @hengck23.\n6. v2m model as actually trained on 4 class study data. We used the `negative` predictions as `none` and merged these with the `binary` predictions from the above models.\n7. Hflip TTA was used during inference.\n\n\n**Part 3: Detection (`opacity` predictions)**\n\n1. We used an ensemble of 5x five fold models.\n2. Models used = `efficientdetD3, efficientdetD5, yolov5x, yolov5l6, retinanet_x101_64x4d_fpn`\n3. image sizes used = `896, 512, 620, 620, (1330, 800)`\n4. we first trained d5, d3, yolov5x, yolov5l6 on only opacity predictions. We used WBF (iou=0.62) to generate pseudo lables of public test. We then used these labels to train `d3, yolov5x, yolov5l6` models again. \n5. In the pseudo label training d3 was trained with ema and yolov5 models were trained with a few `none` images as well. \n6. D5 was trained with EMA from the beginning and pseudo labelling did not bring any improvements to both D5 and RetinanNet so we did not include them in the final ensemble.\n7. At inference predictions from the 5 models were merged using WBF(iou=0.625).\n\n\n**Part 4: Postprocessing**\n1. Power blending with maximum opacity confidence from part 3 and none predictions from part 2\n2. We optimized both `none` predictions as well as `opacity` confidences.\n3. This was inspired from @cdeotte 's [post](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637).\n\n\n*We have released our inference kernel : https://www.kaggle.com/nischaydnk/604e8587410a-v2m-bin-weighted and in the coming days we will release our full code as well. We didn't had time to prepare written solution during competitions, loads of work had to be done during the competition and also we need to clean our code a lot*\n\n**Update : Code will be available [here](https://github.com/benihime91/SIIM-COVID19-DETECTION-KAGGLE)**", "votes": null }, { "id": "1464558", "postDate": "08/10/2021 16:02:44", "content": "

Our single models performance on oof and public leaderboard.
\n\"Screenshot-2021-08-10-at-9-25-54-PM\"

", "rawMarkdown": "**Our single models performance on oof and public leaderboard.**\n\"Screenshot-2021-08-10-at-9-25-54-PM\"", "votes": null }, { "id": "1464729", "postDate": "08/10/2021 17:21:01", "content": "

This is Ranzcr, vinbigdata and chestxpath solutions blended into one haha, I see @nischaydnk using all his experience. Well done @benihime91 great solution

", "rawMarkdown": "This is Ranzcr, vinbigdata and chestxpath solutions blended into one haha, I see @nischaydnk using all his experience. Well done @benihime91 great solution", "votes": null }, { "id": "1464746", "postDate": "08/10/2021 17:29:05", "content": "

Thanks a lot πŸ˜ƒ

", "rawMarkdown": "Thanks a lot πŸ˜ƒ", "votes": null }, { "id": "1465386", "postDate": "08/11/2021 03:31:04", "content": "

Congrats on 5th place and thank you for sharing your solutions. I want to know what is \"EMA\" u mention in Detection section ? Also may i ask your private score on Study Level and Image Level

", "rawMarkdown": "Congrats on 5th place and thank you for sharing your solutions. I want to know what is \"EMA\" u mention in Detection section ? Also may i ask your private score on Study Level and Image Level", "votes": null }, { "id": "1465592", "postDate": "08/11/2021 05:26:55", "content": "

EMA is Exponential Moving Average

\n\n

And regarding the private score on Study Level and Image Level, we didn't really check them separately.

", "rawMarkdown": "EMA is Exponential Moving Average \n- https://www.tensorflow.org/api_docs/python/tf/train/ExponentialMovingAverage\n- https://github.com/rwightman/pytorch-image-models/blob/master/timm/utils/model_ema.py\n\nAnd regarding the `private score on Study Level and Image Level`, we didn't really check them separately.", "votes": null }, { "id": "1465722", "postDate": "08/11/2021 06:39:46", "content": "

Thank you for anwsering

", "rawMarkdown": "Thank you for anwsering", "votes": null }, { "id": "1466127", "postDate": "08/11/2021 10:23:49", "content": "

Thanks for sharing!
\nQ: Can you explain what is Hflip TTA?

", "rawMarkdown": "Thanks for sharing!\nQ: Can you explain what is Hflip TTA?", "votes": null }, { "id": "1466183", "postDate": "08/11/2021 10:56:20", "content": "

TTA is test time augmentation and Hflip is horizontal-flip.

\n

So, basically during infernce we predicted results for the origninal image as well as the horizontally flipped image and took the mean of the predictions

", "rawMarkdown": "TTA is test time augmentation and Hflip is horizontal-flip.\n\nSo, basically during infernce we predicted results for the origninal image as well as the horizontally flipped image and took the mean of the predictions", "votes": null }, { "id": "1466243", "postDate": "08/11/2021 11:27:18", "content": "

Congratulation @benihime91. Thanks for sharing.

", "rawMarkdown": "Congratulation @benihime91. Thanks for sharing.", "votes": null }, { "id": "1466518", "postDate": "08/11/2021 13:50:24", "content": "

Thanks for the helpful information!

", "rawMarkdown": "Thanks for the helpful information!", "votes": null }, { "id": "1466707", "postDate": "08/11/2021 15:23:51", "content": "

Congratulation πŸŽ‰. Would you be able to share the source of efficientnetv2 ? We tried it but didn't work for us.

", "rawMarkdown": "Congratulation πŸŽ‰. Would you be able to share the source of `efficientnetv2` ? We tried it but didn't work for us.", "votes": null }, { "id": "1466727", "postDate": "08/11/2021 15:32:22", "content": "

Congrats to you and your team as well. I am waiting on your solution.

\n

Which part of v2 do you need ? We'll eventually be releasing all our code.

", "rawMarkdown": "Congrats to you and your team as well. I am waiting on your solution.\n\nWhich part of v2 do you need ? We'll eventually be releasing all our code.", "votes": null }, { "id": "1466739", "postDate": "08/11/2021 15:39:12", "content": "

I just want to know the GitHub source. If you guys used PyTorch then it should be from Timm models.
\nBtw I've published our solution here

", "rawMarkdown": "I just want to know the GitHub source. If you guys used **PyTorch** then it should be from **Timm** models.\nBtw I've published our solution [here](https://www.kaggle.com/c/siim-covid19-detection/discussion/264243)", "votes": null }, { "id": "1466743", "postDate": "08/11/2021 15:40:32", "content": "

Yes we indeed used PyTorch and timm models. I tried Tensorflow at the start but couldn't find a reliable way to add aux loss so I switched to PyTorch just before merging with my teammates.

", "rawMarkdown": "Yes we indeed used PyTorch and timm models. I tried Tensorflow at the start but couldn't find a reliable way to add aux loss so I switched to PyTorch just before merging with my teammates.", "votes": null }, { "id": "1466749", "postDate": "08/11/2021 15:42:51", "content": "

thanks, for the info

", "rawMarkdown": "thanks, for the info", "votes": null }, { "id": "1468978", "postDate": "08/12/2021 16:48:20", "content": "

Congrats @benihime91 and team, Axom'r naam aru agot loi ana!!

", "rawMarkdown": "Congrats @benihime91 and team, Axom'r naam aru agot loi ana!!", "votes": null }, { "id": "1469145", "postDate": "08/12/2021 18:18:41", "content": "

Congratulations!

", "rawMarkdown": "Congratulations!", "votes": null }, { "id": "1470120", "postDate": "08/13/2021 09:55:22", "content": "

Great job Brother! Very happy for you. Thanks for the details on the solution!

", "rawMarkdown": "Great job Brother! Very happy for you. Thanks for the details on the solution!", "votes": null }, { "id": "1470961", "postDate": "08/13/2021 20:19:43", "content": "

I saw your dedication to your parents and wanted to give you a big thumbs up! πŸ‘ Keep at it @benihime91 and team !

", "rawMarkdown": "I saw your dedication to your parents and wanted to give you a big thumbs up! πŸ‘ Keep at it @benihime91 and team !", "votes": null }, { "id": "1470978", "postDate": "08/13/2021 20:40:05", "content": "

Thanks a lot 😊

", "rawMarkdown": "Thanks a lot 😊", "votes": null }, { "id": "1471201", "postDate": "08/14/2021 03:30:46", "content": "

Congrats πŸ‘πŸ»

", "rawMarkdown": "Congrats πŸ‘πŸ»", "votes": null }, { "id": "1477433", "postDate": "08/17/2021 13:30:33", "content": "

Great job Brother! . Thanks for the details on the solution!

", "rawMarkdown": "Great job Brother! . Thanks for the details on the solution!", "votes": null }, { "id": "1477437", "postDate": "08/17/2021 13:32:01", "content": "

Great Work …Sorry for the loss of your mother
\nMay your mother's soul be blessed
\nKeep the hard work up…….

", "rawMarkdown": "Great Work …Sorry for the loss of your mother\nMay your mother's soul be blessed\nKeep the hard work up.......", "votes": null } ], "comments": [ { "id": 1464558, "author_name": "nischaydnk", "author_url": "", "post_date": "08/10/2021 16:02:44", "content": "

Our single models performance on oof and public leaderboard.
\n\"Screenshot-2021-08-10-at-9-25-54-PM\"

", "votes": null, "replies": [] }, { "id": 1464729, "author_name": "tanulsingh077", "author_url": "", "post_date": "08/10/2021 17:21:01", "content": "

This is Ranzcr, vinbigdata and chestxpath solutions blended into one haha, I see @nischaydnk using all his experience. Well done @benihime91 great solution

", "votes": null, "replies": [ { "id": 1464746, "author_name": "benihime91", "author_url": "", "post_date": "08/10/2021 17:29:05", "content": "

Thanks a lot πŸ˜ƒ

", "votes": null, "replies": [] } ] }, { "id": 1465386, "author_name": "researchbntz", "author_url": "", "post_date": "08/11/2021 03:31:04", "content": "

Congrats on 5th place and thank you for sharing your solutions. I want to know what is \"EMA\" u mention in Detection section ? Also may i ask your private score on Study Level and Image Level

", "votes": null, "replies": [ { "id": 1465592, "author_name": "benihime91", "author_url": "", "post_date": "08/11/2021 05:26:55", "content": "

EMA is Exponential Moving Average

\n\n

And regarding the private score on Study Level and Image Level, we didn't really check them separately.

", "votes": null, "replies": [] }, { "id": 1465722, "author_name": "researchbntz", "author_url": "", "post_date": "08/11/2021 06:39:46", "content": "

Thank you for anwsering

", "votes": null, "replies": [] } ] }, { "id": 1466127, "author_name": "promrcan1", "author_url": "", "post_date": "08/11/2021 10:23:49", "content": "

Thanks for sharing!
\nQ: Can you explain what is Hflip TTA?

", "votes": null, "replies": [ { "id": 1466183, "author_name": "benihime91", "author_url": "", "post_date": "08/11/2021 10:56:20", "content": "

TTA is test time augmentation and Hflip is horizontal-flip.

\n

So, basically during infernce we predicted results for the origninal image as well as the horizontally flipped image and took the mean of the predictions

", "votes": null, "replies": [] }, { "id": 1466518, "author_name": "promrcan1", "author_url": "", "post_date": "08/11/2021 13:50:24", "content": "

Thanks for the helpful information!

", "votes": null, "replies": [] } ] }, { "id": 1466243, "author_name": "kag0306", "author_url": "", "post_date": "08/11/2021 11:27:18", "content": "

Congratulation @benihime91. Thanks for sharing.

", "votes": null, "replies": [] }, { "id": 1466707, "author_name": "awsaf49", "author_url": "", "post_date": "08/11/2021 15:23:51", "content": "

Congratulation πŸŽ‰. Would you be able to share the source of efficientnetv2 ? We tried it but didn't work for us.

", "votes": null, "replies": [ { "id": 1466727, "author_name": "benihime91", "author_url": "", "post_date": "08/11/2021 15:32:22", "content": "

Congrats to you and your team as well. I am waiting on your solution.

\n

Which part of v2 do you need ? We'll eventually be releasing all our code.

", "votes": null, "replies": [] }, { "id": 1466739, "author_name": "awsaf49", "author_url": "", "post_date": "08/11/2021 15:39:12", "content": "

I just want to know the GitHub source. If you guys used PyTorch then it should be from Timm models.
\nBtw I've published our solution here

", "votes": null, "replies": [] }, { "id": 1466743, "author_name": "benihime91", "author_url": "", "post_date": "08/11/2021 15:40:32", "content": "

Yes we indeed used PyTorch and timm models. I tried Tensorflow at the start but couldn't find a reliable way to add aux loss so I switched to PyTorch just before merging with my teammates.

", "votes": null, "replies": [] }, { "id": 1466749, "author_name": "awsaf49", "author_url": "", "post_date": "08/11/2021 15:42:51", "content": "

thanks, for the info

", "votes": null, "replies": [] } ] }, { "id": 1468978, "author_name": "mrinath", "author_url": "", "post_date": "08/12/2021 16:48:20", "content": "

Congrats @benihime91 and team, Axom'r naam aru agot loi ana!!

", "votes": null, "replies": [] }, { "id": 1469145, "author_name": "mustafasadiq", "author_url": "", "post_date": "08/12/2021 18:18:41", "content": "

Congratulations!

", "votes": null, "replies": [] }, { "id": 1470120, "author_name": "shravankoninti", "author_url": "", "post_date": "08/13/2021 09:55:22", "content": "

Great job Brother! Very happy for you. Thanks for the details on the solution!

", "votes": null, "replies": [] }, { "id": 1470961, "author_name": "nabilbeit", "author_url": "", "post_date": "08/13/2021 20:19:43", "content": "

I saw your dedication to your parents and wanted to give you a big thumbs up! πŸ‘ Keep at it @benihime91 and team !

", "votes": null, "replies": [ { "id": 1470978, "author_name": "benihime91", "author_url": "", "post_date": "08/13/2021 20:40:05", "content": "

Thanks a lot 😊

", "votes": null, "replies": [] } ] }, { "id": 1471201, "author_name": "emresengul", "author_url": "", "post_date": "08/14/2021 03:30:46", "content": "

Congrats πŸ‘πŸ»

", "votes": null, "replies": [] }, { "id": 1477433, "author_name": "mansoorabbasi", "author_url": "", "post_date": "08/17/2021 13:30:33", "content": "

Great job Brother! . Thanks for the details on the solution!

", "votes": null, "replies": [] }, { "id": 1477437, "author_name": "mansoorabbasi", "author_url": "", "post_date": "08/17/2021 13:32:01", "content": "

Great Work …Sorry for the loss of your mother
\nMay your mother's soul be blessed
\nKeep the hard work up…….

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1464554": "First of all I would like to thank my teammates @nischaydnk and @shivamcyborg without whom this achievement would not have been possible. We merged on On 12th July and from then onwards we explored and implemented a lot of new ideas.\n\nI would also like to thank Kaggle, SIIM, FISABIO, RSNA for this great competition, as well has everyone here who has shared great discussions and notebooks. Congrats to all the winners!\n\nTrusting our maximised cv score worked once again for us. Most of our time were spent on Classification models which were the reason we secured a decent position on leaderboard. \n\nWe did not use any external training data. All the models were trained on the competition data after removing the duplicates as suggested [here](https://www.kaggle.com/c/siim-covid19-detection/discussion/246597).\n\n**Cross-validation:** Stratified Group KFOLD\n\n**Part 1: Study**\n\n1. We used an ensemble of 11 models.\n2. Base architectures = *Efficientnet v2m, Efficientnet v2l, Efficientnet B5, Efficientnet B7*\n3. Models were trained on different image sizes ranging from [512, 720] and different augmentations.\n4. Segmentation as aux loss suggested by @hengck23 \n5. v2 models used a custom [pcam](https://github.com/jfhealthcare/Chexpert) pool head + attention.\n6. b5, b7 used multi head + concat pool + attention suggested by @ttahara .\n7. We also used noisy student training method suggested by @moewie94 in [here](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/226616).\n8. Hflip TTA was used during inference.\n\n\n**Part 2: Binary (`none` class predictions)**\n\n1. We used an ensemble of 2x five fold models (efficientnetv2m , efficientnetb6)\n2. Models were trained on 512 image size.\n3. Segmentation as aux loss suggested by @hengck23 \n4. v2m models used a custom [pcam](https://github.com/jfhealthcare/Chexpert) pool head + attention.\n5. b6 was trained using the pipeline provided by @hengck23.\n6. v2m model as actually trained on 4 class study data. We used the `negative` predictions as `none` and merged these with the `binary` predictions from the above models.\n7. Hflip TTA was used during inference.\n\n\n**Part 3: Detection (`opacity` predictions)**\n\n1. We used an ensemble of 5x five fold models.\n2. Models used = `efficientdetD3, efficientdetD5, yolov5x, yolov5l6, retinanet_x101_64x4d_fpn`\n3. image sizes used = `896, 512, 620, 620, (1330, 800)`\n4. we first trained d5, d3, yolov5x, yolov5l6 on only opacity predictions. We used WBF (iou=0.62) to generate pseudo lables of public test. We then used these labels to train `d3, yolov5x, yolov5l6` models again. \n5. In the pseudo label training d3 was trained with ema and yolov5 models were trained with a few `none` images as well. \n6. D5 was trained with EMA from the beginning and pseudo labelling did not bring any improvements to both D5 and RetinanNet so we did not include them in the final ensemble.\n7. At inference predictions from the 5 models were merged using WBF(iou=0.625).\n\n\n**Part 4: Postprocessing**\n1. Power blending with maximum opacity confidence from part 3 and none predictions from part 2\n2. We optimized both `none` predictions as well as `opacity` confidences.\n3. This was inspired from @cdeotte 's [post](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637).\n\n\n*We have released our inference kernel : https://www.kaggle.com/nischaydnk/604e8587410a-v2m-bin-weighted and in the coming days we will release our full code as well. We didn't had time to prepare written solution during competitions, loads of work had to be done during the competition and also we need to clean our code a lot*\n\n**Update : Code will be available [here](https://github.com/benihime91/SIIM-COVID19-DETECTION-KAGGLE)**", "1464558": "**Our single models performance on oof and public leaderboard.**\n\"Screenshot-2021-08-10-at-9-25-54-PM\"", "1464729": "This is Ranzcr, vinbigdata and chestxpath solutions blended into one haha, I see @nischaydnk using all his experience. Well done @benihime91 great solution", "1464746": "Thanks a lot πŸ˜ƒ", "1465386": "Congrats on 5th place and thank you for sharing your solutions. I want to know what is \"EMA\" u mention in Detection section ? Also may i ask your private score on Study Level and Image Level", "1465592": "EMA is Exponential Moving Average \n- https://www.tensorflow.org/api_docs/python/tf/train/ExponentialMovingAverage\n- https://github.com/rwightman/pytorch-image-models/blob/master/timm/utils/model_ema.py\n\nAnd regarding the `private score on Study Level and Image Level`, we didn't really check them separately.", "1465722": "Thank you for anwsering", "1466127": "Thanks for sharing!\nQ: Can you explain what is Hflip TTA?", "1466183": "TTA is test time augmentation and Hflip is horizontal-flip.\n\nSo, basically during infernce we predicted results for the origninal image as well as the horizontally flipped image and took the mean of the predictions", "1466243": "Congratulation @benihime91. Thanks for sharing.", "1466518": "Thanks for the helpful information!", "1466707": "Congratulation πŸŽ‰. Would you be able to share the source of `efficientnetv2` ? We tried it but didn't work for us.", "1466727": "Congrats to you and your team as well. I am waiting on your solution.\n\nWhich part of v2 do you need ? We'll eventually be releasing all our code.", "1466739": "I just want to know the GitHub source. If you guys used **PyTorch** then it should be from **Timm** models.\nBtw I've published our solution [here](https://www.kaggle.com/c/siim-covid19-detection/discussion/264243)", "1466743": "Yes we indeed used PyTorch and timm models. I tried Tensorflow at the start but couldn't find a reliable way to add aux loss so I switched to PyTorch just before merging with my teammates.", "1466749": "thanks, for the info", "1468978": "Congrats @benihime91 and team, Axom'r naam aru agot loi ana!!", "1469145": "Congratulations!", "1470120": "Great job Brother! Very happy for you. Thanks for the details on the solution!", "1470961": "I saw your dedication to your parents and wanted to give you a big thumbs up! πŸ‘ Keep at it @benihime91 and team !", "1470978": "Thanks a lot 😊", "1471201": "Congrats πŸ‘πŸ»", "1477433": "Great job Brother! . Thanks for the details on the solution!", "1477437": "Great Work …Sorry for the loss of your mother\nMay your mother's soul be blessed\nKeep the hard work up......." }, "source": "meta" }