{
  "id": 263674,
  "title": "2nd place summary",
  "url": "/competitions/siim-covid19-detection/writeups/a-team-2nd-place-summary",
  "author_name": "",
  "post_date": "2021-08-18T05:12:50.157Z",
  "votes": 76,
  "comment_count": 34,
  "views": 0,
  "content": "<p>Thanks to SIIM, FISABIO, RSNA and Kaggle for hosting this interesting competition. I am grateful to my teammates <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> <a href=\"https://www.kaggle.com/dvtoan7997\" target=\"_blank\">@dvtoan7997</a> and <a href=\"https://www.kaggle.com/underwearfitting\" target=\"_blank\">@underwearfitting</a> all did a very good job and finally we finished in 2nd place. </p>\n<p><em>The following is our approach to this competition.</em></p>\n<p><strong>Study level:</strong> we mainly work on 2 pipelines and then ensemble.</p>\n<p><strong><em>Pipeline 1:</em></strong> NFnet, Cait: eca_nfnet_l1, eca_nfnet_l2, dm_nfnet_f2, dm_nfnet_f3, cait_xs24_384.</p>\n<ul>\n<li>Pretraining on NIH dataset: Similar to other team, pre-training improves our CV and LB</li>\n<li>5 class classification: we added none as a 5th class.</li>\n<li>We didn’t use aux heads in this pipeline since it does not improve our CV score.</li>\n<li>To add diversity we use different preprocessing methods (histogram equalization, ben’s preprocessing, and adding a lung segmentation channel) and different model architectures: </li>\n<li>No external data</li>\n<li>All model are trained at 384x384</li>\n</ul>\n<p><strong><em>Pipeline 2:</em></strong> EfficientnetV2m </p>\n<ul>\n<li>Use segmentation aux head</li>\n<li>To add diversity we use 5 different augmentation, 4 models trained with external data and BCE loss, 5 models training with CE loss.</li>\n<li>Model are trained at 512x512.</li>\n</ul>\n<p><strong>Image level</strong><br>\n<strong><em>None:</em></strong> We ensemble none probability from 3 sources.</p>\n<ul>\n<li>2 class classification model to detect none</li>\n<li>None probability from pipeline 1.</li>\n<li>Negative probability from pipeline 2.</li>\n</ul>\n<p><strong><em>Opacity:</em></strong></p>\n<ul>\n<li>Yolov5: we use 5 different backbone (resnet52, resnet101, yolov5m, yolov5x, eca_nfnet_l0). All models are train at 384x384</li>\n<li>YoloX: We use 2 backbone (yolox-m and yolox-d). Model are trained at 384x384</li>\n<li>EffDet D5: trained at 512x512</li>\n</ul>\n<h1>Result</h1>\n<p><strong>Study Level</strong></p>\n<table>\n<thead>\n<tr>\n<th>Pipeline</th>\n<th>CV</th>\n<th>Public LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Pipeline 1</td>\n<td>0.596</td>\n<td>0.406</td>\n</tr>\n<tr>\n<td>Pipeline 2</td>\n<td>0.598</td>\n<td>0.407</td>\n</tr>\n<tr>\n<td>Ensemble</td>\n<td>0.604</td>\n<td>0.410</td>\n</tr>\n</tbody>\n</table>\n<p><strong>None</strong></p>\n<table>\n<thead>\n<tr>\n<th>Pipeline</th>\n<th>CV</th>\n<th>Public LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Ensemble</td>\n<td>0.822</td>\n<td>0.134</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Opacity</strong></p>\n<table>\n<thead>\n<tr>\n<th>Pipeline</th>\n<th>CV</th>\n<th>Public LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>yolov5</td>\n<td>0.56</td>\n<td>0.098</td>\n</tr>\n<tr>\n<td>yolox</td>\n<td>0.55</td>\n<td>-</td>\n</tr>\n<tr>\n<td>effdetD5</td>\n<td>0.53</td>\n<td>0.093</td>\n</tr>\n<tr>\n<td>Ensemble</td>\n<td>0.59</td>\n<td>0.100</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Final submission</strong></p>\n<table>\n<thead>\n<tr>\n<th>Pipeline</th>\n<th>None CV</th>\n<th>opacity CV</th>\n<th>study level CV</th>\n<th>CV</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>ensemble</td>\n<td>0.82</td>\n<td>0.59</td>\n<td>0.604</td>\n<td>0.636</td>\n<td>0.645</td>\n<td>0.634</td>\n</tr>\n</tbody>\n</table>\n<h1>Source code:</h1>\n<ul>\n<li>Training: <a href=\"https://github.com/nvnnghia/siim2021\" target=\"_blank\">https://github.com/nvnnghia/siim2021</a></li>\n<li>Inference: <a href=\"https://www.kaggle.com/nvnnghia/siim2021-final-sub2\" target=\"_blank\">https://www.kaggle.com/nvnnghia/siim2021-final-sub2</a></li>\n</ul>",
  "messages": [
    {
      "id": "1462755",
      "postDate": "08/10/2021 01:47:06",
      "content": "<p>Thanks to SIIM, FISABIO, RSNA and Kaggle for hosting this interesting competition. I am grateful to my teammates <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> <a href=\"https://www.kaggle.com/dvtoan7997\" target=\"_blank\">@dvtoan7997</a> and <a href=\"https://www.kaggle.com/underwearfitting\" target=\"_blank\">@underwearfitting</a> all did a very good job and finally we finished in 2nd place. </p>\n<p><em>The following is our approach to this competition.</em></p>\n<p><strong>Study level:</strong> we mainly work on 2 pipelines and then ensemble.</p>\n<p><strong><em>Pipeline 1:</em></strong> NFnet, Cait: eca_nfnet_l1, eca_nfnet_l2, dm_nfnet_f2, dm_nfnet_f3, cait_xs24_384.</p>\n<ul>\n<li>Pretraining on NIH dataset: Similar to other team, pre-training improves our CV and LB</li>\n<li>5 class classification: we added none as a 5th class.</li>\n<li>We didn’t use aux heads in this pipeline since it does not improve our CV score.</li>\n<li>To add diversity we use different preprocessing methods (histogram equalization, ben’s preprocessing, and adding a lung segmentation channel) and different model architectures: </li>\n<li>No external data</li>\n<li>All model are trained at 384x384</li>\n</ul>\n<p><strong><em>Pipeline 2:</em></strong> EfficientnetV2m </p>\n<ul>\n<li>Use segmentation aux head</li>\n<li>To add diversity we use 5 different augmentation, 4 models trained with external data and BCE loss, 5 models training with CE loss.</li>\n<li>Model are trained at 512x512.</li>\n</ul>\n<p><strong>Image level</strong><br>\n<strong><em>None:</em></strong> We ensemble none probability from 3 sources.</p>\n<ul>\n<li>2 class classification model to detect none</li>\n<li>None probability from pipeline 1.</li>\n<li>Negative probability from pipeline 2.</li>\n</ul>\n<p><strong><em>Opacity:</em></strong></p>\n<ul>\n<li>Yolov5: we use 5 different backbone (resnet52, resnet101, yolov5m, yolov5x, eca_nfnet_l0). All models are train at 384x384</li>\n<li>YoloX: We use 2 backbone (yolox-m and yolox-d). Model are trained at 384x384</li>\n<li>EffDet D5: trained at 512x512</li>\n</ul>\n<h1>Result</h1>\n<p><strong>Study Level</strong></p>\n<table>\n<thead>\n<tr>\n<th>Pipeline</th>\n<th>CV</th>\n<th>Public LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Pipeline 1</td>\n<td>0.596</td>\n<td>0.406</td>\n</tr>\n<tr>\n<td>Pipeline 2</td>\n<td>0.598</td>\n<td>0.407</td>\n</tr>\n<tr>\n<td>Ensemble</td>\n<td>0.604</td>\n<td>0.410</td>\n</tr>\n</tbody>\n</table>\n<p><strong>None</strong></p>\n<table>\n<thead>\n<tr>\n<th>Pipeline</th>\n<th>CV</th>\n<th>Public LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Ensemble</td>\n<td>0.822</td>\n<td>0.134</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Opacity</strong></p>\n<table>\n<thead>\n<tr>\n<th>Pipeline</th>\n<th>CV</th>\n<th>Public LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>yolov5</td>\n<td>0.56</td>\n<td>0.098</td>\n</tr>\n<tr>\n<td>yolox</td>\n<td>0.55</td>\n<td>-</td>\n</tr>\n<tr>\n<td>effdetD5</td>\n<td>0.53</td>\n<td>0.093</td>\n</tr>\n<tr>\n<td>Ensemble</td>\n<td>0.59</td>\n<td>0.100</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Final submission</strong></p>\n<table>\n<thead>\n<tr>\n<th>Pipeline</th>\n<th>None CV</th>\n<th>opacity CV</th>\n<th>study level CV</th>\n<th>CV</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>ensemble</td>\n<td>0.82</td>\n<td>0.59</td>\n<td>0.604</td>\n<td>0.636</td>\n<td>0.645</td>\n<td>0.634</td>\n</tr>\n</tbody>\n</table>\n<h1>Source code:</h1>\n<ul>\n<li>Training: <a href=\"https://github.com/nvnnghia/siim2021\" target=\"_blank\">https://github.com/nvnnghia/siim2021</a></li>\n<li>Inference: <a href=\"https://www.kaggle.com/nvnnghia/siim2021-final-sub2\" target=\"_blank\">https://www.kaggle.com/nvnnghia/siim2021-final-sub2</a></li>\n</ul>",
      "rawMarkdown": "Thanks to SIIM, FISABIO, RSNA and Kaggle for hosting this interesting competition. I am grateful to my teammates @steamedsheep @haqishen @dvtoan7997 and @underwearfitting all did a very good job and finally we finished in 2nd place. \n\n*The following is our approach to this competition.*\n\n**Study level:** we mainly work on 2 pipelines and then ensemble.\n\n***Pipeline 1:*** NFnet, Cait: eca_nfnet_l1, eca_nfnet_l2, dm_nfnet_f2, dm_nfnet_f3, cait_xs24_384.\n- Pretraining on NIH dataset: Similar to other team, pre-training improves our CV and LB\n- 5 class classification: we added none as a 5th class.\n- We didn’t use aux heads in this pipeline since it does not improve our CV score.\n- To add diversity we use different preprocessing methods (histogram equalization, ben’s preprocessing, and adding a lung segmentation channel) and different model architectures: \n- No external data\n- All model are trained at 384x384\n\n***Pipeline 2:*** EfficientnetV2m \n- Use segmentation aux head\n- To add diversity we use 5 different augmentation, 4 models trained with external data and BCE loss, 5 models training with CE loss.\n- Model are trained at 512x512.\n\n**Image level**\n***None:*** We ensemble none probability from 3 sources.\n- 2 class classification model to detect none\n- None probability from pipeline 1.\n- Negative probability from pipeline 2.\n\n***Opacity:***\n- Yolov5: we use 5 different backbone (resnet52, resnet101, yolov5m, yolov5x, eca_nfnet_l0). All models are train at 384x384\n- YoloX: We use 2 backbone (yolox-m and yolox-d). Model are trained at 384x384\n- EffDet D5: trained at 512x512\n\n#Result\n**Study Level**\nPipeline| CV| Public LB \n--- | --- | --- \nPipeline 1| 0.596 | 0.406\nPipeline 2| 0.598 | 0.407\nEnsemble| 0.604| 0.410\n\n**None**\nPipeline| CV| Public LB \n--- | --- | --- \nEnsemble| 0.822| 0.134\n\n**Opacity**\nPipeline| CV| Public LB \n--- | --- | --- \nyolov5| 0.56| 0.098\nyolox| 0.55| -\neffdetD5| 0.53| 0.093\nEnsemble| 0.59| 0.100\n\n**Final submission**\nPipeline|None CV|opacity CV|study level CV| CV| Public LB | Private LB \n--- | --- | --- | --- | --- | --- \nensemble| 0.82| 0.59| 0.604| 0.636| 0.645| 0.634\n\n#Source code: \n* Training: https://github.com/nvnnghia/siim2021\n* Inference: https://www.kaggle.com/nvnnghia/siim2021-final-sub2",
      "votes": null
    },
    {
      "id": "1462762",
      "postDate": "08/10/2021 01:53:57",
      "content": "<p>Congratulation and thanks for sharing, 6 gold medals/1 year 💯</p>",
      "rawMarkdown": "Congratulation and thanks for sharing, 6 gold medals/1 year 💯",
      "votes": null
    },
    {
      "id": "1462770",
      "postDate": "08/10/2021 01:59:13",
      "content": "<p>Thanks. congrats to you too. winning a solo 1st place a second time.</p>",
      "rawMarkdown": "Thanks. congrats to you too. winning a solo 1st place a second time.",
      "votes": null
    },
    {
      "id": "1462776",
      "postDate": "08/10/2021 02:01:12",
      "content": "<p>Congratulations!</p>\n<p>thanks for the nice write-up. </p>\n<p>I think your write-up tells me where I have gone wrong.</p>\n<ul>\n<li>\"We didn’t use aux heads in this pipeline since it does not improve our CV score.\"</li>\n<li>I didn't get the improvement expected from external data maybe because I did not separate it from aux loss. I will try your method again.</li>\n</ul>\n<p>By the way, did you apply to pretraining to detection model as well?<br>\nyour results are close to mine.</p>\n<p>But other teams have slightly higher results and they reported using pertaining for detection model as well.</p>\n<p>Do you train with positive images (i.e. remove none images) only for detection?</p>",
      "rawMarkdown": "Congratulations!\n\nthanks for the nice write-up. \n\nI think your write-up tells me where I have gone wrong.\n- \"We didn’t use aux heads in this pipeline since it does not improve our CV score.\"\n- I didn't get the improvement expected from external data maybe because I did not separate it from aux loss. I will try your method again.\n\nBy the way, did you apply to pretraining to detection model as well?\nyour results are close to mine.\n\nBut other teams have slightly higher results and they reported using pertaining for detection model as well.\n\nDo you train with positive images (i.e. remove none images) only for detection?",
      "votes": null
    },
    {
      "id": "1462780",
      "postDate": "08/10/2021 02:03:53",
      "content": "<p>Thanks. <br>\nwe didn't apply pretraining to detection model. We use coco pretrained weights. </p>",
      "rawMarkdown": "Thanks. \nwe didn't apply pretraining to detection model. We use coco pretrained weights.",
      "votes": null
    },
    {
      "id": "1462825",
      "postDate": "08/10/2021 02:25:54",
      "content": "<p>Congrats <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> and team!</p>",
      "rawMarkdown": "Congrats @nvnnghia and team!",
      "votes": null
    },
    {
      "id": "1462914",
      "postDate": "08/10/2021 03:24:39",
      "content": "<p>Congratulations! See you again in another competition. I'm looking forward to teaming up with you someday.</p>",
      "rawMarkdown": "Congratulations! See you again in another competition. I'm looking forward to teaming up with you someday.",
      "votes": null
    },
    {
      "id": "1462947",
      "postDate": "08/10/2021 03:45:35",
      "content": "<blockquote>\n  <p>I didn't get the improvement expected from external data maybe because I did not separate it from aux loss. I will try your method again.</p>\n</blockquote>\n<p>In pipeline 2, I get 0.01 CV boost from pseudo label and mask of bimcv data.  </p>",
      "rawMarkdown": "> I didn't get the improvement expected from external data maybe because I did not separate it from aux loss. I will try your method again.\n\nIn pipeline 2, I get 0.01 CV boost from pseudo label and mask of bimcv data.",
      "votes": null
    },
    {
      "id": "1463009",
      "postDate": "08/10/2021 04:11:31",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/yujiariyasu\" target=\"_blank\">@yujiariyasu</a> . congrats to you too. See you in another competition.</p>",
      "rawMarkdown": "Thanks @yujiariyasu . congrats to you too. See you in another competition.",
      "votes": null
    },
    {
      "id": "1463341",
      "postDate": "08/10/2021 07:03:42",
      "content": "<p>That is great. Between our study level loss along hit 0.46 in Public LB. how does that transforms to yours. <br>\nand image level was sitting around 0.286 at public LB&gt;</p>",
      "rawMarkdown": "That is great. Between our study level loss along hit 0.46 in Public LB. how does that transforms to yours. \nand image level was sitting around 0.286 at public LB>",
      "votes": null
    },
    {
      "id": "1463568",
      "postDate": "08/10/2021 08:35:37",
      "content": "<p>Super interesting pipeline! Did you use simple 5 fold CV? If yes did you average the 5 mAPs or do one mAP for the combined OOF predictions?</p>",
      "rawMarkdown": "Super interesting pipeline! Did you use simple 5 fold CV? If yes did you average the 5 mAPs or do one mAP for the combined OOF predictions?",
      "votes": null
    },
    {
      "id": "1463571",
      "postDate": "08/10/2021 08:36:04",
      "content": "<p>Congrats 🎉 <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> <a href=\"https://www.kaggle.com/dvtoan7997\" target=\"_blank\">@dvtoan7997</a> and <a href=\"https://www.kaggle.com/underwearfitting\" target=\"_blank\">@underwearfitting</a>. <br>\nBtw will you guys be nominating yourselves for the <strong>Student Prize</strong>?</p>",
      "rawMarkdown": "Congrats 🎉 @nvnnghia @steamedsheep @haqishen @dvtoan7997 and @underwearfitting. \nBtw will you guys be nominating yourselves for the **Student Prize**?",
      "votes": null
    },
    {
      "id": "1463662",
      "postDate": "08/10/2021 09:19:42",
      "content": "<p>congratulations <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> </p>",
      "rawMarkdown": "congratulations @nvnnghia",
      "votes": null
    },
    {
      "id": "1463828",
      "postDate": "08/10/2021 10:48:38",
      "content": "<p>Yes, we use 5 fold CV and we calculate mAP once for the whole oof.</p>",
      "rawMarkdown": "Yes, we use 5 fold CV and we calculate mAP once for the whole oof.",
      "votes": null
    },
    {
      "id": "1463831",
      "postDate": "08/10/2021 10:49:47",
      "content": "<p>Congrats to you and team too <a href=\"https://www.kaggle.com/awsaf49\" target=\"_blank\">@awsaf49</a> <br>\nNo, we don't qualify for student prize.</p>",
      "rawMarkdown": "Congrats to you and team too @awsaf49 \nNo, we don't qualify for student prize.",
      "votes": null
    },
    {
      "id": "1463958",
      "postDate": "08/10/2021 11:59:20",
      "content": "<p>Thanks. Looking forward to teaming up someday :D</p>",
      "rawMarkdown": "Thanks. Looking forward to teaming up someday :D",
      "votes": null
    },
    {
      "id": "1464132",
      "postDate": "08/10/2021 13:09:52",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/nvnn\" target=\"_blank\">@nvnn</a> and thanks for the detailed explanation.</p>",
      "rawMarkdown": "Congratulations @nvnn and thanks for the detailed explanation.",
      "votes": null
    },
    {
      "id": "1464327",
      "postDate": "08/10/2021 14:34:25",
      "content": "<p>Congrats for 2nd place. May i ask your private score on each part: Pipeline 1, Pipeline 2, Final study level, 2class classification and Final image level ?</p>",
      "rawMarkdown": "Congrats for 2nd place. May i ask your private score on each part: Pipeline 1, Pipeline 2, Final study level, 2class classification and Final image level ?",
      "votes": null
    },
    {
      "id": "1464361",
      "postDate": "08/10/2021 14:45:14",
      "content": "<p>Hi. we didn't check private score for those parts. Most of our submission was for public test dataset only.</p>",
      "rawMarkdown": "Hi. we didn't check private score for those parts. Most of our submission was for public test dataset only.",
      "votes": null
    },
    {
      "id": "1464367",
      "postDate": "08/10/2021 14:46:16",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> 🎉🎉</p>",
      "rawMarkdown": "Congratulations @nvnnghia 🎉🎉",
      "votes": null
    },
    {
      "id": "1464385",
      "postDate": "08/10/2021 14:53:00",
      "content": "<p>Okie, thanks for anwsering</p>",
      "rawMarkdown": "Okie, thanks for anwsering",
      "votes": null
    },
    {
      "id": "1466102",
      "postDate": "08/11/2021 10:14:01",
      "content": "<p>Congrats <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> and team for 2nd place! </p>",
      "rawMarkdown": "Congrats @nvnnghia and team for 2nd place!",
      "votes": null
    },
    {
      "id": "1466149",
      "postDate": "08/11/2021 10:35:29",
      "content": "<p>Congrats@nvnnghia  , what optimizer and learning rate strategy did you use when training EfficientnetV2m ???</p>",
      "rawMarkdown": "Congrats@nvnnghia  , what optimizer and learning rate strategy did you use when training EfficientnetV2m ???",
      "votes": null
    },
    {
      "id": "1466651",
      "postDate": "08/11/2021 14:55:05",
      "content": "<p>Adam and CosineAnnealingLR with initial lr 0.0004 and train 16-24 epochs.</p>",
      "rawMarkdown": "Adam and CosineAnnealingLR with initial lr 0.0004 and train 16-24 epochs.",
      "votes": null
    },
    {
      "id": "1467314",
      "postDate": "08/11/2021 23:35:47",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> 👌👌</p>",
      "rawMarkdown": "Congratulations @nvnnghia 👌👌",
      "votes": null
    },
    {
      "id": "1467888",
      "postDate": "08/12/2021 07:06:13",
      "content": "<p>i am now doing experiments using pretained  NIH data. I could not verify if my results are correct.<br>\nHere are some questions:</p>\n<ol>\n<li>does the quality of classification of NIH 14 class affects Siim 4-class results?<br>\n2.how about using another  NIH 14class model to pseudo-label the Siim dataset. then use this pseudo-label as aux loss (e.g. KL loss )? should it work if pretraining is confirmed to work?</li>\n</ol>",
      "rawMarkdown": "i am now doing experiments using pretained  NIH data. I could not verify if my results are correct.\nHere are some questions:\n\n1. does the quality of classification of NIH 14 class affects Siim 4-class results?\n2.how about using another  NIH 14class model to pseudo-label the Siim dataset. then use this pseudo-label as aux loss (e.g. KL loss )? should it work if pretraining is confirmed to work?",
      "votes": null
    },
    {
      "id": "1468434",
      "postDate": "08/12/2021 12:20:29",
      "content": "<ul>\n<li>the quality of pretrained weights affects Siim 4-class. I got 0.36 AUC NIH 14-class, another model CV 0.29 has significant lower performance on Siim.</li>\n<li>Nice idea. I think it should work too.</li>\n</ul>",
      "rawMarkdown": "the quality of pretrained weights affects Siim 4-class. I got 0.36 AUC NIH 14-class, another model CV 0.29 has significant lower performance on Siim.\n- Nice idea. I think it should work too.",
      "votes": null
    },
    {
      "id": "1468459",
      "postDate": "08/12/2021 12:30:52",
      "content": "<p><a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> did you also use your modified wbf here too? We tried it here but it didn't give us any boost, perhaps we did it wrong…</p>",
      "rawMarkdown": "nvnnghia did you also use your modified wbf here too? We tried it here but it didn't give us any boost, perhaps we did it wrong...",
      "votes": null
    },
    {
      "id": "1468482",
      "postDate": "08/12/2021 12:45:31",
      "content": "<p>Congrats on your gold!</p>\n<blockquote>\n  <p>5 class classification: we added none as a 5th class.</p>\n</blockquote>\n<p>is there a difference between negative and none class?</p>",
      "rawMarkdown": "Congrats on your gold!\n> 5 class classification: we added none as a 5th class.\n\nis there a difference between negative and none class?",
      "votes": null
    },
    {
      "id": "1468493",
      "postDate": "08/12/2021 12:49:55",
      "content": "<p>yes. none includes some none-negative images (none-negative images without boxes).  we use has_box/not_has_box as 5th class</p>",
      "rawMarkdown": "yes. none includes some none-negative images (none-negative images without boxes).  we use has_box/not_has_box as 5th class",
      "votes": null
    },
    {
      "id": "1468494",
      "postDate": "08/12/2021 12:50:43",
      "content": "<p>yes. I use my modified wbf from past competition. </p>",
      "rawMarkdown": "yes. I use my modified wbf from past competition.",
      "votes": null
    },
    {
      "id": "1469868",
      "postDate": "08/13/2021 06:48:50",
      "content": "<p>Congratulations</p>",
      "rawMarkdown": "Congratulations",
      "votes": null
    },
    {
      "id": "1469984",
      "postDate": "08/13/2021 07:59:58",
      "content": "<p>congratulations <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> 👍👍👍</p>",
      "rawMarkdown": "congratulations @nvnnghia 👍👍👍",
      "votes": null
    },
    {
      "id": "1479172",
      "postDate": "08/18/2021 10:24:42",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> for sharing your solution. This is a very valuable information to learn from</p>",
      "rawMarkdown": "Thank you @nvnnghia for sharing your solution. This is a very valuable information to learn from",
      "votes": null
    },
    {
      "id": "1484593",
      "postDate": "08/21/2021 12:52:55",
      "content": "<p>Congratulation. Thanks for sharing.</p>",
      "rawMarkdown": "Congratulation. Thanks for sharing.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1462762,
      "author_name": "nguyenbadung",
      "author_url": "",
      "post_date": "08/10/2021 01:53:57",
      "content": "<p>Congratulation and thanks for sharing, 6 gold medals/1 year 💯</p>",
      "votes": null,
      "replies": [
        {
          "id": 1462770,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/10/2021 01:59:13",
          "content": "<p>Thanks. congrats to you too. winning a solo 1st place a second time.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1462776,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "08/10/2021 02:01:12",
      "content": "<p>Congratulations!</p>\n<p>thanks for the nice write-up. </p>\n<p>I think your write-up tells me where I have gone wrong.</p>\n<ul>\n<li>\"We didn’t use aux heads in this pipeline since it does not improve our CV score.\"</li>\n<li>I didn't get the improvement expected from external data maybe because I did not separate it from aux loss. I will try your method again.</li>\n</ul>\n<p>By the way, did you apply to pretraining to detection model as well?<br>\nyour results are close to mine.</p>\n<p>But other teams have slightly higher results and they reported using pertaining for detection model as well.</p>\n<p>Do you train with positive images (i.e. remove none images) only for detection?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1462780,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/10/2021 02:03:53",
          "content": "<p>Thanks. <br>\nwe didn't apply pretraining to detection model. We use coco pretrained weights. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1462947,
          "author_name": "steamedsheep",
          "author_url": "",
          "post_date": "08/10/2021 03:45:35",
          "content": "<blockquote>\n  <p>I didn't get the improvement expected from external data maybe because I did not separate it from aux loss. I will try your method again.</p>\n</blockquote>\n<p>In pipeline 2, I get 0.01 CV boost from pseudo label and mask of bimcv data.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1462825,
      "author_name": "duykhanh99",
      "author_url": "",
      "post_date": "08/10/2021 02:25:54",
      "content": "<p>Congrats <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> and team!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1462914,
      "author_name": "yujiariyasu",
      "author_url": "",
      "post_date": "08/10/2021 03:24:39",
      "content": "<p>Congratulations! See you again in another competition. I'm looking forward to teaming up with you someday.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1463009,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/10/2021 04:11:31",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/yujiariyasu\" target=\"_blank\">@yujiariyasu</a> . congrats to you too. See you in another competition.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1463341,
      "author_name": "jaideepvalani",
      "author_url": "",
      "post_date": "08/10/2021 07:03:42",
      "content": "<p>That is great. Between our study level loss along hit 0.46 in Public LB. how does that transforms to yours. <br>\nand image level was sitting around 0.286 at public LB&gt;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1463568,
      "author_name": "simon111",
      "author_url": "",
      "post_date": "08/10/2021 08:35:37",
      "content": "<p>Super interesting pipeline! Did you use simple 5 fold CV? If yes did you average the 5 mAPs or do one mAP for the combined OOF predictions?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1463828,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/10/2021 10:48:38",
          "content": "<p>Yes, we use 5 fold CV and we calculate mAP once for the whole oof.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1463571,
      "author_name": "awsaf49",
      "author_url": "",
      "post_date": "08/10/2021 08:36:04",
      "content": "<p>Congrats 🎉 <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> <a href=\"https://www.kaggle.com/steamedsheep\" target=\"_blank\">@steamedsheep</a> <a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> <a href=\"https://www.kaggle.com/dvtoan7997\" target=\"_blank\">@dvtoan7997</a> and <a href=\"https://www.kaggle.com/underwearfitting\" target=\"_blank\">@underwearfitting</a>. <br>\nBtw will you guys be nominating yourselves for the <strong>Student Prize</strong>?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1463831,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/10/2021 10:49:47",
          "content": "<p>Congrats to you and team too <a href=\"https://www.kaggle.com/awsaf49\" target=\"_blank\">@awsaf49</a> <br>\nNo, we don't qualify for student prize.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1463958,
          "author_name": "awsaf49",
          "author_url": "",
          "post_date": "08/10/2021 11:59:20",
          "content": "<p>Thanks. Looking forward to teaming up someday :D</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1463662,
      "author_name": "givkashi",
      "author_url": "",
      "post_date": "08/10/2021 09:19:42",
      "content": "<p>congratulations <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1464132,
      "author_name": "furcifer",
      "author_url": "",
      "post_date": "08/10/2021 13:09:52",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/nvnn\" target=\"_blank\">@nvnn</a> and thanks for the detailed explanation.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1464327,
      "author_name": "researchbntz",
      "author_url": "",
      "post_date": "08/10/2021 14:34:25",
      "content": "<p>Congrats for 2nd place. May i ask your private score on each part: Pipeline 1, Pipeline 2, Final study level, 2class classification and Final image level ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1464361,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/10/2021 14:45:14",
          "content": "<p>Hi. we didn't check private score for those parts. Most of our submission was for public test dataset only.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1464385,
          "author_name": "researchbntz",
          "author_url": "",
          "post_date": "08/10/2021 14:53:00",
          "content": "<p>Okie, thanks for anwsering</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1464367,
      "author_name": "jarupula",
      "author_url": "",
      "post_date": "08/10/2021 14:46:16",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> 🎉🎉</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1466102,
      "author_name": "promrcan1",
      "author_url": "",
      "post_date": "08/11/2021 10:14:01",
      "content": "<p>Congrats <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> and team for 2nd place! </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1466149,
      "author_name": "biglafe",
      "author_url": "",
      "post_date": "08/11/2021 10:35:29",
      "content": "<p>Congrats@nvnnghia  , what optimizer and learning rate strategy did you use when training EfficientnetV2m ???</p>",
      "votes": null,
      "replies": [
        {
          "id": 1466651,
          "author_name": "steamedsheep",
          "author_url": "",
          "post_date": "08/11/2021 14:55:05",
          "content": "<p>Adam and CosineAnnealingLR with initial lr 0.0004 and train 16-24 epochs.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1467314,
      "author_name": "iniestamoh",
      "author_url": "",
      "post_date": "08/11/2021 23:35:47",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> 👌👌</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1467888,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "08/12/2021 07:06:13",
      "content": "<p>i am now doing experiments using pretained  NIH data. I could not verify if my results are correct.<br>\nHere are some questions:</p>\n<ol>\n<li>does the quality of classification of NIH 14 class affects Siim 4-class results?<br>\n2.how about using another  NIH 14class model to pseudo-label the Siim dataset. then use this pseudo-label as aux loss (e.g. KL loss )? should it work if pretraining is confirmed to work?</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 1468434,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/12/2021 12:20:29",
          "content": "<ul>\n<li>the quality of pretrained weights affects Siim 4-class. I got 0.36 AUC NIH 14-class, another model CV 0.29 has significant lower performance on Siim.</li>\n<li>Nice idea. I think it should work too.</li>\n</ul>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1468459,
      "author_name": "awsaf49",
      "author_url": "",
      "post_date": "08/12/2021 12:30:52",
      "content": "<p><a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> did you also use your modified wbf here too? We tried it here but it didn't give us any boost, perhaps we did it wrong…</p>",
      "votes": null,
      "replies": [
        {
          "id": 1468494,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/12/2021 12:50:43",
          "content": "<p>yes. I use my modified wbf from past competition. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1468482,
      "author_name": "kyoshioka47",
      "author_url": "",
      "post_date": "08/12/2021 12:45:31",
      "content": "<p>Congrats on your gold!</p>\n<blockquote>\n  <p>5 class classification: we added none as a 5th class.</p>\n</blockquote>\n<p>is there a difference between negative and none class?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1468493,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "08/12/2021 12:49:55",
          "content": "<p>yes. none includes some none-negative images (none-negative images without boxes).  we use has_box/not_has_box as 5th class</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1469868,
      "author_name": "emresengul",
      "author_url": "",
      "post_date": "08/13/2021 06:48:50",
      "content": "<p>Congratulations</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1469984,
      "author_name": "legendarypark",
      "author_url": "",
      "post_date": "08/13/2021 07:59:58",
      "content": "<p>congratulations <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> 👍👍👍</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1479172,
      "author_name": "mikecho",
      "author_url": "",
      "post_date": "08/18/2021 10:24:42",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/nvnnghia\" target=\"_blank\">@nvnnghia</a> for sharing your solution. This is a very valuable information to learn from</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1484593,
      "author_name": "minhtien1405",
      "author_url": "",
      "post_date": "08/21/2021 12:52:55",
      "content": "<p>Congratulation. Thanks for sharing.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1462755": "Thanks to SIIM, FISABIO, RSNA and Kaggle for hosting this interesting competition. I am grateful to my teammates @steamedsheep @haqishen @dvtoan7997 and @underwearfitting all did a very good job and finally we finished in 2nd place. \n\n*The following is our approach to this competition.*\n\n**Study level:** we mainly work on 2 pipelines and then ensemble.\n\n***Pipeline 1:*** NFnet, Cait: eca_nfnet_l1, eca_nfnet_l2, dm_nfnet_f2, dm_nfnet_f3, cait_xs24_384.\n- Pretraining on NIH dataset: Similar to other team, pre-training improves our CV and LB\n- 5 class classification: we added none as a 5th class.\n- We didn’t use aux heads in this pipeline since it does not improve our CV score.\n- To add diversity we use different preprocessing methods (histogram equalization, ben’s preprocessing, and adding a lung segmentation channel) and different model architectures: \n- No external data\n- All model are trained at 384x384\n\n***Pipeline 2:*** EfficientnetV2m \n- Use segmentation aux head\n- To add diversity we use 5 different augmentation, 4 models trained with external data and BCE loss, 5 models training with CE loss.\n- Model are trained at 512x512.\n\n**Image level**\n***None:*** We ensemble none probability from 3 sources.\n- 2 class classification model to detect none\n- None probability from pipeline 1.\n- Negative probability from pipeline 2.\n\n***Opacity:***\n- Yolov5: we use 5 different backbone (resnet52, resnet101, yolov5m, yolov5x, eca_nfnet_l0). All models are train at 384x384\n- YoloX: We use 2 backbone (yolox-m and yolox-d). Model are trained at 384x384\n- EffDet D5: trained at 512x512\n\n#Result\n**Study Level**\nPipeline| CV| Public LB \n--- | --- | --- \nPipeline 1| 0.596 | 0.406\nPipeline 2| 0.598 | 0.407\nEnsemble| 0.604| 0.410\n\n**None**\nPipeline| CV| Public LB \n--- | --- | --- \nEnsemble| 0.822| 0.134\n\n**Opacity**\nPipeline| CV| Public LB \n--- | --- | --- \nyolov5| 0.56| 0.098\nyolox| 0.55| -\neffdetD5| 0.53| 0.093\nEnsemble| 0.59| 0.100\n\n**Final submission**\nPipeline|None CV|opacity CV|study level CV| CV| Public LB | Private LB \n--- | --- | --- | --- | --- | --- \nensemble| 0.82| 0.59| 0.604| 0.636| 0.645| 0.634\n\n#Source code: \n* Training: https://github.com/nvnnghia/siim2021\n* Inference: https://www.kaggle.com/nvnnghia/siim2021-final-sub2",
    "1462762": "Congratulation and thanks for sharing, 6 gold medals/1 year 💯",
    "1462770": "Thanks. congrats to you too. winning a solo 1st place a second time.",
    "1462776": "Congratulations!\n\nthanks for the nice write-up. \n\nI think your write-up tells me where I have gone wrong.\n- \"We didn’t use aux heads in this pipeline since it does not improve our CV score.\"\n- I didn't get the improvement expected from external data maybe because I did not separate it from aux loss. I will try your method again.\n\nBy the way, did you apply to pretraining to detection model as well?\nyour results are close to mine.\n\nBut other teams have slightly higher results and they reported using pertaining for detection model as well.\n\nDo you train with positive images (i.e. remove none images) only for detection?",
    "1462780": "Thanks. \nwe didn't apply pretraining to detection model. We use coco pretrained weights.",
    "1462825": "Congrats @nvnnghia and team!",
    "1462914": "Congratulations! See you again in another competition. I'm looking forward to teaming up with you someday.",
    "1462947": "> I didn't get the improvement expected from external data maybe because I did not separate it from aux loss. I will try your method again.\n\nIn pipeline 2, I get 0.01 CV boost from pseudo label and mask of bimcv data.",
    "1463009": "Thanks @yujiariyasu . congrats to you too. See you in another competition.",
    "1463341": "That is great. Between our study level loss along hit 0.46 in Public LB. how does that transforms to yours. \nand image level was sitting around 0.286 at public LB>",
    "1463568": "Super interesting pipeline! Did you use simple 5 fold CV? If yes did you average the 5 mAPs or do one mAP for the combined OOF predictions?",
    "1463571": "Congrats 🎉 @nvnnghia @steamedsheep @haqishen @dvtoan7997 and @underwearfitting. \nBtw will you guys be nominating yourselves for the **Student Prize**?",
    "1463662": "congratulations @nvnnghia",
    "1463828": "Yes, we use 5 fold CV and we calculate mAP once for the whole oof.",
    "1463831": "Congrats to you and team too @awsaf49 \nNo, we don't qualify for student prize.",
    "1463958": "Thanks. Looking forward to teaming up someday :D",
    "1464132": "Congratulations @nvnn and thanks for the detailed explanation.",
    "1464327": "Congrats for 2nd place. May i ask your private score on each part: Pipeline 1, Pipeline 2, Final study level, 2class classification and Final image level ?",
    "1464361": "Hi. we didn't check private score for those parts. Most of our submission was for public test dataset only.",
    "1464367": "Congratulations @nvnnghia 🎉🎉",
    "1464385": "Okie, thanks for anwsering",
    "1466102": "Congrats @nvnnghia and team for 2nd place!",
    "1466149": "Congrats@nvnnghia  , what optimizer and learning rate strategy did you use when training EfficientnetV2m ???",
    "1466651": "Adam and CosineAnnealingLR with initial lr 0.0004 and train 16-24 epochs.",
    "1467314": "Congratulations @nvnnghia 👌👌",
    "1467888": "i am now doing experiments using pretained  NIH data. I could not verify if my results are correct.\nHere are some questions:\n\n1. does the quality of classification of NIH 14 class affects Siim 4-class results?\n2.how about using another  NIH 14class model to pseudo-label the Siim dataset. then use this pseudo-label as aux loss (e.g. KL loss )? should it work if pretraining is confirmed to work?",
    "1468434": "the quality of pretrained weights affects Siim 4-class. I got 0.36 AUC NIH 14-class, another model CV 0.29 has significant lower performance on Siim.\n- Nice idea. I think it should work too.",
    "1468459": "nvnnghia did you also use your modified wbf here too? We tried it here but it didn't give us any boost, perhaps we did it wrong...",
    "1468482": "Congrats on your gold!\n> 5 class classification: we added none as a 5th class.\n\nis there a difference between negative and none class?",
    "1468493": "yes. none includes some none-negative images (none-negative images without boxes).  we use has_box/not_has_box as 5th class",
    "1468494": "yes. I use my modified wbf from past competition.",
    "1469868": "Congratulations",
    "1469984": "congratulations @nvnnghia 👍👍👍",
    "1479172": "Thank you @nvnnghia for sharing your solution. This is a very valuable information to learn from",
    "1484593": "Congratulation. Thanks for sharing."
  },
  "source": "meta"
}