{
  "id": 264858,
  "title": "6th place solution",
  "url": "/competitions/siim-covid19-detection/writeups/quanta-ai-lab-6th-place-solution",
  "author_name": "",
  "post_date": "2021-08-26T08:02:09.300Z",
  "votes": 12,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I would like to thank Kaggle, SIIM, FISABIO, RSNA for hosting this exciting competition for saving many people's lives! I also thank to the other team members <a href=\"https://www.kaggle.com/sleepywill\" target=\"_blank\">@sleepywill</a> and <a href=\"https://www.kaggle.com/beckymo\" target=\"_blank\">@beckymo</a> and all Kagglers for great help. <br>\nI am very surprised noticing we get 6th place on private LB while we only rank 33rd place on public LB. If somebody has an idea why our model has such great potential on generalization, please kindly share your thoughts. 😂</p>\n<p><em>The following is our approach to this competition.</em></p>\n<p><strong>Solution Summary</strong><br>\n<a href=\"https://github.com/b02202050/2021-SIIM-COVID19-Detection#solution-summary\" target=\"_blank\">https://github.com/b02202050/2021-SIIM-COVID19-Detection#solution-summary</a></p>\n<p><strong>General Settings</strong></p>\n<ul>\n<li>Classification:<ul>\n<li>Architecture: EfficientNet-B7 (This is the only backbone architecture we use.)</li>\n<li>Input size: 512</li></ul></li>\n<li>Detection:<ul>\n<li>Architecture: FasterRCNN-EfficientNet-B7-FPN</li>\n<li>Loss: classical RPN and RoI loss</li>\n<li>Input size: 800</li></ul></li>\n</ul>\n<p><strong>Classification pre-training</strong></p>\n<ul>\n<li>Datasets: kaggle_chest_xray_covid19_pneumonia + kaggle_covidx_cxr2 + kaggle_chest_xray_pneumonia + kaggle_curated_chest_xray_image_dataset_for_covid19 + kaggle_covid19_xray_two_proposed_databases + kaggle_ricord_covid19_xray_positive_tests + CXR14 + CheXpert</li>\n<li>Architecture: Shared-backbone multi-head classification</li>\n<li>Loss: CE Loss</li>\n<li>Early stopping: We choose the model of the best average validation AUC of all pneumonia and COVID related tasks.</li>\n<li>Augmentations: Flip (H/V) + rotation + color jitter + shift + aspect ratio jitter + scale jitter</li>\n</ul>\n<p><strong>Study-level training</strong></p>\n<ul>\n<li>ensembles: 5-fold CV</li>\n<li>TTA: horizontal flip</li>\n<li>Training tricks:<ul>\n<li>Inverse focal loss: replace <code>(1-p_t) ** gamma</code> with <code>p_t ** gamma</code> in the original focal loss to suppress outlier samples</li>\n<li>Use sigmoid instead of softmax (increase ~0.003 mAP on my validation set)</li>\n<li>Sharpness-aware minimization</li>\n<li>Augmentations: Flip (H/V) + rotation + color jitter + shift + RandomResizedCrop + random_perspective + elastic_deformation + RandAugment</li></ul></li>\n</ul>\n<p><strong>Detection pre-training</strong></p>\n<ul>\n<li>Datasets: RSNA pneumonia</li>\n<li>Augmentations: Flip (H/V) + rotation + color jitter + shift</li>\n</ul>\n<p><strong>Image-level training</strong></p>\n<ul>\n<li>ensembles: 5-fold CV</li>\n<li>TTA: horizontal flip</li>\n<li>Training tricks:<ul>\n<li>Stochastic weight averaging</li>\n<li>Sharpness-aware minimization</li>\n<li>Attentional-guided context FPN</li>\n<li>Attentional feature fusion</li>\n<li>Fixed feature attention: use feature pyramid from the classification model for attention</li>\n<li>Augmentations: Flip (H/V) + rotation + color jitter + shift + RandomResizedCrop + random_perspective + elastic_deformation</li></ul></li>\n</ul>\n<p><strong>Results</strong></p>\n<table>\n<thead>\n<tr>\n<th>image-level ensemble mAP\\@0.5</th>\n<th>CV</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>w/o fixed feature attention</td>\n<td>0.563</td>\n</tr>\n<tr>\n<td>with fixed feature attention</td>\n<td>0.567</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>CV</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Study-level ensemble mAP</td>\n<td>0.585</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<table>\n<thead>\n<tr>\n<th>Study-level ensemble + Image-level ensemble</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>w/o TTA w/o fixed feature attention</td>\n<td>0.633</td>\n<td>0.621</td>\n</tr>\n<tr>\n<td>w/o TTA</td>\n<td>0.633</td>\n<td>0.625</td>\n</tr>\n<tr>\n<td>study-level TTA</td>\n<td>0.634</td>\n<td>0.626</td>\n</tr>\n<tr>\n<td>image-level TTA</td>\n<td>0.635</td>\n<td>0.627</td>\n</tr>\n<tr>\n<td>both TTA</td>\n<td>0.636</td>\n<td>0.628</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p>For more algorithmic and implementation details, please refer to:</p>\n<ul>\n<li>our code in this github repository: <a href=\"https://github.com/b02202050/2021-SIIM-COVID19-Detection\" target=\"_blank\">https://github.com/b02202050/2021-SIIM-COVID19-Detection</a></li>\n<li>our Kaggle submission kernel: <a href=\"https://www.kaggle.com/terenceythsu/siim-covid19-2021-6th-place?scriptVersionId=70677701\" target=\"_blank\">https://www.kaggle.com/terenceythsu/siim-covid19-2021-6th-place?scriptVersionId=70677701</a></li>\n</ul>",
  "messages": [
    {
      "id": "1470450",
      "postDate": "08/13/2021 14:17:39",
      "content": "<p>I would like to thank Kaggle, SIIM, FISABIO, RSNA for hosting this exciting competition for saving many people's lives! I also thank to the other team members <a href=\"https://www.kaggle.com/sleepywill\" target=\"_blank\">@sleepywill</a> and <a href=\"https://www.kaggle.com/beckymo\" target=\"_blank\">@beckymo</a> and all Kagglers for great help. <br>\nI am very surprised noticing we get 6th place on private LB while we only rank 33rd place on public LB. If somebody has an idea why our model has such great potential on generalization, please kindly share your thoughts. 😂</p>\n<p><em>The following is our approach to this competition.</em></p>\n<p><strong>Solution Summary</strong><br>\n<a href=\"https://github.com/b02202050/2021-SIIM-COVID19-Detection#solution-summary\" target=\"_blank\">https://github.com/b02202050/2021-SIIM-COVID19-Detection#solution-summary</a></p>\n<p><strong>General Settings</strong></p>\n<ul>\n<li>Classification:<ul>\n<li>Architecture: EfficientNet-B7 (This is the only backbone architecture we use.)</li>\n<li>Input size: 512</li></ul></li>\n<li>Detection:<ul>\n<li>Architecture: FasterRCNN-EfficientNet-B7-FPN</li>\n<li>Loss: classical RPN and RoI loss</li>\n<li>Input size: 800</li></ul></li>\n</ul>\n<p><strong>Classification pre-training</strong></p>\n<ul>\n<li>Datasets: kaggle_chest_xray_covid19_pneumonia + kaggle_covidx_cxr2 + kaggle_chest_xray_pneumonia + kaggle_curated_chest_xray_image_dataset_for_covid19 + kaggle_covid19_xray_two_proposed_databases + kaggle_ricord_covid19_xray_positive_tests + CXR14 + CheXpert</li>\n<li>Architecture: Shared-backbone multi-head classification</li>\n<li>Loss: CE Loss</li>\n<li>Early stopping: We choose the model of the best average validation AUC of all pneumonia and COVID related tasks.</li>\n<li>Augmentations: Flip (H/V) + rotation + color jitter + shift + aspect ratio jitter + scale jitter</li>\n</ul>\n<p><strong>Study-level training</strong></p>\n<ul>\n<li>ensembles: 5-fold CV</li>\n<li>TTA: horizontal flip</li>\n<li>Training tricks:<ul>\n<li>Inverse focal loss: replace <code>(1-p_t) ** gamma</code> with <code>p_t ** gamma</code> in the original focal loss to suppress outlier samples</li>\n<li>Use sigmoid instead of softmax (increase ~0.003 mAP on my validation set)</li>\n<li>Sharpness-aware minimization</li>\n<li>Augmentations: Flip (H/V) + rotation + color jitter + shift + RandomResizedCrop + random_perspective + elastic_deformation + RandAugment</li></ul></li>\n</ul>\n<p><strong>Detection pre-training</strong></p>\n<ul>\n<li>Datasets: RSNA pneumonia</li>\n<li>Augmentations: Flip (H/V) + rotation + color jitter + shift</li>\n</ul>\n<p><strong>Image-level training</strong></p>\n<ul>\n<li>ensembles: 5-fold CV</li>\n<li>TTA: horizontal flip</li>\n<li>Training tricks:<ul>\n<li>Stochastic weight averaging</li>\n<li>Sharpness-aware minimization</li>\n<li>Attentional-guided context FPN</li>\n<li>Attentional feature fusion</li>\n<li>Fixed feature attention: use feature pyramid from the classification model for attention</li>\n<li>Augmentations: Flip (H/V) + rotation + color jitter + shift + RandomResizedCrop + random_perspective + elastic_deformation</li></ul></li>\n</ul>\n<p><strong>Results</strong></p>\n<table>\n<thead>\n<tr>\n<th>image-level ensemble mAP\\@0.5</th>\n<th>CV</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>w/o fixed feature attention</td>\n<td>0.563</td>\n</tr>\n<tr>\n<td>with fixed feature attention</td>\n<td>0.567</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>CV</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Study-level ensemble mAP</td>\n<td>0.585</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<table>\n<thead>\n<tr>\n<th>Study-level ensemble + Image-level ensemble</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>w/o TTA w/o fixed feature attention</td>\n<td>0.633</td>\n<td>0.621</td>\n</tr>\n<tr>\n<td>w/o TTA</td>\n<td>0.633</td>\n<td>0.625</td>\n</tr>\n<tr>\n<td>study-level TTA</td>\n<td>0.634</td>\n<td>0.626</td>\n</tr>\n<tr>\n<td>image-level TTA</td>\n<td>0.635</td>\n<td>0.627</td>\n</tr>\n<tr>\n<td>both TTA</td>\n<td>0.636</td>\n<td>0.628</td>\n</tr>\n</tbody>\n</table>\n<hr>\n<p>For more algorithmic and implementation details, please refer to:</p>\n<ul>\n<li>our code in this github repository: <a href=\"https://github.com/b02202050/2021-SIIM-COVID19-Detection\" target=\"_blank\">https://github.com/b02202050/2021-SIIM-COVID19-Detection</a></li>\n<li>our Kaggle submission kernel: <a href=\"https://www.kaggle.com/terenceythsu/siim-covid19-2021-6th-place?scriptVersionId=70677701\" target=\"_blank\">https://www.kaggle.com/terenceythsu/siim-covid19-2021-6th-place?scriptVersionId=70677701</a></li>\n</ul>",
      "rawMarkdown": "I would like to thank Kaggle, SIIM, FISABIO, RSNA for hosting this exciting competition for saving many people's lives! I also thank to the other team members @sleepywill and @beckymo and all Kagglers for great help. \nI am very surprised noticing we get 6th place on private LB while we only rank 33rd place on public LB. If somebody has an idea why our model has such great potential on generalization, please kindly share your thoughts. 😂\n\n*The following is our approach to this competition.*\n\n**Solution Summary**\nhttps://github.com/b02202050/2021-SIIM-COVID19-Detection#solution-summary\n\n**General Settings**\n* Classification:\n  * Architecture: EfficientNet-B7 (This is the only backbone architecture we use.)\n  * Input size: 512\n* Detection:\n  * Architecture: FasterRCNN-EfficientNet-B7-FPN\n  * Loss: classical RPN and RoI loss\n  * Input size: 800\n\n**Classification pre-training**\n* Datasets: kaggle_chest_xray_covid19_pneumonia + kaggle_covidx_cxr2 + kaggle_chest_xray_pneumonia + kaggle_curated_chest_xray_image_dataset_for_covid19 + kaggle_covid19_xray_two_proposed_databases + kaggle_ricord_covid19_xray_positive_tests + CXR14 + CheXpert\n* Architecture: Shared-backbone multi-head classification\n* Loss: CE Loss\n* Early stopping: We choose the model of the best average validation AUC of all pneumonia and COVID related tasks.\n* Augmentations: Flip (H/V) + rotation + color jitter + shift + aspect ratio jitter + scale jitter\n\n**Study-level training**\n* ensembles: 5-fold CV\n* TTA: horizontal flip\n* Training tricks:\n  * Inverse focal loss: replace `(1-p_t) ** gamma` with `p_t ** gamma` in the original focal loss to suppress outlier samples\n  * Use sigmoid instead of softmax (increase ~0.003 mAP on my validation set)\n  * Sharpness-aware minimization\n  * Augmentations: Flip (H/V) + rotation + color jitter + shift + RandomResizedCrop + random_perspective + elastic_deformation + RandAugment\n\n**Detection pre-training**\n* Datasets: RSNA pneumonia\n* Augmentations: Flip (H/V) + rotation + color jitter + shift\n\n**Image-level training**\n* ensembles: 5-fold CV\n* TTA: horizontal flip\n* Training tricks:\n  * Stochastic weight averaging\n  * Sharpness-aware minimization\n  * Attentional-guided context FPN\n  * Attentional feature fusion\n  * Fixed feature attention: use feature pyramid from the classification model for attention\n  * Augmentations: Flip (H/V) + rotation + color jitter + shift + RandomResizedCrop + random_perspective + elastic_deformation\n\n**Results**\n\n|  image-level ensemble mAP\\@0.5  |  CV |\n| ---------------------------- | -------------- |\n| w/o fixed feature attention  | 0.563          |\n| with fixed feature attention | 0.567          |\n\n<hr>\n\n|  |  CV |\n| -------- | -------------- |\n| Study-level ensemble mAP | 0.585          |\n\n<hr>\n\n| Study-level ensemble + Image-level ensemble | Public LB | Private LB |\n| ------------------------------------------- | --------- | ---------- |\n| w/o TTA w/o fixed feature attention         | 0.633     | 0.621      |\n| w/o TTA                                     | 0.633     | 0.625      |\n| study-level TTA                             | 0.634     | 0.626      |\n| image-level TTA                             | 0.635     | 0.627      |\n| both TTA                                    | 0.636     | 0.628      |\n\n<hr>\n\nFor more algorithmic and implementation details, please refer to:\n* our code in this github repository: https://github.com/b02202050/2021-SIIM-COVID19-Detection\n* our Kaggle submission kernel: https://www.kaggle.com/terenceythsu/siim-covid19-2021-6th-place?scriptVersionId=70677701",
      "votes": null
    },
    {
      "id": "1470674",
      "postDate": "08/13/2021 16:45:33",
      "content": "<p>Congratulation~~~</p>\n<p>Could you share your checkpoints ?</p>",
      "rawMarkdown": "Congratulation~~~\n\nCould you share your checkpoints ?",
      "votes": null
    },
    {
      "id": "1471122",
      "postDate": "08/14/2021 01:15:24",
      "content": "<p>Hi, I may release the checkpoints after getting the permission from the host. If you want to quickly train a model that can be run with our inference kernel, you can ignore the external datasets pre-training. Also modify the config files to ignore loading pre-trained weights as described in my repository, Thanks.</p>",
      "rawMarkdown": "Hi, I may release the checkpoints after getting the permission from the host. If you want to quickly train a model that can be run with our inference kernel, you can ignore the external datasets pre-training. Also modify the config files to ignore loading pre-trained weights as described in my repository, Thanks.",
      "votes": null
    },
    {
      "id": "1471125",
      "postDate": "08/14/2021 01:29:06",
      "content": "<p>What a nice work !!<br>\nI have some questions, it would be nice if you can give me more detail on how your pretrain the clf model</p>\n<ol>\n<li>Did you use all the external dataset to pretrain at once?</li>\n<li>How did you synchronize the label between datasets?</li>\n<li>Multihead classification means each dataset has its own label set. And you created N classifcation heads for N datasets, right? And early stopping was based on the avg AUC of pneumonia and covid-related tasks?</li>\n</ol>",
      "rawMarkdown": "What a nice work !!\nI have some questions, it would be nice if you can give me more detail on how your pretrain the clf model\n1. Did you use all the external dataset to pretrain at once?\n2. How did you synchronize the label between datasets?\n3. Multihead classification means each dataset has its own label set. And you created N classifcation heads for N datasets, right? And early stopping was based on the avg AUC of pneumonia and covid-related tasks?",
      "votes": null
    },
    {
      "id": "1471181",
      "postDate": "08/14/2021 02:43:05",
      "content": "<p>Thanks for your appreciation.</p>\n<ol>\n<li>Yes. Each batch of images are sampled from all datasets and trained for each iteration.</li>\n<li>I map some similar pathologies from CheXpert to CXR14 in <a href=\"https://github.com/b02202050/2021-SIIM-COVID19-Detection/blob/main/dataset/CheXpert-v1.0/CheXpert_task_map_to_NIH.txt\" target=\"_blank\">this file</a>.</li>\n<li><ol>\n<li>Except that some CheXpert pathologies may share heads with CXR14, all the other dataset has their own classifier head.</li>\n<li>Yes, I use CXR14, CheXpert pnaumonia and all the other Kaggle datasets for validation. Here I made <a href=\"https://github.com/b02202050/2021-SIIM-COVID19-Detection/blob/57e7875ffabd4db8251181d9014bb71837336973/src/classification/train_multitask_classification.py#L307\" target=\"_blank\">a mistake</a> that I only use \"AUROC of class 1\" even if there exists multiple classes.</li></ol></li>\n</ol>",
      "rawMarkdown": "Thanks for your appreciation.\n1. Yes. Each batch of images are sampled from all datasets and trained for each iteration.\n1. I map some similar pathologies from CheXpert to CXR14 in [this file](https://github.com/b02202050/2021-SIIM-COVID19-Detection/blob/main/dataset/CheXpert-v1.0/CheXpert_task_map_to_NIH.txt).\n1. \n  1. Except that some CheXpert pathologies may share heads with CXR14, all the other dataset has their own classifier head.\n  1. Yes, I use CXR14, CheXpert pnaumonia and all the other Kaggle datasets for validation. Here I made [a mistake](https://github.com/b02202050/2021-SIIM-COVID19-Detection/blob/57e7875ffabd4db8251181d9014bb71837336973/src/classification/train_multitask_classification.py#L307) that I only use \"AUROC of class 1\" even if there exists multiple classes.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1470674,
      "author_name": "atom1231",
      "author_url": "",
      "post_date": "08/13/2021 16:45:33",
      "content": "<p>Congratulation~~~</p>\n<p>Could you share your checkpoints ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1471122,
          "author_name": "terenceythsu",
          "author_url": "",
          "post_date": "08/14/2021 01:15:24",
          "content": "<p>Hi, I may release the checkpoints after getting the permission from the host. If you want to quickly train a model that can be run with our inference kernel, you can ignore the external datasets pre-training. Also modify the config files to ignore loading pre-trained weights as described in my repository, Thanks.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1471125,
      "author_name": "namgalielei",
      "author_url": "",
      "post_date": "08/14/2021 01:29:06",
      "content": "<p>What a nice work !!<br>\nI have some questions, it would be nice if you can give me more detail on how your pretrain the clf model</p>\n<ol>\n<li>Did you use all the external dataset to pretrain at once?</li>\n<li>How did you synchronize the label between datasets?</li>\n<li>Multihead classification means each dataset has its own label set. And you created N classifcation heads for N datasets, right? And early stopping was based on the avg AUC of pneumonia and covid-related tasks?</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 1471181,
          "author_name": "terenceythsu",
          "author_url": "",
          "post_date": "08/14/2021 02:43:05",
          "content": "<p>Thanks for your appreciation.</p>\n<ol>\n<li>Yes. Each batch of images are sampled from all datasets and trained for each iteration.</li>\n<li>I map some similar pathologies from CheXpert to CXR14 in <a href=\"https://github.com/b02202050/2021-SIIM-COVID19-Detection/blob/main/dataset/CheXpert-v1.0/CheXpert_task_map_to_NIH.txt\" target=\"_blank\">this file</a>.</li>\n<li><ol>\n<li>Except that some CheXpert pathologies may share heads with CXR14, all the other dataset has their own classifier head.</li>\n<li>Yes, I use CXR14, CheXpert pnaumonia and all the other Kaggle datasets for validation. Here I made <a href=\"https://github.com/b02202050/2021-SIIM-COVID19-Detection/blob/57e7875ffabd4db8251181d9014bb71837336973/src/classification/train_multitask_classification.py#L307\" target=\"_blank\">a mistake</a> that I only use \"AUROC of class 1\" even if there exists multiple classes.</li></ol></li>\n</ol>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1470450": "I would like to thank Kaggle, SIIM, FISABIO, RSNA for hosting this exciting competition for saving many people's lives! I also thank to the other team members @sleepywill and @beckymo and all Kagglers for great help. \nI am very surprised noticing we get 6th place on private LB while we only rank 33rd place on public LB. If somebody has an idea why our model has such great potential on generalization, please kindly share your thoughts. 😂\n\n*The following is our approach to this competition.*\n\n**Solution Summary**\nhttps://github.com/b02202050/2021-SIIM-COVID19-Detection#solution-summary\n\n**General Settings**\n* Classification:\n  * Architecture: EfficientNet-B7 (This is the only backbone architecture we use.)\n  * Input size: 512\n* Detection:\n  * Architecture: FasterRCNN-EfficientNet-B7-FPN\n  * Loss: classical RPN and RoI loss\n  * Input size: 800\n\n**Classification pre-training**\n* Datasets: kaggle_chest_xray_covid19_pneumonia + kaggle_covidx_cxr2 + kaggle_chest_xray_pneumonia + kaggle_curated_chest_xray_image_dataset_for_covid19 + kaggle_covid19_xray_two_proposed_databases + kaggle_ricord_covid19_xray_positive_tests + CXR14 + CheXpert\n* Architecture: Shared-backbone multi-head classification\n* Loss: CE Loss\n* Early stopping: We choose the model of the best average validation AUC of all pneumonia and COVID related tasks.\n* Augmentations: Flip (H/V) + rotation + color jitter + shift + aspect ratio jitter + scale jitter\n\n**Study-level training**\n* ensembles: 5-fold CV\n* TTA: horizontal flip\n* Training tricks:\n  * Inverse focal loss: replace `(1-p_t) ** gamma` with `p_t ** gamma` in the original focal loss to suppress outlier samples\n  * Use sigmoid instead of softmax (increase ~0.003 mAP on my validation set)\n  * Sharpness-aware minimization\n  * Augmentations: Flip (H/V) + rotation + color jitter + shift + RandomResizedCrop + random_perspective + elastic_deformation + RandAugment\n\n**Detection pre-training**\n* Datasets: RSNA pneumonia\n* Augmentations: Flip (H/V) + rotation + color jitter + shift\n\n**Image-level training**\n* ensembles: 5-fold CV\n* TTA: horizontal flip\n* Training tricks:\n  * Stochastic weight averaging\n  * Sharpness-aware minimization\n  * Attentional-guided context FPN\n  * Attentional feature fusion\n  * Fixed feature attention: use feature pyramid from the classification model for attention\n  * Augmentations: Flip (H/V) + rotation + color jitter + shift + RandomResizedCrop + random_perspective + elastic_deformation\n\n**Results**\n\n|  image-level ensemble mAP\\@0.5  |  CV |\n| ---------------------------- | -------------- |\n| w/o fixed feature attention  | 0.563          |\n| with fixed feature attention | 0.567          |\n\n<hr>\n\n|  |  CV |\n| -------- | -------------- |\n| Study-level ensemble mAP | 0.585          |\n\n<hr>\n\n| Study-level ensemble + Image-level ensemble | Public LB | Private LB |\n| ------------------------------------------- | --------- | ---------- |\n| w/o TTA w/o fixed feature attention         | 0.633     | 0.621      |\n| w/o TTA                                     | 0.633     | 0.625      |\n| study-level TTA                             | 0.634     | 0.626      |\n| image-level TTA                             | 0.635     | 0.627      |\n| both TTA                                    | 0.636     | 0.628      |\n\n<hr>\n\nFor more algorithmic and implementation details, please refer to:\n* our code in this github repository: https://github.com/b02202050/2021-SIIM-COVID19-Detection\n* our Kaggle submission kernel: https://www.kaggle.com/terenceythsu/siim-covid19-2021-6th-place?scriptVersionId=70677701",
    "1470674": "Congratulation~~~\n\nCould you share your checkpoints ?",
    "1471122": "Hi, I may release the checkpoints after getting the permission from the host. If you want to quickly train a model that can be run with our inference kernel, you can ignore the external datasets pre-training. Also modify the config files to ignore loading pre-trained weights as described in my repository, Thanks.",
    "1471125": "What a nice work !!\nI have some questions, it would be nice if you can give me more detail on how your pretrain the clf model\n1. Did you use all the external dataset to pretrain at once?\n2. How did you synchronize the label between datasets?\n3. Multihead classification means each dataset has its own label set. And you created N classifcation heads for N datasets, right? And early stopping was based on the avg AUC of pneumonia and covid-related tasks?",
    "1471181": "Thanks for your appreciation.\n1. Yes. Each batch of images are sampled from all datasets and trained for each iteration.\n1. I map some similar pathologies from CheXpert to CXR14 in [this file](https://github.com/b02202050/2021-SIIM-COVID19-Detection/blob/main/dataset/CheXpert-v1.0/CheXpert_task_map_to_NIH.txt).\n1. \n  1. Except that some CheXpert pathologies may share heads with CXR14, all the other dataset has their own classifier head.\n  1. Yes, I use CXR14, CheXpert pnaumonia and all the other Kaggle datasets for validation. Here I made [a mistake](https://github.com/b02202050/2021-SIIM-COVID19-Detection/blob/57e7875ffabd4db8251181d9014bb71837336973/src/classification/train_multitask_classification.py#L307) that I only use \"AUROC of class 1\" even if there exists multiple classes."
  },
  "source": "meta"
}