First of all, congratulations to all the winners, and thanks so much for hosting such an interesting competition.
\nAlso really appreciate for my teammates, @ryunosukeishizaki and @rinnqd !!
I'm an anesthesiologist in Japan.
\nCOVID-19 patients are increasing here and there, facing with many chest X-rays.
\nClassifying and detecting COVID-19 pneumonia by machine learning is quite interesting and meaningful, which is my motivation to participate in this task.
Here I’ll share my part of the solution.
\nWhat I did consists of mainly 3 parts, 2 types of baseline classification modeling & opacity detection modeling & post-processing.
\nAll experiments and submissions were done using 512x512 images.
\nI selected which images should be used for training, base on the discussion (https://www.kaggle.com/c/siim-covid19-detection/discussion/246597), picked up 1 image_id per study_id (if there are several images per study_id, only image with bbox are used)
\nCross Validation: multilabel stratified kfold
Surely models with aux-loss were better than ones without it.
\nMy baseline models are 2 types of classification EfficientNets with 5 classes (Negative, Typical, Indeterminate, Atypical, and None).
・Type1
\nclass Model(nn.Module):
\n def --init--(self):
\n super(Model, self).--init--()
\n ###blocks###
\n self.logit = nn.Linear(n_features, 5)
\n self.mask = …
\n def forward(self, x):
\n ###blocks###
\n mask = self.mask(x)
\n logit = self.logit(x)
\n return logit, mask
・Type2
\nclass Model(nn.Module):
\n def --init--(self):
\n super(Model, self).--init--()
\n ###blocks###
\n self.logit1 = nn.Linear(n_features, 4)
\n self.logit2 = nn.Sequential(
\n nn.Softmax(),
\n nn.Linear(4, 4),
\n nn.ReLU(),
\n nn.Linear(4, 1))
\n self.mask = …
\n def forward(self, x):
\n ###blocks###
\n logit1 = self.logit1(x)
\n logit2 = self.logit2(logit1)
\n return logit1, logit2, mask
I made custom loss function:
\nCCE * 0.8 (4 class loss) + BCE * 0.1 (1 class loss) + BCE * 0.1 (segmentation_loss)
Augmentations: horizontalflip, shiftscalerotate, randombrightness, cutout
\nAfter team merge, they improved my baseline models using pretraining method!
\nAlso @ryunosukeishizaki tried various types of Efnets, which was necessary for ensemble. Thank you.
\nFinal sub includes 4x efnetb3 and 3x efnetv2l.
Final Ensemble CV: 0.521 / 0.833 (5 folds 7 models ensemble)
\nPublic: ???
Because this competition was my first time to tackle object detection, I spent much time here.
\n5 folds 4 models were finally prepared: YOLOv5x, EfficientDetD3, EfficientDetD4, EfficientDetD5.
\nAfter pretraining all models with pneumonia detection dataset (https://www.kaggle.com/c/rsna-pneumonia-detection-challenge), each model training was done using train data I picked up above.
\nYOLO augmentations: degrees, translate, scale, fliplr
\nEfdet augmentations: horizontalflip, randombrightness, cutout
Each CV is the followings (5 folds mean):
\nYOLOv5x: 0.5428 / 1.0 (0.0904 / 0.1666)
\nEfdetD3: 0.5526 / 1.0 (0.0921 / 0.1666)
\nEfdetD4: 0.5436 / 1.0 (0.0906 / 0.1666)
\nEfdetD5: 0.5375 / 1.0 (0.0895 / 0.1666)
For ensemble, I used WBF to maximize the CV score.
\nWhile transforming bbox info into string, I removed unnecessary bboxes like too small, too big, around corner area, and boxes overriding the midline.
\nI set these values based on train data to maximize the CV score.
・small box
\n((xmax - xmin) * (ymax - ymin)) / (img_width * img_height) < 0.001
・large box
\n((xmax - xmin) * (ymax - ymin)) / (img_width * img_height) > 0.362
・edge box
\n((xmax + xmin)/2 < img_width * 0.05) | ((xmax + xmin)/2 > img_width * 0.95) | ((ymax + ymin)/2 < img_height * 0.05) | ((ymax + ymin)/2 > img_height * 0.95)
・horizontal box
\n(xmax - xmin) > img_width * 0.6
Final Ensemble CV: 0.0981 / 0.1666
\nPublic: 0.100
I hypothesized that classification and detection result should be mutually related.
\ntest[’Negative’] *= (1 - test[‘max_bbox_confidence’])
\ntest[’None’] *= (1 - test[‘max_bbox_confidence’])
\nThis improved the CV by 0.002.
And, Negative and None values were joined to one value per image.
\nnn_values = test['Negative'] * 0.55 + test['None'] * 0.45
\ntest['Negative'] = na_values
\ntest['None'] = nn_values
\nThis improved the CV by 0.001.
Final Ensemble CV: 0.525 / 0.833 (5 folds 7 models ensemble)
\nPublic: 0.537
CV: 0.525+0.098=0.623
\nPublic: 0.637
\nPrivate: 0.621
\nI was really confused about debugging process in using 2 timm versions inside the same notebook for >1wk, and @rinnqd kindly gave me a solution. Thank you.
・High resolution (1024x1024) for both classification and detection
\n・Pseudo labeling including external and public test
\n・Denoising & Relabeling
\n・Mixup
\n・BCELoss for classification
\n・Focal loss
\n・Lovasz hinge loss
\n・Label smoothing
\n・Image cropping around lung area using detection output
\n・lung segmentation with opacity
\n・Using detection result as input too (4 or 5 or 6 ch)
\n・4 class detection modeling to postprocess classification result
First of all, congratulations to all the winners, and thanks so much for hosting such an interesting competition.
\nAlso really appreciate for my teammates, @ryunosukeishizaki and @rinnqd !!
I'm an anesthesiologist in Japan.
\nCOVID-19 patients are increasing here and there, facing with many chest X-rays.
\nClassifying and detecting COVID-19 pneumonia by machine learning is quite interesting and meaningful, which is my motivation to participate in this task.
Here I’ll share my part of the solution.
\nWhat I did consists of mainly 3 parts, 2 types of baseline classification modeling & opacity detection modeling & post-processing.
\nAll experiments and submissions were done using 512x512 images.
\nI selected which images should be used for training, base on the discussion (https://www.kaggle.com/c/siim-covid19-detection/discussion/246597), picked up 1 image_id per study_id (if there are several images per study_id, only image with bbox are used)
\nCross Validation: multilabel stratified kfold
Surely models with aux-loss were better than ones without it.
\nMy baseline models are 2 types of classification EfficientNets with 5 classes (Negative, Typical, Indeterminate, Atypical, and None).
・Type1
\nclass Model(nn.Module):
\n def --init--(self):
\n super(Model, self).--init--()
\n ###blocks###
\n self.logit = nn.Linear(n_features, 5)
\n self.mask = …
\n def forward(self, x):
\n ###blocks###
\n mask = self.mask(x)
\n logit = self.logit(x)
\n return logit, mask
・Type2
\nclass Model(nn.Module):
\n def --init--(self):
\n super(Model, self).--init--()
\n ###blocks###
\n self.logit1 = nn.Linear(n_features, 4)
\n self.logit2 = nn.Sequential(
\n nn.Softmax(),
\n nn.Linear(4, 4),
\n nn.ReLU(),
\n nn.Linear(4, 1))
\n self.mask = …
\n def forward(self, x):
\n ###blocks###
\n logit1 = self.logit1(x)
\n logit2 = self.logit2(logit1)
\n return logit1, logit2, mask
I made custom loss function:
\nCCE * 0.8 (4 class loss) + BCE * 0.1 (1 class loss) + BCE * 0.1 (segmentation_loss)
Augmentations: horizontalflip, shiftscalerotate, randombrightness, cutout
\nAfter team merge, they improved my baseline models using pretraining method!
\nAlso @ryunosukeishizaki tried various types of Efnets, which was necessary for ensemble. Thank you.
\nFinal sub includes 4x efnetb3 and 3x efnetv2l.
Final Ensemble CV: 0.521 / 0.833 (5 folds 7 models ensemble)
\nPublic: ???
Because this competition was my first time to tackle object detection, I spent much time here.
\n5 folds 4 models were finally prepared: YOLOv5x, EfficientDetD3, EfficientDetD4, EfficientDetD5.
\nAfter pretraining all models with pneumonia detection dataset (https://www.kaggle.com/c/rsna-pneumonia-detection-challenge), each model training was done using train data I picked up above.
\nYOLO augmentations: degrees, translate, scale, fliplr
\nEfdet augmentations: horizontalflip, randombrightness, cutout
Each CV is the followings (5 folds mean):
\nYOLOv5x: 0.5428 / 1.0 (0.0904 / 0.1666)
\nEfdetD3: 0.5526 / 1.0 (0.0921 / 0.1666)
\nEfdetD4: 0.5436 / 1.0 (0.0906 / 0.1666)
\nEfdetD5: 0.5375 / 1.0 (0.0895 / 0.1666)
For ensemble, I used WBF to maximize the CV score.
\nWhile transforming bbox info into string, I removed unnecessary bboxes like too small, too big, around corner area, and boxes overriding the midline.
\nI set these values based on train data to maximize the CV score.
・small box
\n((xmax - xmin) * (ymax - ymin)) / (img_width * img_height) < 0.001
・large box
\n((xmax - xmin) * (ymax - ymin)) / (img_width * img_height) > 0.362
・edge box
\n((xmax + xmin)/2 < img_width * 0.05) | ((xmax + xmin)/2 > img_width * 0.95) | ((ymax + ymin)/2 < img_height * 0.05) | ((ymax + ymin)/2 > img_height * 0.95)
・horizontal box
\n(xmax - xmin) > img_width * 0.6
Final Ensemble CV: 0.0981 / 0.1666
\nPublic: 0.100
I hypothesized that classification and detection result should be mutually related.
\ntest[’Negative’] *= (1 - test[‘max_bbox_confidence’])
\ntest[’None’] *= (1 - test[‘max_bbox_confidence’])
\nThis improved the CV by 0.002.
And, Negative and None values were joined to one value per image.
\nnn_values = test['Negative'] * 0.55 + test['None'] * 0.45
\ntest['Negative'] = na_values
\ntest['None'] = nn_values
\nThis improved the CV by 0.001.
Final Ensemble CV: 0.525 / 0.833 (5 folds 7 models ensemble)
\nPublic: 0.537
CV: 0.525+0.098=0.623
\nPublic: 0.637
\nPrivate: 0.621
\nI was really confused about debugging process in using 2 timm versions inside the same notebook for >1wk, and @rinnqd kindly gave me a solution. Thank you.
・High resolution (1024x1024) for both classification and detection
\n・Pseudo labeling including external and public test
\n・Denoising & Relabeling
\n・Mixup
\n・BCELoss for classification
\n・Focal loss
\n・Lovasz hinge loss
\n・Label smoothing
\n・Image cropping around lung area using detection output
\n・lung segmentation with opacity
\n・Using detection result as input too (4 or 5 or 6 ch)
\n・4 class detection modeling to postprocess classification result
Is it possible for me to look at your code?
", "rawMarkdown": "Is it possible for me to look at your code?", "votes": null }, { "id": "1463552", "postDate": "08/10/2021 08:25:25", "content": "Thanks for you sharing and congrats on good finish. btw how much boost did you got with post-processing part on lb?
", "rawMarkdown": "Thanks for you sharing and congrats on good finish. btw how much boost did you got with post-processing part on lb?", "votes": null }, { "id": "1463562", "postDate": "08/10/2021 08:32:57", "content": "Thanks for the comment.
\nAbout +0.002 I guess.
Congratulations. 🎉
\nHow much boost did you got by adding classification(None) head on CV / LB?
", "rawMarkdown": "Congratulations. 🎉 \n\nHow much boost did you got by adding classification(None) head on CV / LB?", "votes": null }, { "id": "1467249", "postDate": "08/11/2021 22:04:22", "content": "Thank you!
\nActually score is almost the same, but we can save more inference time if predicted 5 classes at the same time than 4+1 ones.
Thanks for your reply. Nice Work!
", "rawMarkdown": "Thanks for your reply. Nice Work!", "votes": null }, { "id": "1470216", "postDate": "08/13/2021 11:33:29", "content": "https://www.kaggle.com/drtausamaru/final-submission-ver001?scriptVersionId=70376836
\nfinal submission notebook.
Is it possible for me to look at your code?
", "votes": null, "replies": [ { "id": 1470216, "author_name": "drtausamaru", "author_url": "", "post_date": "08/13/2021 11:33:29", "content": "https://www.kaggle.com/drtausamaru/final-submission-ver001?scriptVersionId=70376836
\nfinal submission notebook.
Thanks for you sharing and congrats on good finish. btw how much boost did you got with post-processing part on lb?
", "votes": null, "replies": [ { "id": 1463562, "author_name": "drtausamaru", "author_url": "", "post_date": "08/10/2021 08:32:57", "content": "Thanks for the comment.
\nAbout +0.002 I guess.
Congratulations. 🎉
\nHow much boost did you got by adding classification(None) head on CV / LB?
", "votes": null, "replies": [ { "id": 1467249, "author_name": "drtausamaru", "author_url": "", "post_date": "08/11/2021 22:04:22", "content": "Thank you!
\nActually score is almost the same, but we can save more inference time if predicted 5 classes at the same time than 4+1 ones.
Thanks for your reply. Nice Work!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1462693": "First of all, congratulations to all the winners, and thanks so much for hosting such an interesting competition.\nAlso really appreciate for my teammates, @ryunosukeishizaki and @rinnqd !!\n\nI'm an anesthesiologist in Japan.\nCOVID-19 patients are increasing here and there, facing with many chest X-rays.\nClassifying and detecting COVID-19 pneumonia by machine learning is quite interesting and meaningful, which is my motivation to participate in this task.\n\nHere I’ll share my part of the solution.\n\nWhat I did consists of mainly 3 parts, 2 types of baseline classification modeling & opacity detection modeling & post-processing.\n\nAll experiments and submissions were done using 512x512 images.\nI selected which images should be used for training, base on the discussion (https://www.kaggle.com/c/siim-covid19-detection/discussion/246597), picked up 1 image_id per study_id (if there are several images per study_id, only image with bbox are used)\nCross Validation: multilabel stratified kfold\n\n# Classification\nSurely models with aux-loss were better than ones without it.\nMy baseline models are 2 types of classification EfficientNets with 5 classes (Negative, Typical, Indeterminate, Atypical, and None).\n\n・Type1\nclass Model(nn.Module):\n def --init--(self):\n super(Model, self).--init--()\n ###blocks###\n self.logit = nn.Linear(n_features, 5)\n self.mask = ...\n def forward(self, x):\n ###blocks###\n mask = self.mask(x)\n logit = self.logit(x)\n return logit, mask\n\n・Type2\nclass Model(nn.Module):\n def --init--(self):\n super(Model, self).--init--()\n ###blocks###\n self.logit1 = nn.Linear(n_features, 4)\n self.logit2 = nn.Sequential(\n nn.Softmax(),\n nn.Linear(4, 4),\n nn.ReLU(),\n nn.Linear(4, 1))\n self.mask = ...\n def forward(self, x):\n ###blocks###\n logit1 = self.logit1(x)\n logit2 = self.logit2(logit1)\n return logit1, logit2, mask\n\nI made custom loss function:\nCCE * 0.8 (4 class loss) + BCE * 0.1 (1 class loss) + BCE * 0.1 (segmentation_loss)\n\nAugmentations: horizontalflip, shiftscalerotate, randombrightness, cutout\n\nAfter team merge, they improved my baseline models using pretraining method!\nAlso @ryunosukeishizaki tried various types of Efnets, which was necessary for ensemble. Thank you.\nFinal sub includes 4x efnetb3 and 3x efnetv2l.\n\n**Final Ensemble CV: 0.521 / 0.833 (5 folds 7 models ensemble)**\n**Public: ???**\n\n# Detection\nBecause this competition was my first time to tackle object detection, I spent much time here.\n5 folds 4 models were finally prepared: YOLOv5x, EfficientDetD3, EfficientDetD4, EfficientDetD5.\nAfter pretraining all models with pneumonia detection dataset (https://www.kaggle.com/c/rsna-pneumonia-detection-challenge), each model training was done using train data I picked up above.\nYOLO augmentations: degrees, translate, scale, fliplr\nEfdet augmentations: horizontalflip, randombrightness, cutout\n\nEach CV is the followings (5 folds mean):\nYOLOv5x: 0.5428 / 1.0 (0.0904 / 0.1666)\nEfdetD3: 0.5526 / 1.0 (0.0921 / 0.1666)\nEfdetD4: 0.5436 / 1.0 (0.0906 / 0.1666)\nEfdetD5: 0.5375 / 1.0 (0.0895 / 0.1666)\n\nFor ensemble, I used WBF to maximize the CV score.\nWhile transforming bbox info into string, I removed unnecessary bboxes like too small, too big, around corner area, and boxes overriding the midline.\nI set these values based on train data to maximize the CV score.\n\n・small box\n((xmax - xmin) * (ymax - ymin)) / (img_width * img_height) < 0.001\n\n・large box\n((xmax - xmin) * (ymax - ymin)) / (img_width * img_height) > 0.362\n\n・edge box\n((xmax + xmin)/2 < img_width * 0.05) | ((xmax + xmin)/2 > img_width * 0.95) | ((ymax + ymin)/2 < img_height * 0.05) | ((ymax + ymin)/2 > img_height * 0.95)\n\n・horizontal box\n(xmax - xmin) > img_width * 0.6\n\n**Final Ensemble CV: 0.0981 / 0.1666**\n**Public: 0.100**\n\n# Post-Processing\nI hypothesized that classification and detection result should be mutually related.\ntest[’Negative’] *= (1 - test[‘max_bbox_confidence’])\ntest[’None’] *= (1 - test[‘max_bbox_confidence’])\nThis improved the CV by 0.002.\n\nAnd, Negative and None values were joined to one value per image.\nnn_values = test['Negative'] * 0.55 + test['None'] * 0.45\ntest['Negative'] = na_values\ntest['None'] = nn_values\nThis improved the CV by 0.001.\n\n**Final Ensemble CV: 0.525 / 0.833 (5 folds 7 models ensemble)**\n**Public: 0.537**\n\n# Final Submission\nCV: 0.525+0.098=0.623\nPublic: 0.637\nPrivate: 0.621\nI was really confused about debugging process in using 2 timm versions inside the same notebook for >1wk, and @rinnqd kindly gave me a solution. Thank you.\n\n# What didn’t work for me\n・High resolution (1024x1024) for both classification and detection\n・Pseudo labeling including external and public test\n・Denoising & Relabeling\n・Mixup\n・BCELoss for classification\n・Focal loss\n・Lovasz hinge loss\n・Label smoothing\n・Image cropping around lung area using detection output\n・lung segmentation with opacity\n・Using detection result as input too (4 or 5 or 6 ch)\n・4 class detection modeling to postprocess classification result", "1463201": "Is it possible for me to look at your code?", "1463552": "Thanks for you sharing and congrats on good finish. btw how much boost did you got with post-processing part on lb?", "1463562": "Thanks for the comment.\nAbout +0.002 I guess.", "1466432": "Congratulations. 🎉 \n\nHow much boost did you got by adding classification(None) head on CV / LB?", "1467249": "Thank you!\nActually score is almost the same, but we can save more inference time if predicted 5 classes at the same time than 4+1 ones.", "1467421": "Thanks for your reply. Nice Work!", "1470216": "https://www.kaggle.com/drtausamaru/final-submission-ver001?scriptVersionId=70376836\nfinal submission notebook." }, "source": "meta" }