![\"\"](\"https://i.imgur.com/2gE74RX.png\")
First of all, I would really like to thank my teammates @yannmajewski and @mo5mami for their hard work and commitment . And I would like to thank all the competitors that shared their codes, ideas, discussions.
\nHere is the overview of our pipeline:
\n
Final submission(Public: 0.969, Private: 0.972)
\nWe used different image size for our models. [(512x512),(640x640),(736x736)]
Our plan is to choose one submission with setting weights referring to LB Scores , and another based on CV scores. But Both Scored 0.972 with some differences ( submission based on LB give us 36th Place)
\nTo Be Continued …
", "messages": [ { "id": 1242882, "postDate": "2021-03-17T22:07:29.593Z", "content": "First of all, I would really like to thank my teammates @yannmajewski and @mo5mami for their hard work and commitment . And I would like to thank all the competitors that shared their codes, ideas, discussions.
\nHere is the overview of our pipeline:
\n
Final submission(Public: 0.969, Private: 0.972)
\nWe used different image size for our models. [(512x512),(640x640),(736x736)]
Our plan is to choose one submission with setting weights referring to LB Scores , and another based on CV scores. But Both Scored 0.972 with some differences ( submission based on LB give us 36th Place)
\nTo Be Continued …
", "rawMarkdown": "First of all, I would really like to thank my teammates @yannmajewski and @mo5mami for their hard work and commitment . And I would like to thank all the competitors that shared their codes, ideas, discussions.\n\nHere is the overview of our pipeline:\n \n\n \n### Summary\n \nFinal submission(Public: 0.969, Private: 0.972) \nWe used different image size for our models. [(512x512),(640x640),(736x736)]\n\n * M1: Multi resnet200d (Public: 0.967, Private: 0.971) :\n * details will be provided later by @yannmajewski\nDual Resnet200d – Multi resolution input
\nThe idea of having multiple resolution images as input was to learn different features of the same image and combine all of them into one ensemble classifier to make one more robust prediction.
\nTraining pipeline (greatly inspired by ammarli):
\nStage 1:
\nStage 2:
\nStage 3:
\nRepeat stage 1 to stage 3 with different image resolutions. I trained 2 models, one with 640x640 image size and the other with 736x736 image size. More models could’ve been used but time was lacking.
\nFinal stage:
\nTraining augmentations used:
\n return A.Compose([\n A.RandomResizedCrop(img_size, img_size, scale=(0.9, 1.0)),\n A.CLAHE(clip_limit=4, p=0.5),\n A.HorizontalFlip(p=0.5),\n A.ShiftScaleRotate (shift_limit=0.025, scale_limit=0, rotate_limit=5,border_mode=0, p=0.5),\n A.OneOf([\n A.RandomContrast(p=1),\n A.RandomGamma(p=1),\n A.RandomBrightness(p=1),\n ], p=0.5),\n A.OneOf([\n A.MultiplicativeNoise(p=1),\n A.IAASharpen(p=1),\n ], p=0.5),\n A.Normalize(),\n ToTensorV2(),\n ])\nThanks to my teamates and everyone in this awesome kaggle community!
", "rawMarkdown": "**Dual Resnet200d – Multi resolution input**\n\nThe idea of having multiple resolution images as input was to learn different features of the same image and combine all of them into one ensemble classifier to make one more robust prediction.\n\n**Training pipeline (greatly inspired by ammarli):**\n\n**Stage 1:**\n- Train a resnet200d on competition training data. \n- Loss: BCELoss() + 0.1*AsymmetricLossMultiLabel() + FocalLoss()\n- 8 epochs\n- lr 3e-4\n\n**Stage 2:**\n- Use stage 1 model as teacher model to generate feature maps of NIH chest X ray dataset.\n- Use the teacher feature maps as labels to pretrain the student model.\n- Loss: MSELoss()\n- 3 epochs\n- lr 3e-4\n\n**Stage 3:**\n- Finetune the pretrained model on competition data.\n- CV:+-0.965 \n- Loss: BCELoss() + 0.1*AsymmetricLossMultiLabel() + FocalLoss()\n- 6 epochs\n- Lr 1e-4 to 5e-5 depending on batch size.\n\n**Repeat stage 1 to stage 3 with different image resolutions. I trained 2 models, one with 640x640 image size and the other with 736x736 image size. More models could’ve been used but time was lacking.**\n\n**Final stage:**\n\n- Freeze encoders and add a classifier that has concatenated embeddings as input.\n- Finally, finetune the classifier.\n- Loss: BCELoss() + 0.1*AsymmetricLossMultiLabel() + FocalLoss()\n- CV:+-0.9675 \n- Public lb: 0.967\n- Private lb: 0.971\n- 6 epochs\n- lr 1e-4 to 5e-5 depending on batch size.\n\n**Training augmentations used:**\n\n```\n return A.Compose([\n A.RandomResizedCrop(img_size, img_size, scale=(0.9, 1.0)),\n A.CLAHE(clip_limit=4, p=0.5),\n A.HorizontalFlip(p=0.5),\n A.ShiftScaleRotate (shift_limit=0.025, scale_limit=0, rotate_limit=5,border_mode=0, p=0.5),\n A.OneOf([\n A.RandomContrast(p=1),\n A.RandomGamma(p=1),\n A.RandomBrightness(p=1),\n ], p=0.5),\n A.OneOf([\n A.MultiplicativeNoise(p=1),\n A.IAASharpen(p=1),\n ], p=0.5),\n A.Normalize(),\n ToTensorV2(),\n ])\n```\n\nThanks to my teamates and everyone in this awesome kaggle community! \n", "votes": 6 }, { "id": 1242894, "postDate": "2021-03-17T22:15:39.453Z", "content": "Congrats on getting your first medal!
", "rawMarkdown": "Congrats on getting your first medal!", "votes": 4, "replies": [ { "id": 1242921, "postDate": "2021-03-17T22:57:26.250Z", "content": "Thanks a lot @underwearfitting For you Contribution in Almost Kaggle Competitions ,
\nAnd Congrats For the second place !
Congrats on your first silver medal @ksouriazer and thanks for sharing solution
", "rawMarkdown": "Congrats on your first silver medal @ksouriazer and thanks for sharing solution ", "votes": 1 }, { "id": 1243036, "postDate": "2021-03-18T01:12:24.073Z", "content": "@ksouriazer Congratulations on Silver Finish . Great Detailed writeup . Thanks so much for sharing
", "rawMarkdown": "@ksouriazer Congratulations on Silver Finish . Great Detailed writeup . Thanks so much for sharing ", "votes": 1, "replies": [ { "id": 1243044, "postDate": "2021-03-18T01:20:32.500Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1243022, "postDate": "2021-03-18T00:53:35.373Z", "content": "Congratulations for your silver medal! Nice solution.
", "rawMarkdown": "Congratulations for your silver medal! Nice solution.", "votes": 1, "replies": [ { "id": 1243037, "postDate": "2021-03-18T01:16:13.090Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1243632, "postDate": "2021-03-18T11:30:15.510Z", "content": "Hi @ksouriazer congrats on the silver, thanks for the detailed write up , can you please share the intuition behind using dual resolution inputs?
", "rawMarkdown": "Hi @ksouriazer congrats on the silver, thanks for the detailed write up , can you please share the intuition behind using dual resolution inputs?", "replies": [ { "id": 1243738, "postDate": "2021-03-18T13:07:00.987Z", "content": "Hi! The idea is to train multiple encoders with different resolutions and then train a ensemble classifier that is trained with multiple resolutions in one single foward pass. This way the classifier learns features at multiple resolutions and can make a better prediction! you can check my comment lower is this comment section about the model! Its nothing special but it did increase the performance by a good margin!
", "rawMarkdown": "Hi! The idea is to train multiple encoders with different resolutions and then train a ensemble classifier that is trained with multiple resolutions in one single foward pass. This way the classifier learns features at multiple resolutions and can make a better prediction! you can check my comment lower is this comment section about the model! Its nothing special but it did increase the performance by a good margin!", "votes": 1 } ] }, { "id": 1246817, "postDate": "2021-03-21T06:18:55.710Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1242909, "author_name": "Yann Majewski", "author_url": "", "post_date": "2021-03-17T22:31:16.170000", "content": "Dual Resnet200d – Multi resolution input
\nThe idea of having multiple resolution images as input was to learn different features of the same image and combine all of them into one ensemble classifier to make one more robust prediction.
\nTraining pipeline (greatly inspired by ammarli):
\nStage 1:
\nStage 2:
\nStage 3:
\nRepeat stage 1 to stage 3 with different image resolutions. I trained 2 models, one with 640x640 image size and the other with 736x736 image size. More models could’ve been used but time was lacking.
\nFinal stage:
\nTraining augmentations used:
\n return A.Compose([\n A.RandomResizedCrop(img_size, img_size, scale=(0.9, 1.0)),\n A.CLAHE(clip_limit=4, p=0.5),\n A.HorizontalFlip(p=0.5),\n A.ShiftScaleRotate (shift_limit=0.025, scale_limit=0, rotate_limit=5,border_mode=0, p=0.5),\n A.OneOf([\n A.RandomContrast(p=1),\n A.RandomGamma(p=1),\n A.RandomBrightness(p=1),\n ], p=0.5),\n A.OneOf([\n A.MultiplicativeNoise(p=1),\n A.IAASharpen(p=1),\n ], p=0.5),\n A.Normalize(),\n ToTensorV2(),\n ])\nThanks to my teamates and everyone in this awesome kaggle community!
", "votes": 6, "replies": [] }, { "id": 1242894, "author_name": "sin", "author_url": "", "post_date": "2021-03-17T22:15:39.453000", "content": "Congrats on getting your first medal!
", "votes": 4, "replies": [ { "id": 1242921, "author_name": "ASSAZZIN", "author_url": "", "post_date": "2021-03-17T22:57:26.250000", "content": "Thanks a lot @underwearfitting For you Contribution in Almost Kaggle Competitions ,
\nAnd Congrats For the second place !
Congrats on your first silver medal @ksouriazer and thanks for sharing solution
", "votes": 1, "replies": [] }, { "id": 1243036, "author_name": "Tensor Girl", "author_url": "", "post_date": "2021-03-18T01:12:24.073000", "content": "@ksouriazer Congratulations on Silver Finish . Great Detailed writeup . Thanks so much for sharing
", "votes": 1, "replies": [ { "id": 1243044, "author_name": "", "author_url": "", "post_date": "2021-03-18T01:20:32.500000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1243022, "author_name": "Miyatti", "author_url": "", "post_date": "2021-03-18T00:53:35.373000", "content": "Congratulations for your silver medal! Nice solution.
", "votes": 1, "replies": [ { "id": 1243037, "author_name": "", "author_url": "", "post_date": "2021-03-18T01:16:13.090000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1243632, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2021-03-18T11:30:15.510000", "content": "Hi @ksouriazer congrats on the silver, thanks for the detailed write up , can you please share the intuition behind using dual resolution inputs?
", "votes": 0, "replies": [ { "id": 1243738, "author_name": "Yann Majewski", "author_url": "", "post_date": "2021-03-18T13:07:00.987000", "content": "Hi! The idea is to train multiple encoders with different resolutions and then train a ensemble classifier that is trained with multiple resolutions in one single foward pass. This way the classifier learns features at multiple resolutions and can make a better prediction! you can check my comment lower is this comment section about the model! Its nothing special but it did increase the performance by a good margin!
", "votes": 1, "replies": [] } ] }, { "id": 1246817, "author_name": "", "author_url": "", "post_date": "2021-03-21T06:18:55.710000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1242882": "First of all, I would really like to thank my teammates @yannmajewski and @mo5mami for their hard work and commitment . And I would like to thank all the competitors that shared their codes, ideas, discussions.\n\nHere is the overview of our pipeline:\n \n\n \n### Summary\n \nFinal submission(Public: 0.969, Private: 0.972) \nWe used different image size for our models. [(512x512),(640x640),(736x736)]\n\n * M1: Multi resnet200d (Public: 0.967, Private: 0.971) :\n * details will be provided later by @yannmajewski\n