First of all, i congratulate all the winners and thank community for lots of ideas and tricks, that i've learnt from you during this competition. 😊
\nThat was my 1st kaggle medal, and i am glad that i received it in my 1st Computer Vision competition.
\nI didn't try some ideas, because i had only last 3 weeks for participation and limited hardware 😅
\nCompetition data only. Just for 2020.
\nI also tried to use 2019 dataset, but it didn't make any improvement for my CV.
5-fold stratified.
\nI used imagenet mean and std. But i also tried mean and std from dataset, but it didn't make any improvement for my CV. It seems that dataset is quite small for this when we do transfer learning.
\nFor ensembling i just averaged predictions.
\nI also replaced fully connected layer for ResNet with some sequential stuff.
\nI used mixed precision for allocating memory, even though it decreased my CV, it allowed me to train batch size 32 with size 512.
\nClass weights or upsampling didn't give me smth.
\nImage Size: 512
\nEpochs: 10
\nEarly Stopping: No (Saved Best Epoch for loss, not accuracy)
\nLoss Function: Taylor CE Loss with Label Smoothing
\nOptimizer: Adam. (Didn't have time to try others)
\nLR Scheduler: Cosine Annealing with Warm Restarts (Didn't have time to try others)
\nBatch Size: 32
\nAugmentations: Variants of standard (No CutMix, FMix or SnapMix)
RandomResizedCrop,\n Transpose,\n HorizontalFlip,\n VerticalFlip,\n ShiftScaleRotate,\n HueSaturationValue,\n RandomBrightnessContrast,\n CoarseDropout,\n Cutout,
RandomResizedCrop,\n Transpose,\n HorizontalFlip,\n VerticalFlip,\n IAAAdditiveGaussianNoise,\n GaussNoise,\n ShiftScaleRotate,\n CLAHE,\n IAASharpen,\n IAAEmboss,\n RandomBrightnessContrast, \n HueSaturationValue
No TTA for final submissions.
\nFor one of my ensembled resnets i used denoising, based on geometric mean of false confidence of my 2 other resnets and effb3.
\nI also choosed for my final submissions only from models which i trained exactly on kaggle notebook, because i don't know if it is allowed to upload pretrained models with custom weights for inference. So, as me not a risky person, i used only models from my notebook output.
\nSingle CVs for my models were below 0.9 (except denoised model, but it's not comparable).
\nPublic score of my final ensemble was 0.9 , and in Private i have 0.899.
So, i moved from 1094th place to 170th place and got my 1st medal.
", "messages": [ { "id": "1210376", "postDate": "02/19/2021 11:38:28", "content": "First of all, i congratulate all the winners and thank community for lots of ideas and tricks, that i've learnt from you during this competition. 😊
\nThat was my 1st kaggle medal, and i am glad that i received it in my 1st Computer Vision competition.
\nI didn't try some ideas, because i had only last 3 weeks for participation and limited hardware 😅
\nCompetition data only. Just for 2020.
\nI also tried to use 2019 dataset, but it didn't make any improvement for my CV.
5-fold stratified.
\nI used imagenet mean and std. But i also tried mean and std from dataset, but it didn't make any improvement for my CV. It seems that dataset is quite small for this when we do transfer learning.
\nFor ensembling i just averaged predictions.
\nI also replaced fully connected layer for ResNet with some sequential stuff.
\nI used mixed precision for allocating memory, even though it decreased my CV, it allowed me to train batch size 32 with size 512.
\nClass weights or upsampling didn't give me smth.
\nImage Size: 512
\nEpochs: 10
\nEarly Stopping: No (Saved Best Epoch for loss, not accuracy)
\nLoss Function: Taylor CE Loss with Label Smoothing
\nOptimizer: Adam. (Didn't have time to try others)
\nLR Scheduler: Cosine Annealing with Warm Restarts (Didn't have time to try others)
\nBatch Size: 32
\nAugmentations: Variants of standard (No CutMix, FMix or SnapMix)
RandomResizedCrop,\n Transpose,\n HorizontalFlip,\n VerticalFlip,\n ShiftScaleRotate,\n HueSaturationValue,\n RandomBrightnessContrast,\n CoarseDropout,\n Cutout,
RandomResizedCrop,\n Transpose,\n HorizontalFlip,\n VerticalFlip,\n IAAAdditiveGaussianNoise,\n GaussNoise,\n ShiftScaleRotate,\n CLAHE,\n IAASharpen,\n IAAEmboss,\n RandomBrightnessContrast, \n HueSaturationValue
No TTA for final submissions.
\nFor one of my ensembled resnets i used denoising, based on geometric mean of false confidence of my 2 other resnets and effb3.
\nI also choosed for my final submissions only from models which i trained exactly on kaggle notebook, because i don't know if it is allowed to upload pretrained models with custom weights for inference. So, as me not a risky person, i used only models from my notebook output.
\nSingle CVs for my models were below 0.9 (except denoised model, but it's not comparable).
\nPublic score of my final ensemble was 0.9 , and in Private i have 0.899.
So, i moved from 1094th place to 170th place and got my 1st medal.
", "rawMarkdown": "First of all, i congratulate all the winners and thank community for lots of ideas and tricks, that i've learnt from you during this competition. 😊\n\nThat was my 1st kaggle medal, and i am glad that i received it in my 1st Computer Vision competition. \n\nI didn't try some ideas, because i had only last 3 weeks for participation and limited hardware 😅\n\n# Final Solution\n## Dataset\nCompetition data only. Just for 2020.\nI also tried to use 2019 dataset, but it didn't make any improvement for my CV.\n\n## CV strategy\n5-fold stratified. \n\n## Normalization\nI used imagenet mean and std. But i also tried mean and std from dataset, but it didn't make any improvement for my CV. It seems that dataset is quite small for this when we do transfer learning.\n\n## Models (pretrained):\n1. 2*resnext50_32x4d (one with custom 95 percentile denoise + w/o it)\n2. efficientnet_b3_ns\n\nFor ensembling i just averaged predictions.\n\nI also replaced fully connected layer for ResNet with some sequential stuff.\n\nI used mixed precision for allocating memory, even though it decreased my CV, it allowed me to train batch size 32 with size 512.\n\nClass weights or upsampling didn't give me smth.\n\n## Train Setting:\nImage Size: 512\nEpochs: 10\nEarly Stopping: No (Saved Best Epoch for loss, not accuracy)\nLoss Function: Taylor CE Loss with Label Smoothing\nOptimizer: Adam. (Didn't have time to try others)\nLR Scheduler: Cosine Annealing with Warm Restarts (Didn't have time to try others)\nBatch Size: 32\nAugmentations: Variants of standard (No CutMix, FMix or SnapMix)\n\n### Augmentations 1:\n` RandomResizedCrop,\n Transpose,\n HorizontalFlip,\n VerticalFlip,\n ShiftScaleRotate,\n HueSaturationValue,\n RandomBrightnessContrast,\n CoarseDropout,\n Cutout,`\n\n### Augmentations 2:\n` RandomResizedCrop,\n Transpose,\n HorizontalFlip,\n VerticalFlip,\n IAAAdditiveGaussianNoise,\n GaussNoise,\n ShiftScaleRotate,\n CLAHE,\n IAASharpen,\n IAAEmboss,\n RandomBrightnessContrast, \n HueSaturationValue`\n\nNo TTA for final submissions.\n\n## Denoising:\nFor one of my ensembled resnets i used denoising, based on geometric mean of false confidence of my 2 other resnets and effb3.\n\nI also choosed for my final submissions only from models which i trained exactly on kaggle notebook, because i don't know if it is allowed to upload pretrained models with custom weights for inference. So, as me not a risky person, i used only models from my notebook output.\n\nSingle CVs for my models were below 0.9 (except denoised model, but it's not comparable).\nPublic score of my final ensemble was 0.9 , and in Private i have 0.899. \n\nSo, i moved from 1094th place to 170th place and got my 1st medal.", "votes": null }, { "id": "1210418", "postDate": "02/19/2021 12:17:25", "content": "That's quite a huge leap forward. Nice work. Did you use Pytorch ?
", "rawMarkdown": "That's quite a huge leap forward. Nice work. Did you use Pytorch ?", "votes": null }, { "id": "1210424", "postDate": "02/19/2021 12:22:41", "content": "Thank you! Yes, PyTorch. I started to learn Computer Vision and neural networks only 3 months ago. So, I had to choose one framework first. And it was PyTorch 😄
", "rawMarkdown": "Thank you! Yes, PyTorch. I started to learn Computer Vision and neural networks only 3 months ago. So, I had to choose one framework first. And it was PyTorch 😄", "votes": null } ], "comments": [ { "id": 1210418, "author_name": "mohneesh7", "author_url": "", "post_date": "02/19/2021 12:17:25", "content": "That's quite a huge leap forward. Nice work. Did you use Pytorch ?
", "votes": null, "replies": [ { "id": 1210424, "author_name": "sergeydvindenko", "author_url": "", "post_date": "02/19/2021 12:22:41", "content": "Thank you! Yes, PyTorch. I started to learn Computer Vision and neural networks only 3 months ago. So, I had to choose one framework first. And it was PyTorch 😄
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1210376": "First of all, i congratulate all the winners and thank community for lots of ideas and tricks, that i've learnt from you during this competition. 😊\n\nThat was my 1st kaggle medal, and i am glad that i received it in my 1st Computer Vision competition. \n\nI didn't try some ideas, because i had only last 3 weeks for participation and limited hardware 😅\n\n# Final Solution\n## Dataset\nCompetition data only. Just for 2020.\nI also tried to use 2019 dataset, but it didn't make any improvement for my CV.\n\n## CV strategy\n5-fold stratified. \n\n## Normalization\nI used imagenet mean and std. But i also tried mean and std from dataset, but it didn't make any improvement for my CV. It seems that dataset is quite small for this when we do transfer learning.\n\n## Models (pretrained):\n1. 2*resnext50_32x4d (one with custom 95 percentile denoise + w/o it)\n2. efficientnet_b3_ns\n\nFor ensembling i just averaged predictions.\n\nI also replaced fully connected layer for ResNet with some sequential stuff.\n\nI used mixed precision for allocating memory, even though it decreased my CV, it allowed me to train batch size 32 with size 512.\n\nClass weights or upsampling didn't give me smth.\n\n## Train Setting:\nImage Size: 512\nEpochs: 10\nEarly Stopping: No (Saved Best Epoch for loss, not accuracy)\nLoss Function: Taylor CE Loss with Label Smoothing\nOptimizer: Adam. (Didn't have time to try others)\nLR Scheduler: Cosine Annealing with Warm Restarts (Didn't have time to try others)\nBatch Size: 32\nAugmentations: Variants of standard (No CutMix, FMix or SnapMix)\n\n### Augmentations 1:\n` RandomResizedCrop,\n Transpose,\n HorizontalFlip,\n VerticalFlip,\n ShiftScaleRotate,\n HueSaturationValue,\n RandomBrightnessContrast,\n CoarseDropout,\n Cutout,`\n\n### Augmentations 2:\n` RandomResizedCrop,\n Transpose,\n HorizontalFlip,\n VerticalFlip,\n IAAAdditiveGaussianNoise,\n GaussNoise,\n ShiftScaleRotate,\n CLAHE,\n IAASharpen,\n IAAEmboss,\n RandomBrightnessContrast, \n HueSaturationValue`\n\nNo TTA for final submissions.\n\n## Denoising:\nFor one of my ensembled resnets i used denoising, based on geometric mean of false confidence of my 2 other resnets and effb3.\n\nI also choosed for my final submissions only from models which i trained exactly on kaggle notebook, because i don't know if it is allowed to upload pretrained models with custom weights for inference. So, as me not a risky person, i used only models from my notebook output.\n\nSingle CVs for my models were below 0.9 (except denoised model, but it's not comparable).\nPublic score of my final ensemble was 0.9 , and in Private i have 0.899. \n\nSo, i moved from 1094th place to 170th place and got my 1st medal.", "1210418": "That's quite a huge leap forward. Nice work. Did you use Pytorch ?", "1210424": "Thank you! Yes, PyTorch. I started to learn Computer Vision and neural networks only 3 months ago. So, I had to choose one framework first. And it was PyTorch 😄" }, "source": "meta" }