![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F974295%2F0c878fc4e2b2156904d2ff3ff3cbb05c%2FLocalCV_publicLB.jpeg?generation=1569486918078902&alt=media\")
Acknowledgements
\nThanks to Kaggle for organizing this competition.
\nI learned a tremendous amount of tricks in this competition and it would not have been able if it wasn’t for all this generous sharing of top solutions and extremely talented teammates ( @cdeotte @hanson0910 and @zlanan ) i was lucky to have along the way. thanks to my well collaborative team mates. Together we learned a lot 💪
We did not have any magic there, only diverse models with highest CV
\nour aim was to design an ensemble using top pytorch + tensorflow models with different image size,different TTA and diverse models (it worked well on public lb and also on private)
\nour 32nd place submission uses,
\nmodel 1 -> nf-resnet50:
\n[0.897, 0.903, 0.899. 0.8932, 0.8946]
model 2 ->ResNext50(image size 512):
\n[0.891, 0.897, 0.892, 0.886, 0.892]
model 3 -> 0.8913 vit. + 0.886 +0.884 +0.883 noisy tf B4. Intentionally trained three relatively low CV noisy tf B4 but works, quite strange.
\nmodel 4 and 5 ->
\n(tf_efficientnet_b4_ns + Vit-B16) with 2019 years of data, so CV has no reference value but it's around 0.9
model 6 ->
\nnot sure what the CV is for 512 image size efficientnet-b0 (tensorflow model) but public lb was 0.895
model 7 ->
\ntf_efficientnet_b4_ns_fold_0_8: cv 0.8914
\ntf_efficientnet_b4_ns_fold_0_5: cv 0.8932
only 1 fold of vit base from this notebook : ViT - Pytorch xla (TPU) for leaf disease
\nvit_base_patch16_384_fold_4: cv0.89175
\nonly 2 fold of vit large from this notebook : Faster Pytorch TPU baseline for CLD(cv 0.9)
\nvit_large_patch16_384_fold_2: cv 0.89568
\nvit_large_patch16_384_fold_1: cv 0.89826
used following tta (3 step):
\ndef get_inference_transforms(image_size = image_size):\n return Compose([\n RandomResizedCrop(image_size, image_size),\n Transpose(p=0.5),\n HorizontalFlip(p=0.5),\n VerticalFlip(p=0.5),\n HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n ToTensorV2(p=1.0),\n ], p=1.)\nand for vit large :
\ninference_transforms = albumentations.Compose([\n #albumentations.RandomResizedCrop(image_size, image_size),\n albumentations.Resize(image_size, image_size),\n albumentations.Transpose(p=0.5),\n albumentations.HorizontalFlip(p=0.5),\n albumentations.VerticalFlip(p=0.5),\n albumentations.HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n albumentations.RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n ]) \n \n@zlanan found these as best augmentations :
\n RandomResizedCrop(CFG['img_size'], CFG['img_size']),\n Transpose(p=0.4),\n HorizontalFlip(p=0.4),\n VerticalFlip(p=0.4),\n ShiftScaleRotate(p=0.3),\n MedianBlur(blur_limit=7,always_apply=False, p=0.3),\n IAAAdditiveGaussianNoise(scale=(0, 0.15*255),p=0.5),\n HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.4),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.4),\n Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n CoarseDropout(p=1.0),\n Cutout(p=0.4),\nwe blended all models that had cv close to 0.9 or more than that, with that we got public lb 0.8978 and private lb 0.9019 (but couldn't select it because of low lb score)
\n
Final note
\nwe did observe a rise on both LB with lighter TTA, but also a drop. we did observe a improve with different image size model but also a drop. we can not say anything is really useful, except avg ensemble.
\nsome of our early experiment results can be found here : https://docs.google.com/spreadsheets/d/1HSMuTrMmwB5-8GXeJ-h68xeOIEeffcRo2W6HcmfXCzs/edit#gid=0
\nthank you for reading
", "messages": [ { "id": "1210881", "postDate": "02/19/2021 19:36:45", "content": "Acknowledgements
\nThanks to Kaggle for organizing this competition.
\nI learned a tremendous amount of tricks in this competition and it would not have been able if it wasn’t for all this generous sharing of top solutions and extremely talented teammates ( @cdeotte @hanson0910 and @zlanan ) i was lucky to have along the way. thanks to my well collaborative team mates. Together we learned a lot 💪
We did not have any magic there, only diverse models with highest CV
\nour aim was to design an ensemble using top pytorch + tensorflow models with different image size,different TTA and diverse models (it worked well on public lb and also on private)
\nour 32nd place submission uses,
\nmodel 1 -> nf-resnet50:
\n[0.897, 0.903, 0.899. 0.8932, 0.8946]
model 2 ->ResNext50(image size 512):
\n[0.891, 0.897, 0.892, 0.886, 0.892]
model 3 -> 0.8913 vit. + 0.886 +0.884 +0.883 noisy tf B4. Intentionally trained three relatively low CV noisy tf B4 but works, quite strange.
\nmodel 4 and 5 ->
\n(tf_efficientnet_b4_ns + Vit-B16) with 2019 years of data, so CV has no reference value but it's around 0.9
model 6 ->
\nnot sure what the CV is for 512 image size efficientnet-b0 (tensorflow model) but public lb was 0.895
model 7 ->
\ntf_efficientnet_b4_ns_fold_0_8: cv 0.8914
\ntf_efficientnet_b4_ns_fold_0_5: cv 0.8932
only 1 fold of vit base from this notebook : ViT - Pytorch xla (TPU) for leaf disease
\nvit_base_patch16_384_fold_4: cv0.89175
\nonly 2 fold of vit large from this notebook : Faster Pytorch TPU baseline for CLD(cv 0.9)
\nvit_large_patch16_384_fold_2: cv 0.89568
\nvit_large_patch16_384_fold_1: cv 0.89826
used following tta (3 step):
\ndef get_inference_transforms(image_size = image_size):\n return Compose([\n RandomResizedCrop(image_size, image_size),\n Transpose(p=0.5),\n HorizontalFlip(p=0.5),\n VerticalFlip(p=0.5),\n HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n ToTensorV2(p=1.0),\n ], p=1.)\nand for vit large :
\ninference_transforms = albumentations.Compose([\n #albumentations.RandomResizedCrop(image_size, image_size),\n albumentations.Resize(image_size, image_size),\n albumentations.Transpose(p=0.5),\n albumentations.HorizontalFlip(p=0.5),\n albumentations.VerticalFlip(p=0.5),\n albumentations.HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n albumentations.RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n ]) \n \n@zlanan found these as best augmentations :
\n RandomResizedCrop(CFG['img_size'], CFG['img_size']),\n Transpose(p=0.4),\n HorizontalFlip(p=0.4),\n VerticalFlip(p=0.4),\n ShiftScaleRotate(p=0.3),\n MedianBlur(blur_limit=7,always_apply=False, p=0.3),\n IAAAdditiveGaussianNoise(scale=(0, 0.15*255),p=0.5),\n HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.4),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.4),\n Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n CoarseDropout(p=1.0),\n Cutout(p=0.4),\nwe blended all models that had cv close to 0.9 or more than that, with that we got public lb 0.8978 and private lb 0.9019 (but couldn't select it because of low lb score)
\n
Final note
\nwe did observe a rise on both LB with lighter TTA, but also a drop. we did observe a improve with different image size model but also a drop. we can not say anything is really useful, except avg ensemble.
\nsome of our early experiment results can be found here : https://docs.google.com/spreadsheets/d/1HSMuTrMmwB5-8GXeJ-h68xeOIEeffcRo2W6HcmfXCzs/edit#gid=0
\nthank you for reading
", "rawMarkdown": "**Acknowledgements**\nThanks to Kaggle for organizing this competition.\nI learned a tremendous amount of tricks in this competition and it would not have been able if it wasn’t for all this generous sharing of top solutions and extremely talented teammates ( @cdeotte @hanson0910 and @zlanan ) i was lucky to have along the way. thanks to my well collaborative team mates. **Together we learned a lot** 💪\n\n**We did not have any magic there, only diverse models with highest CV**\n\nour aim was to design an ensemble using top pytorch + tensorflow models with different image size,different TTA and diverse models (it worked well on public lb and also on private)\n\nour 32nd place submission uses,\n\n**model 1 -> nf-resnet50:**\n[0.897, 0.903, 0.899. 0.8932, 0.8946]\n\n**model 2 ->ResNext50(image size 512):**\n[0.891, 0.897, 0.892, 0.886, 0.892]\n\n**model 3 -> 0.8913 vit. + 0.886 +0.884 +0.883 noisy tf B4. Intentionally trained three relatively low CV noisy tf B4 but works, quite strange.**\n\n**model 4 and 5 ->**\n(tf_efficientnet_b4_ns + Vit-B16) with 2019 years of data, so CV has no reference value but it's around 0.9\n\n**model 6 ->**\nnot sure what the CV is for 512 image size efficientnet-b0 **(tensorflow model)** but public lb was 0.895 \n\n**model 7 ->**\n\ntf_efficientnet_b4_ns_fold_0_8: cv 0.8914\ntf_efficientnet_b4_ns_fold_0_5: cv 0.8932\n\nonly 1 fold of vit base from this notebook : [ViT - Pytorch xla (TPU) for leaf disease](https://www.kaggle.com/mobassir/vit-pytorch-xla-tpu-for-leaf-disease)\n\nvit_base_patch16_384_fold_4: cv0.89175\n\nonly 2 fold of vit large from this notebook : [Faster Pytorch TPU baseline for CLD(cv 0.9)](https://www.kaggle.com/mobassir/faster-pytorch-tpu-baseline-for-cld-cv-0-9)\n\n\nvit_large_patch16_384_fold_2: cv 0.89568\nvit_large_patch16_384_fold_1: cv 0.89826\n\nused following tta (3 step):\n\n```\ndef get_inference_transforms(image_size = image_size):\n return Compose([\n RandomResizedCrop(image_size, image_size),\n Transpose(p=0.5),\n HorizontalFlip(p=0.5),\n VerticalFlip(p=0.5),\n HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n ToTensorV2(p=1.0),\n ], p=1.)\n```\n\nand for vit large : \n\n```\ninference_transforms = albumentations.Compose([\n #albumentations.RandomResizedCrop(image_size, image_size),\n albumentations.Resize(image_size, image_size),\n albumentations.Transpose(p=0.5),\n albumentations.HorizontalFlip(p=0.5),\n albumentations.VerticalFlip(p=0.5),\n albumentations.HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n albumentations.RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n ]) \n``` \n @zlanan found these as best augmentations : \n\n RandomResizedCrop(CFG['img_size'], CFG['img_size']),\n Transpose(p=0.4),\n HorizontalFlip(p=0.4),\n VerticalFlip(p=0.4),\n ShiftScaleRotate(p=0.3),\n MedianBlur(blur_limit=7,always_apply=False, p=0.3),\n IAAAdditiveGaussianNoise(scale=(0, 0.15*255),p=0.5),\n HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.4),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.4),\n Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n CoarseDropout(p=1.0),\n Cutout(p=0.4),\n\nwe blended all models that had cv close to 0.9 or more than that, with that we got public lb 0.8978 and **private lb 0.9019** **(but couldn't select it because of low lb score)**\n\n\n\n\n**Final note**\n\nwe did observe a rise on both LB with lighter TTA, but also a drop. we did observe a improve with different image size model but also a drop. we can not say anything is really useful, except **avg ensemble.**\n\nsome of our early experiment results can be found here : https://docs.google.com/spreadsheets/d/1HSMuTrMmwB5-8GXeJ-h68xeOIEeffcRo2W6HcmfXCzs/edit#gid=0\n\nthank you for reading", "votes": null }, { "id": "1210894", "postDate": "02/19/2021 19:45:19", "content": "Simple but not simple solution. Thank you for sharing the solution and experiment result. :)
\n@mobassir
thank you for your all great discussion posts and kernels of this competition @piantic
", "rawMarkdown": "thank you for your all great discussion posts and kernels of this competition @piantic", "votes": null }, { "id": "1210937", "postDate": "02/19/2021 20:06:02", "content": "Thank you for sharing your solution and especially experiment guts).
\nI see, you had different loss functions for your models. So the question, by which factor you chose you best epochs: loss or acc? Or even maybe f1 or another additional metric?)
\nThis question was unobvious for me, but from some moment i decided to stand on loss comparison.
we always tracked validation multi class accuracy
", "rawMarkdown": "we always tracked validation multi class accuracy", "votes": null }, { "id": "1211285", "postDate": "02/20/2021 05:50:52", "content": "Was witing for the TOP 3 solutions. Bad luck this time for your leap back. Nice work and learned a lot.
", "rawMarkdown": "Was witing for the TOP 3 solutions. Bad luck this time for your leap back. Nice work and learned a lot.", "votes": null }, { "id": "1211474", "postDate": "02/20/2021 09:09:27", "content": "great work thanks!
", "rawMarkdown": "great work thanks!", "votes": null }, { "id": "1211584", "postDate": "02/20/2021 10:40:02", "content": "Congratz @mobassir, that's a nice finish nonetheless !
", "rawMarkdown": "Congratz @mobassir, that's a nice finish nonetheless !", "votes": null }, { "id": "1211698", "postDate": "02/20/2021 13:09:47", "content": "Which tool did you use to track your validation accuracy?
", "rawMarkdown": "Which tool did you use to track your validation accuracy?", "votes": null }, { "id": "1211721", "postDate": "02/20/2021 13:35:12", "content": "Sorry i do not understand your question, we monitor validation multi claas accuracy which is a metric i Don't know where the word \"tools\" comes here,you can check this notebook : https://www.kaggle.com/mobassir/faster-pytorch-tpu-baseline-for-cld-cv-0-9
\nFor understanding how to monitor validation accuracy.
\nFor keeping track of best validation accuract we use a simple code like this
\nIf Current_accuracy > best_accuracy:
Thank you for sharing!
", "rawMarkdown": "Thank you for sharing!", "votes": null }, { "id": "1211975", "postDate": "02/20/2021 18:01:15", "content": "Good job, and congratulation
", "rawMarkdown": "Good job, and congratulation", "votes": null }, { "id": "1212269", "postDate": "02/21/2021 04:01:17", "content": "I learned a lot from your notebook, thank you for your sharing!
", "rawMarkdown": "I learned a lot from your notebook, thank you for your sharing!", "votes": null }, { "id": "1212524", "postDate": "02/21/2021 09:50:12", "content": "Congrats @mobassir on the gold medal and thank you for sharing your solution. I learnt a lot from your notebooks as well as @cdeotte posts. Just wanted suggestions on how to tune hyper parameters - I was tuning them manually and lost a bit of experimentation time. Also, any suggestion to keep track of models - I use kaggle kernels only and have hard time syncing between my git repo and kaggle experiments.
", "rawMarkdown": "Congrats @mobassir on the gold medal and thank you for sharing your solution. I learnt a lot from your notebooks as well as @cdeotte posts. Just wanted suggestions on how to tune hyper parameters - I was tuning them manually and lost a bit of experimentation time. Also, any suggestion to keep track of models - I use kaggle kernels only and have hard time syncing between my git repo and kaggle experiments.", "votes": null }, { "id": "1212527", "postDate": "02/21/2021 09:58:39", "content": "Ah sorry for not being clear. Since you mentioned \"tracking validation accuracy\" I wondered if you used any ML experiment tracking tool. I use Weights and Biases for that matter.
\nYes I am familiar with validation multi-class accucary. :)
", "rawMarkdown": "Ah sorry for not being clear. Since you mentioned \"tracking validation accuracy\" I wondered if you used any ML experiment tracking tool. I use Weights and Biases for that matter. \n\nYes I am familiar with validation multi-class accucary. :)", "votes": null }, { "id": "1212540", "postDate": "02/21/2021 10:13:29", "content": "sorry, we didn't use any ML experiment tracking tool,just tracked and saved weights based on best validation multi-class accuracy
", "rawMarkdown": "sorry, we didn't use any ML experiment tracking tool,just tracked and saved weights based on best validation multi-class accuracy", "votes": null }, { "id": "1212547", "postDate": "02/21/2021 10:20:05", "content": "hi @suryajrrafl
\nwe haven't got gold unfortunately but regarding your question on hyperparameter tuning,i think you can try something like optuna : https://www.analyticsvidhya.com/blog/2020/11/hyperparameter-tuning-using-optuna/
i am not a good hyperparameter tuner,but as a beginner i used to do a lot of hyperparameter tuning and waste a lot of time+gpu hours,later i realized that it's a skill where i should put less focus and more focus on novelty and designing good solutions.i think most of the kaggle beginners start with hyperparameter tuning like i did :)
\nfor tracking your experiments and models performance you can fill up a sheet like this : https://docs.google.com/spreadsheets/d/1HSMuTrMmwB5-8GXeJ-h68xeOIEeffcRo2W6HcmfXCzs/edit#gid=0
\nit really helps a lot and i learned it from my old good team mate @theoviel
\nmy suggestion(based on my past mistakes) : spend less time on tuning and more time on trying to implement something that looks promising,,because with tuning you can get very smallllll amount of boost most of the times and most of the times they can overfit too,but if you can come up with an different idea+implementation that very few people will try then you can stand out from the crowd with ease,thank you :)
", "rawMarkdown": "hi @suryajrrafl \nwe haven't got gold unfortunately but regarding your question on hyperparameter tuning,i think you can try something like optuna : https://www.analyticsvidhya.com/blog/2020/11/hyperparameter-tuning-using-optuna/\n\ni am not a good hyperparameter tuner,but as a beginner i used to do a lot of hyperparameter tuning and waste a lot of time+gpu hours,later i realized that it's a skill where i should put less focus and more focus on novelty and designing good solutions.i think most of the kaggle beginners start with hyperparameter tuning like i did :)\n\nfor tracking your experiments and models performance you can fill up a sheet like this : https://docs.google.com/spreadsheets/d/1HSMuTrMmwB5-8GXeJ-h68xeOIEeffcRo2W6HcmfXCzs/edit#gid=0\n\nit really helps a lot and i learned it from my old good team mate @theoviel\n\nmy suggestion(based on my past mistakes) : spend less time on tuning and more time on trying to implement something that looks promising,,because with tuning you can get very smallllll amount of boost most of the times and most of the times they can overfit too,but if you can come up with an different idea+implementation that very few people will try then you can stand out from the crowd with ease,thank you :)", "votes": null }, { "id": "1212800", "postDate": "02/21/2021 16:02:12", "content": "Thanks @mobassir for your patient reply. I too did track my experiments using a sheet like this, but as you said, I focussed on tuning to gain marginal improvements. Your insight gives a better clarity on where to experiment and spend more time on. Thanks once again for your help
", "rawMarkdown": "Thanks @mobassir for your patient reply. I too did track my experiments using a sheet like this, but as you said, I focussed on tuning to gain marginal improvements. Your insight gives a better clarity on where to experiment and spend more time on. Thanks once again for your help", "votes": null } ], "comments": [ { "id": 1210894, "author_name": "piantic", "author_url": "", "post_date": "02/19/2021 19:45:19", "content": "Simple but not simple solution. Thank you for sharing the solution and experiment result. :)
\n@mobassir
thank you for your all great discussion posts and kernels of this competition @piantic
", "votes": null, "replies": [] } ] }, { "id": 1210937, "author_name": "sergeydvindenko", "author_url": "", "post_date": "02/19/2021 20:06:02", "content": "Thank you for sharing your solution and especially experiment guts).
\nI see, you had different loss functions for your models. So the question, by which factor you chose you best epochs: loss or acc? Or even maybe f1 or another additional metric?)
\nThis question was unobvious for me, but from some moment i decided to stand on loss comparison.
we always tracked validation multi class accuracy
", "votes": null, "replies": [] }, { "id": 1211698, "author_name": "ayuraj", "author_url": "", "post_date": "02/20/2021 13:09:47", "content": "Which tool did you use to track your validation accuracy?
", "votes": null, "replies": [] }, { "id": 1211721, "author_name": "mobassir", "author_url": "", "post_date": "02/20/2021 13:35:12", "content": "Sorry i do not understand your question, we monitor validation multi claas accuracy which is a metric i Don't know where the word \"tools\" comes here,you can check this notebook : https://www.kaggle.com/mobassir/faster-pytorch-tpu-baseline-for-cld-cv-0-9
\nFor understanding how to monitor validation accuracy.
\nFor keeping track of best validation accuract we use a simple code like this
\nIf Current_accuracy > best_accuracy:
Ah sorry for not being clear. Since you mentioned \"tracking validation accuracy\" I wondered if you used any ML experiment tracking tool. I use Weights and Biases for that matter.
\nYes I am familiar with validation multi-class accucary. :)
", "votes": null, "replies": [] }, { "id": 1212540, "author_name": "mobassir", "author_url": "", "post_date": "02/21/2021 10:13:29", "content": "sorry, we didn't use any ML experiment tracking tool,just tracked and saved weights based on best validation multi-class accuracy
", "votes": null, "replies": [] } ] }, { "id": 1211285, "author_name": "mohneesh7", "author_url": "", "post_date": "02/20/2021 05:50:52", "content": "Was witing for the TOP 3 solutions. Bad luck this time for your leap back. Nice work and learned a lot.
", "votes": null, "replies": [] }, { "id": 1211474, "author_name": "wonjunpark", "author_url": "", "post_date": "02/20/2021 09:09:27", "content": "great work thanks!
", "votes": null, "replies": [] }, { "id": 1211584, "author_name": "theoviel", "author_url": "", "post_date": "02/20/2021 10:40:02", "content": "Congratz @mobassir, that's a nice finish nonetheless !
", "votes": null, "replies": [] }, { "id": 1211815, "author_name": "jacoporepossi", "author_url": "", "post_date": "02/20/2021 15:29:54", "content": "Thank you for sharing!
", "votes": null, "replies": [] }, { "id": 1211975, "author_name": "bessenyeiszilrd", "author_url": "", "post_date": "02/20/2021 18:01:15", "content": "Good job, and congratulation
", "votes": null, "replies": [] }, { "id": 1212269, "author_name": "cuteffff", "author_url": "", "post_date": "02/21/2021 04:01:17", "content": "I learned a lot from your notebook, thank you for your sharing!
", "votes": null, "replies": [] }, { "id": 1212524, "author_name": "suryajrrafl", "author_url": "", "post_date": "02/21/2021 09:50:12", "content": "Congrats @mobassir on the gold medal and thank you for sharing your solution. I learnt a lot from your notebooks as well as @cdeotte posts. Just wanted suggestions on how to tune hyper parameters - I was tuning them manually and lost a bit of experimentation time. Also, any suggestion to keep track of models - I use kaggle kernels only and have hard time syncing between my git repo and kaggle experiments.
", "votes": null, "replies": [ { "id": 1212547, "author_name": "mobassir", "author_url": "", "post_date": "02/21/2021 10:20:05", "content": "hi @suryajrrafl
\nwe haven't got gold unfortunately but regarding your question on hyperparameter tuning,i think you can try something like optuna : https://www.analyticsvidhya.com/blog/2020/11/hyperparameter-tuning-using-optuna/
i am not a good hyperparameter tuner,but as a beginner i used to do a lot of hyperparameter tuning and waste a lot of time+gpu hours,later i realized that it's a skill where i should put less focus and more focus on novelty and designing good solutions.i think most of the kaggle beginners start with hyperparameter tuning like i did :)
\nfor tracking your experiments and models performance you can fill up a sheet like this : https://docs.google.com/spreadsheets/d/1HSMuTrMmwB5-8GXeJ-h68xeOIEeffcRo2W6HcmfXCzs/edit#gid=0
\nit really helps a lot and i learned it from my old good team mate @theoviel
\nmy suggestion(based on my past mistakes) : spend less time on tuning and more time on trying to implement something that looks promising,,because with tuning you can get very smallllll amount of boost most of the times and most of the times they can overfit too,but if you can come up with an different idea+implementation that very few people will try then you can stand out from the crowd with ease,thank you :)
", "votes": null, "replies": [ { "id": 1212800, "author_name": "suryajrrafl", "author_url": "", "post_date": "02/21/2021 16:02:12", "content": "Thanks @mobassir for your patient reply. I too did track my experiments using a sheet like this, but as you said, I focussed on tuning to gain marginal improvements. Your insight gives a better clarity on where to experiment and spend more time on. Thanks once again for your help
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "1210881": "**Acknowledgements**\nThanks to Kaggle for organizing this competition.\nI learned a tremendous amount of tricks in this competition and it would not have been able if it wasn’t for all this generous sharing of top solutions and extremely talented teammates ( @cdeotte @hanson0910 and @zlanan ) i was lucky to have along the way. thanks to my well collaborative team mates. **Together we learned a lot** 💪\n\n**We did not have any magic there, only diverse models with highest CV**\n\nour aim was to design an ensemble using top pytorch + tensorflow models with different image size,different TTA and diverse models (it worked well on public lb and also on private)\n\nour 32nd place submission uses,\n\n**model 1 -> nf-resnet50:**\n[0.897, 0.903, 0.899. 0.8932, 0.8946]\n\n**model 2 ->ResNext50(image size 512):**\n[0.891, 0.897, 0.892, 0.886, 0.892]\n\n**model 3 -> 0.8913 vit. + 0.886 +0.884 +0.883 noisy tf B4. Intentionally trained three relatively low CV noisy tf B4 but works, quite strange.**\n\n**model 4 and 5 ->**\n(tf_efficientnet_b4_ns + Vit-B16) with 2019 years of data, so CV has no reference value but it's around 0.9\n\n**model 6 ->**\nnot sure what the CV is for 512 image size efficientnet-b0 **(tensorflow model)** but public lb was 0.895 \n\n**model 7 ->**\n\ntf_efficientnet_b4_ns_fold_0_8: cv 0.8914\ntf_efficientnet_b4_ns_fold_0_5: cv 0.8932\n\nonly 1 fold of vit base from this notebook : [ViT - Pytorch xla (TPU) for leaf disease](https://www.kaggle.com/mobassir/vit-pytorch-xla-tpu-for-leaf-disease)\n\nvit_base_patch16_384_fold_4: cv0.89175\n\nonly 2 fold of vit large from this notebook : [Faster Pytorch TPU baseline for CLD(cv 0.9)](https://www.kaggle.com/mobassir/faster-pytorch-tpu-baseline-for-cld-cv-0-9)\n\n\nvit_large_patch16_384_fold_2: cv 0.89568\nvit_large_patch16_384_fold_1: cv 0.89826\n\nused following tta (3 step):\n\n```\ndef get_inference_transforms(image_size = image_size):\n return Compose([\n RandomResizedCrop(image_size, image_size),\n Transpose(p=0.5),\n HorizontalFlip(p=0.5),\n VerticalFlip(p=0.5),\n HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n ToTensorV2(p=1.0),\n ], p=1.)\n```\n\nand for vit large : \n\n```\ninference_transforms = albumentations.Compose([\n #albumentations.RandomResizedCrop(image_size, image_size),\n albumentations.Resize(image_size, image_size),\n albumentations.Transpose(p=0.5),\n albumentations.HorizontalFlip(p=0.5),\n albumentations.VerticalFlip(p=0.5),\n albumentations.HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.5),\n albumentations.RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n ]) \n``` \n @zlanan found these as best augmentations : \n\n RandomResizedCrop(CFG['img_size'], CFG['img_size']),\n Transpose(p=0.4),\n HorizontalFlip(p=0.4),\n VerticalFlip(p=0.4),\n ShiftScaleRotate(p=0.3),\n MedianBlur(blur_limit=7,always_apply=False, p=0.3),\n IAAAdditiveGaussianNoise(scale=(0, 0.15*255),p=0.5),\n HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit=0.2, val_shift_limit=0.2, p=0.4),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.4),\n Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n CoarseDropout(p=1.0),\n Cutout(p=0.4),\n\nwe blended all models that had cv close to 0.9 or more than that, with that we got public lb 0.8978 and **private lb 0.9019** **(but couldn't select it because of low lb score)**\n\n\n\n\n**Final note**\n\nwe did observe a rise on both LB with lighter TTA, but also a drop. we did observe a improve with different image size model but also a drop. we can not say anything is really useful, except **avg ensemble.**\n\nsome of our early experiment results can be found here : https://docs.google.com/spreadsheets/d/1HSMuTrMmwB5-8GXeJ-h68xeOIEeffcRo2W6HcmfXCzs/edit#gid=0\n\nthank you for reading", "1210894": "Simple but not simple solution. Thank you for sharing the solution and experiment result. :)\n@mobassir", "1210900": "thank you for your all great discussion posts and kernels of this competition @piantic", "1210937": "Thank you for sharing your solution and especially experiment guts). \nI see, you had different loss functions for your models. So the question, by which factor you chose you best epochs: loss or acc? Or even maybe f1 or another additional metric?) \nThis question was unobvious for me, but from some moment i decided to stand on loss comparison.", "1210940": "we always tracked validation multi class accuracy", "1211285": "Was witing for the TOP 3 solutions. Bad luck this time for your leap back. Nice work and learned a lot.", "1211474": "great work thanks!", "1211584": "Congratz @mobassir, that's a nice finish nonetheless !", "1211698": "Which tool did you use to track your validation accuracy?", "1211721": "Sorry i do not understand your question, we monitor validation multi claas accuracy which is a metric i Don't know where the word \"tools\" comes here,you can check this notebook : https://www.kaggle.com/mobassir/faster-pytorch-tpu-baseline-for-cld-cv-0-9\n\nFor understanding how to monitor validation accuracy. \nFor keeping track of best validation accuract we use a simple code like this \nIf Current_accuracy > best_accuracy:\n#save weight", "1211815": "Thank you for sharing!", "1211975": "Good job, and congratulation", "1212269": "I learned a lot from your notebook, thank you for your sharing!", "1212524": "Congrats @mobassir on the gold medal and thank you for sharing your solution. I learnt a lot from your notebooks as well as @cdeotte posts. Just wanted suggestions on how to tune hyper parameters - I was tuning them manually and lost a bit of experimentation time. Also, any suggestion to keep track of models - I use kaggle kernels only and have hard time syncing between my git repo and kaggle experiments.", "1212527": "Ah sorry for not being clear. Since you mentioned \"tracking validation accuracy\" I wondered if you used any ML experiment tracking tool. I use Weights and Biases for that matter. \n\nYes I am familiar with validation multi-class accucary. :)", "1212540": "sorry, we didn't use any ML experiment tracking tool,just tracked and saved weights based on best validation multi-class accuracy", "1212547": "hi @suryajrrafl \nwe haven't got gold unfortunately but regarding your question on hyperparameter tuning,i think you can try something like optuna : https://www.analyticsvidhya.com/blog/2020/11/hyperparameter-tuning-using-optuna/\n\ni am not a good hyperparameter tuner,but as a beginner i used to do a lot of hyperparameter tuning and waste a lot of time+gpu hours,later i realized that it's a skill where i should put less focus and more focus on novelty and designing good solutions.i think most of the kaggle beginners start with hyperparameter tuning like i did :)\n\nfor tracking your experiments and models performance you can fill up a sheet like this : https://docs.google.com/spreadsheets/d/1HSMuTrMmwB5-8GXeJ-h68xeOIEeffcRo2W6HcmfXCzs/edit#gid=0\n\nit really helps a lot and i learned it from my old good team mate @theoviel\n\nmy suggestion(based on my past mistakes) : spend less time on tuning and more time on trying to implement something that looks promising,,because with tuning you can get very smallllll amount of boost most of the times and most of the times they can overfit too,but if you can come up with an different idea+implementation that very few people will try then you can stand out from the crowd with ease,thank you :)", "1212800": "Thanks @mobassir for your patient reply. I too did track my experiments using a sheet like this, but as you said, I focussed on tuning to gain marginal improvements. Your insight gives a better clarity on where to experiment and spend more time on. Thanks once again for your help" }, "source": "meta" }