![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2034058%2F3d0e3062cda7a0bfd1c6715503f28612%2F39th.png?generation=1598015353924037&alt=media\")
a soft reminder.
\nas @hengck23 suggested in bengali.ai competition what to do after competition?
\nplease follow the instructions for this competition too.
write a summary of your method and make a post. regardless of your rank.will provide our simple solution soon.
\nif your worked hard honest approach didn't give you any medal then share your approach in the comment box please,it can help many researchers interested for doing further research using this competitions data.thank you everyone
a soft reminder.
\nas @hengck23 suggested in bengali.ai competition what to do after competition?
\nplease follow the instructions for this competition too.
write a summary of your method and make a post. regardless of your rank.will provide our simple solution soon.
\nif your worked hard honest approach didn't give you any medal then share your approach in the comment box please,it can help many researchers interested for doing further research using this competitions data.thank you everyone
i will share part of our 39th place solution
\nmodels of @kcotton21 san
Default Setting
\nidia from kaeruru san
\nSorry. I lost the tracking of the submission score.
\nBut most of them have not been submitted.
\nmarkdown
| model | \ncv | \n
|---|---|
| siim-efb5-512-weights | \n0.92622 | \n
| siim-efb4-512-weights | \n0.92946 | \n
| siim-efb4-384-weights | \n0.91714 | \n
| siim-efb5-384-noisy-weights | \n0.93030 | \n
| siim-efb5-480-noisy-student-coarse | \n0.93368 | \n
| siim-2ndstage-efb5-384 | \n0.91308 | \n
| siim-2ndstage-efb5-384-pretrain-wo-2020 | \n0.92106 | \n
| siim-seresnext50-192-v2-upsample | \n0.91291 | \n
(cv score)
\nalmost : power << rank(0.94890) < simple(0.95096) < log(0.95100)
See the notebook for the code.
\nsimple 0.95096 → 0.95147
\ncode from abhishek thakur ml book
\nhttps://www.amazon.com/Approaching-Almost-Machine-Learning-Problem-ebook/dp/B089P13QHT
\nIn order to minimize the variance of the weights, I split the train set and averaged out the ones with smaller deviations between validation and train scores.(idia from 2019 jigsaw comp 3rd place). but, I don’t know if it worked.
\nplus meta prediction 0.95147 → 0.95158
\n0.95(ensemble prediction) + 0.05(meta prediction)
below image shows our full 39th place solution :
\n
Congratulations on getting a silver medal. Learned some tricks from your solutions. Hope they will come in handy for me in the future. Trust in your CV, they always say. But for the first time, it neither worked out for us in the Public LB nor on the Private. So strange!
Fantastic. This a great solution. You built many wonderful diverse models. Well done Mobassir and team.
", "rawMarkdown": "Fantastic. This a great solution. You built many wonderful diverse models. Well done Mobassir and team.", "votes": 1 } ] }, { "id": 974946, "postDate": "2020-08-18T05:08:11.860Z", "content": "Congrats Mobassir and team. You all did great. What was your teams' CV and what unique things did you use with your models?
", "rawMarkdown": "Congrats Mobassir and team. You all did great. What was your teams' CV and what unique things did you use with your models?", "votes": 3 }, { "id": 975227, "postDate": "2020-08-18T07:55:22.060Z", "content": "@cdeotte sir
\nthank you a lot @cdeotte sir
\nit was not possible to design a silver solution for us without your high quality discussions and notebooks.
\nwe(all 5 team mates) worked on your kernel and analyzed your discussion posts carefully.
\nthe key to success was diversity and strong models and most importantly \"trusting CV\" as you mentioned here : https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175344
more than 1 month we started with very simple tweaking and noted down some experiments result here : https://drive.google.com/file/d/1PQ6yNPGzzwlpNcwR9qBCSabYqZ5dD59a/view?usp=sharing
\ni shared my first plan to team that \"we will design ensemble kernel based on oof\" i mean 1 submission will be based on \"highest ensemble CV\"
\nthis was the plan for 1 submission more than 1 month ago and we worked on that for most of the times.
\nall team mates worked on different modeling and following different strategy with your kernels and this is what helped us the most.
\nThe diversity of the team made the ensemble a success.
\nThanks for the good models!
\nmost of my experiments were based on 512 and 768 images(768 mostly), most of ktr's experiments were based on 480,384,512 images and attempting models like inceptionresnetv2,hair augmentations,xception model etc on top of your baseline tripple stratified kernel,kaerururu san tried coarse dropout and other things for getting boost in simple model, toru san tried b2,b5 together to design another diverge model.
\nand then after lot of experiments i found mixed_loss and it worked well for me,,suggested it to team mates to try and see if it improves their best diverge models or not as it gave me good boost
they tried and got little more boost in their ensemble!
\ngreat team work here was the key to success,everyone worked on diverge model to get strong but diverge models
\nwe did, and finally we blended our best models and got
\nensemble cv: 0.951474,pubic lb 0.9511 and private lb 0.9437
what worked?
\n-> diversity and different strong models
\n-> coarse dropout,hair aug,mixed_loss,b2-b5,with-tabular using pretrained with 2018 etc
what didn't work?
\n-> bad nested blendings
\n-> some losses from this list : https://github.com/subeeshvasu/Awesome-Learning-with-Label-Noise
\n-> gn,frn etc
lessons from past 4/5 consecutive competitions :
\ntrust your CV and don't love public lb position too much!
thank you :)
", "rawMarkdown": "@cdeotte sir\n\nthank you a lot @cdeotte sir\nit was not possible to design a silver solution for us without your high quality discussions and notebooks.\nwe(all 5 team mates) worked on your kernel and analyzed your discussion posts carefully.\nthe key to success was diversity and strong models and most importantly \"trusting CV\" as you mentioned here : https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175344\n\nmore than 1 month we started with very simple tweaking and noted down some experiments result here : https://drive.google.com/file/d/1PQ6yNPGzzwlpNcwR9qBCSabYqZ5dD59a/view?usp=sharing\n\ni shared my first plan to team that \"we will design ensemble kernel based on oof\" i mean 1 submission will be based on \"highest ensemble CV\"\n\nthis was the plan for 1 submission more than 1 month ago and we worked on that for most of the times.\n\nall team mates worked on different modeling and following different strategy with your kernels and this is what helped us the most.\nThe diversity of the team made the ensemble a success.\nThanks for the good models!\nmost of my experiments were based on 512 and 768 images(768 mostly), most of ktr's experiments were based on 480,384,512 images and attempting models like inceptionresnetv2,hair augmentations,xception model etc on top of your baseline tripple stratified kernel,kaerururu san tried coarse dropout and other things for getting boost in simple model, toru san tried b2,b5 together to design another diverge model.\nand then after lot of experiments i found mixed_loss and it worked well for me,,suggested it to team mates to try and see if it improves their best diverge models or not as it gave me good boost\n\nthey tried and got little more boost in their ensemble!\ngreat team work here was the key to success,everyone worked on diverge model to get strong but diverge models\nwe did, and finally we blended our best models and got \nensemble cv: 0.951474,pubic lb 0.9511 and private lb 0.9437\n\nwhat worked? \n-> diversity and different strong models\n-> coarse dropout,hair aug,mixed_loss,b2-b5,with-tabular using pretrained with 2018 etc\n\nwhat didn't work? \n-> bad nested blendings\n-> some losses from this list : https://github.com/subeeshvasu/Awesome-Learning-with-Label-Noise\n-> gn,frn etc\n\nlessons from past 4/5 consecutive competitions : \ntrust your CV and don't love public lb position too much!\n\nthank you :)\n\n\n\n\n\n", "votes": 2 }, { "id": 980950, "postDate": "2020-08-22T03:33:06.223Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ], "comments": [ { "id": 980284, "author_name": "Mobassir", "author_url": "", "post_date": "2020-08-21T13:10:09.150000", "content": "i will share part of our 39th place solution
\nmodels of @kcotton21 san
Default Setting
\nidia from kaeruru san
\nSorry. I lost the tracking of the submission score.
\nBut most of them have not been submitted.
\nmarkdown
| model | \ncv | \n
|---|---|
| siim-efb5-512-weights | \n0.92622 | \n
| siim-efb4-512-weights | \n0.92946 | \n
| siim-efb4-384-weights | \n0.91714 | \n
| siim-efb5-384-noisy-weights | \n0.93030 | \n
| siim-efb5-480-noisy-student-coarse | \n0.93368 | \n
| siim-2ndstage-efb5-384 | \n0.91308 | \n
| siim-2ndstage-efb5-384-pretrain-wo-2020 | \n0.92106 | \n
| siim-seresnext50-192-v2-upsample | \n0.91291 | \n
(cv score)
\nalmost : power << rank(0.94890) < simple(0.95096) < log(0.95100)
See the notebook for the code.
\nsimple 0.95096 → 0.95147
\ncode from abhishek thakur ml book
\nhttps://www.amazon.com/Approaching-Almost-Machine-Learning-Problem-ebook/dp/B089P13QHT
\nIn order to minimize the variance of the weights, I split the train set and averaged out the ones with smaller deviations between validation and train scores.(idia from 2019 jigsaw comp 3rd place). but, I don’t know if it worked.
\nplus meta prediction 0.95147 → 0.95158
\n0.95(ensemble prediction) + 0.05(meta prediction)
below image shows our full 39th place solution :
\n
Congratulations on getting a silver medal. Learned some tricks from your solutions. Hope they will come in handy for me in the future. Trust in your CV, they always say. But for the first time, it neither worked out for us in the Public LB nor on the Private. So strange!
Fantastic. This a great solution. You built many wonderful diverse models. Well done Mobassir and team.
", "votes": 1, "replies": [] } ] }, { "id": 974946, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-08-18T05:08:11.860000", "content": "Congrats Mobassir and team. You all did great. What was your teams' CV and what unique things did you use with your models?
", "votes": 3, "replies": [] }, { "id": 975227, "author_name": "Mobassir", "author_url": "", "post_date": "2020-08-18T07:55:22.060000", "content": "@cdeotte sir
\nthank you a lot @cdeotte sir
\nit was not possible to design a silver solution for us without your high quality discussions and notebooks.
\nwe(all 5 team mates) worked on your kernel and analyzed your discussion posts carefully.
\nthe key to success was diversity and strong models and most importantly \"trusting CV\" as you mentioned here : https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175344
more than 1 month we started with very simple tweaking and noted down some experiments result here : https://drive.google.com/file/d/1PQ6yNPGzzwlpNcwR9qBCSabYqZ5dD59a/view?usp=sharing
\ni shared my first plan to team that \"we will design ensemble kernel based on oof\" i mean 1 submission will be based on \"highest ensemble CV\"
\nthis was the plan for 1 submission more than 1 month ago and we worked on that for most of the times.
\nall team mates worked on different modeling and following different strategy with your kernels and this is what helped us the most.
\nThe diversity of the team made the ensemble a success.
\nThanks for the good models!
\nmost of my experiments were based on 512 and 768 images(768 mostly), most of ktr's experiments were based on 480,384,512 images and attempting models like inceptionresnetv2,hair augmentations,xception model etc on top of your baseline tripple stratified kernel,kaerururu san tried coarse dropout and other things for getting boost in simple model, toru san tried b2,b5 together to design another diverge model.
\nand then after lot of experiments i found mixed_loss and it worked well for me,,suggested it to team mates to try and see if it improves their best diverge models or not as it gave me good boost
they tried and got little more boost in their ensemble!
\ngreat team work here was the key to success,everyone worked on diverge model to get strong but diverge models
\nwe did, and finally we blended our best models and got
\nensemble cv: 0.951474,pubic lb 0.9511 and private lb 0.9437
what worked?
\n-> diversity and different strong models
\n-> coarse dropout,hair aug,mixed_loss,b2-b5,with-tabular using pretrained with 2018 etc
what didn't work?
\n-> bad nested blendings
\n-> some losses from this list : https://github.com/subeeshvasu/Awesome-Learning-with-Label-Noise
\n-> gn,frn etc
lessons from past 4/5 consecutive competitions :
\ntrust your CV and don't love public lb position too much!
thank you :)
", "votes": 2, "replies": [] }, { "id": 980950, "author_name": "", "author_url": "", "post_date": "2020-08-22T03:33:06.223000", "content": "", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "974918": "a soft reminder.\nas @hengck23 suggested in bengali.ai competition [**what to do after competition?**](https://www.kaggle.com/c/bengaliai-cv19/discussion/136027)\nplease follow the instructions for this competition too.\n\n**write a summary of your method and make a post. regardless of your rank**.will provide our simple solution soon.\nif your **worked hard honest approach** didn't give you any medal then share your approach in the comment box please,it can help many researchers interested for doing further research using this competitions data.thank you everyone", "980284": "i will share part of our 39th place solution\nmodels of @kcotton21 san\n\nDefault Setting\n- validation: Chris’s triple stratified leak-free CV\n - with 2018 data\n- augmentation:\npython\nimg = transform(img)\nimg = tf.image.random_flip_left_right(img)\nimg = tf.image.random_flip_up_down(img)\nimg = tf.image.rot90(img, tf.random.uniform(shape=[], minval=0, maxval=4, dtype=tf.int32))\nimg = tf.image.random_hue(img, 0.01)\nimg = tf.image.random_saturation(img, 0.7, 1.3)\nimg = tf.image.random_contrast(img, 0.8, 1.2)\nimg = tf.image.random_brightness(img, 0.1)\n- TTA: 11\n## Default Models\n### magenet\n- siim-efb5-512-weights\n- siim-efb4-512-weights\n- siim-efb4-384-weights\n### noisy-student\n- siim-efb5-384-noisy-weights\n- siim-efb5-480-noisy-student-coarse\n - +coarse dropout augmentation\n### pretraining models\nidia from kaeruru san\n- siim-2ndstage-efb5-384\n - include everything data pretrain: 2018, 2019, 2020, upsample data\n - fine-tuning: 2020 data\n- siim-2ndstage-efb5-384-pretrain-wo-2020\n - without 2020 data pratrain : 2018, 2019, upsample data\n - fine-tuning: 2020 data\n### other\n- siim-seresnext50-192-v2-upsample\n - seresnext50 size 192x192\n - train with upsampling data\n### Meta\n- LightGBM (target Stratified-5Fold)\n - with image size Public/Private: 0.8616/0.8532\n - I removed the image size after submission because the image size correlation may not be the same in the test data\n## Scores\nSorry. I lost the tracking of the submission score.\nBut most of them have not been submitted.\nmarkdown\n| model | cv |\n|-----------------------------------------|---------|\n| siim-efb5-512-weights | 0.92622 |\n| siim-efb4-512-weights | 0.92946 |\n| siim-efb4-384-weights | 0.91714 |\n| siim-efb5-384-noisy-weights | 0.93030 |\n| siim-efb5-480-noisy-student-coarse | 0.93368 |\n| siim-2ndstage-efb5-384 | 0.91308 |\n| siim-2ndstage-efb5-384-pretrain-wo-2020 | 0.92106 |\n| siim-seresnext50-192-v2-upsample | 0.91291 |\n## Ensemble\n(cv score)\nalmost : power << rank(0.94890) < simple(0.95096) < log(0.95100)\n### Optimize AUC\nSee the notebook for the code.\nsimple 0.95096 → 0.95147\ncode from abhishek thakur ml book\n[https://www.amazon.com/Approaching-Almost-Machine-Learning-Problem-ebook/dp/B089P13QHT](https://www.amazon.com/Approaching-Almost-Machine-Learning-Problem-ebook/dp/B089P13QHT)\nIn order to minimize the variance of the weights, I split the train set and averaged out the ones with smaller deviations between validation and train scores.(idia from 2019 jigsaw comp 3rd place). but, I don’t know if it worked.\nplus meta prediction 0.95147 → 0.95158\n0.95*(ensemble prediction) + 0.05*(meta prediction)\n\nbelow image shows our full 39th place solution :\n\n", "974946": "Congrats Mobassir and team. You all did great. What was your teams' CV and what unique things did you use with your models?", "975227": "@cdeotte sir\n\nthank you a lot @cdeotte sir\nit was not possible to design a silver solution for us without your high quality discussions and notebooks.\nwe(all 5 team mates) worked on your kernel and analyzed your discussion posts carefully.\nthe key to success was diversity and strong models and most importantly \"trusting CV\" as you mentioned here : https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175344\n\nmore than 1 month we started with very simple tweaking and noted down some experiments result here : https://drive.google.com/file/d/1PQ6yNPGzzwlpNcwR9qBCSabYqZ5dD59a/view?usp=sharing\n\ni shared my first plan to team that \"we will design ensemble kernel based on oof\" i mean 1 submission will be based on \"highest ensemble CV\"\n\nthis was the plan for 1 submission more than 1 month ago and we worked on that for most of the times.\n\nall team mates worked on different modeling and following different strategy with your kernels and this is what helped us the most.\nThe diversity of the team made the ensemble a success.\nThanks for the good models!\nmost of my experiments were based on 512 and 768 images(768 mostly), most of ktr's experiments were based on 480,384,512 images and attempting models like inceptionresnetv2,hair augmentations,xception model etc on top of your baseline tripple stratified kernel,kaerururu san tried coarse dropout and other things for getting boost in simple model, toru san tried b2,b5 together to design another diverge model.\nand then after lot of experiments i found mixed_loss and it worked well for me,,suggested it to team mates to try and see if it improves their best diverge models or not as it gave me good boost\n\nthey tried and got little more boost in their ensemble!\ngreat team work here was the key to success,everyone worked on diverge model to get strong but diverge models\nwe did, and finally we blended our best models and got \nensemble cv: 0.951474,pubic lb 0.9511 and private lb 0.9437\n\nwhat worked? \n-> diversity and different strong models\n-> coarse dropout,hair aug,mixed_loss,b2-b5,with-tabular using pretrained with 2018 etc\n\nwhat didn't work? \n-> bad nested blendings\n-> some losses from this list : https://github.com/subeeshvasu/Awesome-Learning-with-Label-Noise\n-> gn,frn etc\n\nlessons from past 4/5 consecutive competitions : \ntrust your CV and don't love public lb position too much!\n\nthank you :)\n\n\n\n\n\n", "980950": "" } }