Trust your CV
\nI entered this competition after Panda and am surprised by the result.
\nThe most important things here should be your CV. Always trust it. A single positive sample in public testset can have 0.0064 effect on public LB and the effect is very likely to accumulate during ensembling and selecting.
\nI used @cdeotte's JPEG files and his triple stratifed CV. They help me quickly start trainning and build a strong CV. Thank to his great work.
Best Single Model
\nI used pytorch and 2xRTX2080 to generate my result and used no meta data.
\nThe best single model is efficientnet-b5 on 512x512 resolution. I firstly train model on 2019 data with 8 classes. Then I finetune it with 2020+2018/2017 data. This method improves 0.01 on CV,0.005 on private LB but no improvement on public LB.
\nAugmentations: ImageCompression, Flips, ShiftRotateScale, HueSaturationValue, RandomBrightnetnessContrast, CutOut.
\nThis model achieves 0.944 on CV (before TTA) and 0.9447-0.9462 on private LB(different averaging method). Sadly I picked the lowest one. xD.
Ensemble
\nI ensembled this model with some of my previous models. Most of these overfitted.
\nThe ensembles scores 0.9452-0.9492 on private LB. Made the wrong choise again. xD.
Trust your CV
\nI entered this competition after Panda and am surprised by the result.
\nThe most important things here should be your CV. Always trust it. A single positive sample in public testset can have 0.0064 effect on public LB and the effect is very likely to accumulate during ensembling and selecting.
\nI used @cdeotte's JPEG files and his triple stratifed CV. They help me quickly start trainning and build a strong CV. Thank to his great work.
Best Single Model
\nI used pytorch and 2xRTX2080 to generate my result and used no meta data.
\nThe best single model is efficientnet-b5 on 512x512 resolution. I firstly train model on 2019 data with 8 classes. Then I finetune it with 2020+2018/2017 data. This method improves 0.01 on CV,0.005 on private LB but no improvement on public LB.
\nAugmentations: ImageCompression, Flips, ShiftRotateScale, HueSaturationValue, RandomBrightnetnessContrast, CutOut.
\nThis model achieves 0.944 on CV (before TTA) and 0.9447-0.9462 on private LB(different averaging method). Sadly I picked the lowest one. xD.
Ensemble
\nI ensembled this model with some of my previous models. Most of these overfitted.
\nThe ensembles scores 0.9452-0.9492 on private LB. Made the wrong choise again. xD.
Congratulations. Pretraining on 8 classes was a smart idea. Increasing CV by 0.01 is big. Great job.
", "rawMarkdown": "Congratulations. Pretraining on 8 classes was a smart idea. Increasing CV by 0.01 is big. Great job.", "votes": null }, { "id": "975012", "postDate": "08/18/2020 05:44:31", "content": "Congrats @rguo97 and thanks for sharing.
", "rawMarkdown": "Congrats @rguo97 and thanks for sharing.", "votes": null }, { "id": "975060", "postDate": "08/18/2020 06:09:24", "content": "Thanks! could you share your code to change the triple stratified CV code to classify 8 classes instead of 2.
", "rawMarkdown": "Thanks! could you share your code to change the triple stratified CV code to classify 8 classes instead of 2.", "votes": null }, { "id": "975091", "postDate": "08/18/2020 06:25:10", "content": "I didn’t make another stratified csv for 2019 data. Just used Chris’s. The diagnosis column is the label for 8 class classification.
", "rawMarkdown": "I didn’t make another stratified csv for 2019 data. Just used Chris’s. The diagnosis column is the label for 8 class classification.", "votes": null }, { "id": "975093", "postDate": "08/18/2020 06:30:41", "content": "Thanks. I started on that and then thought that I should make it a 3 class problem instead of 8 because some of the classes had low frequencies. DId not know how to work with TFR records to make new groups or a new variable.
", "rawMarkdown": "Thanks. I started on that and then thought that I should make it a 3 class problem instead of 8 because some of the classes had low frequencies. DId not know how to work with TFR records to make new groups or a new variable.", "votes": null }, { "id": "975323", "postDate": "08/18/2020 08:44:13", "content": "Congratulations, how many epochs did you train?
", "rawMarkdown": "Congratulations, how many epochs did you train?", "votes": null }, { "id": "976148", "postDate": "08/18/2020 17:02:17", "content": "Congrats
", "rawMarkdown": "Congrats", "votes": null }, { "id": "976260", "postDate": "08/18/2020 18:45:38", "content": "20 for pretrain,10 for finetune
", "rawMarkdown": "20 for pretrain,10 for finetune", "votes": null }, { "id": "976312", "postDate": "08/18/2020 19:28:41", "content": "@rguo97 how do you manage to run b5 512 on RTX 2080? is it due to using half-precision?
", "rawMarkdown": "rguo97 how do you manage to run b5 512 on RTX 2080? is it due to using half-precision?", "votes": null }, { "id": "976434", "postDate": "08/18/2020 21:04:00", "content": "A single card can run batch size 10, with 2 cards, it's 20.
", "rawMarkdown": "A single card can run batch size 10, with 2 cards, it's 20.", "votes": null }, { "id": "977400", "postDate": "08/19/2020 13:05:01", "content": "@rguo97 congrats , which loss function used , focal loss , or BCE
", "rawMarkdown": "rguo97 congrats , which loss function used , focal loss , or BCE", "votes": null }, { "id": "978133", "postDate": "08/20/2020 01:15:02", "content": "I used BCE
", "rawMarkdown": "I used BCE", "votes": null }, { "id": "978138", "postDate": "08/20/2020 01:23:11", "content": "congratulation
", "rawMarkdown": "congratulation", "votes": null }, { "id": "980207", "postDate": "08/21/2020 12:02:17", "content": "Thank you for sharing!
\nWhen finetuning, did you change last linear layer dimension to 3(unknown,n,mel) from 8? or leave as is?
Congratulations. Pretraining on 8 classes was a smart idea. Increasing CV by 0.01 is big. Great job.
", "votes": null, "replies": [] }, { "id": 975012, "author_name": "piantic", "author_url": "", "post_date": "08/18/2020 05:44:31", "content": "Congrats @rguo97 and thanks for sharing.
", "votes": null, "replies": [] }, { "id": 975060, "author_name": "sebastianji", "author_url": "", "post_date": "08/18/2020 06:09:24", "content": "Thanks! could you share your code to change the triple stratified CV code to classify 8 classes instead of 2.
", "votes": null, "replies": [ { "id": 975091, "author_name": "rguo97", "author_url": "", "post_date": "08/18/2020 06:25:10", "content": "I didn’t make another stratified csv for 2019 data. Just used Chris’s. The diagnosis column is the label for 8 class classification.
", "votes": null, "replies": [] }, { "id": 975093, "author_name": "sebastianji", "author_url": "", "post_date": "08/18/2020 06:30:41", "content": "Thanks. I started on that and then thought that I should make it a 3 class problem instead of 8 because some of the classes had low frequencies. DId not know how to work with TFR records to make new groups or a new variable.
", "votes": null, "replies": [] } ] }, { "id": 975323, "author_name": "aliabdin1", "author_url": "", "post_date": "08/18/2020 08:44:13", "content": "Congratulations, how many epochs did you train?
", "votes": null, "replies": [ { "id": 976260, "author_name": "rguo97", "author_url": "", "post_date": "08/18/2020 18:45:38", "content": "20 for pretrain,10 for finetune
", "votes": null, "replies": [] } ] }, { "id": 976312, "author_name": "yimacs", "author_url": "", "post_date": "08/18/2020 19:28:41", "content": "@rguo97 how do you manage to run b5 512 on RTX 2080? is it due to using half-precision?
", "votes": null, "replies": [ { "id": 976434, "author_name": "rguo97", "author_url": "", "post_date": "08/18/2020 21:04:00", "content": "A single card can run batch size 10, with 2 cards, it's 20.
", "votes": null, "replies": [] } ] }, { "id": 977400, "author_name": "rajnishe", "author_url": "", "post_date": "08/19/2020 13:05:01", "content": "@rguo97 congrats , which loss function used , focal loss , or BCE
", "votes": null, "replies": [ { "id": 978133, "author_name": "rguo97", "author_url": "", "post_date": "08/20/2020 01:15:02", "content": "I used BCE
", "votes": null, "replies": [] } ] }, { "id": 978138, "author_name": "zengyaner", "author_url": "", "post_date": "08/20/2020 01:23:11", "content": "congratulation
", "votes": null, "replies": [] }, { "id": 980207, "author_name": "ajtryt2", "author_url": "", "post_date": "08/21/2020 12:02:17", "content": "Thank you for sharing!
\nWhen finetuning, did you change last linear layer dimension to 3(unknown,n,mel) from 8? or leave as is?
Congrats
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "974994": "**Trust your CV**\nI entered this competition after Panda and am surprised by the result.\nThe most important things here should be your CV. Always trust it. A single positive sample in public testset can have 0.0064 effect on public LB and the effect is very likely to accumulate during ensembling and selecting.\nI used @cdeotte's JPEG files and his triple stratifed CV. They help me quickly start trainning and build a strong CV. Thank to his great work.\n\n**Best Single Model**\nI used pytorch and 2xRTX2080 to generate my result and used no meta data.\nThe best single model is efficientnet-b5 on 512x512 resolution. I firstly train model on 2019 data with 8 classes. Then I finetune it with 2020+2018/2017 data. This method improves 0.01 on CV,0.005 on private LB but no improvement on public LB.\nAugmentations: ImageCompression, Flips, ShiftRotateScale, HueSaturationValue, RandomBrightnetnessContrast, CutOut.\nThis model achieves 0.944 on CV (before TTA) and 0.9447-0.9462 on private LB(different averaging method). Sadly I picked the lowest one. xD.\n\n**Ensemble**\nI ensembled this model with some of my previous models. Most of these overfitted.\nThe ensembles scores 0.9452-0.9492 on private LB. Made the wrong choise again. xD.", "974996": "Congratulations. Pretraining on 8 classes was a smart idea. Increasing CV by 0.01 is big. Great job.", "975012": "Congrats @rguo97 and thanks for sharing.", "975060": "Thanks! could you share your code to change the triple stratified CV code to classify 8 classes instead of 2.", "975091": "I didn’t make another stratified csv for 2019 data. Just used Chris’s. The diagnosis column is the label for 8 class classification.", "975093": "Thanks. I started on that and then thought that I should make it a 3 class problem instead of 8 because some of the classes had low frequencies. DId not know how to work with TFR records to make new groups or a new variable.", "975323": "Congratulations, how many epochs did you train?", "976148": "Congrats", "976260": "20 for pretrain,10 for finetune", "976312": "rguo97 how do you manage to run b5 512 on RTX 2080? is it due to using half-precision?", "976434": "A single card can run batch size 10, with 2 cards, it's 20.", "977400": "rguo97 congrats , which loss function used , focal loss , or BCE", "978133": "I used BCE", "978138": "congratulation", "980207": "Thank you for sharing!\nWhen finetuning, did you change last linear layer dimension to 3(unknown,n,mel) from 8? or leave as is?" }, "source": "meta" }