Code available here: https://github.com/Masdevallia/3rd-place-kaggle-siim-isic-melanoma-classification
\nHello everybody!
\nWell⦠I am speechless. I am quite new at Kaggle and was not expecting such a good result, it has taken me completely by surprise.
\nFirst of all, I need to deeply thank the entire Kaggle community. I've learned a lot throughout the entire competition thanks to all the knowledge and insights you have generously shared. Thank you also to the organizers and Kaggle for hosting the competition.
\nI am away on vacation with a limited internet connection, but I will share my solution as soon as I can.
\nAs a quick summary, my main submission was an ensemble of 8 different models built with various combinations of image sizes (256, 384, 512, 768). Many thanks to @vbhargav875, whose notebook \"EfficientNet-B5_B6_B7 TF-Keras\" was an incredible starting point. I used CV to implement some experiments, but my finals models where obtained with all available data (without validation). I used 2017-2018-2019 + 2020 TFrecords (huge thanks to @cdeotte), hair augmentation (thanks to @nroman and @graf10a), heavy TTA, EfficientNet-B6 models and metadata (thanks to @titericz). This approach scored: 0.9481 private LB, 0.9596 public LB.
\nHowever, I wanted to try to ensemble some public notebooks too, in order to add some diversity. I decided to go ahead with these two amazing notebooks, which introduced some juicy techniques that I didn't have time to test:
\nThis approach scored: 0.9484 private LB, 0.9620 public LB.
\nCongratulations to all participants! I know that I still have a long way to go, but I am looking forward to continuing to grow alongside this incredible community.
", "messages": [ { "id": "976433", "postDate": "08/18/2020 21:03:58", "content": "Code available here: https://github.com/Masdevallia/3rd-place-kaggle-siim-isic-melanoma-classification
\nHello everybody!
\nWell⦠I am speechless. I am quite new at Kaggle and was not expecting such a good result, it has taken me completely by surprise.
\nFirst of all, I need to deeply thank the entire Kaggle community. I've learned a lot throughout the entire competition thanks to all the knowledge and insights you have generously shared. Thank you also to the organizers and Kaggle for hosting the competition.
\nI am away on vacation with a limited internet connection, but I will share my solution as soon as I can.
\nAs a quick summary, my main submission was an ensemble of 8 different models built with various combinations of image sizes (256, 384, 512, 768). Many thanks to @vbhargav875, whose notebook \"EfficientNet-B5_B6_B7 TF-Keras\" was an incredible starting point. I used CV to implement some experiments, but my finals models where obtained with all available data (without validation). I used 2017-2018-2019 + 2020 TFrecords (huge thanks to @cdeotte), hair augmentation (thanks to @nroman and @graf10a), heavy TTA, EfficientNet-B6 models and metadata (thanks to @titericz). This approach scored: 0.9481 private LB, 0.9596 public LB.
\nHowever, I wanted to try to ensemble some public notebooks too, in order to add some diversity. I decided to go ahead with these two amazing notebooks, which introduced some juicy techniques that I didn't have time to test:
\nThis approach scored: 0.9484 private LB, 0.9620 public LB.
\nCongratulations to all participants! I know that I still have a long way to go, but I am looking forward to continuing to grow alongside this incredible community.
", "rawMarkdown": "Code available here: https://github.com/Masdevallia/3rd-place-kaggle-siim-isic-melanoma-classification\n\nHello everybody!\n\nWell... I am speechless. I am quite new at Kaggle and was not expecting such a good result, it has taken me completely by surprise.\n\nFirst of all, I need to deeply thank the entire Kaggle community. I've learned a lot throughout the entire competition thanks to all the knowledge and insights you have generously shared. Thank you also to the organizers and Kaggle for hosting the competition.\n\nI am away on vacation with a limited internet connection, but I will share my solution as soon as I can.\n\nAs a quick summary, my main submission was an ensemble of 8 different models built with various combinations of image sizes (256, 384, 512, 768). Many thanks to @vbhargav875, whose notebook \"EfficientNet-B5_B6_B7 TF-Keras\" was an incredible starting point. I used CV to implement some experiments, but my finals models where obtained with all available data (without validation). I used 2017-2018-2019 + 2020 TFrecords (huge thanks to @cdeotte), hair augmentation (thanks to @nroman and @graf10a), heavy TTA, EfficientNet-B6 models and metadata (thanks to @titericz). This approach scored: 0.9481 private LB, 0.9596 public LB.\n\nHowever, I wanted to try to ensemble some public notebooks too, in order to add some diversity. I decided to go ahead with these two amazing notebooks, which introduced some juicy techniques that I didn't have time to test:\n- https://www.kaggle.com/rajnishe/rc-fork-siim-isic-melanoma-384x384 (thanks to @rajnishe).\n- https://www.kaggle.com/ajaykumar7778/efficientnet-cv (thanks to @ajaykumar7778).\n\nThis approach scored: 0.9484 private LB, 0.9620 public LB.\n\nCongratulations to all participants! I know that I still have a long way to go, but I am looking forward to continuing to grow alongside this incredible community.", "votes": null }, { "id": "976438", "postDate": "08/18/2020 21:08:14", "content": "Congratulations! Cant wait to see your code and learn from youπ
", "rawMarkdown": "Congratulations! Cant wait to see your code and learn from youπ", "votes": null }, { "id": "976553", "postDate": "08/19/2020 00:16:04", "content": "Great models Masdevallia. Congratulations on your amazing accomplishment !
", "rawMarkdown": "Great models Masdevallia. Congratulations on your amazing accomplishment !", "votes": null }, { "id": "976793", "postDate": "08/19/2020 05:17:31", "content": "Congratulations @masdevallia for your great win .
", "rawMarkdown": "Congratulations @masdevallia for your great win .", "votes": null }, { "id": "976803", "postDate": "08/19/2020 05:28:18", "content": "Congratulations a great result!
", "rawMarkdown": "Congratulations a great result!", "votes": null }, { "id": "976806", "postDate": "08/19/2020 05:31:56", "content": "Wow, wow π
\nGreat win, Congratulations @masdevallia π
@cdeotte thanks for your datasets which help us in model creation part.
\nIt was great learning from your notebooks during competition and also notebook of forward ensemble is gem. May be few words you would like to share on how to upskill from here and how you build yourself .
\nIt will be a great roadmap for all of us learner here.
\nThanks again
Congratulations @masdevallia, you chose the right way to achieve a great result.
\nExcellent debut here at Kaggle Competitions!
Great work!
", "rawMarkdown": "Great work!", "votes": null }, { "id": "977010", "postDate": "08/19/2020 08:24:59", "content": "Congrats!
", "rawMarkdown": "Congrats!", "votes": null }, { "id": "977024", "postDate": "08/19/2020 08:39:51", "content": "Congratulations on the great result @masdevallia! You were smart about selecting only a few of the public notebooks that could add diversity to your ensemble and further improve the score. Nice job!
\nYou said you were training your final models on full data without validation. How did you select the number of epochs? Was it based on when the early stopping was occurring in your previous CV results?
", "rawMarkdown": "Congratulations on the great result @masdevallia! You were smart about selecting only a few of the public notebooks that could add diversity to your ensemble and further improve the score. Nice job!\n\nYou said you were training your final models on full data without validation. How did you select the number of epochs? Was it based on when the early stopping was occurring in your previous CV results?", "votes": null }, { "id": "977150", "postDate": "08/19/2020 10:19:26", "content": "Congratulations!
\n\n\nmy finals models where obtained with all available data (without validation
\n
That's something I wanted to do and forgot unfortunately. I am not surprised it worked well.
", "rawMarkdown": "Congratulations!\n\n> my finals models where obtained with all available data (without validation\n\nThat's something I wanted to do and forgot unfortunately. I am not surprised it worked well.", "votes": null }, { "id": "977180", "postDate": "08/19/2020 10:38:45", "content": "Worked worse for us :)
", "rawMarkdown": "Worked worse for us :)", "votes": null }, { "id": "977243", "postDate": "08/19/2020 11:22:01", "content": "Great job!
\nIncluding all data (2020+2019+2018/17) seems to have worked better. I did not try it based on premise that 2020+19 was worse in CV than 2020+18/17. Stupid me.
", "rawMarkdown": "Great job!\n\nIncluding all data (2020+2019+2018/17) seems to have worked better. I did not try it based on premise that 2020+19 was worse in CV than 2020+18/17. Stupid me.", "votes": null }, { "id": "977434", "postDate": "08/19/2020 13:24:48", "content": "Congrats @masdevallia and your solution and results! Cool to see you started with a different baseline model than most others. When I saw you get 3rd place I noticed you had this public dataset: melanoma384x384nohair - Did you use hair removal in your final solution at all? I didn't see anything about it in your writeup.
\n\n\nI used CV to implement some experiments, but my finals models where obtained with all available data (without validation).
\n
This is a smart technique. Unfortunately our team did not have time to do this at the end. How did you determine the epochs you trained on for your final models? Did you set blending weights based on CV?
", "rawMarkdown": "Congrats @masdevallia and your solution and results! Cool to see you started with a different baseline model than most others. When I saw you get 3rd place I noticed you had this public dataset: [melanoma384x384nohair](https://www.kaggle.com/masdevallia/melanoma384x384nohair) - Did you use hair removal in your final solution at all? I didn't see anything about it in your writeup.\n\n> I used CV to implement some experiments, but my finals models where obtained with all available data (without validation).\n\nThis is a smart technique. Unfortunately our team did not have time to do this at the end. How did you determine the epochs you trained on for your final models? Did you set blending weights based on CV?", "votes": null }, { "id": "977436", "postDate": "08/19/2020 13:27:32", "content": "\n\nWorked worse for us :)
\n
Interesting. Any idea why @philippsinger? I've always assumed that if you could determine the correct epochs to not over/under fit then training a model on all data would be better than multiple CV split models.
", "rawMarkdown": "> Worked worse for us :)\n\nInteresting. Any idea why @philippsinger? I've always assumed that if you could determine the correct epochs to not over/under fit then training a model on all data would be better than multiple CV split models.", "votes": null }, { "id": "977466", "postDate": "08/19/2020 13:47:29", "content": "Yes, private LB is also random to some degree :)
\nIt is also not exact same models and blending method in both \"comparable\" subs. That said k-fold blend can be better in theory if diverse sub-samples blend better, I personally do not believe this to be the case here - also fullfits always worked better on pub LB for us here.
", "rawMarkdown": "Yes, private LB is also random to some degree :)\n\nIt is also not exact same models and blending method in both \"comparable\" subs. That said k-fold blend can be better in theory if diverse sub-samples blend better, I personally do not believe this to be the case here - also fullfits always worked better on pub LB for us here.", "votes": null }, { "id": "977796", "postDate": "08/19/2020 17:54:26", "content": "Congrats! Waiting for your detailed solution~
", "rawMarkdown": "Congrats! Waiting for your detailed solution~", "votes": null }, { "id": "977842", "postDate": "08/19/2020 18:35:03", "content": "@masdevallia congrats for 3rd place!
\ncan you explain a bit on this part
\n\" I used CV to implement some experiments, but my finals models where obtained with all available data (without validation)\"
\ndo you mean that you did not used k-fold-split-training ? and how does this idea help? looking for some insightsβ¦
Congratulations
", "rawMarkdown": "Congratulations", "votes": null }, { "id": "979197", "postDate": "08/20/2020 17:27:18", "content": "Congrats. It is inspiring to see new faces winning big time. If possible, do share your solution whenever you get the time and if possible a more detailed account of your experiences. All the very best for your future competitions
", "rawMarkdown": "Congrats. It is inspiring to see new faces winning big time. If possible, do share your solution whenever you get the time and if possible a more detailed account of your experiences. All the very best for your future competitions", "votes": null }, { "id": "979218", "postDate": "08/20/2020 17:42:06", "content": "Excellent! We look forward to seeing more from you.
", "rawMarkdown": "Excellent! We look forward to seeing more from you.", "votes": null }, { "id": "982344", "postDate": "08/23/2020 09:39:50", "content": "Congratulations for the high score!
", "rawMarkdown": "Congratulations for the high score!", "votes": null }, { "id": "982682", "postDate": "08/23/2020 15:19:56", "content": "Hello
", "rawMarkdown": "Hello", "votes": null }, { "id": "983575", "postDate": "08/24/2020 12:15:05", "content": "Hi @pawankumarsahu ,
\nI used k-fold cross-validation to estimate the hyperparameters of the model (hyperparameter tuning) and then used those optimized hyperparameters to fit a model to the whole dataset. This procedure is based on the idea that the more data you use the more likely it is to have a robust model that generalizes well.
Exactly @kozodoi! Although I had to reduce the \"optimal\" number of epochs on some occasions to avoid exceeding the limit of Kaggle's TPU (since I was using more data and also hair augmentation, which was quite computationally expensive).
", "rawMarkdown": "Exactly @kozodoi! Although I had to reduce the \"optimal\" number of epochs on some occasions to avoid exceeding the limit of Kaggle's TPU (since I was using more data and also hair augmentation, which was quite computationally expensive).", "votes": null }, { "id": "983612", "postDate": "08/24/2020 12:49:18", "content": "Thank you very much for all the comments! :)
", "rawMarkdown": "Thank you very much for all the comments! :)", "votes": null }, { "id": "983643", "postDate": "08/24/2020 13:18:16", "content": "Geat work , congrats
", "rawMarkdown": "Geat work , congrats", "votes": null }, { "id": "983669", "postDate": "08/24/2020 13:45:44", "content": "Thank you, @robikscube! I did not use hair removal in my final solution. I tested it and it worked worse for me than hair augmentation, so I decided to go this other way. The number of epochs was mostly determined based on when the early stopping was occurring in my previous CV results (with some adjustments to avoid exceeding the limit of Kaggle's TPU). Regarding my ensembles, I tried several things, but my final solution is actually a simple average of the 10 models (8 own models + 2 public notebooks). Finally I added metadata to this ensemble with a weighted average. I know I have a lot to learn about ensemble techniques and that I was pretty lucky to get such a good result with such a simple approach.
", "rawMarkdown": "Thank you, @robikscube! I did not use hair removal in my final solution. I tested it and it worked worse for me than hair augmentation, so I decided to go this other way. The number of epochs was mostly determined based on when the early stopping was occurring in my previous CV results (with some adjustments to avoid exceeding the limit of Kaggle's TPU). Regarding my ensembles, I tried several things, but my final solution is actually a simple average of the 10 models (8 own models + 2 public notebooks). Finally I added metadata to this ensemble with a weighted average. I know I have a lot to learn about ensemble techniques and that I was pretty lucky to get such a good result with such a simple approach.", "votes": null }, { "id": "983702", "postDate": "08/24/2020 14:24:12", "content": "Try not to sell yourself short, you did several things top competitors didn't try and your instincts were clearly good.
\nI'd be interested to see how the leaderboard score compares for a model trained on all of the data and a model trained with cross-validation. I was uncomfortable trying this as I was fearful of submitting a model without validating it. But I guess it wouldn't change too much as you had validated, just with a little less training data. I'll make sure to try this on future image comps!
", "rawMarkdown": "Try not to sell yourself short, you did several things top competitors didn't try and your instincts were clearly good. \n\nI'd be interested to see how the leaderboard score compares for a model trained on all of the data and a model trained with cross-validation. I was uncomfortable trying this as I was fearful of submitting a model without validating it. But I guess it wouldn't change too much as you had validated, just with a little less training data. I'll make sure to try this on future image comps!", "votes": null }, { "id": "985844", "postDate": "08/26/2020 03:52:29", "content": "WOW!!
\nNice and clear
congratulations for you first gold.
", "rawMarkdown": "congratulations for you first gold.", "votes": null }, { "id": "1002574", "postDate": "09/08/2020 08:23:50", "content": "Iβm very deloitted that you won!
", "rawMarkdown": "Iβm very deloitted that you won!", "votes": null }, { "id": "1003178", "postDate": "09/08/2020 17:54:18", "content": "πππ Thank you!
", "rawMarkdown": "πππ Thank you!", "votes": null }, { "id": "1008698", "postDate": "09/13/2020 10:20:57", "content": "Congratulation !!
", "rawMarkdown": "Congratulation !!", "votes": null }, { "id": "1008727", "postDate": "09/13/2020 10:48:56", "content": "Thank you and congrats!
", "rawMarkdown": "Thank you and congrats!", "votes": null }, { "id": "1008845", "postDate": "09/13/2020 12:43:01", "content": "Awesome! great work
", "rawMarkdown": "Awesome! great work", "votes": null }, { "id": "1011665", "postDate": "09/15/2020 16:03:14", "content": "Congratulations on the great result @masdevallia!
\nInteresting to see your solution! Do I understand your notebook right, so that you just mixed B6 models with variable image sizes and image data? Well done!
Yes, exactly! Thank you!
", "rawMarkdown": "Yes, exactly! Thank you!", "votes": null }, { "id": "1312652", "postDate": "05/18/2021 07:05:27", "content": "@masdevallia Thanks for sharing ! Amazing work .
", "rawMarkdown": "masdevallia Thanks for sharing ! Amazing work .", "votes": null }, { "id": "2017955", "postDate": "11/05/2022 09:01:57", "content": "EDIT: Already figured out the question asked. So I will just keep the greeting to you :). Great resource to explain the crop resize reason found here: https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175344
\nReally great job @masdevallia, your notebook has really helped me to understand and make sense of the TPU image detection model building.
\nThanks in advance, and (late) congratulations on your work!
", "rawMarkdown": "EDIT: Already figured out the question asked. So I will just keep the greeting to you :). Great resource to explain the crop resize reason found here: https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175344\n\nReally great job @masdevallia, your notebook has really helped me to understand and make sense of the TPU image detection model building.\n\nThanks in advance, and (late) congratulations on your work!", "votes": null }, { "id": "2388153", "postDate": "08/13/2023 07:14:05", "content": "Hey, Congratulations @masdevallia. It was so great for you to achive this, I am sorry I know that I am late to congralute you. Can you help me in some of the topics that you have used in this competition to win. Because I am totally new to these advanced concepts, it would be great that if I learned these concepts from you :)
", "rawMarkdown": "Hey, Congratulations @masdevallia. It was so great for you to achive this, I am sorry I know that I am late to congralute you. Can you help me in some of the topics that you have used in this competition to win. Because I am totally new to these advanced concepts, it would be great that if I learned these concepts from you :)", "votes": null }, { "id": "3068695", "postDate": "12/10/2024 15:09:26", "content": "Congratulations on the great result @masdevallia!
\nI would like to ask if you have used metadata, and if so, at which specific location in the code have you used it?
Congratulations! Cant wait to see your code and learn from youπ
", "votes": null, "replies": [] }, { "id": 976553, "author_name": "cdeotte", "author_url": "", "post_date": "08/19/2020 00:16:04", "content": "Great models Masdevallia. Congratulations on your amazing accomplishment !
", "votes": null, "replies": [ { "id": 976839, "author_name": "rajnishe", "author_url": "", "post_date": "08/19/2020 06:05:45", "content": "@cdeotte thanks for your datasets which help us in model creation part.
\nIt was great learning from your notebooks during competition and also notebook of forward ensemble is gem. May be few words you would like to share on how to upskill from here and how you build yourself .
\nIt will be a great roadmap for all of us learner here.
\nThanks again
Congratulations @masdevallia for your great win .
", "votes": null, "replies": [] }, { "id": 976803, "author_name": "fchmiel", "author_url": "", "post_date": "08/19/2020 05:28:18", "content": "Congratulations a great result!
", "votes": null, "replies": [] }, { "id": 976806, "author_name": "rahim3", "author_url": "", "post_date": "08/19/2020 05:31:56", "content": "Wow, wow π
\nGreat win, Congratulations @masdevallia π
Congratulations @masdevallia, you chose the right way to achieve a great result.
\nExcellent debut here at Kaggle Competitions!
Great work!
", "votes": null, "replies": [] }, { "id": 977010, "author_name": "abisheksudarshan", "author_url": "", "post_date": "08/19/2020 08:24:59", "content": "Congrats!
", "votes": null, "replies": [] }, { "id": 977024, "author_name": "kozodoi", "author_url": "", "post_date": "08/19/2020 08:39:51", "content": "Congratulations on the great result @masdevallia! You were smart about selecting only a few of the public notebooks that could add diversity to your ensemble and further improve the score. Nice job!
\nYou said you were training your final models on full data without validation. How did you select the number of epochs? Was it based on when the early stopping was occurring in your previous CV results?
", "votes": null, "replies": [ { "id": 983591, "author_name": "masdevallia", "author_url": "", "post_date": "08/24/2020 12:33:17", "content": "Exactly @kozodoi! Although I had to reduce the \"optimal\" number of epochs on some occasions to avoid exceeding the limit of Kaggle's TPU (since I was using more data and also hair augmentation, which was quite computationally expensive).
", "votes": null, "replies": [] } ] }, { "id": 977150, "author_name": "cpmpml", "author_url": "", "post_date": "08/19/2020 10:19:26", "content": "Congratulations!
\n\n\nmy finals models where obtained with all available data (without validation
\n
That's something I wanted to do and forgot unfortunately. I am not surprised it worked well.
", "votes": null, "replies": [ { "id": 977180, "author_name": "philippsinger", "author_url": "", "post_date": "08/19/2020 10:38:45", "content": "Worked worse for us :)
", "votes": null, "replies": [] }, { "id": 977436, "author_name": "robikscube", "author_url": "", "post_date": "08/19/2020 13:27:32", "content": "\n\nWorked worse for us :)
\n
Interesting. Any idea why @philippsinger? I've always assumed that if you could determine the correct epochs to not over/under fit then training a model on all data would be better than multiple CV split models.
", "votes": null, "replies": [] }, { "id": 977466, "author_name": "philippsinger", "author_url": "", "post_date": "08/19/2020 13:47:29", "content": "Yes, private LB is also random to some degree :)
\nIt is also not exact same models and blending method in both \"comparable\" subs. That said k-fold blend can be better in theory if diverse sub-samples blend better, I personally do not believe this to be the case here - also fullfits always worked better on pub LB for us here.
", "votes": null, "replies": [] } ] }, { "id": 977243, "author_name": "manjeshg03", "author_url": "", "post_date": "08/19/2020 11:22:01", "content": "Great job!
\nIncluding all data (2020+2019+2018/17) seems to have worked better. I did not try it based on premise that 2020+19 was worse in CV than 2020+18/17. Stupid me.
", "votes": null, "replies": [] }, { "id": 977434, "author_name": "robikscube", "author_url": "", "post_date": "08/19/2020 13:24:48", "content": "Congrats @masdevallia and your solution and results! Cool to see you started with a different baseline model than most others. When I saw you get 3rd place I noticed you had this public dataset: melanoma384x384nohair - Did you use hair removal in your final solution at all? I didn't see anything about it in your writeup.
\n\n\nI used CV to implement some experiments, but my finals models where obtained with all available data (without validation).
\n
This is a smart technique. Unfortunately our team did not have time to do this at the end. How did you determine the epochs you trained on for your final models? Did you set blending weights based on CV?
", "votes": null, "replies": [ { "id": 983669, "author_name": "masdevallia", "author_url": "", "post_date": "08/24/2020 13:45:44", "content": "Thank you, @robikscube! I did not use hair removal in my final solution. I tested it and it worked worse for me than hair augmentation, so I decided to go this other way. The number of epochs was mostly determined based on when the early stopping was occurring in my previous CV results (with some adjustments to avoid exceeding the limit of Kaggle's TPU). Regarding my ensembles, I tried several things, but my final solution is actually a simple average of the 10 models (8 own models + 2 public notebooks). Finally I added metadata to this ensemble with a weighted average. I know I have a lot to learn about ensemble techniques and that I was pretty lucky to get such a good result with such a simple approach.
", "votes": null, "replies": [] }, { "id": 983702, "author_name": "fchmiel", "author_url": "", "post_date": "08/24/2020 14:24:12", "content": "Try not to sell yourself short, you did several things top competitors didn't try and your instincts were clearly good.
\nI'd be interested to see how the leaderboard score compares for a model trained on all of the data and a model trained with cross-validation. I was uncomfortable trying this as I was fearful of submitting a model without validating it. But I guess it wouldn't change too much as you had validated, just with a little less training data. I'll make sure to try this on future image comps!
", "votes": null, "replies": [] } ] }, { "id": 977796, "author_name": "yuanlin08", "author_url": "", "post_date": "08/19/2020 17:54:26", "content": "Congrats! Waiting for your detailed solution~
", "votes": null, "replies": [] }, { "id": 977842, "author_name": "pawankumarsahu", "author_url": "", "post_date": "08/19/2020 18:35:03", "content": "@masdevallia congrats for 3rd place!
\ncan you explain a bit on this part
\n\" I used CV to implement some experiments, but my finals models where obtained with all available data (without validation)\"
\ndo you mean that you did not used k-fold-split-training ? and how does this idea help? looking for some insightsβ¦
Hi @pawankumarsahu ,
\nI used k-fold cross-validation to estimate the hyperparameters of the model (hyperparameter tuning) and then used those optimized hyperparameters to fit a model to the whole dataset. This procedure is based on the idea that the more data you use the more likely it is to have a robust model that generalizes well.
Congrats. It is inspiring to see new faces winning big time. If possible, do share your solution whenever you get the time and if possible a more detailed account of your experiences. All the very best for your future competitions
", "votes": null, "replies": [] }, { "id": 979218, "author_name": "brianfeeny", "author_url": "", "post_date": "08/20/2020 17:42:06", "content": "Excellent! We look forward to seeing more from you.
", "votes": null, "replies": [] }, { "id": 982344, "author_name": "rubix9821", "author_url": "", "post_date": "08/23/2020 09:39:50", "content": "Congratulations for the high score!
", "votes": null, "replies": [] }, { "id": 983612, "author_name": "masdevallia", "author_url": "", "post_date": "08/24/2020 12:49:18", "content": "Thank you very much for all the comments! :)
", "votes": null, "replies": [] }, { "id": 983643, "author_name": "sahnoun", "author_url": "", "post_date": "08/24/2020 13:18:16", "content": "Geat work , congrats
", "votes": null, "replies": [] }, { "id": 985844, "author_name": "balaramk", "author_url": "", "post_date": "08/26/2020 03:52:29", "content": "WOW!!
\nNice and clear
congratulations for you first gold.
", "votes": null, "replies": [] }, { "id": 1002574, "author_name": "khahuras", "author_url": "", "post_date": "09/08/2020 08:23:50", "content": "Iβm very deloitted that you won!
", "votes": null, "replies": [ { "id": 1003178, "author_name": "masdevallia", "author_url": "", "post_date": "09/08/2020 17:54:18", "content": "πππ Thank you!
", "votes": null, "replies": [] } ] }, { "id": 1011665, "author_name": "romanweilguny", "author_url": "", "post_date": "09/15/2020 16:03:14", "content": "Congratulations on the great result @masdevallia!
\nInteresting to see your solution! Do I understand your notebook right, so that you just mixed B6 models with variable image sizes and image data? Well done!
Yes, exactly! Thank you!
", "votes": null, "replies": [] } ] }, { "id": 1312652, "author_name": "sayedathar11", "author_url": "", "post_date": "05/18/2021 07:05:27", "content": "@masdevallia Thanks for sharing ! Amazing work .
", "votes": null, "replies": [] }, { "id": 2017955, "author_name": "oriolbielsa", "author_url": "", "post_date": "11/05/2022 09:01:57", "content": "EDIT: Already figured out the question asked. So I will just keep the greeting to you :). Great resource to explain the crop resize reason found here: https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175344
\nReally great job @masdevallia, your notebook has really helped me to understand and make sense of the TPU image detection model building.
\nThanks in advance, and (late) congratulations on your work!
", "votes": null, "replies": [] }, { "id": 2388153, "author_name": "karthikajntua", "author_url": "", "post_date": "08/13/2023 07:14:05", "content": "Hey, Congratulations @masdevallia. It was so great for you to achive this, I am sorry I know that I am late to congralute you. Can you help me in some of the topics that you have used in this competition to win. Because I am totally new to these advanced concepts, it would be great that if I learned these concepts from you :)
", "votes": null, "replies": [] }, { "id": 3068695, "author_name": "dujinggjing", "author_url": "", "post_date": "12/10/2024 15:09:26", "content": "Congratulations on the great result @masdevallia!
\nI would like to ask if you have used metadata, and if so, at which specific location in the code have you used it?
Congratulations
", "votes": null, "replies": [] }, { "id": 982682, "author_name": "ruslanmemmedov", "author_url": "", "post_date": "08/23/2020 15:19:56", "content": "Hello
", "votes": null, "replies": [] }, { "id": 1008698, "author_name": "rjmanoj", "author_url": "", "post_date": "09/13/2020 10:20:57", "content": "Congratulation !!
", "votes": null, "replies": [] }, { "id": 1008727, "author_name": "alkattan", "author_url": "", "post_date": "09/13/2020 10:48:56", "content": "Thank you and congrats!
", "votes": null, "replies": [] }, { "id": 1008845, "author_name": "sonyrajan", "author_url": "", "post_date": "09/13/2020 12:43:01", "content": "Awesome! great work
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "976433": "Code available here: https://github.com/Masdevallia/3rd-place-kaggle-siim-isic-melanoma-classification\n\nHello everybody!\n\nWell... I am speechless. I am quite new at Kaggle and was not expecting such a good result, it has taken me completely by surprise.\n\nFirst of all, I need to deeply thank the entire Kaggle community. I've learned a lot throughout the entire competition thanks to all the knowledge and insights you have generously shared. Thank you also to the organizers and Kaggle for hosting the competition.\n\nI am away on vacation with a limited internet connection, but I will share my solution as soon as I can.\n\nAs a quick summary, my main submission was an ensemble of 8 different models built with various combinations of image sizes (256, 384, 512, 768). Many thanks to @vbhargav875, whose notebook \"EfficientNet-B5_B6_B7 TF-Keras\" was an incredible starting point. I used CV to implement some experiments, but my finals models where obtained with all available data (without validation). I used 2017-2018-2019 + 2020 TFrecords (huge thanks to @cdeotte), hair augmentation (thanks to @nroman and @graf10a), heavy TTA, EfficientNet-B6 models and metadata (thanks to @titericz). This approach scored: 0.9481 private LB, 0.9596 public LB.\n\nHowever, I wanted to try to ensemble some public notebooks too, in order to add some diversity. I decided to go ahead with these two amazing notebooks, which introduced some juicy techniques that I didn't have time to test:\n- https://www.kaggle.com/rajnishe/rc-fork-siim-isic-melanoma-384x384 (thanks to @rajnishe).\n- https://www.kaggle.com/ajaykumar7778/efficientnet-cv (thanks to @ajaykumar7778).\n\nThis approach scored: 0.9484 private LB, 0.9620 public LB.\n\nCongratulations to all participants! I know that I still have a long way to go, but I am looking forward to continuing to grow alongside this incredible community.", "976438": "Congratulations! Cant wait to see your code and learn from youπ", "976553": "Great models Masdevallia. Congratulations on your amazing accomplishment !", "976793": "Congratulations @masdevallia for your great win .", "976803": "Congratulations a great result!", "976806": "Wow, wow π \nGreat win, Congratulations @masdevallia π", "976839": "cdeotte thanks for your datasets which help us in model creation part.\nIt was great learning from your notebooks during competition and also notebook of forward ensemble is gem. May be few words you would like to share on how to upskill from here and how you build yourself .\nIt will be a great roadmap for all of us learner here.\nThanks again", "976903": "Congratulations @masdevallia, you chose the right way to achieve a great result. \nExcellent debut here at Kaggle Competitions!", "976981": "Great work!", "977010": "Congrats!", "977024": "Congratulations on the great result @masdevallia! You were smart about selecting only a few of the public notebooks that could add diversity to your ensemble and further improve the score. Nice job!\n\nYou said you were training your final models on full data without validation. How did you select the number of epochs? Was it based on when the early stopping was occurring in your previous CV results?", "977150": "Congratulations!\n\n> my finals models where obtained with all available data (without validation\n\nThat's something I wanted to do and forgot unfortunately. I am not surprised it worked well.", "977180": "Worked worse for us :)", "977243": "Great job!\n\nIncluding all data (2020+2019+2018/17) seems to have worked better. I did not try it based on premise that 2020+19 was worse in CV than 2020+18/17. Stupid me.", "977434": "Congrats @masdevallia and your solution and results! Cool to see you started with a different baseline model than most others. When I saw you get 3rd place I noticed you had this public dataset: [melanoma384x384nohair](https://www.kaggle.com/masdevallia/melanoma384x384nohair) - Did you use hair removal in your final solution at all? I didn't see anything about it in your writeup.\n\n> I used CV to implement some experiments, but my finals models where obtained with all available data (without validation).\n\nThis is a smart technique. Unfortunately our team did not have time to do this at the end. How did you determine the epochs you trained on for your final models? Did you set blending weights based on CV?", "977436": "> Worked worse for us :)\n\nInteresting. Any idea why @philippsinger? I've always assumed that if you could determine the correct epochs to not over/under fit then training a model on all data would be better than multiple CV split models.", "977466": "Yes, private LB is also random to some degree :)\n\nIt is also not exact same models and blending method in both \"comparable\" subs. That said k-fold blend can be better in theory if diverse sub-samples blend better, I personally do not believe this to be the case here - also fullfits always worked better on pub LB for us here.", "977796": "Congrats! Waiting for your detailed solution~", "977842": "masdevallia congrats for 3rd place!\ncan you explain a bit on this part \n\" I used CV to implement some experiments, but my finals models where obtained with all available data (without validation)\"\ndo you mean that you did not used k-fold-split-training ? and how does this idea help? looking for some insights...", "978722": "Congratulations", "979197": "Congrats. It is inspiring to see new faces winning big time. If possible, do share your solution whenever you get the time and if possible a more detailed account of your experiences. All the very best for your future competitions", "979218": "Excellent! We look forward to seeing more from you.", "982344": "Congratulations for the high score!", "982682": "Hello", "983575": "Hi @pawankumarsahu ,\nI used k-fold cross-validation to estimate the hyperparameters of the model (hyperparameter tuning) and then used those optimized hyperparameters to fit a model to the whole dataset. This procedure is based on the idea that the more data you use the more likely it is to have a robust model that generalizes well.", "983591": "Exactly @kozodoi! Although I had to reduce the \"optimal\" number of epochs on some occasions to avoid exceeding the limit of Kaggle's TPU (since I was using more data and also hair augmentation, which was quite computationally expensive).", "983612": "Thank you very much for all the comments! :)", "983643": "Geat work , congrats", "983669": "Thank you, @robikscube! I did not use hair removal in my final solution. I tested it and it worked worse for me than hair augmentation, so I decided to go this other way. The number of epochs was mostly determined based on when the early stopping was occurring in my previous CV results (with some adjustments to avoid exceeding the limit of Kaggle's TPU). Regarding my ensembles, I tried several things, but my final solution is actually a simple average of the 10 models (8 own models + 2 public notebooks). Finally I added metadata to this ensemble with a weighted average. I know I have a lot to learn about ensemble techniques and that I was pretty lucky to get such a good result with such a simple approach.", "983702": "Try not to sell yourself short, you did several things top competitors didn't try and your instincts were clearly good. \n\nI'd be interested to see how the leaderboard score compares for a model trained on all of the data and a model trained with cross-validation. I was uncomfortable trying this as I was fearful of submitting a model without validating it. But I guess it wouldn't change too much as you had validated, just with a little less training data. I'll make sure to try this on future image comps!", "985844": "WOW!!\nNice and clear", "987216": "congratulations for you first gold.", "1002574": "Iβm very deloitted that you won!", "1003178": "πππ Thank you!", "1008698": "Congratulation !!", "1008727": "Thank you and congrats!", "1008845": "Awesome! great work", "1011665": "Congratulations on the great result @masdevallia! \nInteresting to see your solution! Do I understand your notebook right, so that you just mixed B6 models with variable image sizes and image data? Well done!", "1026514": "Yes, exactly! Thank you!", "1312652": "masdevallia Thanks for sharing ! Amazing work .", "2017955": "EDIT: Already figured out the question asked. So I will just keep the greeting to you :). Great resource to explain the crop resize reason found here: https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/175344\n\nReally great job @masdevallia, your notebook has really helped me to understand and make sense of the TPU image detection model building.\n\nThanks in advance, and (late) congratulations on your work!", "2388153": "Hey, Congratulations @masdevallia. It was so great for you to achive this, I am sorry I know that I am late to congralute you. Can you help me in some of the topics that you have used in this competition to win. Because I am totally new to these advanced concepts, it would be great that if I learned these concepts from you :)", "3068695": "Congratulations on the great result @masdevallia! \nI would like to ask if you have used metadata, and if so, at which specific location in the code have you used it?" }, "source": "meta" }