Congrats to the winners and all the participants !!\nI really want to thank @drhabib and @ratthachat and everyone else who shared their insights and work in notebooks and discussion forums, it helped me a lot. \nWell my solution is pretty simple, the two things that really worked for me were : \n1. Test Time Augmentation\n2. Ensembling 5 models
\n\nOverview\n\ni used the Efficientnet B3 with image size 300, pre-trained on the old competition data and then fine-tuned on this competition data. I used a lot of augmentations - zoom_range(0.25-0.35), horizontal and vertical flips, and rotation(360).\nTTA - used a very soft tta - getting predictions on test images, randomly rotating 1-6 degrees and making predictions again and then again rotating original image to 7-12 degrees. Did the same for all 5 models, and then taking mode of predictions for each type of tta from each model.\nthis gave me three lists of predictions - one for each tta, and then again taking mode of predictions of those three lists.\nModels used for ensembling with TTA, scored - 0.784, 0.790, 0.794, 0.797 and another model with 0.797 on public lb.
\n\nTTA gave me a boost of about 0.02 as these above models scored around 0.77-0.78 on public lb.\nEnsembling gave me a huge boost.
\n\nThanks to Kaggle and APTOS for this wonderful competition, it taught me a lot.
", "messages": [ { "id": "620943", "postDate": "09/08/2019 04:58:32", "content": "Congrats to the winners and all the participants !!\nI really want to thank @drhabib and @ratthachat and everyone else who shared their insights and work in notebooks and discussion forums, it helped me a lot. \nWell my solution is pretty simple, the two things that really worked for me were : \n1. Test Time Augmentation\n2. Ensembling 5 models
\n\nOverview\n\ni used the Efficientnet B3 with image size 300, pre-trained on the old competition data and then fine-tuned on this competition data. I used a lot of augmentations - zoom_range(0.25-0.35), horizontal and vertical flips, and rotation(360).\nTTA - used a very soft tta - getting predictions on test images, randomly rotating 1-6 degrees and making predictions again and then again rotating original image to 7-12 degrees. Did the same for all 5 models, and then taking mode of predictions for each type of tta from each model.\nthis gave me three lists of predictions - one for each tta, and then again taking mode of predictions of those three lists.\nModels used for ensembling with TTA, scored - 0.784, 0.790, 0.794, 0.797 and another model with 0.797 on public lb.
\n\nTTA gave me a boost of about 0.02 as these above models scored around 0.77-0.78 on public lb.\nEnsembling gave me a huge boost.
\n\nThanks to Kaggle and APTOS for this wonderful competition, it taught me a lot.
", "rawMarkdown": "Congrats to the winners and all the participants !!\nI really want to thank @drhabib and @ratthachat and everyone else who shared their insights and work in notebooks and discussion forums, it helped me a lot. \nWell my solution is pretty simple, the two things that really worked for me were : \n1. Test Time Augmentation\n2. Ensembling 5 models\n\n##### **Overview** \n- Ensemble 5 models - all EfficientNets B3 with image size 300.\n- a lot of augmentation.\n- soft TTA.\n- Preprocessing - only cropping the black borders.\n- using Keras.\n\n##### **Score** \n- The model that gave me best results scored 0.810 on the public LB and 0.921 on the private LB.\n- Best performing model on Public lb (0.814) gave 0.917 on the Private LB.\n\n##### **Training** -\ni used the Efficientnet B3 with image size 300, pre-trained on the old competition data and then fine-tuned on this competition data. I used a lot of augmentations - zoom_range(0.25-0.35), horizontal and vertical flips, and rotation(360).\nTTA - used a very soft tta - getting predictions on test images, randomly rotating 1-6 degrees and making predictions again and then again rotating original image to 7-12 degrees. Did the same for all 5 models, and then taking mode of predictions for each type of tta from each model.\nthis gave me three lists of predictions - one for each tta, and then again taking mode of predictions of those three lists.\nModels used for ensembling with TTA, scored - 0.784, 0.790, 0.794, 0.797 and another model with 0.797 on public lb.\n\nTTA gave me a boost of about 0.02 as these above models scored around 0.77-0.78 on public lb.\nEnsembling gave me a huge boost.\n\nThanks to Kaggle and APTOS for this wonderful competition, it taught me a lot.", "votes": null }, { "id": "620955", "postDate": "09/08/2019 05:10:38", "content": "Congratulations\nGreat work\nAnd Thanks for sharing your approach and insights.!! @modojj
", "rawMarkdown": "Congratulations\nGreat work\nAnd Thanks for sharing your approach and insights.!! @modojj", "votes": null }, { "id": "620960", "postDate": "09/08/2019 05:24:08", "content": "Hi Rohit, If possible can you please share you Keras kernals.
", "rawMarkdown": "Hi Rohit, If possible can you please share you Keras kernals.", "votes": null }, { "id": "620963", "postDate": "09/08/2019 05:29:41", "content": "Hello @mayank17, Yes i will share my kernels, actually i have created 3 kernels one for training on old data, another one for training the same model on new data and one for inference.
", "rawMarkdown": "Hello @mayank17, Yes i will share my kernels, actually i have created 3 kernels one for training on old data, another one for training the same model on new data and one for inference.", "votes": null }, { "id": "620968", "postDate": "09/08/2019 05:39:55", "content": "Thanks @modojj . I asked this for two reasons.
\n\n1) I am very new to ML and wanted to learn from others implementation.\n2) I also used keras but couldn't cross .76.
\n\nThanks for all the help in advance !! :)
", "rawMarkdown": "Thanks @modojj . I asked this for two reasons. \n\n1) I am very new to ML and wanted to learn from others implementation.\n2) I also used keras but couldn't cross .76. \n\nThanks for all the help in advance !! :)", "votes": null }, { "id": "620975", "postDate": "09/08/2019 05:48:20", "content": "@mayank17 Did you use the old data ? because i was also not able to cross the 0.762 score without using the old data.
", "rawMarkdown": "mayank17 Did you use the old data ? because i was also not able to cross the 0.762 score without using the old data.", "votes": null }, { "id": "621028", "postDate": "09/08/2019 06:51:50", "content": "yes I did use old data also.. but not sure where it went wrong.. May be once I see your then I will be able to find issue with mine.
\n\nhttps://www.kaggle.com/mayank17/efficientnet-old-data?scriptVersionId=19874474
\n\nI am very new to ML. :)
", "rawMarkdown": "yes I did use old data also.. but not sure where it went wrong.. May be once I see your then I will be able to find issue with mine.\n\n[https://www.kaggle.com/mayank17/efficientnet-old-data?scriptVersionId=19874474](https://www.kaggle.com/mayank17/efficientnet-old-data?scriptVersionId=19874474)\n\n\nI am very new to ML. :)", "votes": null }, { "id": "621056", "postDate": "09/08/2019 07:16:13", "content": "@mayank17 here Keras EfficientNetB3 + TTA, i combined the 2019 data training phase and the inference in a single kernel, note - i couldn't combine the old data training but old data training and configuration is exactly same as in this kernel.
", "rawMarkdown": "mayank17 here [Keras EfficientNetB3 + TTA](https://www.kaggle.com/modojj/keras-efficientnetb3-tta), i combined the 2019 data training phase and the inference in a single kernel, note - i couldn't combine the old data training but old data training and configuration is exactly same as in this kernel.", "votes": null } ], "comments": [ { "id": 620955, "author_name": "veeralakrishna", "author_url": "", "post_date": "09/08/2019 05:10:38", "content": "Congratulations\nGreat work\nAnd Thanks for sharing your approach and insights.!! @modojj
", "votes": null, "replies": [] }, { "id": 620960, "author_name": "mayank17", "author_url": "", "post_date": "09/08/2019 05:24:08", "content": "Hi Rohit, If possible can you please share you Keras kernals.
", "votes": null, "replies": [ { "id": 620963, "author_name": "modojj", "author_url": "", "post_date": "09/08/2019 05:29:41", "content": "Hello @mayank17, Yes i will share my kernels, actually i have created 3 kernels one for training on old data, another one for training the same model on new data and one for inference.
", "votes": null, "replies": [] }, { "id": 620968, "author_name": "mayank17", "author_url": "", "post_date": "09/08/2019 05:39:55", "content": "Thanks @modojj . I asked this for two reasons.
\n\n1) I am very new to ML and wanted to learn from others implementation.\n2) I also used keras but couldn't cross .76.
\n\nThanks for all the help in advance !! :)
", "votes": null, "replies": [] }, { "id": 620975, "author_name": "modojj", "author_url": "", "post_date": "09/08/2019 05:48:20", "content": "@mayank17 Did you use the old data ? because i was also not able to cross the 0.762 score without using the old data.
", "votes": null, "replies": [] }, { "id": 621028, "author_name": "mayank17", "author_url": "", "post_date": "09/08/2019 06:51:50", "content": "yes I did use old data also.. but not sure where it went wrong.. May be once I see your then I will be able to find issue with mine.
\n\nhttps://www.kaggle.com/mayank17/efficientnet-old-data?scriptVersionId=19874474
\n\nI am very new to ML. :)
", "votes": null, "replies": [] }, { "id": 621056, "author_name": "modojj", "author_url": "", "post_date": "09/08/2019 07:16:13", "content": "@mayank17 here Keras EfficientNetB3 + TTA, i combined the 2019 data training phase and the inference in a single kernel, note - i couldn't combine the old data training but old data training and configuration is exactly same as in this kernel.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "620943": "Congrats to the winners and all the participants !!\nI really want to thank @drhabib and @ratthachat and everyone else who shared their insights and work in notebooks and discussion forums, it helped me a lot. \nWell my solution is pretty simple, the two things that really worked for me were : \n1. Test Time Augmentation\n2. Ensembling 5 models\n\n##### **Overview** \n- Ensemble 5 models - all EfficientNets B3 with image size 300.\n- a lot of augmentation.\n- soft TTA.\n- Preprocessing - only cropping the black borders.\n- using Keras.\n\n##### **Score** \n- The model that gave me best results scored 0.810 on the public LB and 0.921 on the private LB.\n- Best performing model on Public lb (0.814) gave 0.917 on the Private LB.\n\n##### **Training** -\ni used the Efficientnet B3 with image size 300, pre-trained on the old competition data and then fine-tuned on this competition data. I used a lot of augmentations - zoom_range(0.25-0.35), horizontal and vertical flips, and rotation(360).\nTTA - used a very soft tta - getting predictions on test images, randomly rotating 1-6 degrees and making predictions again and then again rotating original image to 7-12 degrees. Did the same for all 5 models, and then taking mode of predictions for each type of tta from each model.\nthis gave me three lists of predictions - one for each tta, and then again taking mode of predictions of those three lists.\nModels used for ensembling with TTA, scored - 0.784, 0.790, 0.794, 0.797 and another model with 0.797 on public lb.\n\nTTA gave me a boost of about 0.02 as these above models scored around 0.77-0.78 on public lb.\nEnsembling gave me a huge boost.\n\nThanks to Kaggle and APTOS for this wonderful competition, it taught me a lot.", "620955": "Congratulations\nGreat work\nAnd Thanks for sharing your approach and insights.!! @modojj", "620960": "Hi Rohit, If possible can you please share you Keras kernals.", "620963": "Hello @mayank17, Yes i will share my kernels, actually i have created 3 kernels one for training on old data, another one for training the same model on new data and one for inference.", "620968": "Thanks @modojj . I asked this for two reasons. \n\n1) I am very new to ML and wanted to learn from others implementation.\n2) I also used keras but couldn't cross .76. \n\nThanks for all the help in advance !! :)", "620975": "mayank17 Did you use the old data ? because i was also not able to cross the 0.762 score without using the old data.", "621028": "yes I did use old data also.. but not sure where it went wrong.. May be once I see your then I will be able to find issue with mine.\n\n[https://www.kaggle.com/mayank17/efficientnet-old-data?scriptVersionId=19874474](https://www.kaggle.com/mayank17/efficientnet-old-data?scriptVersionId=19874474)\n\n\nI am very new to ML. :)", "621056": "mayank17 here [Keras EfficientNetB3 + TTA](https://www.kaggle.com/modojj/keras-efficientnetb3-tta), i combined the 2019 data training phase and the inference in a single kernel, note - i couldn't combine the old data training but old data training and configuration is exactly same as in this kernel." }, "source": "meta" }