{ "id": 108017, "title": "37th Place Solution(And Let's talk about CV and private score)", "url": "/competitions/aptos2019-blindness-detection/writeups/am-ranthos-37th-place-solution-and-let-s-talk-abou", "author_name": "", "post_date": "2019-09-12T19:36:14.687Z", "votes": 8, "comment_count": 7, "views": 0, "content": "

Since some of the top place solutions are already available, I'm not sure if this note is going to add anything new. Just sharing my experiences. :)

\n\n

My final LB score is an ensemble of 5 Efficientnet models. Here is the overview:

\n\n\n\n

As @rishabhiitbhu showed that the class distribution is quite similar in different datasets, I had a hunch that it wouldn't be that much different for our large test set. This is why I trusted my stratified validation data(can't even imagine what would have happened if I were wrong).

\n\n

My Validation and Public scores didn't seem to be correlated but now the private scores resemble my val scores. Here is a summary of the models I used:

\n\n

| Model | Image Size | Val Kappa | Public Kappa | Private Kappa |\n|-----------------|------------|-----------|--------------|---------------|\n| Efficientnet B2 | 256 | 0.922 | 0.807 | 0.918 |\n| Efficientnet B1 | 256 | 0.926 | 0.804 | 0.921 |\n| Efficientnet B0 | 256 | 0.919 | 0.816 | 0.914 |\n| Efficientnet B3 | 256 | 0.921 | 0.812 | 0.917 |\n| Efficientnet B5 | 300 | 0.920 | 0.802 | 0.916 |\n| Ensemble | | 0.932 | 0.826 | 0.926 |

\n\n

Please share and let us know about your CV and Private scores. Also congratulations to the ones who survived the LB shakeup. :)

", "messages": [ { "id": "621415", "postDate": "09/08/2019 13:46:55", "content": "

Since some of the top place solutions are already available, I'm not sure if this note is going to add anything new. Just sharing my experiences. :)

\n\n

My final LB score is an ensemble of 5 Efficientnet models. Here is the overview:

\n\n\n\n

As @rishabhiitbhu showed that the class distribution is quite similar in different datasets, I had a hunch that it wouldn't be that much different for our large test set. This is why I trusted my stratified validation data(can't even imagine what would have happened if I were wrong).

\n\n

My Validation and Public scores didn't seem to be correlated but now the private scores resemble my val scores. Here is a summary of the models I used:

\n\n

| Model | Image Size | Val Kappa | Public Kappa | Private Kappa |\n|-----------------|------------|-----------|--------------|---------------|\n| Efficientnet B2 | 256 | 0.922 | 0.807 | 0.918 |\n| Efficientnet B1 | 256 | 0.926 | 0.804 | 0.921 |\n| Efficientnet B0 | 256 | 0.919 | 0.816 | 0.914 |\n| Efficientnet B3 | 256 | 0.921 | 0.812 | 0.917 |\n| Efficientnet B5 | 300 | 0.920 | 0.802 | 0.916 |\n| Ensemble | | 0.932 | 0.826 | 0.926 |

\n\n

Please share and let us know about your CV and Private scores. Also congratulations to the ones who survived the LB shakeup. :)

", "rawMarkdown": "Since some of the top place solutions are already available, I'm not sure if this note is going to add anything new. Just sharing my experiences. :)\n\nMy final LB score is an ensemble of 5 Efficientnet models. Here is the overview:\n\n\n- **Preprocessing**: Only Circle Cropping from [this](https://www.kaggle.com/tanlikesmath/diabetic-retinopathy-resnet50-binary-cropped) kernel by @tanlikesmath \n\n- **Augmentations**: Horizontal and Vertical Flip, 360 rotation, zoom 20%-25%, lighting 50%. Used Fast.ai transformations.\n\n- **Training**: Mostly followed @drhabib's [method](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815#latest-582005). Did 5+10 training using `2015` data as train and `2019` as validation set. Then I split the `2019` data into 80:20 ratio using `sklearn`'s Stratified Split function and finetuned the previously trained model over it for 15 epochs. I took the best model based on validation loss which didn't seem to be overfitted and used it for submission. I used a fixed seed so that I can compare different model performances over the same val set. Didn't use CV or TTA.\n\nAs @rishabhiitbhu [showed](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/106844#latest-620958) that the class distribution is quite similar in different datasets, I had a hunch that it wouldn't be that much different for our large test set. This is why I trusted my stratified validation data(can't even imagine what would have happened if I were wrong). \n\nMy Validation and Public scores didn't seem to be correlated but now the private scores resemble my val scores. Here is a summary of the models I used:\n\n| Model | Image Size | Val Kappa | Public Kappa | Private Kappa |\n|-----------------|------------|-----------|--------------|---------------|\n| Efficientnet B2 | 256 | 0\\.922 | 0\\.807 | 0\\.918 |\n| Efficientnet B1 | 256 | 0\\.926 | 0\\.804 | 0\\.921 |\n| Efficientnet B0 | 256 | 0\\.919 | 0\\.816 | 0\\.914 |\n| Efficientnet B3 | 256 | 0\\.921 | 0\\.812 | 0\\.917 |\n| Efficientnet B5 | 300 | 0\\.920 | 0\\.802 | 0\\.916 |\n| Ensemble | | 0\\.932 | 0\\.826 | 0\\.926 |\n\nPlease share and let us know about your CV and Private scores. Also congratulations to the ones who survived the LB shakeup. :)", "votes": null }, { "id": "621417", "postDate": "09/08/2019 13:52:02", "content": "

Simplicity and criativity! Good @tahsin.

", "rawMarkdown": "Simplicity and criativity! Good @tahsin.", "votes": null }, { "id": "621421", "postDate": "09/08/2019 13:54:32", "content": "

Thanks :)

", "rawMarkdown": "Thanks :)", "votes": null }, { "id": "621570", "postDate": "09/08/2019 17:08:55", "content": "

amazing insight..that your Val score was correlated to Private lb..thanks a lot and congratulations !!!

", "rawMarkdown": "amazing insight..that your Val score was correlated to Private lb..thanks a lot and congratulations !!!", "votes": null }, { "id": "621914", "postDate": "09/09/2019 04:58:23", "content": "

Congratulations...\nGreat Work...\nThanks for Sharing your Approach & Insights... !! @tahsin

", "rawMarkdown": "Congratulations...\nGreat Work...\nThanks for Sharing your Approach & Insights... !! @tahsin", "votes": null }, { "id": "622322", "postDate": "09/09/2019 13:58:44", "content": "

Congratulations.\nThanks for Sharing your Approach

", "rawMarkdown": "Congratulations.\nThanks for Sharing your Approach", "votes": null }, { "id": "622514", "postDate": "09/09/2019 18:57:54", "content": "

Thanks @jmourad100

", "rawMarkdown": "Thanks @jmourad100", "votes": null }, { "id": "3127286", "postDate": "02/18/2025 08:49:18", "content": "

Dear can you share the code ?

", "rawMarkdown": "Dear can you share the code ?", "votes": null } ], "comments": [ { "id": 3127286, "author_name": "dsshayan", "author_url": "", "post_date": "02/18/2025 08:49:18", "content": "

Dear can you share the code ?

", "votes": null, "replies": [] }, { "id": 621417, "author_name": "brunhs", "author_url": "", "post_date": "09/08/2019 13:52:02", "content": "

Simplicity and criativity! Good @tahsin.

", "votes": null, "replies": [ { "id": 621421, "author_name": "tahsin", "author_url": "", "post_date": "09/08/2019 13:54:32", "content": "

Thanks :)

", "votes": null, "replies": [] } ] }, { "id": 621570, "author_name": "anuragtr", "author_url": "", "post_date": "09/08/2019 17:08:55", "content": "

amazing insight..that your Val score was correlated to Private lb..thanks a lot and congratulations !!!

", "votes": null, "replies": [] }, { "id": 621914, "author_name": "veeralakrishna", "author_url": "", "post_date": "09/09/2019 04:58:23", "content": "

Congratulations...\nGreat Work...\nThanks for Sharing your Approach & Insights... !! @tahsin

", "votes": null, "replies": [] }, { "id": 622322, "author_name": "jmourad100", "author_url": "", "post_date": "09/09/2019 13:58:44", "content": "

Congratulations.\nThanks for Sharing your Approach

", "votes": null, "replies": [ { "id": 622514, "author_name": "tahsin", "author_url": "", "post_date": "09/09/2019 18:57:54", "content": "

Thanks @jmourad100

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "621415": "Since some of the top place solutions are already available, I'm not sure if this note is going to add anything new. Just sharing my experiences. :)\n\nMy final LB score is an ensemble of 5 Efficientnet models. Here is the overview:\n\n\n- **Preprocessing**: Only Circle Cropping from [this](https://www.kaggle.com/tanlikesmath/diabetic-retinopathy-resnet50-binary-cropped) kernel by @tanlikesmath \n\n- **Augmentations**: Horizontal and Vertical Flip, 360 rotation, zoom 20%-25%, lighting 50%. Used Fast.ai transformations.\n\n- **Training**: Mostly followed @drhabib's [method](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815#latest-582005). Did 5+10 training using `2015` data as train and `2019` as validation set. Then I split the `2019` data into 80:20 ratio using `sklearn`'s Stratified Split function and finetuned the previously trained model over it for 15 epochs. I took the best model based on validation loss which didn't seem to be overfitted and used it for submission. I used a fixed seed so that I can compare different model performances over the same val set. Didn't use CV or TTA.\n\nAs @rishabhiitbhu [showed](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/106844#latest-620958) that the class distribution is quite similar in different datasets, I had a hunch that it wouldn't be that much different for our large test set. This is why I trusted my stratified validation data(can't even imagine what would have happened if I were wrong). \n\nMy Validation and Public scores didn't seem to be correlated but now the private scores resemble my val scores. Here is a summary of the models I used:\n\n| Model | Image Size | Val Kappa | Public Kappa | Private Kappa |\n|-----------------|------------|-----------|--------------|---------------|\n| Efficientnet B2 | 256 | 0\\.922 | 0\\.807 | 0\\.918 |\n| Efficientnet B1 | 256 | 0\\.926 | 0\\.804 | 0\\.921 |\n| Efficientnet B0 | 256 | 0\\.919 | 0\\.816 | 0\\.914 |\n| Efficientnet B3 | 256 | 0\\.921 | 0\\.812 | 0\\.917 |\n| Efficientnet B5 | 300 | 0\\.920 | 0\\.802 | 0\\.916 |\n| Ensemble | | 0\\.932 | 0\\.826 | 0\\.926 |\n\nPlease share and let us know about your CV and Private scores. Also congratulations to the ones who survived the LB shakeup. :)", "621417": "Simplicity and criativity! Good @tahsin.", "621421": "Thanks :)", "621570": "amazing insight..that your Val score was correlated to Private lb..thanks a lot and congratulations !!!", "621914": "Congratulations...\nGreat Work...\nThanks for Sharing your Approach & Insights... !! @tahsin", "622322": "Congratulations.\nThanks for Sharing your Approach", "622514": "Thanks @jmourad100", "3127286": "Dear can you share the code ?" }, "source": "meta" }