{ "id": 176154, "title": "6th Place Write-Up", "url": "/competitions/siim-isic-melanoma-classification/discussion/176154", "author_name": "", "post_date": "2020-08-20T17:27:05.505000", "votes": 24, "comment_count": 10, "views": 0, "content": "

Congratatulations to the winners, and thanks to kaggle, competition sponsor and kernel contributors who shared their insights, it helped us a lot. Congratulations to my teammates @masterroshi @upbeatfreak

\n
Our current position standing 6th Private (23rd Public) is totally because of @cdeotte amazing contribution.
\n

Our Approach

\n

We started this competition after 2 months after the start.For a baseline, we used @cdeotte 's Triple Stratified Kfold With Tfrecords for tensorflow and @shonenkov 's [Training CV] Melanoma Starter. thankyou for these amazing kernels.we started working on them.

\n

We used EfficientNet [B0-B6], Resnest,Resnext, with Sizes 192x192 256x256 384x384 512x512 768x768 384x512[HxW]

\n

Summary

\n
What Worked for Us
\n

Heavy TTA (X20)
\nCutmix
\ncoarse dropout
\nSWA
\nloss-Label Smoothing
\noptimizer-AdamW, Adam
\nBCE
\n2018, 2020 and malignant datasets
\n5 checkpoints' prediction averaging(stabalised our model's predictions)
\nsome models were trained with different height width ratios

\n
What didn't Work for Us
\n

loss functions-Focal loss, dice loss
\noptimizer- Ranger
\nhair removal/addition
\npseudo labelling
\n2019 dataset
\npreprocessing techniques from aptos competition
\nprogressive learning

\n

Ensembling techniques

\n

weighted average
\nPower Average
\nminmax ensemble(didn't help)
\n3hr before end of competition I came across rank ensembling and and we did this ensemble and got 0.9697 for our last submission

\n

Our 3 final Submission

\n

we new the shakeup was coming, so we tried to select different approaches

\n
    \n
  1. All pytorch gpu solution models(with context) - 0.9530 (public LB) 0.9380 (private LB) 0.9541 (CV)
    2. \n
  2. All pytorch model (with context) and All tf models (without context) - 0.9627 (public LB) 0.9470 (private LB) 0.9618 (CV)
    3. \n
  3. Blend of public submission with 2nd submission with post proccessing technique - 0.9697 (public LB) 0.9126 (private LB) (overfitted)
    \nall the above were also ensembled with the meta only submission.
    4. \n
\n

we wanted to give a shot to public lb overfitted submission but obviosly didn't work out well.
\nI guess we were lucky enough to select the best private lb submission from our arsenel

\n

we found the discussions and public kernels really fruitful and learnt a lot from this competition.

\n

We are quite new in kaggle and this is our first competition. We will share the code in some days.

", "messages": [ { "id": 979196, "postDate": "2020-08-20T17:27:05.507Z", "content": "

Congratatulations to the winners, and thanks to kaggle, competition sponsor and kernel contributors who shared their insights, it helped us a lot. Congratulations to my teammates @masterroshi @upbeatfreak

\n
Our current position standing 6th Private (23rd Public) is totally because of @cdeotte amazing contribution.
\n

Our Approach

\n

We started this competition after 2 months after the start.For a baseline, we used @cdeotte 's Triple Stratified Kfold With Tfrecords for tensorflow and @shonenkov 's [Training CV] Melanoma Starter. thankyou for these amazing kernels.we started working on them.

\n

We used EfficientNet [B0-B6], Resnest,Resnext, with Sizes 192x192 256x256 384x384 512x512 768x768 384x512[HxW]

\n

Summary

\n
What Worked for Us
\n

Heavy TTA (X20)
\nCutmix
\ncoarse dropout
\nSWA
\nloss-Label Smoothing
\noptimizer-AdamW, Adam
\nBCE
\n2018, 2020 and malignant datasets
\n5 checkpoints' prediction averaging(stabalised our model's predictions)
\nsome models were trained with different height width ratios

\n
What didn't Work for Us
\n

loss functions-Focal loss, dice loss
\noptimizer- Ranger
\nhair removal/addition
\npseudo labelling
\n2019 dataset
\npreprocessing techniques from aptos competition
\nprogressive learning

\n

Ensembling techniques

\n

weighted average
\nPower Average
\nminmax ensemble(didn't help)
\n3hr before end of competition I came across rank ensembling and and we did this ensemble and got 0.9697 for our last submission

\n

Our 3 final Submission

\n

we new the shakeup was coming, so we tried to select different approaches

\n
    \n
  1. All pytorch gpu solution models(with context) - 0.9530 (public LB) 0.9380 (private LB) 0.9541 (CV)
    2. \n
  2. All pytorch model (with context) and All tf models (without context) - 0.9627 (public LB) 0.9470 (private LB) 0.9618 (CV)
    3. \n
  3. Blend of public submission with 2nd submission with post proccessing technique - 0.9697 (public LB) 0.9126 (private LB) (overfitted)
    \nall the above were also ensembled with the meta only submission.
    4. \n
\n

we wanted to give a shot to public lb overfitted submission but obviosly didn't work out well.
\nI guess we were lucky enough to select the best private lb submission from our arsenel

\n

we found the discussions and public kernels really fruitful and learnt a lot from this competition.

\n

We are quite new in kaggle and this is our first competition. We will share the code in some days.

", "rawMarkdown": "Congratatulations to the winners, and thanks to kaggle, competition sponsor and kernel contributors who shared their insights, it helped us a lot. Congratulations to my teammates @masterroshi @upbeatfreak\n##### Our current position standing 6th Private (23rd Public) is totally because of @cdeotte amazing contribution.\n\n#Our Approach\n\nWe started this competition after 2 months after the start.For a baseline, we used @cdeotte 's [Triple Stratified Kfold With Tfrecords](https://www.kaggle.com/cdeotte/triple-stratified-kfold-with-tfrecords) for tensorflow and @shonenkov 's [[Training CV] Melanoma Starter](https://www.kaggle.com/shonenkov/training-cv-melanoma-starter). thankyou for these amazing kernels.we started working on them.\n\n\n\nWe used **EfficientNet [B0-B6]**, **Resnest**,**Resnext**, with Sizes **192x192** **256x256** **384x384** **512x512** **768x768** **384x512**[HxW]\n\n## Summary\n###### What Worked for Us\nHeavy TTA (X20)\nCutmix\ncoarse dropout\nSWA\nloss-Label Smoothing\noptimizer-AdamW, Adam\nBCE\n2018, 2020 and malignant datasets\n5 checkpoints' prediction averaging(stabalised our model's predictions)\nsome models were trained with different height width ratios\n\n###### What didn't Work for Us\nloss functions-Focal loss, dice loss\noptimizer- Ranger \nhair removal/addition\npseudo labelling\n2019 dataset\npreprocessing techniques from aptos competition\nprogressive learning\n\n## Ensembling techniques\nweighted average\nPower Average\nminmax ensemble(didn't help)\n3hr before end of competition I came across rank ensembling and and we did this ensemble and got **0.9697** for our last submission\n\n#Our 3 final Submission\nwe new the shakeup was coming, so we tried to select different approaches\n\n1. All pytorch gpu solution models(with context) - 0.9530 (public LB) 0.9380 (private LB) 0.9541 (CV) \n2. All pytorch model (with context) and All tf models (without context) - 0.9627 (public LB) 0.9470 (private LB) 0.9618 (CV) \n3. Blend of public submission with 2nd submission with post proccessing technique - 0.9697 (public LB) 0.9126 (private LB) (overfitted)\nall the above were also ensembled with the meta only submission.\n\nwe wanted to give a shot to public lb overfitted submission but obviosly didn't work out well.\nI guess we were lucky enough to select the best private lb submission from our arsenel \n\nwe found the discussions and public kernels really fruitful and learnt a lot from this competition. \n\nWe are quite new in kaggle and this is our first competition. We will share the code in some days.\n", "votes": 24 }, { "id": 980730, "postDate": "2020-08-21T20:06:14.520Z", "content": "

Congratulations. Well done. Awesome models and finish!

", "rawMarkdown": "Congratulations. Well done. Awesome models and finish!", "votes": 1, "replies": [ { "id": 980861, "postDate": "2020-08-22T00:00:05.247Z", "content": "

I'm curious about your 384x512 resolution. Is this a rectangle crop, or did you take a square crop and resize to rectangle?

", "rawMarkdown": "I'm curious about your `384x512` resolution. Is this a rectangle crop, or did you take a square crop and resize to rectangle?", "votes": 1 }, { "id": 980972, "postDate": "2020-08-22T04:16:03.160Z", "content": "

If it is 384x512 then first we central Crop into 576x576. Then resize into 384x512.
\nIn pytorch models 384x512 gives second high score(0.941) 512x512 (0.943) 384x384(0.938)

", "rawMarkdown": "If it is 384x512 then first we central Crop into 576x576. Then resize into 384x512.\nIn pytorch models 384x512 gives second high score(0.941) 512x512 (0.943) 384x384(0.938)", "votes": 1 } ] }, { "id": 980308, "postDate": "2020-08-21T13:39:07.427Z", "content": "

Congrats

", "rawMarkdown": "Congrats", "votes": 1 }, { "id": 979880, "postDate": "2020-08-21T07:13:52.830Z", "content": "

Great job 👍
\nCongratulations 😍

", "rawMarkdown": "Great job 👍\nCongratulations 😍", "votes": 1 }, { "id": 979214, "postDate": "2020-08-20T17:39:57.360Z", "content": "

Congrats good job

", "rawMarkdown": "Congrats good job", "votes": 1 }, { "id": 979211, "postDate": "2020-08-20T17:38:36.853Z", "content": "

Congrats @luckymnc and @masterroshi, @upbeatfreak for 6th place and gold medals.
\nAnd thanks for write-up.
\nI'm looking forward to pytorch solution. :)

", "rawMarkdown": "Congrats @luckymnc and @masterroshi, @upbeatfreak for 6th place and gold medals.\nAnd thanks for write-up. \nI'm looking forward to pytorch solution. :)", "votes": 1 }, { "id": 986329, "postDate": "2020-08-26T11:39:28.370Z", "content": "

congrats

", "rawMarkdown": "congrats" }, { "id": 979215, "postDate": "2020-08-20T17:39:57.777Z", "rawMarkdown": "", "isDeleted": true }, { "id": 980582, "postDate": "2020-08-21T17:44:01.380Z", "content": "

Thanks for sharing. Congrats

", "rawMarkdown": "Thanks for sharing. Congrats", "votes": 1 } ], "comments": [ { "id": 980730, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-08-21T20:06:14.520000", "content": "

Congratulations. Well done. Awesome models and finish!

", "votes": 1, "replies": [ { "id": 980861, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-08-22T00:00:05.247000", "content": "

I'm curious about your 384x512 resolution. Is this a rectangle crop, or did you take a square crop and resize to rectangle?

", "votes": 1, "replies": [] }, { "id": 980972, "author_name": "", "author_url": "", "post_date": "2020-08-22T04:16:03.160000", "content": "

If it is 384x512 then first we central Crop into 576x576. Then resize into 384x512.
\nIn pytorch models 384x512 gives second high score(0.941) 512x512 (0.943) 384x384(0.938)

", "votes": 1, "replies": [] } ] }, { "id": 980308, "author_name": "Survesh Chauhan", "author_url": "", "post_date": "2020-08-21T13:39:07.427000", "content": "

Congrats

", "votes": 1, "replies": [] }, { "id": 979880, "author_name": "Abdur Rahim", "author_url": "", "post_date": "2020-08-21T07:13:52.830000", "content": "

Great job 👍
\nCongratulations 😍

", "votes": 1, "replies": [] }, { "id": 979214, "author_name": "Bo Peng", "author_url": "", "post_date": "2020-08-20T17:39:57.360000", "content": "

Congrats good job

", "votes": 1, "replies": [] }, { "id": 979211, "author_name": "Heroseo", "author_url": "", "post_date": "2020-08-20T17:38:36.853000", "content": "

Congrats @luckymnc and @masterroshi, @upbeatfreak for 6th place and gold medals.
\nAnd thanks for write-up.
\nI'm looking forward to pytorch solution. :)

", "votes": 1, "replies": [] }, { "id": 986329, "author_name": "Safoora Naureen ", "author_url": "", "post_date": "2020-08-26T11:39:28.370000", "content": "

congrats

", "votes": 0, "replies": [] }, { "id": 979215, "author_name": "", "author_url": "", "post_date": "2020-08-20T17:39:57.777000", "content": "", "votes": 0, "replies": [] }, { "id": 980582, "author_name": "Raoof Naushad", "author_url": "", "post_date": "2020-08-21T17:44:01.380000", "content": "

Thanks for sharing. Congrats

", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "979196": "Congratatulations to the winners, and thanks to kaggle, competition sponsor and kernel contributors who shared their insights, it helped us a lot. Congratulations to my teammates @masterroshi @upbeatfreak\n##### Our current position standing 6th Private (23rd Public) is totally because of @cdeotte amazing contribution.\n\n#Our Approach\n\nWe started this competition after 2 months after the start.For a baseline, we used @cdeotte 's [Triple Stratified Kfold With Tfrecords](https://www.kaggle.com/cdeotte/triple-stratified-kfold-with-tfrecords) for tensorflow and @shonenkov 's [[Training CV] Melanoma Starter](https://www.kaggle.com/shonenkov/training-cv-melanoma-starter). thankyou for these amazing kernels.we started working on them.\n\n\n\nWe used **EfficientNet [B0-B6]**, **Resnest**,**Resnext**, with Sizes **192x192** **256x256** **384x384** **512x512** **768x768** **384x512**[HxW]\n\n## Summary\n###### What Worked for Us\nHeavy TTA (X20)\nCutmix\ncoarse dropout\nSWA\nloss-Label Smoothing\noptimizer-AdamW, Adam\nBCE\n2018, 2020 and malignant datasets\n5 checkpoints' prediction averaging(stabalised our model's predictions)\nsome models were trained with different height width ratios\n\n###### What didn't Work for Us\nloss functions-Focal loss, dice loss\noptimizer- Ranger \nhair removal/addition\npseudo labelling\n2019 dataset\npreprocessing techniques from aptos competition\nprogressive learning\n\n## Ensembling techniques\nweighted average\nPower Average\nminmax ensemble(didn't help)\n3hr before end of competition I came across rank ensembling and and we did this ensemble and got **0.9697** for our last submission\n\n#Our 3 final Submission\nwe new the shakeup was coming, so we tried to select different approaches\n\n1. All pytorch gpu solution models(with context) - 0.9530 (public LB) 0.9380 (private LB) 0.9541 (CV) \n2. All pytorch model (with context) and All tf models (without context) - 0.9627 (public LB) 0.9470 (private LB) 0.9618 (CV) \n3. Blend of public submission with 2nd submission with post proccessing technique - 0.9697 (public LB) 0.9126 (private LB) (overfitted)\nall the above were also ensembled with the meta only submission.\n\nwe wanted to give a shot to public lb overfitted submission but obviosly didn't work out well.\nI guess we were lucky enough to select the best private lb submission from our arsenel \n\nwe found the discussions and public kernels really fruitful and learnt a lot from this competition. \n\nWe are quite new in kaggle and this is our first competition. We will share the code in some days.\n", "980730": "Congratulations. Well done. Awesome models and finish!", "980308": "Congrats", "979880": "Great job 👍\nCongratulations 😍", "979214": "Congrats good job", "979211": "Congrats @luckymnc and @masterroshi, @upbeatfreak for 6th place and gold medals.\nAnd thanks for write-up. \nI'm looking forward to pytorch solution. :)", "986329": "congrats", "979215": "", "980582": "Thanks for sharing. Congrats" } }