{ "id": 70427, "title": "2nd place solution", "url": "/competitions/rsna-pneumonia-detection-challenge/discussion/70427", "author_name": "Dmytro Poplavskiy", "post_date": "2018-11-03T14:30:24.087000", "votes": 133, "comment_count": 54, "views": 0, "content": "

My solution is based on the modified retinanet based model. Single model, ensembled outputs of 4 folds.

\n\n

I decided to use retinanet as it's much simpler comparing to Faster-RCNN like models or SSD while having comparable results, this allows much easier experiments and debugging/tuning of model.

\n\n

Credits to pytorch-retinanet implementation my solution is based on: https://github.com/yhenon/pytorch-retinanet

\n\n

I scaled the original images to 512x512 resolution, with 256 resolution I have seen results degradation and using the full resolution was not as practical with heavier base models.

\n\n

Modifications I have done to the original pytorch-retinanet implementation:

\n\n\n\n

Augmentation used:\nMild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes. I limited the amount of brightness/gamma augmentations as it was hard for me to verify if it does not invalidate labels. To reduce impact of rotation to bounding box sizes, instead of rotating the corners I rotated two points at each edge, at 1/3 and 2/3 edge length from corner, 8 points in total and calculated the new bounding box as min/max of rotated points.

\n\n

![Rotation box augmentation][1]

\n\n

I used 4 folds, stratified split by class.

\n\n

For submission, I ensembled models trained on each fold and a few checkpoints around CV loss minimum. I have averaged model outputs before applying any thresholds and NMS stages.

\n\n

As everyone else noticed, one of challenges of this competition was different distribution of train and test datasets, most likely due to different labeling methodology. Multiple radiologists have labeled each image with intersection used for similar labels. I'd expect this would lead to more boxes predicted, but of smaller size especially in complex cases.\nI tried to roughly simulate this process, using outputs from different folds. Instead of using the mean output of anchor sizes, I used value at 20 percentile and reduced it even more proportionally to difference between 80 and 20 percentile between models.

\n\n

I have not used any other metadata from images, I felt the orientation field borderlines with being the leak.

\n\n

I trained model for around 12 epochs, around 1 hour per epoch on 1080ti GPU.

\n\n

What I'd like to try:\nI was not able to work on this competition for the last two week before stage 2 starts and I missed the post about NIH dataset allowed to use. It's a bigger dataset but with lower quality of labels, would be very interesting to check if training the model to predict both datasets in interleaved way would improve the result, or at least use it to pretrain the base model.

\n\n

I'd like to thank organisers for this competitions, it was a pleasure to participate and hopefully the solutions would make an impact.

", "messages": [ { "id": 414752, "postDate": "2018-11-03T14:30:24.087Z", "content": "

My solution is based on the modified retinanet based model. Single model, ensembled outputs of 4 folds.

\n\n

I decided to use retinanet as it's much simpler comparing to Faster-RCNN like models or SSD while having comparable results, this allows much easier experiments and debugging/tuning of model.

\n\n

Credits to pytorch-retinanet implementation my solution is based on: https://github.com/yhenon/pytorch-retinanet

\n\n

I scaled the original images to 512x512 resolution, with 256 resolution I have seen results degradation and using the full resolution was not as practical with heavier base models.

\n\n

Modifications I have done to the original pytorch-retinanet implementation:

\n\n\n\n

Augmentation used:\nMild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes. I limited the amount of brightness/gamma augmentations as it was hard for me to verify if it does not invalidate labels. To reduce impact of rotation to bounding box sizes, instead of rotating the corners I rotated two points at each edge, at 1/3 and 2/3 edge length from corner, 8 points in total and calculated the new bounding box as min/max of rotated points.

\n\n

![Rotation box augmentation][1]

\n\n

I used 4 folds, stratified split by class.

\n\n

For submission, I ensembled models trained on each fold and a few checkpoints around CV loss minimum. I have averaged model outputs before applying any thresholds and NMS stages.

\n\n

As everyone else noticed, one of challenges of this competition was different distribution of train and test datasets, most likely due to different labeling methodology. Multiple radiologists have labeled each image with intersection used for similar labels. I'd expect this would lead to more boxes predicted, but of smaller size especially in complex cases.\nI tried to roughly simulate this process, using outputs from different folds. Instead of using the mean output of anchor sizes, I used value at 20 percentile and reduced it even more proportionally to difference between 80 and 20 percentile between models.

\n\n

I have not used any other metadata from images, I felt the orientation field borderlines with being the leak.

\n\n

I trained model for around 12 epochs, around 1 hour per epoch on 1080ti GPU.

\n\n

What I'd like to try:\nI was not able to work on this competition for the last two week before stage 2 starts and I missed the post about NIH dataset allowed to use. It's a bigger dataset but with lower quality of labels, would be very interesting to check if training the model to predict both datasets in interleaved way would improve the result, or at least use it to pretrain the base model.

\n\n

I'd like to thank organisers for this competitions, it was a pleasure to participate and hopefully the solutions would make an impact.

", "rawMarkdown": "My solution is based on the modified retinanet based model. Single model, ensembled outputs of 4 folds.\n\nI decided to use retinanet as it's much simpler comparing to Faster-RCNN like models or SSD while having comparable results, this allows much easier experiments and debugging/tuning of model. \n\nCredits to pytorch-retinanet implementation my solution is based on: https://github.com/yhenon/pytorch-retinanet\n\nI scaled the original images to 512x512 resolution, with 256 resolution I have seen results degradation and using the full resolution was not as practical with heavier base models.\n\nModifications I have done to the original pytorch-retinanet implementation:\n\n* tested different base models, se-resnext101 worked the best, se-resnext50 slightly worse\n\n* added an extra output for smaller anchors (level 2 pyramid layer) to handle smaller boxes\n\n* added another classification output predicting the class of the whole image ('No Lung Opacity / Not Normal', 'Normal', 'Lung Opacity'). I have not used the output but even making the model to predict other related function improved the result.\n\n* I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well.\n\n* As classification outputs overfits much faster comparing to anchors position/size regression outputs, I added dropout to anchor and the whole image class outputs. In addition to extra regularisation, it helped to achieve the optimal classification and regression results around the same epoch.\n\nAugmentation used:\nMild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes. I limited the amount of brightness/gamma augmentations as it was hard for me to verify if it does not invalidate labels. To reduce impact of rotation to bounding box sizes, instead of rotating the corners I rotated two points at each edge, at 1/3 and 2/3 edge length from corner, 8 points in total and calculated the new bounding box as min/max of rotated points.\n\n![Rotation box augmentation][1]\n\nI used 4 folds, stratified split by class.\n\nFor submission, I ensembled models trained on each fold and a few checkpoints around CV loss minimum. I have averaged model outputs before applying any thresholds and NMS stages.\n\nAs everyone else noticed, one of challenges of this competition was different distribution of train and test datasets, most likely due to different labeling methodology. Multiple radiologists have labeled each image with intersection used for similar labels. I'd expect this would lead to more boxes predicted, but of smaller size especially in complex cases.\nI tried to roughly simulate this process, using outputs from different folds. Instead of using the mean output of anchor sizes, I used value at 20 percentile and reduced it even more proportionally to difference between 80 and 20 percentile between models.\n\nI have not used any other metadata from images, I felt the orientation field borderlines with being the leak.\n\nI trained model for around 12 epochs, around 1 hour per epoch on 1080ti GPU.\n\nWhat I'd like to try:\nI was not able to work on this competition for the last two week before stage 2 starts and I missed the post about NIH dataset allowed to use. It's a bigger dataset but with lower quality of labels, would be very interesting to check if training the model to predict both datasets in interleaved way would improve the result, or at least use it to pretrain the base model.\n\n\nI'd like to thank organisers for this competitions, it was a pleasure to participate and hopefully the solutions would make an impact.\n\n", "votes": 133 }, { "id": 414761, "postDate": "2018-11-03T14:46:38.753Z", "content": "

Congratulations on 2nd and Grandmaster! Your solution is very elegant, I really like the idea of adding a classification output for the whole image, and your method to accommodate the different annotation processes is very clever.

\n\n

We pretrained our classification models on the NIH dataset first before fine-tuning on the pneumonia challenge dataset. This resulted in minimal improvement (~1%) over using ImageNet weights, though it did lead to faster convergence. We thought about other ways to leverage that dataset but didn't have much time to experiment.

", "rawMarkdown": "Congratulations on 2nd and Grandmaster! Your solution is very elegant, I really like the idea of adding a classification output for the whole image, and your method to accommodate the different annotation processes is very clever. \n\nWe pretrained our classification models on the NIH dataset first before fine-tuning on the pneumonia challenge dataset. This resulted in minimal improvement (~1%) over using ImageNet weights, though it did lead to faster convergence. We thought about other ways to leverage that dataset but didn't have much time to experiment. ", "votes": 3 }, { "id": 432937, "postDate": "2018-12-04T13:52:23.610Z", "content": "

Hi Dmytro, congratulations! Recently I want to re-implement your solution, but I don't understand how to modify the original pytorch-retinanet. Such as how to add an extra output for small anchors and how to add a classification output to classify the whole image? Could you please share your code? It would be very helpful to me .

", "rawMarkdown": "Hi Dmytro, congratulations! Recently I want to re-implement your solution, but I don't understand how to modify the original pytorch-retinanet. Such as how to add an extra output for small anchors and how to add a classification output to classify the whole image? Could you please share your code? It would be very helpful to me .", "votes": 1 }, { "id": 416649, "postDate": "2018-11-07T03:18:45.040Z", "content": "

Congratulations Dmytro on your elegant solution and your impressive grandmaster achievement! There are lots of creative ideas in your solution that I'm interested in playing with, including adding the whole-image classification output and reducing the effect of rotation on bounding box size.

", "rawMarkdown": "Congratulations Dmytro on your elegant solution and your impressive grandmaster achievement! There are lots of creative ideas in your solution that I'm interested in playing with, including adding the whole-image classification output and reducing the effect of rotation on bounding box size. ", "votes": 1 }, { "id": 415113, "postDate": "2018-11-04T12:37:49.453Z", "content": "

Congratulations

", "rawMarkdown": "Congratulations", "votes": 1 }, { "id": 415085, "postDate": "2018-11-04T11:26:55.677Z", "content": "

Congratulations. Very elegant and insightful.

", "rawMarkdown": "Congratulations. Very elegant and insightful.", "votes": 1 }, { "id": 415009, "postDate": "2018-11-04T05:25:37.110Z", "content": "

Congratulations and thanks for sharing your approach

", "rawMarkdown": "Congratulations and thanks for sharing your approach", "votes": 1 }, { "id": 414995, "postDate": "2018-11-04T04:36:38.937Z", "content": "

Congrats Dmytro and thanks for sharing your solution. I really like these 2 comments :

\n\n
\n

I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well.

\n \n

As classification outputs overfits much faster comparing to anchors position/size regression outputs, I added dropout to anchor and the whole image class outputs. In addition to extra regularisation, it helped to achieve the optimal classification and regression results around the same epoch.

\n
\n\n

It looks so clean and effective. We struggled ourselves to get a good balance between classification and regression precision and you got a very elegant solution.

", "rawMarkdown": "Congrats Dmytro and thanks for sharing your solution. I really like these 2 comments : \n\n> I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well.\n\n>As classification outputs overfits much faster comparing to anchors position/size regression outputs, I added dropout to anchor and the whole image class outputs. In addition to extra regularisation, it helped to achieve the optimal classification and regression results around the same epoch.\n\nIt looks so clean and effective. We struggled ourselves to get a good balance between classification and regression precision and you got a very elegant solution.\n\n", "votes": 1 }, { "id": 414978, "postDate": "2018-11-04T02:25:22.367Z", "content": "

congratulations

", "rawMarkdown": "congratulations", "votes": 1 }, { "id": 414968, "postDate": "2018-11-04T01:48:37.753Z", "content": "

Congratulations and thanks for sharing your approach.

", "rawMarkdown": "Congratulations and thanks for sharing your approach.", "votes": 1 }, { "id": 414903, "postDate": "2018-11-03T21:31:31.400Z", "content": "

Congratulations @Dmytro Poplavskiy, well deserved. Kudos on your brand new grand master status and thanks for sharing.

", "rawMarkdown": "Congratulations @Dmytro Poplavskiy, well deserved. Kudos on your brand new grand master status and thanks for sharing.", "votes": 1 }, { "id": 414766, "postDate": "2018-11-03T14:53:00.010Z", "content": "

amazing find about retinanet's loss behavior. congrats!

", "rawMarkdown": "amazing find about retinanet's loss behavior. congrats!", "votes": 1 }, { "id": 414843, "postDate": "2018-11-03T18:07:40.293Z", "content": "

Thanks for sharing, really insightful and great to see a single model approach do so well.

", "rawMarkdown": "Thanks for sharing, really insightful and great to see a single model approach do so well.", "votes": 2 }, { "id": 459111, "postDate": "2019-01-21T07:51:12.450Z", "content": "

Can you please share your code.

", "rawMarkdown": "Can you please share your code.", "votes": 1 }, { "id": 497399, "postDate": "2019-03-23T14:09:43.983Z", "content": "

would you please share your code. thank you soooo much!

", "rawMarkdown": "would you please share your code. thank you soooo much!" }, { "id": 433471, "postDate": "2018-12-05T05:37:52.843Z", "content": "

Hi Dmytro, congratulations! I'm a noobie to computer vision. I'm wondering when an image contains multiple bounding boxes, how it influences anchors position/size regression. To my understanding, an image corresponds to one (x, y, w, h) value. Then if an images does not contain any bounding boxes, or contain more than one, what should I expect? Please correct me if my understanding is wrong. Thank you very much~

", "rawMarkdown": "Hi Dmytro, congratulations! I'm a noobie to computer vision. I'm wondering when an image contains multiple bounding boxes, how it influences anchors position/size regression. To my understanding, an image corresponds to one (x, y, w, h) value. Then if an images does not contain any bounding boxes, or contain more than one, what should I expect? Please correct me if my understanding is wrong. Thank you very much~" }, { "id": 423859, "postDate": "2018-11-19T06:42:12.007Z", "content": "

Congratulations!And I hope that I can study from your code.

", "rawMarkdown": "Congratulations!And I hope that I can study from your code. " }, { "id": 419683, "postDate": "2018-11-12T11:50:36.940Z", "content": "

Great job!

", "rawMarkdown": "Great job!" }, { "id": 419682, "postDate": "2018-11-12T11:50:26.533Z", "content": "

Great job@!

", "rawMarkdown": "Great job@!" }, { "id": 417103, "postDate": "2018-11-07T18:50:21.290Z", "content": "

This looks awesome! Thanks for sharing! Congrats too!

", "rawMarkdown": "This looks awesome! Thanks for sharing! Congrats too!" }, { "id": 416506, "postDate": "2018-11-06T19:01:14.233Z", "content": "

Wow, that's awesome! Great job!

", "rawMarkdown": "Wow, that's awesome! Great job!" }, { "id": 416424, "postDate": "2018-11-06T16:28:40.727Z", "content": "

Congrats!

", "rawMarkdown": "Congrats!" }, { "id": 416304, "postDate": "2018-11-06T14:23:21.190Z", "content": "

Excellent work, thank you for sharing your solution !

", "rawMarkdown": "Excellent work, thank you for sharing your solution !" }, { "id": 416270, "postDate": "2018-11-06T13:59:36.250Z", "content": "

Thank you for sharing your elegant solution.

", "rawMarkdown": "Thank you for sharing your elegant solution. " }, { "id": 416097, "postDate": "2018-11-06T07:30:18.267Z", "content": "

congratulations and thanks for sharing. Looking forward to your code.

", "rawMarkdown": "congratulations and thanks for sharing. Looking forward to your code." }, { "id": 416042, "postDate": "2018-11-06T04:27:54.750Z", "content": "

First of all congratulations :)

\n\n

Secondly, I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well; I'm assuming it's the Focal Loss. Does this mean you're using two classes? Positive and Non-Positive class during during?

", "rawMarkdown": "First of all congratulations :)\n\nSecondly, I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well; I'm assuming it's the Focal Loss. Does this mean you're using two classes? Positive and Non-Positive class during during?" }, { "id": 415809, "postDate": "2018-11-05T18:27:08.050Z", "content": "

Congratulations! And thank you for the code !!!

", "rawMarkdown": "Congratulations! And thank you for the code !!!" }, { "id": 415763, "postDate": "2018-11-05T16:51:36.310Z", "content": "

Congrats Dmytro for the second place and for the GrandMaster title! Very well deserved.

", "rawMarkdown": "Congrats Dmytro for the second place and for the GrandMaster title! Very well deserved." }, { "id": 415758, "postDate": "2018-11-05T16:26:18.180Z", "content": "

Congrats on the result, the elegance of the solution, and for the well deserved Grandmaster title!

", "rawMarkdown": "Congrats on the result, the elegance of the solution, and for the well deserved Grandmaster title!" }, { "id": 415558, "postDate": "2018-11-05T10:11:34.690Z", "content": "

Congratulations Grandmaster! Your solution is very elegant

", "rawMarkdown": "Congratulations Grandmaster! Your solution is very elegant " }, { "id": 415291, "postDate": "2018-11-04T20:24:36.453Z", "content": "

congratulations!

", "rawMarkdown": "congratulations!" }, { "id": 415281, "postDate": "2018-11-04T20:04:25.870Z", "content": "

congrats

", "rawMarkdown": "congrats" }, { "id": 415130, "postDate": "2018-11-04T13:38:43.083Z", "content": "

congrats

", "rawMarkdown": "congrats" }, { "id": 415129, "postDate": "2018-11-04T13:38:24.803Z", "content": "

exciting

", "rawMarkdown": "exciting " }, { "id": 414887, "postDate": "2018-11-03T20:35:25.210Z", "content": "

can you please share your code.

", "rawMarkdown": "can you please share your code.", "replies": [ { "id": 414890, "postDate": "2018-11-03T20:48:50.047Z", "content": "

I'll share my code shortly, will do some cleanup etc. first

", "rawMarkdown": "I'll share my code shortly, will do some cleanup etc. first", "votes": 17 }, { "id": 415078, "postDate": "2018-11-04T11:12:18.367Z", "content": "

Thanks

", "rawMarkdown": "Thanks" }, { "id": 419732, "postDate": "2018-11-12T13:37:04.707Z", "content": "

Hi Dmytro. Congratulations on the 2nd spot and thanks for sharing details on your implementation. How soon can we expect to see a grandmasters work?

", "rawMarkdown": "Hi Dmytro. Congratulations on the 2nd spot and thanks for sharing details on your implementation. How soon can we expect to see a grandmasters work?", "votes": 4 }, { "id": 438548, "postDate": "2018-12-13T21:40:00.587Z", "content": "

Is the code already available? If so, can you share a link?

", "rawMarkdown": "Is the code already available? If so, can you share a link?", "votes": 4 } ] }, { "id": 417360, "postDate": "2018-11-08T06:31:56.800Z", "rawMarkdown": "", "isDeleted": true }, { "id": 415018, "postDate": "2018-11-04T06:54:58.470Z", "content": "

Thanks and congrats!

", "rawMarkdown": "Thanks and congrats!\n", "votes": 1 }, { "id": 415016, "postDate": "2018-11-04T06:10:56.993Z", "content": "

congratulations and thanks for sharing

", "rawMarkdown": "congratulations and thanks for sharing", "votes": 1 }, { "id": 414787, "postDate": "2018-11-03T15:46:51.227Z", "content": "

Thanks for sharing this.

", "rawMarkdown": "Thanks for sharing this. ", "votes": 1 }, { "id": 414758, "postDate": "2018-11-03T14:44:10.790Z", "content": "

Thank you for sharing this.

", "rawMarkdown": "Thank you for sharing this.", "votes": 1 }, { "id": 422422, "postDate": "2018-11-16T07:45:35.410Z", "content": "

Congratulations and thanks for sharing!

", "rawMarkdown": "Congratulations and thanks for sharing!" }, { "id": 421068, "postDate": "2018-11-14T14:31:41.133Z", "content": "

Thanks for sharing!

", "rawMarkdown": "Thanks for sharing!" }, { "id": 419877, "postDate": "2018-11-12T17:25:51.053Z", "content": "

Thanks for sharing!

", "rawMarkdown": "Thanks for sharing!" }, { "id": 419190, "postDate": "2018-11-11T13:28:41.670Z", "content": "

Thanks for sharing this.

", "rawMarkdown": "Thanks for sharing this. " }, { "id": 418026, "postDate": "2018-11-09T06:13:22.523Z", "content": "

Thank you for sharing your solution.

", "rawMarkdown": "Thank you for sharing your solution." }, { "id": 417211, "postDate": "2018-11-08T00:12:19.220Z", "content": "

Brilliant work! Thanks for sharing!

", "rawMarkdown": "Brilliant work! Thanks for sharing!" }, { "id": 416960, "postDate": "2018-11-07T14:27:56.773Z", "content": "

Thanks so much

", "rawMarkdown": "Thanks so much" }, { "id": 416182, "postDate": "2018-11-06T10:49:45.157Z", "content": "

thanks for sharing the solution

", "rawMarkdown": "thanks for sharing the solution" }, { "id": 416147, "postDate": "2018-11-06T09:11:55.517Z", "content": "

Thanks for sharing.

", "rawMarkdown": "Thanks for sharing." }, { "id": 415968, "postDate": "2018-11-06T01:32:57.177Z", "content": "

Thank you for sharing your work.

", "rawMarkdown": "Thank you for sharing your work." }, { "id": 415548, "postDate": "2018-11-05T09:53:40.990Z", "content": "

Thank you for sharing this!

", "rawMarkdown": "Thank you for sharing this!" } ], "comments": [ { "id": 414761, "author_name": "Ian Pan", "author_url": "", "post_date": "2018-11-03T14:46:38.753000", "content": "

Congratulations on 2nd and Grandmaster! Your solution is very elegant, I really like the idea of adding a classification output for the whole image, and your method to accommodate the different annotation processes is very clever.

\n\n

We pretrained our classification models on the NIH dataset first before fine-tuning on the pneumonia challenge dataset. This resulted in minimal improvement (~1%) over using ImageNet weights, though it did lead to faster convergence. We thought about other ways to leverage that dataset but didn't have much time to experiment.

", "votes": 3, "replies": [] }, { "id": 432937, "author_name": "Rayfong Kwong", "author_url": "", "post_date": "2018-12-04T13:52:23.610000", "content": "

Hi Dmytro, congratulations! Recently I want to re-implement your solution, but I don't understand how to modify the original pytorch-retinanet. Such as how to add an extra output for small anchors and how to add a classification output to classify the whole image? Could you please share your code? It would be very helpful to me .

", "votes": 1, "replies": [] }, { "id": 416649, "author_name": "Phillip Cheng", "author_url": "", "post_date": "2018-11-07T03:18:45.040000", "content": "

Congratulations Dmytro on your elegant solution and your impressive grandmaster achievement! There are lots of creative ideas in your solution that I'm interested in playing with, including adding the whole-image classification output and reducing the effect of rotation on bounding box size.

", "votes": 1, "replies": [] }, { "id": 415113, "author_name": "Fábio Wakisaka", "author_url": "", "post_date": "2018-11-04T12:37:49.453000", "content": "

Congratulations

", "votes": 1, "replies": [] }, { "id": 415085, "author_name": "baggins", "author_url": "", "post_date": "2018-11-04T11:26:55.677000", "content": "

Congratulations. Very elegant and insightful.

", "votes": 1, "replies": [] }, { "id": 415009, "author_name": "ai-gen", "author_url": "", "post_date": "2018-11-04T05:25:37.110000", "content": "

Congratulations and thanks for sharing your approach

", "votes": 1, "replies": [] }, { "id": 414995, "author_name": "Alexandre Cadrin-Chênevert", "author_url": "", "post_date": "2018-11-04T04:36:38.937000", "content": "

Congrats Dmytro and thanks for sharing your solution. I really like these 2 comments :

\n\n
\n

I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well.

\n \n

As classification outputs overfits much faster comparing to anchors position/size regression outputs, I added dropout to anchor and the whole image class outputs. In addition to extra regularisation, it helped to achieve the optimal classification and regression results around the same epoch.

\n
\n\n

It looks so clean and effective. We struggled ourselves to get a good balance between classification and regression precision and you got a very elegant solution.

", "votes": 1, "replies": [] }, { "id": 414978, "author_name": "William Green", "author_url": "", "post_date": "2018-11-04T02:25:22.367000", "content": "

congratulations

", "votes": 1, "replies": [] }, { "id": 414968, "author_name": "Tim H", "author_url": "", "post_date": "2018-11-04T01:48:37.753000", "content": "

Congratulations and thanks for sharing your approach.

", "votes": 1, "replies": [] }, { "id": 414903, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2018-11-03T21:31:31.400000", "content": "

Congratulations @Dmytro Poplavskiy, well deserved. Kudos on your brand new grand master status and thanks for sharing.

", "votes": 1, "replies": [] }, { "id": 414766, "author_name": "raddar", "author_url": "", "post_date": "2018-11-03T14:53:00.010000", "content": "

amazing find about retinanet's loss behavior. congrats!

", "votes": 1, "replies": [] }, { "id": 414843, "author_name": "Tom Aindow", "author_url": "", "post_date": "2018-11-03T18:07:40.293000", "content": "

Thanks for sharing, really insightful and great to see a single model approach do so well.

", "votes": 2, "replies": [] }, { "id": 459111, "author_name": "Negative25", "author_url": "", "post_date": "2019-01-21T07:51:12.450000", "content": "

Can you please share your code.

", "votes": 1, "replies": [] }, { "id": 497399, "author_name": "Negative25", "author_url": "", "post_date": "2019-03-23T14:09:43.983000", "content": "

would you please share your code. thank you soooo much!

", "votes": 0, "replies": [] }, { "id": 433471, "author_name": "Zhen", "author_url": "", "post_date": "2018-12-05T05:37:52.843000", "content": "

Hi Dmytro, congratulations! I'm a noobie to computer vision. I'm wondering when an image contains multiple bounding boxes, how it influences anchors position/size regression. To my understanding, an image corresponds to one (x, y, w, h) value. Then if an images does not contain any bounding boxes, or contain more than one, what should I expect? Please correct me if my understanding is wrong. Thank you very much~

", "votes": 0, "replies": [] }, { "id": 423859, "author_name": "silly_boy", "author_url": "", "post_date": "2018-11-19T06:42:12.007000", "content": "

Congratulations!And I hope that I can study from your code.

", "votes": 0, "replies": [] }, { "id": 419683, "author_name": "Luiz Eduardo", "author_url": "", "post_date": "2018-11-12T11:50:36.940000", "content": "

Great job!

", "votes": 0, "replies": [] }, { "id": 419682, "author_name": "Luiz Eduardo", "author_url": "", "post_date": "2018-11-12T11:50:26.533000", "content": "

Great job@!

", "votes": 0, "replies": [] }, { "id": 417103, "author_name": "Annie Qiu", "author_url": "", "post_date": "2018-11-07T18:50:21.290000", "content": "

This looks awesome! Thanks for sharing! Congrats too!

", "votes": 0, "replies": [] }, { "id": 416506, "author_name": "Matthew D. Scarborough", "author_url": "", "post_date": "2018-11-06T19:01:14.233000", "content": "

Wow, that's awesome! Great job!

", "votes": 0, "replies": [] }, { "id": 416424, "author_name": "shivee chauhan", "author_url": "", "post_date": "2018-11-06T16:28:40.727000", "content": "

Congrats!

", "votes": 0, "replies": [] }, { "id": 416304, "author_name": "neumann", "author_url": "", "post_date": "2018-11-06T14:23:21.190000", "content": "

Excellent work, thank you for sharing your solution !

", "votes": 0, "replies": [] }, { "id": 416270, "author_name": "Ivan Ivani", "author_url": "", "post_date": "2018-11-06T13:59:36.250000", "content": "

Thank you for sharing your elegant solution.

", "votes": 0, "replies": [] }, { "id": 416097, "author_name": "chenglong", "author_url": "", "post_date": "2018-11-06T07:30:18.267000", "content": "

congratulations and thanks for sharing. Looking forward to your code.

", "votes": 0, "replies": [] }, { "id": 416042, "author_name": "Chan", "author_url": "", "post_date": "2018-11-06T04:27:54.750000", "content": "

First of all congratulations :)

\n\n

Secondly, I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well; I'm assuming it's the Focal Loss. Does this mean you're using two classes? Positive and Non-Positive class during during?

", "votes": 0, "replies": [] }, { "id": 415809, "author_name": "Blonde", "author_url": "", "post_date": "2018-11-05T18:27:08.050000", "content": "

Congratulations! And thank you for the code !!!

", "votes": 0, "replies": [] }, { "id": 415763, "author_name": "Giba", "author_url": "", "post_date": "2018-11-05T16:51:36.310000", "content": "

Congrats Dmytro for the second place and for the GrandMaster title! Very well deserved.

", "votes": 0, "replies": [] }, { "id": 415758, "author_name": "CPMP", "author_url": "", "post_date": "2018-11-05T16:26:18.180000", "content": "

Congrats on the result, the elegance of the solution, and for the well deserved Grandmaster title!

", "votes": 0, "replies": [] }, { "id": 415558, "author_name": "Vishnu Dhas", "author_url": "", "post_date": "2018-11-05T10:11:34.690000", "content": "

Congratulations Grandmaster! Your solution is very elegant

", "votes": 0, "replies": [] }, { "id": 415291, "author_name": "", "author_url": "", "post_date": "2018-11-04T20:24:36.453000", "content": "", "votes": 0, "replies": [] }, { "id": 415281, "author_name": "", "author_url": "", "post_date": "2018-11-04T20:04:25.870000", "content": "", "votes": 0, "replies": [] }, { "id": 415130, "author_name": "", "author_url": "", "post_date": "2018-11-04T13:38:43.083000", "content": "", "votes": 0, "replies": [] }, { "id": 415129, "author_name": "", "author_url": "", "post_date": "2018-11-04T13:38:24.803000", "content": "", "votes": 0, "replies": [] }, { "id": 414887, "author_name": "", "author_url": "", "post_date": "2018-11-03T20:35:25.210000", "content": "", "votes": 0, "replies": [ { "id": 414890, "author_name": "", "author_url": "", "post_date": "2018-11-03T20:48:50.047000", "content": "", "votes": 17, "replies": [] }, { "id": 415078, "author_name": "", "author_url": "", "post_date": "2018-11-04T11:12:18.367000", "content": "", "votes": 0, "replies": [] }, { "id": 419732, "author_name": "", "author_url": "", "post_date": "2018-11-12T13:37:04.707000", "content": "", "votes": 4, "replies": [] }, { "id": 438548, "author_name": "", "author_url": "", "post_date": "2018-12-13T21:40:00.587000", "content": "", "votes": 4, "replies": [] } ] }, { "id": 417360, "author_name": "", "author_url": "", "post_date": "2018-11-08T06:31:56.800000", "content": "", "votes": 0, "replies": [] }, { "id": 415018, "author_name": "", "author_url": "", "post_date": "2018-11-04T06:54:58.470000", "content": "", "votes": 1, "replies": [] }, { "id": 415016, "author_name": "", "author_url": "", "post_date": "2018-11-04T06:10:56.993000", "content": "", "votes": 1, "replies": [] }, { "id": 414787, "author_name": "", "author_url": "", "post_date": "2018-11-03T15:46:51.227000", "content": "", "votes": 1, "replies": [] }, { "id": 414758, "author_name": "", "author_url": "", "post_date": "2018-11-03T14:44:10.790000", "content": "", "votes": 1, "replies": [] }, { "id": 422422, "author_name": "", "author_url": "", "post_date": "2018-11-16T07:45:35.410000", "content": "", "votes": 0, "replies": [] }, { "id": 421068, "author_name": "", "author_url": "", "post_date": "2018-11-14T14:31:41.133000", "content": "", "votes": 0, "replies": [] }, { "id": 419877, "author_name": "", "author_url": "", "post_date": "2018-11-12T17:25:51.053000", "content": "", "votes": 0, "replies": [] }, { "id": 419190, "author_name": "", "author_url": "", "post_date": "2018-11-11T13:28:41.670000", "content": "", "votes": 0, "replies": [] }, { "id": 418026, "author_name": "", "author_url": "", "post_date": "2018-11-09T06:13:22.523000", "content": "", "votes": 0, "replies": [] }, { "id": 417211, "author_name": "", "author_url": "", "post_date": "2018-11-08T00:12:19.220000", "content": "", "votes": 0, "replies": [] }, { "id": 416960, "author_name": "", "author_url": "", "post_date": "2018-11-07T14:27:56.773000", "content": "", "votes": 0, "replies": [] }, { "id": 416182, "author_name": "", "author_url": "", "post_date": "2018-11-06T10:49:45.157000", "content": "", "votes": 0, "replies": [] }, { "id": 416147, "author_name": "", "author_url": "", "post_date": "2018-11-06T09:11:55.517000", "content": "", "votes": 0, "replies": [] }, { "id": 415968, "author_name": "", "author_url": "", "post_date": "2018-11-06T01:32:57.177000", "content": "", "votes": 0, "replies": [] }, { "id": 415548, "author_name": "", "author_url": "", "post_date": "2018-11-05T09:53:40.990000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "414752": "My solution is based on the modified retinanet based model. Single model, ensembled outputs of 4 folds.\n\nI decided to use retinanet as it's much simpler comparing to Faster-RCNN like models or SSD while having comparable results, this allows much easier experiments and debugging/tuning of model. \n\nCredits to pytorch-retinanet implementation my solution is based on: https://github.com/yhenon/pytorch-retinanet\n\nI scaled the original images to 512x512 resolution, with 256 resolution I have seen results degradation and using the full resolution was not as practical with heavier base models.\n\nModifications I have done to the original pytorch-retinanet implementation:\n\n* tested different base models, se-resnext101 worked the best, se-resnext50 slightly worse\n\n* added an extra output for smaller anchors (level 2 pyramid layer) to handle smaller boxes\n\n* added another classification output predicting the class of the whole image ('No Lung Opacity / Not Normal', 'Normal', 'Lung Opacity'). I have not used the output but even making the model to predict other related function improved the result.\n\n* I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well.\n\n* As classification outputs overfits much faster comparing to anchors position/size regression outputs, I added dropout to anchor and the whole image class outputs. In addition to extra regularisation, it helped to achieve the optimal classification and regression results around the same epoch.\n\nAugmentation used:\nMild rotations (up to 6 deg), shift, scale, shear and h_flip, for some images random level of blur and noise and gamma changes. I limited the amount of brightness/gamma augmentations as it was hard for me to verify if it does not invalidate labels. To reduce impact of rotation to bounding box sizes, instead of rotating the corners I rotated two points at each edge, at 1/3 and 2/3 edge length from corner, 8 points in total and calculated the new bounding box as min/max of rotated points.\n\n![Rotation box augmentation][1]\n\nI used 4 folds, stratified split by class.\n\nFor submission, I ensembled models trained on each fold and a few checkpoints around CV loss minimum. I have averaged model outputs before applying any thresholds and NMS stages.\n\nAs everyone else noticed, one of challenges of this competition was different distribution of train and test datasets, most likely due to different labeling methodology. Multiple radiologists have labeled each image with intersection used for similar labels. I'd expect this would lead to more boxes predicted, but of smaller size especially in complex cases.\nI tried to roughly simulate this process, using outputs from different folds. Instead of using the mean output of anchor sizes, I used value at 20 percentile and reduced it even more proportionally to difference between 80 and 20 percentile between models.\n\nI have not used any other metadata from images, I felt the orientation field borderlines with being the leak.\n\nI trained model for around 12 epochs, around 1 hour per epoch on 1080ti GPU.\n\nWhat I'd like to try:\nI was not able to work on this competition for the last two week before stage 2 starts and I missed the post about NIH dataset allowed to use. It's a bigger dataset but with lower quality of labels, would be very interesting to check if training the model to predict both datasets in interleaved way would improve the result, or at least use it to pretrain the base model.\n\n\nI'd like to thank organisers for this competitions, it was a pleasure to participate and hopefully the solutions would make an impact.\n\n", "414761": "Congratulations on 2nd and Grandmaster! Your solution is very elegant, I really like the idea of adding a classification output for the whole image, and your method to accommodate the different annotation processes is very clever. \n\nWe pretrained our classification models on the NIH dataset first before fine-tuning on the pneumonia challenge dataset. This resulted in minimal improvement (~1%) over using ImageNet weights, though it did lead to faster convergence. We thought about other ways to leverage that dataset but didn't have much time to experiment. ", "432937": "Hi Dmytro, congratulations! Recently I want to re-implement your solution, but I don't understand how to modify the original pytorch-retinanet. Such as how to add an extra output for small anchors and how to add a classification output to classify the whole image? Could you please share your code? It would be very helpful to me .", "416649": "Congratulations Dmytro on your elegant solution and your impressive grandmaster achievement! There are lots of creative ideas in your solution that I'm interested in playing with, including adding the whole-image classification output and reducing the effect of rotation on bounding box size. ", "415113": "Congratulations", "415085": "Congratulations. Very elegant and insightful.", "415009": "Congratulations and thanks for sharing your approach", "414995": "Congrats Dmytro and thanks for sharing your solution. I really like these 2 comments : \n\n> I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well.\n\n>As classification outputs overfits much faster comparing to anchors position/size regression outputs, I added dropout to anchor and the whole image class outputs. In addition to extra regularisation, it helped to achieve the optimal classification and regression results around the same epoch.\n\nIt looks so clean and effective. We struggled ourselves to get a good balance between classification and regression precision and you got a very elegant solution.\n\n", "414978": "congratulations", "414968": "Congratulations and thanks for sharing your approach.", "414903": "Congratulations @Dmytro Poplavskiy, well deserved. Kudos on your brand new grand master status and thanks for sharing.", "414766": "amazing find about retinanet's loss behavior. congrats!", "414843": "Thanks for sharing, really insightful and great to see a single model approach do so well.", "459111": "Can you please share your code.", "497399": "would you please share your code. thank you soooo much!", "433471": "Hi Dmytro, congratulations! I'm a noobie to computer vision. I'm wondering when an image contains multiple bounding boxes, how it influences anchors position/size regression. To my understanding, an image corresponds to one (x, y, w, h) value. Then if an images does not contain any bounding boxes, or contain more than one, what should I expect? Please correct me if my understanding is wrong. Thank you very much~", "423859": "Congratulations!And I hope that I can study from your code. ", "419683": "Great job!", "419682": "Great job@!", "417103": "This looks awesome! Thanks for sharing! Congrats too!", "416506": "Wow, that's awesome! Great job!", "416424": "Congrats!", "416304": "Excellent work, thank you for sharing your solution !", "416270": "Thank you for sharing your elegant solution. ", "416097": "congratulations and thanks for sharing. Looking forward to your code.", "416042": "First of all congratulations :)\n\nSecondly, I found the original pytorch-retinanet implementation ignored images with no boxes, I changed it to calculate loss on them as well; I'm assuming it's the Focal Loss. Does this mean you're using two classes? Positive and Non-Positive class during during?", "415809": "Congratulations! And thank you for the code !!!", "415763": "Congrats Dmytro for the second place and for the GrandMaster title! Very well deserved.", "415758": "Congrats on the result, the elegance of the solution, and for the well deserved Grandmaster title!", "415558": "Congratulations Grandmaster! Your solution is very elegant ", "415291": "congratulations!", "415281": "congrats", "415130": "congrats", "415129": "exciting ", "414887": "can you please share your code.", "417360": "", "415018": "Thanks and congrats!\n", "415016": "congratulations and thanks for sharing", "414787": "Thanks for sharing this. ", "414758": "Thank you for sharing this.", "422422": "Congratulations and thanks for sharing!", "421068": "Thanks for sharing!", "419877": "Thanks for sharing!", "419190": "Thanks for sharing this. ", "418026": "Thank you for sharing your solution.", "417211": "Brilliant work! Thanks for sharing!", "416960": "Thanks so much", "416182": "thanks for sharing the solution", "416147": "Thanks for sharing.", "415968": "Thank you for sharing your work.", "415548": "Thank you for sharing this!" } }