![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F505196%2F953479a06dbb81adbf16320caadf44e7%2Fcompressor.png?generation=1574249239505220&alt=media\")
Hello to everyone participating in the competition, congratulations to all who won and thanks to kaggle for the excellent competition.
\n\nHere I will give a general solution to the problem, I will talk about techniques that helped and those ideas that did not work.
\n\nMost recently, I participated in kaggle segmentation contests \nSIIM-ACR Pneumothorax Segmentation and Severstal: Steel Defect Detection\nTherefore, I have gained decent experience in solving such problems. I already had an idea of what could work and what couldn’t.
\n\nLooking at the data, I saw that the images have a dimension of 1400x2100 and it was not a good idea to put such data to the network directly. Of course, it was possible to resize the image to 2 or 4 times, but obviously, we will definitely lose something from the data.\nI came up with a compromise. Use a small network - a compressor, that extracts significant features from the data and reduces the image size.\nIt looks something like this:
\n\n
To build models, i used Keras 2, Tensorflow 1.4 and the library https://github.com/qubvel/segmentation_models (thank you very much Pavel Yakubovskiy)
\n\nIn order to build an effective ensemble, we must use models with the least possible correlation between predictions. I decided to use such combinations of model parameters:
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F505196%2Fe0d9fd528dc14968070094b3bd2dd595%2Fmodels_grid.png?generation=1574249329683540&alt=media\")
All models had a Unet decoder.
\n\ntraining parameters:\nOptimizer: Adam\nLoss Function: FocalLoss\nBatch Size: 4
\n\nHard albumentation: \nHflip, VFlip, Equalize, CLAHE, RandomBrightnessContrast, RandomGamma, Cutout\nShiftScaleRotate, GridDistortion, GaussNoise
\n\n30 epochs on a two-cycle learning profile. It looks something like this:
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F505196%2F2f8acf3439d405a4d13cde0fa28d6378%2FLearning%20profile.png?generation=1574249719812217&alt=media\")
For training models, I used 2xP3.2 Amazon instance
\n\nPostprocessing. \nMean average all models -> raw probability\nAll tasks for segmenting objects with a DICE metric are very sensitive to FalsePositive errors. In some cases, training a separate classifier model for detect of a mask in the image very helps. In my case, the classifiers did not help much and I used the Triple rule method, which I first saw in the first place solution about competition SIIM-ACR Pneumothorax Segmentation. \nThanks so much for the idea of Aimoldin Anuar https://www.kaggle.com/sneddy\nThe description of this approach can be understood from here Kaggle SIIM-ACR Pneumothorax Challenge - 1st place solution - Anuar Aimoldin
\n\nThe triple rule parameters (threshold1, minsize, threshold2) were searched by global optimization methods.
\n\nBasically, this is all that helped in solving the task.
\n\nWhat didn't work:\n- Mask classifiers\n- mmdetection / FasterRCNN\n- BCE-DICE, lovasz, triple_loss\n- Adversarial validation\n- Pseudo labeling
\n\n> \nThanks for watching
", "messages": [ { "id": 677612, "postDate": "2019-11-20T11:46:55.523Z", "content": "Hello to everyone participating in the competition, congratulations to all who won and thanks to kaggle for the excellent competition.
\n\nHere I will give a general solution to the problem, I will talk about techniques that helped and those ideas that did not work.
\n\nMost recently, I participated in kaggle segmentation contests \nSIIM-ACR Pneumothorax Segmentation and Severstal: Steel Defect Detection\nTherefore, I have gained decent experience in solving such problems. I already had an idea of what could work and what couldn’t.
\n\nLooking at the data, I saw that the images have a dimension of 1400x2100 and it was not a good idea to put such data to the network directly. Of course, it was possible to resize the image to 2 or 4 times, but obviously, we will definitely lose something from the data.\nI came up with a compromise. Use a small network - a compressor, that extracts significant features from the data and reduces the image size.\nIt looks something like this:
\n\n
To build models, i used Keras 2, Tensorflow 1.4 and the library https://github.com/qubvel/segmentation_models (thank you very much Pavel Yakubovskiy)
\n\nIn order to build an effective ensemble, we must use models with the least possible correlation between predictions. I decided to use such combinations of model parameters:
\n\n
All models had a Unet decoder.
\n\ntraining parameters:\nOptimizer: Adam\nLoss Function: FocalLoss\nBatch Size: 4
\n\nHard albumentation: \nHflip, VFlip, Equalize, CLAHE, RandomBrightnessContrast, RandomGamma, Cutout\nShiftScaleRotate, GridDistortion, GaussNoise
\n\n30 epochs on a two-cycle learning profile. It looks something like this:
\n\n
For training models, I used 2xP3.2 Amazon instance
\n\nPostprocessing. \nMean average all models -> raw probability\nAll tasks for segmenting objects with a DICE metric are very sensitive to FalsePositive errors. In some cases, training a separate classifier model for detect of a mask in the image very helps. In my case, the classifiers did not help much and I used the Triple rule method, which I first saw in the first place solution about competition SIIM-ACR Pneumothorax Segmentation. \nThanks so much for the idea of Aimoldin Anuar https://www.kaggle.com/sneddy\nThe description of this approach can be understood from here Kaggle SIIM-ACR Pneumothorax Challenge - 1st place solution - Anuar Aimoldin
\n\nThe triple rule parameters (threshold1, minsize, threshold2) were searched by global optimization methods.
\n\nBasically, this is all that helped in solving the task.
\n\nWhat didn't work:\n- Mask classifiers\n- mmdetection / FasterRCNN\n- BCE-DICE, lovasz, triple_loss\n- Adversarial validation\n- Pseudo labeling
\n\n> \nThanks for watching
", "rawMarkdown": "Hello to everyone participating in the competition, congratulations to all who won and thanks to kaggle for the excellent competition.\n\nHere I will give a general solution to the problem, I will talk about techniques that helped and those ideas that did not work.\n\nMost recently, I participated in kaggle segmentation contests \n[SIIM-ACR Pneumothorax Segmentation](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation) and [Severstal: Steel Defect Detection](https://www.kaggle.com/c/severstal-steel-defect-detection)\nTherefore, I have gained decent experience in solving such problems. I already had an idea of what could work and what couldn’t.\n** **\n\n### Idea #1\nLooking at the data, I saw that the images have a dimension of 1400x2100 and it was not a good idea to put such data to the network directly. Of course, it was possible to resize the image to 2 or 4 times, but obviously, we will definitely lose something from the data.\nI came up with a compromise. Use a small network - a compressor, that extracts significant features from the data and reduces the image size.\nIt looks something like this:\n\n\n\nTo build models, i used Keras 2, Tensorflow 1.4 and the library [https://github.com/qubvel/segmentation_models](https://github.com/qubvel/segmentation_models) (thank you very much Pavel Yakubovskiy)\n** **\n\n### Idea #2\nIn order to build an effective ensemble, we must use models with the least possible correlation between predictions. I decided to use such combinations of model parameters:\n\n\n\nAll models had a Unet decoder.\n\n** **\n\n**training parameters:**\nOptimizer: Adam\nLoss Function: FocalLoss\nBatch Size: 4\n\nHard albumentation: \nHflip, VFlip, Equalize, CLAHE, RandomBrightnessContrast, RandomGamma, Cutout\nShiftScaleRotate, GridDistortion, GaussNoise\n\n30 epochs on a two-cycle learning profile. It looks something like this:\n\n\n\nFor training models, I used 2xP3.2 Amazon instance\n** **\n\n### Idea #3\n**Postprocessing**. \nMean average all models -> raw probability\nAll tasks for segmenting objects with a DICE metric are very sensitive to FalsePositive errors. In some cases, training a separate classifier model for detect of a mask in the image very helps. In my case, the classifiers did not help much and I used the Triple rule method, which I first saw in the first place solution about competition [SIIM-ACR Pneumothorax Segmentation](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation). \nThanks so much for the idea of **Aimoldin Anuar** https://www.kaggle.com/sneddy\nThe description of this approach can be understood from here [Kaggle SIIM-ACR Pneumothorax Challenge - 1st place solution - Anuar Aimoldin](https://youtu.be/Wuf0wE3Mrxg)\n\n\nThe triple rule parameters (threshold1, minsize, threshold2) were searched by global optimization methods.\n\nBasically, this is all that helped in solving the task.\n\nWhat didn't work:\n- Mask classifiers\n- mmdetection / FasterRCNN\n- BCE-DICE, lovasz, triple_loss\n- Adversarial validation\n- Pseudo labeling\n\n\n> \nThanks for watching\n\n\n\n", "votes": 85 }, { "id": 677621, "postDate": "2019-11-20T12:12:30.070Z", "content": "Thanks for sharing, you got a high score on your first three submissions. How did you do that😁
", "rawMarkdown": "Thanks for sharing, you got a high score on your first three submissions. How did you do that😁 ", "votes": 1, "replies": [ { "id": 677624, "postDate": "2019-11-20T12:20:09.173Z", "content": "It's a magic 😃
", "rawMarkdown": "It's a magic 😃 " }, { "id": 677636, "postDate": "2019-11-20T12:35:59.140Z", "content": "Could you share this amazing magic 😁
", "rawMarkdown": "Could you share this amazing magic 😁 " } ] }, { "id": 679754, "postDate": "2019-11-23T10:19:19.947Z", "content": "Congratulations and Thank you for sharing!\nThe convolution compress is brilliant, I don't know if I understand correctly, the backbone is a intact segment model? you input is 1408x2176 image, compress by convolution, and the model output is 352x544 segment mask?
", "rawMarkdown": "Congratulations and Thank you for sharing!\nThe convolution compress is brilliant, I don't know if I understand correctly, the backbone is a intact segment model? you input is 1408x2176 image, compress by convolution, and the model output is 352x544 segment mask?", "replies": [ { "id": 680134, "postDate": "2019-11-24T04:36:29.243Z", "content": "Yes, right.
", "rawMarkdown": "Yes, right." } ] }, { "id": 679198, "postDate": "2019-11-22T12:29:02.923Z", "content": "Since I'm new to Convolution networks,could you please tell me what is backbone and how to use it with other networks?
", "rawMarkdown": "Since I'm new to Convolution networks,could you please tell me what is backbone and how to use it with other networks?", "replies": [ { "id": 679623, "postDate": "2019-11-23T03:46:21.160Z", "content": "I used two backbones: efficientnetb1, seresnext50. It's encoder. And Unet as decoder. All part ready to use from that library https://github.com/qubvel/segmentation_models
", "rawMarkdown": "I used two backbones: efficientnetb1, seresnext50. It's encoder. And Unet as decoder. All part ready to use from that library https://github.com/qubvel/segmentation_models " }, { "id": 679681, "postDate": "2019-11-23T06:49:54.430Z", "content": "I'm asking what is backbone??
", "rawMarkdown": "I'm asking what is **backbone**??" }, { "id": 680135, "postDate": "2019-11-24T04:39:00.927Z", "content": "I am sorry. Backbone is common name of encoder in segmentation models.
", "rawMarkdown": "I am sorry. Backbone is common name of encoder in segmentation models." }, { "id": 680162, "postDate": "2019-11-24T06:18:26.070Z", "content": "Thank you for the clarification
", "rawMarkdown": "Thank you for the clarification\n" } ] }, { "id": 678189, "postDate": "2019-11-21T04:42:37.740Z", "content": "Congrats Andrey Kiryasov
", "rawMarkdown": "Congrats Andrey Kiryasov " }, { "id": 677846, "postDate": "2019-11-20T16:43:21.787Z", "content": "Triple congrats Andrey on solo Gold, prize money, and becoming Grand Master !!
\n\nI like your idea #1. That's a smart trick of retaining full original information. How did you train it? I'm particularly interested in learning schedule of which layers were frozen and un-frozen, where were your losses applied, learning rates, etc.
", "rawMarkdown": "Triple congrats Andrey on solo Gold, prize money, and becoming Grand Master !!\n\nI like your idea #1. That's a smart trick of retaining full original information. How did you train it? I'm particularly interested in learning schedule of which layers were frozen and un-frozen, where were your losses applied, learning rates, etc.", "replies": [ { "id": 677867, "postDate": "2019-11-20T17:22:14.607Z", "content": "Thank you!
\n\nhi, what is fixed two-cycle learning rate profile ? Is there any shared information
", "rawMarkdown": "hi, what is fixed two-cycle learning rate profile ? Is there any shared information" }, { "id": 818708, "postDate": "2020-04-24T04:14:29.433Z", "content": "I trained the network for 30 epohs and changed the learning rate according to this schedule ![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F505196%2Fd5d16579ad8c3aa96489bf94341e1394%2FLearning%20profile.png?generation=1587701754018812&alt=media\")
That small network to extract features of full size and compress image was brilliant.\nI will keep that in mind
", "rawMarkdown": "That small network to extract features of full size and compress image was brilliant.\nI will keep that in mind" }, { "id": 677713, "postDate": "2019-11-20T14:17:18.820Z", "rawMarkdown": "", "isDeleted": true }, { "id": 677820, "postDate": "2019-11-20T16:11:42.547Z", "content": "Thanks for sharing!!
", "rawMarkdown": "Thanks for sharing!!" }, { "id": 677732, "postDate": "2019-11-20T14:31:08.517Z", "content": "Nice approach and thanks for sharing
", "rawMarkdown": "Nice approach and thanks for sharing" } ], "comments": [ { "id": 677621, "author_name": "He", "author_url": "", "post_date": "2019-11-20T12:12:30.070000", "content": "Thanks for sharing, you got a high score on your first three submissions. How did you do that😁
", "votes": 1, "replies": [ { "id": 677624, "author_name": "Andrey Kiryasov", "author_url": "", "post_date": "2019-11-20T12:20:09.173000", "content": "It's a magic 😃
", "votes": 0, "replies": [] }, { "id": 677636, "author_name": "He", "author_url": "", "post_date": "2019-11-20T12:35:59.140000", "content": "Could you share this amazing magic 😁
", "votes": 0, "replies": [] } ] }, { "id": 679754, "author_name": "Kyle", "author_url": "", "post_date": "2019-11-23T10:19:19.947000", "content": "Congratulations and Thank you for sharing!\nThe convolution compress is brilliant, I don't know if I understand correctly, the backbone is a intact segment model? you input is 1408x2176 image, compress by convolution, and the model output is 352x544 segment mask?
", "votes": 0, "replies": [ { "id": 680134, "author_name": "Andrey Kiryasov", "author_url": "", "post_date": "2019-11-24T04:36:29.243000", "content": "Yes, right.
", "votes": 0, "replies": [] } ] }, { "id": 679198, "author_name": "MhdSharuk", "author_url": "", "post_date": "2019-11-22T12:29:02.923000", "content": "Since I'm new to Convolution networks,could you please tell me what is backbone and how to use it with other networks?
", "votes": 0, "replies": [ { "id": 679623, "author_name": "Andrey Kiryasov", "author_url": "", "post_date": "2019-11-23T03:46:21.160000", "content": "I used two backbones: efficientnetb1, seresnext50. It's encoder. And Unet as decoder. All part ready to use from that library https://github.com/qubvel/segmentation_models
", "votes": 0, "replies": [] }, { "id": 679681, "author_name": "MhdSharuk", "author_url": "", "post_date": "2019-11-23T06:49:54.430000", "content": "I'm asking what is backbone??
", "votes": 0, "replies": [] }, { "id": 680135, "author_name": "Andrey Kiryasov", "author_url": "", "post_date": "2019-11-24T04:39:00.927000", "content": "I am sorry. Backbone is common name of encoder in segmentation models.
", "votes": 0, "replies": [] }, { "id": 680162, "author_name": "MhdSharuk", "author_url": "", "post_date": "2019-11-24T06:18:26.070000", "content": "Thank you for the clarification
", "votes": 0, "replies": [] } ] }, { "id": 678189, "author_name": "S T MOHAMMED", "author_url": "", "post_date": "2019-11-21T04:42:37.740000", "content": "Congrats Andrey Kiryasov
", "votes": 0, "replies": [] }, { "id": 677846, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-11-20T16:43:21.787000", "content": "Triple congrats Andrey on solo Gold, prize money, and becoming Grand Master !!
\n\nI like your idea #1. That's a smart trick of retaining full original information. How did you train it? I'm particularly interested in learning schedule of which layers were frozen and un-frozen, where were your losses applied, learning rates, etc.
", "votes": 0, "replies": [ { "id": 677867, "author_name": "Andrey Kiryasov", "author_url": "", "post_date": "2019-11-20T17:22:14.607000", "content": "Thank you!
\n\nhi, what is fixed two-cycle learning rate profile ? Is there any shared information
", "votes": 0, "replies": [] }, { "id": 818708, "author_name": "Andrey Kiryasov", "author_url": "", "post_date": "2020-04-24T04:14:29.433000", "content": "I trained the network for 30 epohs and changed the learning rate according to this schedule
That small network to extract features of full size and compress image was brilliant.\nI will keep that in mind
", "votes": 0, "replies": [] }, { "id": 677713, "author_name": "", "author_url": "", "post_date": "2019-11-20T14:17:18.820000", "content": "", "votes": 0, "replies": [] }, { "id": 677820, "author_name": "Thiago Sena", "author_url": "", "post_date": "2019-11-20T16:11:42.547000", "content": "Thanks for sharing!!
", "votes": 0, "replies": [] }, { "id": 677732, "author_name": "SeshuRaju 🧘♂️", "author_url": "", "post_date": "2019-11-20T14:31:08.517000", "content": "Nice approach and thanks for sharing
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "677612": "Hello to everyone participating in the competition, congratulations to all who won and thanks to kaggle for the excellent competition.\n\nHere I will give a general solution to the problem, I will talk about techniques that helped and those ideas that did not work.\n\nMost recently, I participated in kaggle segmentation contests \n[SIIM-ACR Pneumothorax Segmentation](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation) and [Severstal: Steel Defect Detection](https://www.kaggle.com/c/severstal-steel-defect-detection)\nTherefore, I have gained decent experience in solving such problems. I already had an idea of what could work and what couldn’t.\n** **\n\n### Idea #1\nLooking at the data, I saw that the images have a dimension of 1400x2100 and it was not a good idea to put such data to the network directly. Of course, it was possible to resize the image to 2 or 4 times, but obviously, we will definitely lose something from the data.\nI came up with a compromise. Use a small network - a compressor, that extracts significant features from the data and reduces the image size.\nIt looks something like this:\n\n\n\nTo build models, i used Keras 2, Tensorflow 1.4 and the library [https://github.com/qubvel/segmentation_models](https://github.com/qubvel/segmentation_models) (thank you very much Pavel Yakubovskiy)\n** **\n\n### Idea #2\nIn order to build an effective ensemble, we must use models with the least possible correlation between predictions. I decided to use such combinations of model parameters:\n\n\n\nAll models had a Unet decoder.\n\n** **\n\n**training parameters:**\nOptimizer: Adam\nLoss Function: FocalLoss\nBatch Size: 4\n\nHard albumentation: \nHflip, VFlip, Equalize, CLAHE, RandomBrightnessContrast, RandomGamma, Cutout\nShiftScaleRotate, GridDistortion, GaussNoise\n\n30 epochs on a two-cycle learning profile. It looks something like this:\n\n\n\nFor training models, I used 2xP3.2 Amazon instance\n** **\n\n### Idea #3\n**Postprocessing**. \nMean average all models -> raw probability\nAll tasks for segmenting objects with a DICE metric are very sensitive to FalsePositive errors. In some cases, training a separate classifier model for detect of a mask in the image very helps. In my case, the classifiers did not help much and I used the Triple rule method, which I first saw in the first place solution about competition [SIIM-ACR Pneumothorax Segmentation](https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation). \nThanks so much for the idea of **Aimoldin Anuar** https://www.kaggle.com/sneddy\nThe description of this approach can be understood from here [Kaggle SIIM-ACR Pneumothorax Challenge - 1st place solution - Anuar Aimoldin](https://youtu.be/Wuf0wE3Mrxg)\n\n\nThe triple rule parameters (threshold1, minsize, threshold2) were searched by global optimization methods.\n\nBasically, this is all that helped in solving the task.\n\nWhat didn't work:\n- Mask classifiers\n- mmdetection / FasterRCNN\n- BCE-DICE, lovasz, triple_loss\n- Adversarial validation\n- Pseudo labeling\n\n\n> \nThanks for watching\n\n\n\n", "677621": "Thanks for sharing, you got a high score on your first three submissions. How did you do that😁 ", "679754": "Congratulations and Thank you for sharing!\nThe convolution compress is brilliant, I don't know if I understand correctly, the backbone is a intact segment model? you input is 1408x2176 image, compress by convolution, and the model output is 352x544 segment mask?", "679198": "Since I'm new to Convolution networks,could you please tell me what is backbone and how to use it with other networks?", "678189": "Congrats Andrey Kiryasov ", "677846": "Triple congrats Andrey on solo Gold, prize money, and becoming Grand Master !!\n\nI like your idea #1. That's a smart trick of retaining full original information. How did you train it? I'm particularly interested in learning schedule of which layers were frozen and un-frozen, where were your losses applied, learning rates, etc.", "677669": "That small network to extract features of full size and compress image was brilliant.\nI will keep that in mind", "677713": "", "677820": "Thanks for sharing!!", "677732": "Nice approach and thanks for sharing" } }