I know it's a bit early for that, but wonder what's the best way to combine/ensemble segmentation models?\nLinear combination of predictions, stacked as transfer learning features, stacked predictions into a new model?\nWhat does the literature say?
\n\nPlease comment with your ideas, and what works or doesn't. Thank you.
", "messages": [ { "id": "373479", "postDate": "08/21/2018 12:26:29", "content": "I know it's a bit early for that, but wonder what's the best way to combine/ensemble segmentation models?\nLinear combination of predictions, stacked as transfer learning features, stacked predictions into a new model?\nWhat does the literature say?
\n\nPlease comment with your ideas, and what works or doesn't. Thank you.
", "rawMarkdown": "I know it's a bit early for that, but wonder what's the best way to combine/ensemble segmentation models?\nLinear combination of predictions, stacked as transfer learning features, stacked predictions into a new model?\nWhat does the literature say?\n\nPlease comment with your ideas, and what works or doesn't. Thank you.", "votes": null }, { "id": "373652", "postDate": "08/21/2018 17:40:42", "content": "In some competitions like the 2018 science bowl winner, they took a simple average of the final layer output between all the ensemble models. You could also try test time augmentation: run each image 4 times, with rotations/flips and take the average. Obviously for the second phase this is going to pay a performance penalty but I'm not sure what the real strategy is. Probably accuracy is most important. Only the top performing teams are eligible to participate in the Speed prize.
", "rawMarkdown": "In some competitions like the 2018 science bowl winner, they took a simple average of the final layer output between all the ensemble models. You could also try test time augmentation: run each image 4 times, with rotations/flips and take the average. Obviously for the second phase this is going to pay a performance penalty but I'm not sure what the real strategy is. Probably accuracy is most important. Only the top performing teams are eligible to participate in the Speed prize.", "votes": null }, { "id": "373793", "postDate": "08/21/2018 23:19:02", "content": "Cool, thanks!
\n\nI've tried a hierarchical approach running first a classification (boat/no-boat) then the segmentation to avoid false positives: \nhttps://www.kaggle.com/hmendonca/classification-and-segmentation
\n\nIt works, however, I believe it's probably possible to get the same result with a single model...
", "rawMarkdown": "Cool, thanks!\n\nI've tried a hierarchical approach running first a classification (boat/no-boat) then the segmentation to avoid false positives: \nhttps://www.kaggle.com/hmendonca/classification-and-segmentation\n\nIt works, however, I believe it's probably possible to get the same result with a single model...", "votes": null }, { "id": "374849", "postDate": "08/24/2018 01:32:39", "content": "People could probably get a good result without any classification beforehand, but you would have to tinker with the distributions of positive and negative as well as loss functions. Did you balance the classes for your classifier?
", "rawMarkdown": "People could probably get a good result without any classification beforehand, but you would have to tinker with the distributions of positive and negative as well as loss functions. Did you balance the classes for your classifier?", "votes": null }, { "id": "375133", "postDate": "08/24/2018 16:44:16", "content": "That kernel is really just an ensemble of the 2 best public models, but there is some class balance. However, it might not be optimal for the ensemble, as each model was not specifically trained for blending. Any tips?
", "rawMarkdown": "That kernel is really just an ensemble of the 2 best public models, but there is some class balance. However, it might not be optimal for the ensemble, as each model was not specifically trained for blending. Any tips?", "votes": null }, { "id": "375170", "postDate": "08/24/2018 17:51:54", "content": "I think the two layer approach is one way of dealing with the class imbalance. Similar to RCNN type approaches where you have an image classifier as a coarse grained filter and another NN to make more fine grain predictions. Here is a good overview of that approach (and iterations on top of that): https://blog.athelas.com/a-brief-history-of-cnns-in-image-segmentation-from-r-cnn-to-mask-r-cnn-34ea83205de4
", "rawMarkdown": "I think the two layer approach is one way of dealing with the class imbalance. Similar to RCNN type approaches where you have an image classifier as a coarse grained filter and another NN to make more fine grain predictions. Here is a good overview of that approach (and iterations on top of that): https://blog.athelas.com/a-brief-history-of-cnns-in-image-segmentation-from-r-cnn-to-mask-r-cnn-34ea83205de4", "votes": null }, { "id": "375572", "postDate": "08/25/2018 14:21:27", "content": "Very interesting article! Thanks Paul!
", "rawMarkdown": "Very interesting article! Thanks Paul!", "votes": null } ], "comments": [ { "id": 373652, "author_name": "oewyn000", "author_url": "", "post_date": "08/21/2018 17:40:42", "content": "In some competitions like the 2018 science bowl winner, they took a simple average of the final layer output between all the ensemble models. You could also try test time augmentation: run each image 4 times, with rotations/flips and take the average. Obviously for the second phase this is going to pay a performance penalty but I'm not sure what the real strategy is. Probably accuracy is most important. Only the top performing teams are eligible to participate in the Speed prize.
", "votes": null, "replies": [ { "id": 373793, "author_name": "hmendonca", "author_url": "", "post_date": "08/21/2018 23:19:02", "content": "Cool, thanks!
\n\nI've tried a hierarchical approach running first a classification (boat/no-boat) then the segmentation to avoid false positives: \nhttps://www.kaggle.com/hmendonca/classification-and-segmentation
\n\nIt works, however, I believe it's probably possible to get the same result with a single model...
", "votes": null, "replies": [] }, { "id": 374849, "author_name": "arpandhatt", "author_url": "", "post_date": "08/24/2018 01:32:39", "content": "People could probably get a good result without any classification beforehand, but you would have to tinker with the distributions of positive and negative as well as loss functions. Did you balance the classes for your classifier?
", "votes": null, "replies": [] }, { "id": 375133, "author_name": "hmendonca", "author_url": "", "post_date": "08/24/2018 16:44:16", "content": "That kernel is really just an ensemble of the 2 best public models, but there is some class balance. However, it might not be optimal for the ensemble, as each model was not specifically trained for blending. Any tips?
", "votes": null, "replies": [] }, { "id": 375170, "author_name": "oewyn000", "author_url": "", "post_date": "08/24/2018 17:51:54", "content": "I think the two layer approach is one way of dealing with the class imbalance. Similar to RCNN type approaches where you have an image classifier as a coarse grained filter and another NN to make more fine grain predictions. Here is a good overview of that approach (and iterations on top of that): https://blog.athelas.com/a-brief-history-of-cnns-in-image-segmentation-from-r-cnn-to-mask-r-cnn-34ea83205de4
", "votes": null, "replies": [] }, { "id": 375572, "author_name": "dhacker", "author_url": "", "post_date": "08/25/2018 14:21:27", "content": "Very interesting article! Thanks Paul!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "373479": "I know it's a bit early for that, but wonder what's the best way to combine/ensemble segmentation models?\nLinear combination of predictions, stacked as transfer learning features, stacked predictions into a new model?\nWhat does the literature say?\n\nPlease comment with your ideas, and what works or doesn't. Thank you.", "373652": "In some competitions like the 2018 science bowl winner, they took a simple average of the final layer output between all the ensemble models. You could also try test time augmentation: run each image 4 times, with rotations/flips and take the average. Obviously for the second phase this is going to pay a performance penalty but I'm not sure what the real strategy is. Probably accuracy is most important. Only the top performing teams are eligible to participate in the Speed prize.", "373793": "Cool, thanks!\n\nI've tried a hierarchical approach running first a classification (boat/no-boat) then the segmentation to avoid false positives: \nhttps://www.kaggle.com/hmendonca/classification-and-segmentation\n\nIt works, however, I believe it's probably possible to get the same result with a single model...", "374849": "People could probably get a good result without any classification beforehand, but you would have to tinker with the distributions of positive and negative as well as loss functions. Did you balance the classes for your classifier?", "375133": "That kernel is really just an ensemble of the 2 best public models, but there is some class balance. However, it might not be optimal for the ensemble, as each model was not specifically trained for blending. Any tips?", "375170": "I think the two layer approach is one way of dealing with the class imbalance. Similar to RCNN type approaches where you have an image classifier as a coarse grained filter and another NN to make more fine grain predictions. Here is a good overview of that approach (and iterations on top of that): https://blog.athelas.com/a-brief-history-of-cnns-in-image-segmentation-from-r-cnn-to-mask-r-cnn-34ea83205de4", "375572": "Very interesting article! Thanks Paul!" }, "source": "meta" }