For segmentation we have:
\n[Input] -> [Model] -> [Mask]
\nAfter this, (for this competition) we get a binary mask or at least we are supposed to. The mask that I get, resembles a binary mask but has some values other than 0s or 1s like 0.0001, or 0.789, etc. To counter this I thought of using thresholding but I think that it's degrading my performance. Please let me know if there are any alternatives to it.
", "messages": [ { "id": "1863811", "postDate": "07/20/2022 14:19:37", "content": "For segmentation we have:
\n[Input] -> [Model] -> [Mask]
\nAfter this, (for this competition) we get a binary mask or at least we are supposed to. The mask that I get, resembles a binary mask but has some values other than 0s or 1s like 0.0001, or 0.789, etc. To counter this I thought of using thresholding but I think that it's degrading my performance. Please let me know if there are any alternatives to it.
", "rawMarkdown": "For segmentation we have:\n\n[Input] -> [Model] -> [Mask]\n\nAfter this, (for this competition) we get a binary mask or at least we are supposed to. The mask that I get, resembles a binary mask but has some values other than 0s or 1s like 0.0001, or 0.789, etc. To counter this I thought of using thresholding but I think that it's degrading my performance. Please let me know if there are any alternatives to it.", "votes": null }, { "id": "1864089", "postDate": "07/20/2022 18:07:05", "content": "You are probably checking your performance with a loss function like BCE, which is sensitive with high confidence predictions, the competition metric is dice score, normally more than 99% of the kaggle uses threshold, if you think that using threshold is degrading your performance, you can tune the threshold to make it optimal, 0.5 is not always the best.
", "rawMarkdown": "You are probably checking your performance with a loss function like BCE, which is sensitive with high confidence predictions, the competition metric is dice score, normally more than 99% of the kaggle uses threshold, if you think that using threshold is degrading your performance, you can tune the threshold to make it optimal, 0.5 is not always the best.", "votes": null }, { "id": "1864418", "postDate": "07/21/2022 03:58:16", "content": "Generally I don't want to mess with threshold and keep it simple (> 0.5). But you can try playing with it to see what kind of CV and LB it produces
", "rawMarkdown": "Generally I don't want to mess with threshold and keep it simple (> 0.5). But you can try playing with it to see what kind of CV and LB it produces", "votes": null }, { "id": "1864439", "postDate": "07/21/2022 04:11:37", "content": "Hello Harshit! Thank you for your response. I am using Dice Loss actually. My validation IoU was 0.90+. I didn't CV. But my score during test time is 0.20 only XD
\nI get it, test set is a whole different beast but it kinda made me suspicious of threshold. Thanks for your response again :D
Hello Quan!
\nMy LB is only 0.20 :(
\nHowever during training, my IoU score on the validation set was 0.90+. I have created the folds ut didn't use CV yet. What was your IoU during training?
How did your IoU get so high? Sorry but I think you may compute IoU incorrectly
", "rawMarkdown": "How did your IoU get so high? Sorry but I think you may compute IoU incorrectly", "votes": null }, { "id": "1864460", "postDate": "07/21/2022 04:24:53", "content": "I was using IoU metric given by Qubvel's \"Segmentation Models PyTorch\" package. But given the test score, I'm inclined to believe what you're saying is right
\nEpochs: 100/100, Training — Loss: 0.141, Precision: 0.847, Recall: 0.754, IoU: 0.874, Validation — Loss: 0.075, Precision: 0.919, Recall: 0.862, IoU: 0.934
", "rawMarkdown": "I was using IoU metric given by Qubvel's \"Segmentation Models PyTorch\" package. But given the test score, I'm inclined to believe what you're saying is right\n\nEpochs: 100/100, Training — Loss: 0.141, Precision: 0.847, Recall: 0.754, IoU: 0.874, Validation — Loss: 0.075, Precision: 0.919, Recall: 0.862, IoU: 0.934", "votes": null }, { "id": "1917688", "postDate": "08/29/2022 01:34:55", "content": "Hello! I have the same problem and I'm also puzzled about this. If you have any solution, please give me a suggestion. Thank you!
", "rawMarkdown": "Hello! I have the same problem and I'm also puzzled about this. If you have any solution, please give me a suggestion. Thank you!", "votes": null }, { "id": "1922082", "postDate": "09/01/2022 08:57:32", "content": "I sort of transposed my mask and it gave me slightly better results.
", "rawMarkdown": "I sort of transposed my mask and it gave me slightly better results.", "votes": null } ], "comments": [ { "id": 1864089, "author_name": "harshitsheoran", "author_url": "", "post_date": "07/20/2022 18:07:05", "content": "You are probably checking your performance with a loss function like BCE, which is sensitive with high confidence predictions, the competition metric is dice score, normally more than 99% of the kaggle uses threshold, if you think that using threshold is degrading your performance, you can tune the threshold to make it optimal, 0.5 is not always the best.
", "votes": null, "replies": [ { "id": 1864439, "author_name": "eddwait", "author_url": "", "post_date": "07/21/2022 04:11:37", "content": "Hello Harshit! Thank you for your response. I am using Dice Loss actually. My validation IoU was 0.90+. I didn't CV. But my score during test time is 0.20 only XD
\nI get it, test set is a whole different beast but it kinda made me suspicious of threshold. Thanks for your response again :D
Generally I don't want to mess with threshold and keep it simple (> 0.5). But you can try playing with it to see what kind of CV and LB it produces
", "votes": null, "replies": [ { "id": 1864441, "author_name": "eddwait", "author_url": "", "post_date": "07/21/2022 04:13:28", "content": "Hello Quan!
\nMy LB is only 0.20 :(
\nHowever during training, my IoU score on the validation set was 0.90+. I have created the folds ut didn't use CV yet. What was your IoU during training?
How did your IoU get so high? Sorry but I think you may compute IoU incorrectly
", "votes": null, "replies": [] }, { "id": 1864460, "author_name": "eddwait", "author_url": "", "post_date": "07/21/2022 04:24:53", "content": "I was using IoU metric given by Qubvel's \"Segmentation Models PyTorch\" package. But given the test score, I'm inclined to believe what you're saying is right
\nEpochs: 100/100, Training — Loss: 0.141, Precision: 0.847, Recall: 0.754, IoU: 0.874, Validation — Loss: 0.075, Precision: 0.919, Recall: 0.862, IoU: 0.934
", "votes": null, "replies": [] } ] }, { "id": 1917688, "author_name": "lyyyfu", "author_url": "", "post_date": "08/29/2022 01:34:55", "content": "Hello! I have the same problem and I'm also puzzled about this. If you have any solution, please give me a suggestion. Thank you!
", "votes": null, "replies": [ { "id": 1922082, "author_name": "eddwait", "author_url": "", "post_date": "09/01/2022 08:57:32", "content": "I sort of transposed my mask and it gave me slightly better results.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1863811": "For segmentation we have:\n\n[Input] -> [Model] -> [Mask]\n\nAfter this, (for this competition) we get a binary mask or at least we are supposed to. The mask that I get, resembles a binary mask but has some values other than 0s or 1s like 0.0001, or 0.789, etc. To counter this I thought of using thresholding but I think that it's degrading my performance. Please let me know if there are any alternatives to it.", "1864089": "You are probably checking your performance with a loss function like BCE, which is sensitive with high confidence predictions, the competition metric is dice score, normally more than 99% of the kaggle uses threshold, if you think that using threshold is degrading your performance, you can tune the threshold to make it optimal, 0.5 is not always the best.", "1864418": "Generally I don't want to mess with threshold and keep it simple (> 0.5). But you can try playing with it to see what kind of CV and LB it produces", "1864439": "Hello Harshit! Thank you for your response. I am using Dice Loss actually. My validation IoU was 0.90+. I didn't CV. But my score during test time is 0.20 only XD\nI get it, test set is a whole different beast but it kinda made me suspicious of threshold. Thanks for your response again :D", "1864441": "Hello Quan!\nMy LB is only 0.20 :(\nHowever during training, my IoU score on the validation set was 0.90+. I have created the folds ut didn't use CV yet. What was your IoU during training?", "1864449": "How did your IoU get so high? Sorry but I think you may compute IoU incorrectly", "1864460": "I was using IoU metric given by Qubvel's \"Segmentation Models PyTorch\" package. But given the test score, I'm inclined to believe what you're saying is right\n\nEpochs: 100/100, Training — Loss: 0.141, Precision: 0.847, Recall: 0.754, IoU: 0.874, Validation — Loss: 0.075, Precision: 0.919, Recall: 0.862, IoU: 0.934", "1917688": "Hello! I have the same problem and I'm also puzzled about this. If you have any solution, please give me a suggestion. Thank you!", "1922082": "I sort of transposed my mask and it gave me slightly better results." }, "source": "meta" }