The competition has concluded, but I hope someone can still help me to understand this.
\nI am trying to learn semantic segmentation using this competition. I trained a Unet and visualized ONE training slice and its predicted mask. Visually I could see most of the segmentation seems largely OK (see the image below, left is predicted, right is the ground truth), but the score (computed by calling the score() function provided by the organizer) was only 0.16. Does that mean to get a descent score (e.g. > 0.5) the predicted mask needs to be super accurate?
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16653207%2F5775017609292d33e2d7a873e5e452f7%2FScreen%20Shot%202024-02-07%20at%2012.10.03%20PM.png?generation=1707325965584333&alt=media\")
The competition has concluded, but I hope someone can still help me to understand this.
\nI am trying to learn semantic segmentation using this competition. I trained a Unet and visualized ONE training slice and its predicted mask. Visually I could see most of the segmentation seems largely OK (see the image below, left is predicted, right is the ground truth), but the score (computed by calling the score() function provided by the organizer) was only 0.16. Does that mean to get a descent score (e.g. > 0.5) the predicted mask needs to be super accurate?
\n
The competition metric is calculated over the 3d volume, not on independent 2d slices. Also, you might want to visualize the predictions of more than one slice as there might be a bug in other predictions.
", "rawMarkdown": "The competition metric is calculated over the 3d volume, not on independent 2d slices. Also, you might want to visualize the predictions of more than one slice as there might be a bug in other predictions.", "votes": null }, { "id": "2642102", "postDate": "02/07/2024 22:29:52", "content": "Yes, this segmentation competition is all about minimizing the false positives and false negatives near the boundaries of the vessels. The 3D surface dice score used in the competition measures your precision in finding the boundaries. Often these false positive and false negative blobs are so tiny (1 to 4 pixels in area), I dilated those blobs to visualize them.
\nSlice 1000 from kidney_1_dense Label Vs Prediction. Prediction is color coded as Green for True Positives, Red for false positives and Blue for false negatives. Also all blobs are dilated 5x5 to observe the tiny blobs visually. Most of the false positives and false negatives are 1 pixel area blobs. However, there are also some 1 pixel area true positive blobs in this slice and many more of those in early slices for example 0100. So, cannot blindly remove 1 pixel blobs.
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18199244%2F3d16023f25a7f35a225c15c2d88cc107%2FSegmentedKidney1_1000.png?generation=1707342563232471&alt=media\")
Slice 0100 from kidney_1_dense Label Vs Prediction. 119 pixels in label, 133 pixels found, 93 pixels true positives, 40 pixels false positives, 26 pixels false negatives. Many of the blob sizes are tiny 1 to 4 pixels area.
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F18199244%2F632bff25ba557b49017796618251f092%2FSegmentedKidney1_0100.png?generation=1707342598764919&alt=media\")
For more context please see my solution post, https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/475311
", "rawMarkdown": "Yes, this segmentation competition is all about minimizing the false positives and false negatives near the boundaries of the vessels. The 3D surface dice score used in the competition measures your precision in finding the boundaries. Often these false positive and false negative blobs are so tiny (1 to 4 pixels in area), I dilated those blobs to visualize them.\n\n## Sample segmentation results on slices for illustration\n\nSlice 1000 from kidney_1_dense Label Vs Prediction. Prediction is color coded as Green for True Positives, Red for false positives and Blue for false negatives. Also all blobs are dilated 5x5 to observe the tiny blobs visually. Most of the false positives and false negatives are 1 pixel area blobs. However, there are also some 1 pixel area true positive blobs in this slice and many more of those in early slices for example 0100. So, cannot blindly remove 1 pixel blobs.\n\n\n\nSlice 0100 from kidney_1_dense Label Vs Prediction. 119 pixels in label, 133 pixels found, 93 pixels true positives, 40 pixels false positives, 26 pixels false negatives. Many of the blob sizes are tiny 1 to 4 pixels area.\n\n\n\nFor more context please see my solution post, https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/475311", "votes": null } ], "comments": [ { "id": 2641832, "author_name": "davidfmora", "author_url": "", "post_date": "02/07/2024 17:40:01", "content": "The competition metric is calculated over the 3d volume, not on independent 2d slices. Also, you might want to visualize the predictions of more than one slice as there might be a bug in other predictions.
", "votes": null, "replies": [] }, { "id": 2642102, "author_name": "velangovan", "author_url": "", "post_date": "02/07/2024 22:29:52", "content": "Yes, this segmentation competition is all about minimizing the false positives and false negatives near the boundaries of the vessels. The 3D surface dice score used in the competition measures your precision in finding the boundaries. Often these false positive and false negative blobs are so tiny (1 to 4 pixels in area), I dilated those blobs to visualize them.
\nSlice 1000 from kidney_1_dense Label Vs Prediction. Prediction is color coded as Green for True Positives, Red for false positives and Blue for false negatives. Also all blobs are dilated 5x5 to observe the tiny blobs visually. Most of the false positives and false negatives are 1 pixel area blobs. However, there are also some 1 pixel area true positive blobs in this slice and many more of those in early slices for example 0100. So, cannot blindly remove 1 pixel blobs.
\n
Slice 0100 from kidney_1_dense Label Vs Prediction. 119 pixels in label, 133 pixels found, 93 pixels true positives, 40 pixels false positives, 26 pixels false negatives. Many of the blob sizes are tiny 1 to 4 pixels area.
\n
For more context please see my solution post, https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/475311
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2641804": "The competition has concluded, but I hope someone can still help me to understand this.\nI am trying to learn semantic segmentation using this competition. I trained a Unet and visualized ONE training slice and its predicted mask. Visually I could see most of the segmentation seems largely OK (see the image below, left is predicted, right is the ground truth), but the score (computed by calling the score() function provided by the organizer) was only 0.16. Does that mean to get a descent score (e.g. > 0.5) the predicted mask needs to be super accurate?\n", "2641832": "The competition metric is calculated over the 3d volume, not on independent 2d slices. Also, you might want to visualize the predictions of more than one slice as there might be a bug in other predictions.", "2642102": "Yes, this segmentation competition is all about minimizing the false positives and false negatives near the boundaries of the vessels. The 3D surface dice score used in the competition measures your precision in finding the boundaries. Often these false positive and false negative blobs are so tiny (1 to 4 pixels in area), I dilated those blobs to visualize them.\n\n## Sample segmentation results on slices for illustration\n\nSlice 1000 from kidney_1_dense Label Vs Prediction. Prediction is color coded as Green for True Positives, Red for false positives and Blue for false negatives. Also all blobs are dilated 5x5 to observe the tiny blobs visually. Most of the false positives and false negatives are 1 pixel area blobs. However, there are also some 1 pixel area true positive blobs in this slice and many more of those in early slices for example 0100. So, cannot blindly remove 1 pixel blobs.\n\n\n\nSlice 0100 from kidney_1_dense Label Vs Prediction. 119 pixels in label, 133 pixels found, 93 pixels true positives, 40 pixels false positives, 26 pixels false negatives. Many of the blob sizes are tiny 1 to 4 pixels area.\n\n\n\nFor more context please see my solution post, https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/475311" }, "source": "meta" }