{ "id": 456288, "title": "One possible approach to address a score of 0", "url": "/competitions/blood-vessel-segmentation/discussion/456288", "author_name": "", "post_date": "2023-11-19T06:04:56.250980Z", "votes": 17, "comment_count": 5, "views": 0, "content": "

Please refer to this link for the problem description.

\n

After several days of dedicated effort, I've finally attained a Public Score that seems reasonable. The key to success for me was adjusting the threshold for the output results. Initially, I had set the threshold quite low, which appeared normal during my local testing. However, the resulting Leaderboard score remained stagnant at 0. Now, having increased the threshold, I'm achieving more sensible scores.

\n

I suspect this could be tied to the competition's metric, so it's crucial to exercise caution when tweaking the threshold. The competition employs the Surface Dice Metric, and I've found a comprehensive explanation of it in the relevant literature. Deep learning to achieve clinically applicable\nsegmentation of head and neck anatomy for\nradiotherapy.
\n\"\"

\n

If you find this helpful, please consider giving it an upvote.

", "messages": [ { "id": "2530411", "postDate": "11/19/2023 06:04:56", "content": "

Please refer to this link for the problem description.

\n

After several days of dedicated effort, I've finally attained a Public Score that seems reasonable. The key to success for me was adjusting the threshold for the output results. Initially, I had set the threshold quite low, which appeared normal during my local testing. However, the resulting Leaderboard score remained stagnant at 0. Now, having increased the threshold, I'm achieving more sensible scores.

\n

I suspect this could be tied to the competition's metric, so it's crucial to exercise caution when tweaking the threshold. The competition employs the Surface Dice Metric, and I've found a comprehensive explanation of it in the relevant literature. Deep learning to achieve clinically applicable\nsegmentation of head and neck anatomy for\nradiotherapy.
\n\"\"

\n

If you find this helpful, please consider giving it an upvote.

", "rawMarkdown": "Please refer to this [link](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/455787) for the problem description. \n\nAfter several days of dedicated effort, I've finally attained a Public Score that seems reasonable. The key to success for me was adjusting the threshold for the output results. Initially, I had set the threshold quite low, which appeared normal during my local testing. However, the resulting Leaderboard score remained stagnant at 0. Now, having increased the threshold, I'm achieving more sensible scores.\n\nI suspect this could be tied to the competition's metric, so it's crucial to exercise caution when tweaking the threshold. The competition employs the Surface Dice Metric, and I've found a comprehensive explanation of it in the relevant literature. [Deep learning to achieve clinically applicable\nsegmentation of head and neck anatomy for\nradiotherapy](https://arxiv.org/pdf/1809.04430.pdf).\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1133510%2Fe7efee6af56c8f19efd130412a23eafc%2FDeepinScreenshot_select-area_20231119135354.png?generation=1700373573072070&alt=media)\n\n\nIf you find this helpful, please consider giving it an upvote.", "votes": null }, { "id": "2531275", "postDate": "11/20/2023 04:48:22", "content": "

Given that this metric is sensitive to the predictions, might there be issues with the interpolation of masks during the resizing process, both from a smaller dimension (like 256x256) to a larger one, and the reverse, back to the image's original size? I suspect that the majority of errors in submissions are due to most models being 2D and built on resampled images of a lower dimension.

\n

Two potential solutions I'm considering are:

\n
    \n
  1. Maintaining the original shape of the image and employing a 3D patch-based model.
    2. \n
  2. Resampling only a select portion of the images to match the model's input size.
    3. \n
\n

Any thoughts or suggestions?

", "rawMarkdown": "Given that this metric is sensitive to the predictions, might there be issues with the interpolation of masks during the resizing process, both from a smaller dimension (like 256x256) to a larger one, and the reverse, back to the image's original size? I suspect that the majority of errors in submissions are due to most models being 2D and built on resampled images of a lower dimension. \n\nTwo potential solutions I'm considering are:\n1. Maintaining the original shape of the image and employing a 3D patch-based model.\n2. Resampling only a select portion of the images to match the model's input size.\n\nAny thoughts or suggestions?", "votes": null }, { "id": "2531346", "postDate": "11/20/2023 06:20:40", "content": "

I think you are certainly onto something! I've implemented the competitions scoring metric into my own notebook and the score goes down considerably when I scale the mask back up to the original image size.

", "rawMarkdown": "I think you are certainly onto something! I've implemented the competitions scoring metric into my own notebook and the score goes down considerably when I scale the mask back up to the original image size.", "votes": null }, { "id": "2532255", "postDate": "11/20/2023 21:41:56", "content": "

Finally got a rank in the competition but my score is 0. I adjusted the threshold to 0.999 i.e. I used sigmoid as my final activation function. Anything above this is considered as a 1 and rest as 0. I agree to the fact that we should lookout for the threshold.

", "rawMarkdown": "Finally got a rank in the competition but my score is 0. I adjusted the threshold to 0.999 i.e. I used sigmoid as my final activation function. Anything above this is considered as a 1 and rest as 0. I agree to the fact that we should lookout for the threshold.", "votes": null }, { "id": "2532362", "postDate": "11/21/2023 01:50:13", "content": "
\n

Finally got a rank in the competition but my score is 0. I adjusted the threshold to 0.999 i.e. I used sigmoid as my final activation function. Anything above this is considered as a 1 and rest as 0. I agree to the fact that we should lookout for the threshold.

\n
\n

Thank you @dhinkris , this work for me

", "rawMarkdown": "> Finally got a rank in the competition but my score is 0. I adjusted the threshold to 0.999 i.e. I used sigmoid as my final activation function. Anything above this is considered as a 1 and rest as 0. I agree to the fact that we should lookout for the threshold.\n\n**Thank you @dhinkris , this work for me**", "votes": null }, { "id": "2536199", "postDate": "11/24/2023 02:48:42", "content": "

could you share your code? I forked public baseline and scoring errors many times.

", "rawMarkdown": "could you share your code? I forked public baseline and scoring errors many times.", "votes": null } ], "comments": [ { "id": 2531275, "author_name": "dhinkris", "author_url": "", "post_date": "11/20/2023 04:48:22", "content": "

Given that this metric is sensitive to the predictions, might there be issues with the interpolation of masks during the resizing process, both from a smaller dimension (like 256x256) to a larger one, and the reverse, back to the image's original size? I suspect that the majority of errors in submissions are due to most models being 2D and built on resampled images of a lower dimension.

\n

Two potential solutions I'm considering are:

\n
    \n
  1. Maintaining the original shape of the image and employing a 3D patch-based model.
    2. \n
  2. Resampling only a select portion of the images to match the model's input size.
    3. \n
\n

Any thoughts or suggestions?

", "votes": null, "replies": [ { "id": 2531346, "author_name": "miltiadesgeneral", "author_url": "", "post_date": "11/20/2023 06:20:40", "content": "

I think you are certainly onto something! I've implemented the competitions scoring metric into my own notebook and the score goes down considerably when I scale the mask back up to the original image size.

", "votes": null, "replies": [] } ] }, { "id": 2532255, "author_name": "dhinkris", "author_url": "", "post_date": "11/20/2023 21:41:56", "content": "

Finally got a rank in the competition but my score is 0. I adjusted the threshold to 0.999 i.e. I used sigmoid as my final activation function. Anything above this is considered as a 1 and rest as 0. I agree to the fact that we should lookout for the threshold.

", "votes": null, "replies": [ { "id": 2532362, "author_name": "dieptam", "author_url": "", "post_date": "11/21/2023 01:50:13", "content": "
\n

Finally got a rank in the competition but my score is 0. I adjusted the threshold to 0.999 i.e. I used sigmoid as my final activation function. Anything above this is considered as a 1 and rest as 0. I agree to the fact that we should lookout for the threshold.

\n
\n

Thank you @dhinkris , this work for me

", "votes": null, "replies": [] } ] }, { "id": 2536199, "author_name": "dragonzhang", "author_url": "", "post_date": "11/24/2023 02:48:42", "content": "

could you share your code? I forked public baseline and scoring errors many times.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2530411": "Please refer to this [link](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/455787) for the problem description. \n\nAfter several days of dedicated effort, I've finally attained a Public Score that seems reasonable. The key to success for me was adjusting the threshold for the output results. Initially, I had set the threshold quite low, which appeared normal during my local testing. However, the resulting Leaderboard score remained stagnant at 0. Now, having increased the threshold, I'm achieving more sensible scores.\n\nI suspect this could be tied to the competition's metric, so it's crucial to exercise caution when tweaking the threshold. The competition employs the Surface Dice Metric, and I've found a comprehensive explanation of it in the relevant literature. [Deep learning to achieve clinically applicable\nsegmentation of head and neck anatomy for\nradiotherapy](https://arxiv.org/pdf/1809.04430.pdf).\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1133510%2Fe7efee6af56c8f19efd130412a23eafc%2FDeepinScreenshot_select-area_20231119135354.png?generation=1700373573072070&alt=media)\n\n\nIf you find this helpful, please consider giving it an upvote.", "2531275": "Given that this metric is sensitive to the predictions, might there be issues with the interpolation of masks during the resizing process, both from a smaller dimension (like 256x256) to a larger one, and the reverse, back to the image's original size? I suspect that the majority of errors in submissions are due to most models being 2D and built on resampled images of a lower dimension. \n\nTwo potential solutions I'm considering are:\n1. Maintaining the original shape of the image and employing a 3D patch-based model.\n2. Resampling only a select portion of the images to match the model's input size.\n\nAny thoughts or suggestions?", "2531346": "I think you are certainly onto something! I've implemented the competitions scoring metric into my own notebook and the score goes down considerably when I scale the mask back up to the original image size.", "2532255": "Finally got a rank in the competition but my score is 0. I adjusted the threshold to 0.999 i.e. I used sigmoid as my final activation function. Anything above this is considered as a 1 and rest as 0. I agree to the fact that we should lookout for the threshold.", "2532362": "> Finally got a rank in the competition but my score is 0. I adjusted the threshold to 0.999 i.e. I used sigmoid as my final activation function. Anything above this is considered as a 1 and rest as 0. I agree to the fact that we should lookout for the threshold.\n\n**Thank you @dhinkris , this work for me**", "2536199": "could you share your code? I forked public baseline and scoring errors many times." }, "source": "meta" }