Hi everyone, I'm using nnUNet for segmentation, but keep getting a score of 0 upon submission. I've confirmed that the output dimensions and data formatting are correct. The model performs decently with a DICE of ≈0.8, but I can't figure out why the score is 0 on Kaggle. Has anyone encountered a similar issue or has any suggestions? Thanks!
", "messages": [ { "id": "2602234", "postDate": "01/15/2024 02:18:47", "content": "Hi everyone, I'm using nnUNet for segmentation, but keep getting a score of 0 upon submission. I've confirmed that the output dimensions and data formatting are correct. The model performs decently with a DICE of ≈0.8, but I can't figure out why the score is 0 on Kaggle. Has anyone encountered a similar issue or has any suggestions? Thanks!
", "rawMarkdown": "Hi everyone, I'm using [nnUNet](https://www.kaggle.com/code/zhrthegreat/nnunet/notebook) for segmentation, but keep getting a score of 0 upon submission. I've confirmed that the output dimensions and data formatting are correct. The model performs decently with a DICE of ≈0.8, but I can't figure out why the score is 0 on Kaggle. Has anyone encountered a similar issue or has any suggestions? Thanks!", "votes": null }, { "id": "2602778", "postDate": "01/15/2024 10:36:52", "content": "If you are using nnUnet's default normalization for CT scans, the clipping parameter of their normalization may cause the problem. They determine the clipping range based on the percentiles of your training data. Since the number of kidneys in training data is very few, the training statistics may poorly cover the intensity of other kidneys. You will get zero score if the intensity of other kidneys is completely outside the range of the clipping parameters. That's what happened in my early experiments. You may want to check if you are meeting something similar.
", "rawMarkdown": "If you are using nnUnet's default normalization for CT scans, the clipping parameter of their normalization may cause the problem. They determine the clipping range based on the percentiles of your training data. Since the number of kidneys in training data is very few, the training statistics may poorly cover the intensity of other kidneys. You will get zero score if the intensity of other kidneys is completely outside the range of the clipping parameters. That's what happened in my early experiments. You may want to check if you are meeting something similar.", "votes": null }, { "id": "2602788", "postDate": "01/15/2024 10:41:36", "content": "Thank you for your answer, I also encountered a similar problem.
", "rawMarkdown": "Thank you for your answer, I also encountered a similar problem.", "votes": null }, { "id": "2602867", "postDate": "01/15/2024 11:49:57", "content": "Thanks a lot！
", "rawMarkdown": "Thanks a lot！", "votes": null }, { "id": "2609064", "postDate": "01/19/2024 08:24:02", "content": "Hi @czuryh Thanks, your input is correct. In my case, using zscore as the normalization method in nnUNet fixed the 0 score issue. My baseline 2d model got a surface dice of 0.795 from a single fold. Thanks again!
Hi, would it be possible for you to share how to install nnunet offline on kaggle?
", "rawMarkdown": "Hi, would it be possible for you to share how to install nnunet offline on kaggle?", "votes": null }, { "id": "2611926", "postDate": "01/21/2024 04:39:21", "content": "Perhaps you could refer to my notebook, the link is in the description of the question above.
", "rawMarkdown": "Perhaps you could refer to my notebook, the link is in the description of the question above.", "votes": null } ], "comments": [ { "id": 2602778, "author_name": "czuryh", "author_url": "", "post_date": "01/15/2024 10:36:52", "content": "If you are using nnUnet's default normalization for CT scans, the clipping parameter of their normalization may cause the problem. They determine the clipping range based on the percentiles of your training data. Since the number of kidneys in training data is very few, the training statistics may poorly cover the intensity of other kidneys. You will get zero score if the intensity of other kidneys is completely outside the range of the clipping parameters. That's what happened in my early experiments. You may want to check if you are meeting something similar.
", "votes": null, "replies": [ { "id": 2602788, "author_name": "jisookimjisoo", "author_url": "", "post_date": "01/15/2024 10:41:36", "content": "Thank you for your answer, I also encountered a similar problem.
", "votes": null, "replies": [] }, { "id": 2602867, "author_name": "zhrthegreat", "author_url": "", "post_date": "01/15/2024 11:49:57", "content": "Thanks a lot！
", "votes": null, "replies": [] }, { "id": 2609064, "author_name": "markpeng", "author_url": "", "post_date": "01/19/2024 08:24:02", "content": "Hi @czuryh Thanks, your input is correct. In my case, using zscore as the normalization method in nnUNet fixed the 0 score issue. My baseline 2d model got a surface dice of 0.795 from a single fold. Thanks again!
Hi, would it be possible for you to share how to install nnunet offline on kaggle?
", "votes": null, "replies": [ { "id": 2611926, "author_name": "zhrthegreat", "author_url": "", "post_date": "01/21/2024 04:39:21", "content": "Perhaps you could refer to my notebook, the link is in the description of the question above.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2602234": "Hi everyone, I'm using [nnUNet](https://www.kaggle.com/code/zhrthegreat/nnunet/notebook) for segmentation, but keep getting a score of 0 upon submission. I've confirmed that the output dimensions and data formatting are correct. The model performs decently with a DICE of ≈0.8, but I can't figure out why the score is 0 on Kaggle. Has anyone encountered a similar issue or has any suggestions? Thanks!", "2602778": "If you are using nnUnet's default normalization for CT scans, the clipping parameter of their normalization may cause the problem. They determine the clipping range based on the percentiles of your training data. Since the number of kidneys in training data is very few, the training statistics may poorly cover the intensity of other kidneys. You will get zero score if the intensity of other kidneys is completely outside the range of the clipping parameters. That's what happened in my early experiments. You may want to check if you are meeting something similar.", "2602788": "Thank you for your answer, I also encountered a similar problem.", "2602867": "Thanks a lot！", "2609064": "Hi @czuryh Thanks, your input is correct. In my case, using `zscore` as the normalization method in nnUNet fixed the 0 score issue. My baseline 2d model got a surface dice of 0.795 from a single fold. Thanks again!", "2611869": "Hi, would it be possible for you to share how to install nnunet offline on kaggle?", "2611926": "Perhaps you could refer to my notebook, the link is in the description of the question above." }, "source": "meta" }