I've seen other people ask this but I didn't understand any of the answers. I'm very new to machine learning. I trained an nn-UNet model without the --npz flag, so it didn't save any softmax outputs, I think. My inference code can output predicted masks, which are binary. I don't think the nn-UNet outputs any sort of confidence in the predictions, but I'm not sure.
\nIn case there's no way for me to get a confidence from the model, can I just put some random number as the confidence? Or is it actually important for the evaluation?
", "messages": [ { "id": "2322209", "postDate": "06/29/2023 06:32:39", "content": "I've seen other people ask this but I didn't understand any of the answers. I'm very new to machine learning. I trained an nn-UNet model without the --npz flag, so it didn't save any softmax outputs, I think. My inference code can output predicted masks, which are binary. I don't think the nn-UNet outputs any sort of confidence in the predictions, but I'm not sure.
\nIn case there's no way for me to get a confidence from the model, can I just put some random number as the confidence? Or is it actually important for the evaluation?
", "rawMarkdown": "I've seen other people ask this but I didn't understand any of the answers. I'm very new to machine learning. I trained an nn-UNet model without the --npz flag, so it didn't save any softmax outputs, I think. My inference code can output predicted masks, which are binary. I don't think the nn-UNet outputs any sort of confidence in the predictions, but I'm not sure.\n\nIn case there's no way for me to get a confidence from the model, can I just put some random number as the confidence? Or is it actually important for the evaluation?", "votes": null }, { "id": "2322223", "postDate": "06/29/2023 06:44:58", "content": "Oh I managed to get the softmax output, actually, by using the --save_probabilities option on the inference command. Do I now just take the average softmax output on ONLY the POSITIVE predicted regions to get the confidence?
", "rawMarkdown": "Oh I managed to get the softmax output, actually, by using the --save_probabilities option on the inference command. Do I now just take the average softmax output on ONLY the POSITIVE predicted regions to get the confidence?", "votes": null }, { "id": "2322699", "postDate": "06/29/2023 12:48:02", "content": "I suppose you got the probabilities for each segmented instance..?
", "rawMarkdown": "I suppose you got the probabilities for each segmented instance..?", "votes": null }, { "id": "2323910", "postDate": "06/30/2023 08:41:37", "content": "I got the probabilities for every pixel, segmented and non-segmented. My current confidence calculation is taking the pixels with probability >0.5 as True, summing the probabilities for all True pixels, then dividing by the number of True pixels, for each tile. I don't do anything \"for each segmented instance\", am I supposed to? In the submission CSV, the RLE string is meant to be for the whole tile, right? So one row per tile, not one row per segmented instance?
", "rawMarkdown": "I got the probabilities for every pixel, segmented and non-segmented. My current confidence calculation is taking the pixels with probability >0.5 as True, summing the probabilities for all True pixels, then dividing by the number of True pixels, for each tile. I don't do anything \"for each segmented instance\", am I supposed to? In the submission CSV, the RLE string is meant to be for the whole tile, right? So one row per tile, not one row per segmented instance?", "votes": null }, { "id": "2323919", "postDate": "06/30/2023 08:47:25", "content": "Nevermind, I just re-read the evaluation rules 😑Sorry about that. Okay, so for each segmentation INSTANCE, I get the confidence by taking the average probability for that instance?
", "rawMarkdown": "Nevermind, I just re-read the evaluation rules 😑Sorry about that. Okay, so for each segmentation INSTANCE, I get the confidence by taking the average probability for that instance?", "votes": null }, { "id": "2323924", "postDate": "06/30/2023 08:52:45", "content": "na it's alright. made me reconsider my own code too. thanks.
", "rawMarkdown": "na it's alright. made me reconsider my own code too. thanks.", "votes": null }, { "id": "2324119", "postDate": "06/30/2023 11:58:21", "content": "For the standard instance segmentation problem, if you want to apply a semantic segmentation model like UNet to this competition, you would need to define the confidence for each mask instance on a given mask yourself
\nsuch as
https://www.kaggle.com/code/kongzhangtang/hubmap-unetplusplus-inference
\nMy suggestion is to consider changing the competition or using a commonly used instance segmentation model like Mask R-CNN, as it would be easier. If you want to know more about the confidence effect ,check the following link
\nhttps://storage.googleapis.com/openimages/web/evaluation.html#instance_segmentation_eval
", "rawMarkdown": "For the standard instance segmentation problem, if you want to apply a semantic segmentation model like UNet to this competition, you would need to define the confidence for each mask instance on a given mask yourself\nsuch as\n\nhttps://www.kaggle.com/code/kongzhangtang/hubmap-unetplusplus-inference\n\n\nMy suggestion is to consider changing the competition or using a commonly used instance segmentation model like Mask R-CNN, as it would be easier. If you want to know more about the confidence effect ,check the following link\n\nhttps://storage.googleapis.com/openimages/web/evaluation.html#instance_segmentation_eval", "votes": null } ], "comments": [ { "id": 2322223, "author_name": "solerikaboman", "author_url": "", "post_date": "06/29/2023 06:44:58", "content": "Oh I managed to get the softmax output, actually, by using the --save_probabilities option on the inference command. Do I now just take the average softmax output on ONLY the POSITIVE predicted regions to get the confidence?
", "votes": null, "replies": [ { "id": 2322699, "author_name": "bhavesjain", "author_url": "", "post_date": "06/29/2023 12:48:02", "content": "I suppose you got the probabilities for each segmented instance..?
", "votes": null, "replies": [ { "id": 2323910, "author_name": "solerikaboman", "author_url": "", "post_date": "06/30/2023 08:41:37", "content": "I got the probabilities for every pixel, segmented and non-segmented. My current confidence calculation is taking the pixels with probability >0.5 as True, summing the probabilities for all True pixels, then dividing by the number of True pixels, for each tile. I don't do anything \"for each segmented instance\", am I supposed to? In the submission CSV, the RLE string is meant to be for the whole tile, right? So one row per tile, not one row per segmented instance?
", "votes": null, "replies": [ { "id": 2323919, "author_name": "solerikaboman", "author_url": "", "post_date": "06/30/2023 08:47:25", "content": "Nevermind, I just re-read the evaluation rules 😑Sorry about that. Okay, so for each segmentation INSTANCE, I get the confidence by taking the average probability for that instance?
", "votes": null, "replies": [ { "id": 2323924, "author_name": "bhavesjain", "author_url": "", "post_date": "06/30/2023 08:52:45", "content": "na it's alright. made me reconsider my own code too. thanks.
", "votes": null, "replies": [] } ] } ] } ] } ] }, { "id": 2324119, "author_name": "atom1231", "author_url": "", "post_date": "06/30/2023 11:58:21", "content": "For the standard instance segmentation problem, if you want to apply a semantic segmentation model like UNet to this competition, you would need to define the confidence for each mask instance on a given mask yourself
\nsuch as
https://www.kaggle.com/code/kongzhangtang/hubmap-unetplusplus-inference
\nMy suggestion is to consider changing the competition or using a commonly used instance segmentation model like Mask R-CNN, as it would be easier. If you want to know more about the confidence effect ,check the following link
\nhttps://storage.googleapis.com/openimages/web/evaluation.html#instance_segmentation_eval
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2322209": "I've seen other people ask this but I didn't understand any of the answers. I'm very new to machine learning. I trained an nn-UNet model without the --npz flag, so it didn't save any softmax outputs, I think. My inference code can output predicted masks, which are binary. I don't think the nn-UNet outputs any sort of confidence in the predictions, but I'm not sure.\n\nIn case there's no way for me to get a confidence from the model, can I just put some random number as the confidence? Or is it actually important for the evaluation?", "2322223": "Oh I managed to get the softmax output, actually, by using the --save_probabilities option on the inference command. Do I now just take the average softmax output on ONLY the POSITIVE predicted regions to get the confidence?", "2322699": "I suppose you got the probabilities for each segmented instance..?", "2323910": "I got the probabilities for every pixel, segmented and non-segmented. My current confidence calculation is taking the pixels with probability >0.5 as True, summing the probabilities for all True pixels, then dividing by the number of True pixels, for each tile. I don't do anything \"for each segmented instance\", am I supposed to? In the submission CSV, the RLE string is meant to be for the whole tile, right? So one row per tile, not one row per segmented instance?", "2323919": "Nevermind, I just re-read the evaluation rules 😑Sorry about that. Okay, so for each segmentation INSTANCE, I get the confidence by taking the average probability for that instance?", "2323924": "na it's alright. made me reconsider my own code too. thanks.", "2324119": "For the standard instance segmentation problem, if you want to apply a semantic segmentation model like UNet to this competition, you would need to define the confidence for each mask instance on a given mask yourself\nsuch as\n\nhttps://www.kaggle.com/code/kongzhangtang/hubmap-unetplusplus-inference\n\n\nMy suggestion is to consider changing the competition or using a commonly used instance segmentation model like Mask R-CNN, as it would be easier. If you want to know more about the confidence effect ,check the following link\n\nhttps://storage.googleapis.com/openimages/web/evaluation.html#instance_segmentation_eval" }, "source": "meta" }