Hi all,
\nI'm just joining, what should we have in submission.csv in the following cases:
PredictionString with empty string?Thanks.
", "messages": [ { "id": 1212930, "postDate": "2021-02-21T18:21:10.783Z", "content": "Hi all,
\nI'm just joining, what should we have in submission.csv in the following cases:
PredictionString with empty string?Thanks.
", "rawMarkdown": "Hi all,\n\nI'm just joining, what should we have in `submission.csv` in the following cases:\n\n- HPA segmentation detects zero cells (and zero nuclei). I assume we fill in `PredictionString` with empty string?\n- HPA segmentation detects cells but our model detects nothing. I assume then we have to fill in with class 18 (Negative) for all cells.\n\nThanks.", "votes": 10 }, { "id": 1217648, "postDate": "2021-02-25T08:10:19.997Z", "content": "The next question is about border cells. Is it recommended or not to remove them? It does not seem to affect mAP score.
\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/221099
\nWhat about if annotator:
I've submitted with these 2 assumptions and I did not get any \"Submission scoring error\". But I'm still wondering if class 18 should be set (or not) on cells with nothing detected by model. It depends how the ground truth is built. What does the annotator/expert do with cells without 0 to 17 labels. Is it 18 by default? If so all HPA segmented cells must have a label.
There are some info in this thread: https://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/215141
\nbut not on class 18.
@lnhtrang or @emmalumpan Could you shed some light on this?
\nThanks a lot.
", "rawMarkdown": "I've submitted with these 2 assumptions and I did not get any \"`Submission scoring error`\". But I'm still wondering if class 18 should be set (or not) on cells with nothing detected by model. It depends how the ground truth is built. What does the annotator/expert do with cells without 0 to 17 labels. Is it 18 by default? If so all HPA segmented cells must have a label.\n\nThere are some info in this thread: https://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/215141\nbut not on class 18.\n\n@lnhtrang or @emmalumpan Could you shed some light on this?\n\nThanks a lot.", "replies": [ { "id": 1214398, "postDate": "2021-02-22T20:13:18.267Z", "content": "I think label 18 is treated the same way as any other label. If your model has found a cell and it predicts with some confidence that the cell doesn't belong to any of the usual labels 0,…17 then it could just predict label 18 with that confidence score. And it can still predict any of the other labels 0, ..,17 for the same cell with their confidence scores.
\nThe segmentation models sometimes make mistakes. In that case the human annotator makes a correction. There are also some cells that are at the boundary of the image that have no labels and are not used for scoring. So it would be OK not giving any prediction for a cell that the segmentation tool has found. That would be equivalent to just saying that your model thinks that there is no significant cell there.
", "rawMarkdown": "I think label 18 is treated the same way as any other label. If your model has found a cell and it predicts with some confidence that the cell doesn't belong to any of the usual labels 0,...17 then it could just predict label 18 with that confidence score. And it can still predict any of the other labels 0, ..,17 for the same cell with their confidence scores.\n\nThe segmentation models sometimes make mistakes. In that case the human annotator makes a correction. There are also some cells that are at the boundary of the image that have no labels and are not used for scoring. So it would be OK not giving any prediction for a cell that the segmentation tool has found. That would be equivalent to just saying that your model thinks that there is no significant cell there. \n\n\n", "votes": 3, "isDeleted": true }, { "id": 1215069, "postDate": "2021-02-23T10:17:47.123Z", "content": "Thanks, my concern is related to data description:
\n\n\nAs the training labels are a collective label for all the cells in an image, it means that each labeled pattern can be seen in the image but not necessarily that each cell within the image expresses the pattern. This imprecise labeling is what we refer to as weak.
\n
I understand that if an image is labeled with, for example, class 16 only, then it does not mean that all cells have class 16 pattern. Some cells have no pattern and are not labeled with class 18. So image-level label is not [16, 18] but 16 only.
\nIs it the same for ground truth? I understand the border cells could be ignored but the question is for non-border cells.
\n@lnhtrang or @emmalumpan Could you clarify? In other word, shoud we provide a label for each segmented cell? If a model detects class 16 pattern for 5 cells on an image with 8 non-border segmented cells. What should we set for the 3 other cells? Nothing or class 18?
\nUpdate: It looks the answer is yes, all segmented cells (except border cells or cells with less than half-size) must be labeled in submission.csv. Cells with no pattern detected must be labeled with class 18 (Negative). A cell labeled with class 18 cannot have additional labels:
\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/216924#1187646
If someone can confirm all the above, I will be happy :-)
", "rawMarkdown": "Thanks, my concern is related to data description:\n> As the training labels are a **collective label for all the cells in an image**, it means that each labeled pattern can be seen in the image but **not necessarily that each cell within the image expresses the pattern**. This imprecise labeling is what we refer to as weak.\n\nI understand that if an image is labeled with, for example, class 16 only, then it does not mean that all cells have class 16 pattern. Some cells have no pattern and are not labeled with class 18. So image-level label is not [16, 18] but 16 only.\n\nIs it the same for ground truth? I understand the border cells could be ignored but the question is for non-border cells.\n\n @lnhtrang or @emmalumpan Could you clarify? In other word, shoud we provide a label for each segmented cell? If a model detects class 16 pattern for 5 cells on an image with 8 non-border segmented cells. What should we set for the 3 other cells? Nothing or class 18?\n\n**Update**: It looks the answer is yes, all segmented cells (except border cells or cells with less than half-size) must be labeled in `submission.csv`. Cells with no pattern detected must be labeled with class 18 (Negative). A cell labeled with class 18 cannot have additional labels:\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/216924#1187646 \n\nIf someone can confirm all the above, I will be happy :-)", "votes": 1 }, { "id": 1215527, "postDate": "2021-02-23T17:40:38.943Z", "content": "I've submitted the same predictions twice: one with class 18 for cells without detected pattern and another with nothing for such cells. Both worked but the second one was lower. So it confirms that class 18 have to be used in that case.
It would be interesting to see what happens if you submit class 0 for these cells (the most frequent class). Would that increase or lower your score?
The next question is about border cells. Is it recommended or not to remove them? It does not seem to affect mAP score.
\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/221099
\nWhat about if annotator:
I've submitted with these 2 assumptions and I did not get any \"Submission scoring error\". But I'm still wondering if class 18 should be set (or not) on cells with nothing detected by model. It depends how the ground truth is built. What does the annotator/expert do with cells without 0 to 17 labels. Is it 18 by default? If so all HPA segmented cells must have a label.
There are some info in this thread: https://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/215141
\nbut not on class 18.
@lnhtrang or @emmalumpan Could you shed some light on this?
\nThanks a lot.
", "votes": 0, "replies": [ { "id": 1214398, "author_name": "", "author_url": "", "post_date": "2021-02-22T20:13:18.267000", "content": "I think label 18 is treated the same way as any other label. If your model has found a cell and it predicts with some confidence that the cell doesn't belong to any of the usual labels 0,…17 then it could just predict label 18 with that confidence score. And it can still predict any of the other labels 0, ..,17 for the same cell with their confidence scores.
\nThe segmentation models sometimes make mistakes. In that case the human annotator makes a correction. There are also some cells that are at the boundary of the image that have no labels and are not used for scoring. So it would be OK not giving any prediction for a cell that the segmentation tool has found. That would be equivalent to just saying that your model thinks that there is no significant cell there.
", "votes": 3, "replies": [] }, { "id": 1215069, "author_name": "MPWARE", "author_url": "", "post_date": "2021-02-23T10:17:47.123000", "content": "Thanks, my concern is related to data description:
\n\n\nAs the training labels are a collective label for all the cells in an image, it means that each labeled pattern can be seen in the image but not necessarily that each cell within the image expresses the pattern. This imprecise labeling is what we refer to as weak.
\n
I understand that if an image is labeled with, for example, class 16 only, then it does not mean that all cells have class 16 pattern. Some cells have no pattern and are not labeled with class 18. So image-level label is not [16, 18] but 16 only.
\nIs it the same for ground truth? I understand the border cells could be ignored but the question is for non-border cells.
\n@lnhtrang or @emmalumpan Could you clarify? In other word, shoud we provide a label for each segmented cell? If a model detects class 16 pattern for 5 cells on an image with 8 non-border segmented cells. What should we set for the 3 other cells? Nothing or class 18?
\nUpdate: It looks the answer is yes, all segmented cells (except border cells or cells with less than half-size) must be labeled in submission.csv. Cells with no pattern detected must be labeled with class 18 (Negative). A cell labeled with class 18 cannot have additional labels:
\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/216924#1187646
If someone can confirm all the above, I will be happy :-)
", "votes": 1, "replies": [] }, { "id": 1215527, "author_name": "MPWARE", "author_url": "", "post_date": "2021-02-23T17:40:38.943000", "content": "I've submitted the same predictions twice: one with class 18 for cells without detected pattern and another with nothing for such cells. Both worked but the second one was lower. So it confirms that class 18 have to be used in that case.
It would be interesting to see what happens if you submit class 0 for these cells (the most frequent class). Would that increase or lower your score?