{ "id": 279790, "title": "Tips in submission and baseline (in the beginning of the competition)", "url": "/competitions/sartorius-cell-instance-segmentation/discussion/279790", "author_name": "Inoichan", "post_date": "2021-10-19T04:32:44.054000", "votes": 205, "comment_count": 39, "views": 0, "content": "

Some competitors seems to have troubles in submission.
\nI could succeed in submission, so I will share the tips.

\n

Instance segmentation

\n

First, the task of this competition is \"instance segmentation\", not \"semantic segmentation\". So, we need to predict cell masks separately. My first impression is that it's better to use a instance segmentation model like a Mask RCNN than to use a U-Net or something semantic segmentation models. My base model which scored over 0.22 in LB is a kind of Mask RCNN.

\n

Update:
\nI read discussions of Data Science Bowl 2018, and found that U-Net based models seems to be also useful. However, in the previous competition, we had to segment the nucleus, not the cells, which is different from this time. It will take some experimentation to see which method is more effective in this competition.
\nData Science Bowl 2018 link

\n

Mask encoding

\n

In submission, we have to encoding mask. Evaluation page show the pixels are numbered from top to bottom, then left to right. But, this is not correct. As you know, the training annotations provided are numbered from left to right, then top to bottom, and this is also the case with submission.
\nAn LB score with encoded mask (top -> bottom, left -> right) was 0.0000, but a score with (left -> right, top -> bottom) was over 0.22.
\nUnfortunately, the evaluation page is just a copy from Data Science Bowl 2018. I hope that the competition hosts will update the evaluation page for this competition.
\n@christoffersartorius @addisonhoward

\n

Update:
\nThe evaluation page was updated. Thanks.

\n

No overlapping

\n

At first few submissions, I got Submission scoring error. The model predictions looks good, but the error occurred. The reason is that each predicted mask is not allowed to overlap the other mask in the same image. By removing overlapped region of each mask, the submission was succeeded.
\nI think this is one of the important points of this competition. We might need to find better post processing to remove overlapping. In my baseline, I set a threshold of nms=0.0000001, so if even 1 pixel overlaps with other masks, the mask with lower score will be removed. I don't think this is the best way, and further improvements are needed.

", "messages": [ { "id": 1549585, "postDate": "2021-10-19T04:32:44.053Z", "content": "

Some competitors seems to have troubles in submission.
\nI could succeed in submission, so I will share the tips.

\n

Instance segmentation

\n

First, the task of this competition is \"instance segmentation\", not \"semantic segmentation\". So, we need to predict cell masks separately. My first impression is that it's better to use a instance segmentation model like a Mask RCNN than to use a U-Net or something semantic segmentation models. My base model which scored over 0.22 in LB is a kind of Mask RCNN.

\n

Update:
\nI read discussions of Data Science Bowl 2018, and found that U-Net based models seems to be also useful. However, in the previous competition, we had to segment the nucleus, not the cells, which is different from this time. It will take some experimentation to see which method is more effective in this competition.
\nData Science Bowl 2018 link

\n

Mask encoding

\n

In submission, we have to encoding mask. Evaluation page show the pixels are numbered from top to bottom, then left to right. But, this is not correct. As you know, the training annotations provided are numbered from left to right, then top to bottom, and this is also the case with submission.
\nAn LB score with encoded mask (top -> bottom, left -> right) was 0.0000, but a score with (left -> right, top -> bottom) was over 0.22.
\nUnfortunately, the evaluation page is just a copy from Data Science Bowl 2018. I hope that the competition hosts will update the evaluation page for this competition.
\n@christoffersartorius @addisonhoward

\n

Update:
\nThe evaluation page was updated. Thanks.

\n

No overlapping

\n

At first few submissions, I got Submission scoring error. The model predictions looks good, but the error occurred. The reason is that each predicted mask is not allowed to overlap the other mask in the same image. By removing overlapped region of each mask, the submission was succeeded.
\nI think this is one of the important points of this competition. We might need to find better post processing to remove overlapping. In my baseline, I set a threshold of nms=0.0000001, so if even 1 pixel overlaps with other masks, the mask with lower score will be removed. I don't think this is the best way, and further improvements are needed.

", "rawMarkdown": "Some competitors seems to have troubles in submission.\nI could succeed in submission, so I will share the tips.\n\n## Instance segmentation\nFirst, the task of this competition is \"instance segmentation\", not \"semantic segmentation\". So, we need to predict cell masks separately. My first impression is that it's better to use a instance segmentation model like a Mask RCNN than to use a U-Net or something semantic segmentation models. My base model which scored over 0.22 in LB is a kind of Mask RCNN.\n\nUpdate:\nI read discussions of Data Science Bowl 2018, and found that U-Net based models seems to be also useful. However, in the previous competition, we had to segment the nucleus, not the cells, which is different from this time. It will take some experimentation to see which method is more effective in this competition.\nData Science Bowl 2018 [link](https://www.kaggle.com/c/data-science-bowl-2018/overview)\n\n## Mask encoding\nIn submission, we have to encoding mask. Evaluation page show the pixels are numbered from top to bottom, then left to right. But, this is not correct. As you know, the training annotations provided are numbered from left to right, then top to bottom, and this is also the case with submission.\nAn LB score with encoded mask (top -> bottom, left -> right) was 0.0000, but a score with (left -> right, top -> bottom) was over 0.22.\nUnfortunately, the evaluation page is just a copy from Data Science Bowl 2018. I hope that the competition hosts will update the evaluation page for this competition.\n@christoffersartorius @addisonhoward \n\nUpdate:\nThe evaluation page was updated. Thanks.\n\n## No overlapping\nAt first few submissions, I got Submission scoring error. The model predictions looks good, but the error occurred. The reason is that each predicted mask is not allowed to overlap the other mask in the same image. By removing overlapped region of each mask, the submission was succeeded.\nI think this is one of the important points of this competition. We might need to find better post processing to remove overlapping. In my baseline, I set a threshold of nms=0.0000001, so if even 1 pixel overlaps with other masks, the mask with lower score will be removed. I don't think this is the best way, and further improvements are needed.", "votes": 203 }, { "id": 1550270, "postDate": "2021-10-19T15:11:47.413Z", "content": "

The third point sure is weird, considering there are overlaps in the ground truth masks. Thanks for the warning

", "rawMarkdown": "The third point sure is weird, considering there are overlaps in the ground truth masks. Thanks for the warning", "votes": 13, "replies": [ { "id": 1550638, "postDate": "2021-10-19T20:53:54.547Z", "content": "

+1 to the fact that there are overlaps in the ground truth masks.

", "rawMarkdown": "+1 to the fact that there are overlaps in the ground truth masks. ", "votes": 4 }, { "id": 1550653, "postDate": "2021-10-19T21:39:50.317Z", "content": "

if masks are broken, its kinda pointless to fight overlap (since in that case we should get another annotation set). But I agree, there should be no overlap in gt masks

", "rawMarkdown": "if masks are broken, its kinda pointless to fight overlap (since in that case we should get another annotation set). But I agree, there should be no overlap in gt masks" }, { "id": 1550666, "postDate": "2021-10-19T22:09:36.970Z", "content": "

The data description has been updated to clarify that point:

\n

Note: while predictions are not allowed to overlap, the training labels are provided in full (with overlapping portions included). This is to ensure that models are provided the full data for each object. Removing overlap in predictions is a task for the competitor.

", "rawMarkdown": "The data description has been updated to clarify that point:\n\n\n```Note: while predictions are not allowed to overlap, the training labels are provided in full (with overlapping portions included). This is to ensure that models are provided the full data for each object. Removing overlap in predictions is a task for the competitor.```", "votes": 7 }, { "id": 1552334, "postDate": "2021-10-21T10:45:01.003Z", "content": "

@sohier Im lost on how to score metric then. Is Evaluation tab up to date? Is it possible to score 1 on this map evaluation? Different methods of removing overlap will lead to different map score am I right? On an unrelated note, why kaggle wont share metric code itself?

", "rawMarkdown": "@sohier Im lost on how to score metric then. Is Evaluation tab up to date? Is it possible to score 1 on this map evaluation? Different methods of removing overlap will lead to different map score am I right? On an unrelated note, why kaggle wont share metric code itself?", "votes": 3 }, { "id": 1556803, "postDate": "2021-10-25T06:39:01.980Z", "content": "
\n

The pixels are one-indexed
\n and numbered from top to bottom, then left to right

\n
\n

I believe, this one should be also updated. Or maybe just pin this thread to the top?

", "rawMarkdown": "> The pixels are one-indexed\nand numbered from top to bottom, then left to right\n\nI believe, this one should be also updated. Or maybe just pin this thread to the top?", "votes": 3 } ] }, { "id": 1551068, "postDate": "2021-10-20T09:01:27.350Z", "content": "

This is my first instance segmentation competition. If I understood correctly, instance segmentation models like Mask RCNN outputs every instance mask separately like an object detection model but semantic segmentation models like U-Net outputs a single mask of logits. In that case, if we can separate every instance in the semantic segmentation model's output, it should also work fine, right?

", "rawMarkdown": "This is my first instance segmentation competition. If I understood correctly, instance segmentation models like Mask RCNN outputs every instance mask separately like an object detection model but semantic segmentation models like U-Net outputs a single mask of logits. In that case, if we can separate every instance in the semantic segmentation model's output, it should also work fine, right?", "votes": 12, "replies": [ { "id": 1551654, "postDate": "2021-10-20T19:41:31.177Z", "content": "

+1 to this, It should not matter at the end if we can separate the Unet's output.

", "rawMarkdown": "+1 to this, It should not matter at the end if we can separate the Unet's output." }, { "id": 1551853, "postDate": "2021-10-20T22:46:01.027Z", "content": "

If you take care of the overlaps of the instances, you can then assign every pixel predicted by the unet to one of the instances.

", "rawMarkdown": "If you take care of the overlaps of the instances, you can then assign every pixel predicted by the unet to one of the instances.", "votes": 1 }, { "id": 1576561, "postDate": "2021-11-09T10:01:31.883Z", "content": "

how to separate every instance in the semantic segmentation model's (unet) output?

", "rawMarkdown": "how to separate every instance in the semantic segmentation model's (unet) output? ", "votes": 2 }, { "id": 1625242, "postDate": "2021-12-21T16:01:31.690Z", "content": "

The topic of the nucleus separation on the output is shortly treated here: https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/288376

\n

The following code do it:
\ndef post_process(mask, min_size=180, shape=(520, 704,)):\n num_component, component = cv2.connectedComponents(mask.astype(np.uint8))\n predictions = []\n for c in range(1, num_component):\n p = (component == c)\n if p.sum() > min_size:\n a_prediction = np.zeros(shape, np.float32)\n a_prediction[p] = 1\n predictions.append(a_prediction)\n return predictions

\n

I have included them on a U-Net model and they actually work.
\nNotice that they are image-shape sensitive so be aware of the size of the mask to submit.

", "rawMarkdown": "The topic of the nucleus separation on the output is shortly treated here: https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/288376\n\nThe following code do it:\n`def post_process(mask, min_size=180, shape=(520, 704,)):\n num_component, component = cv2.connectedComponents(mask.astype(np.uint8))\n predictions = []\n for c in range(1, num_component):\n p = (component == c)\n if p.sum() > min_size:\n a_prediction = np.zeros(shape, np.float32)\n a_prediction[p] = 1\n predictions.append(a_prediction)\n return predictions`\n\n\nI have included them on a U-Net model and they actually work.\nNotice that they are image-shape sensitive so be aware of the size of the mask to submit." } ] }, { "id": 1576571, "postDate": "2021-11-09T10:11:56.330Z", "content": "

\" previous competition, we had to segment the nucleus, not the cells, which is different from this time.\"

\n

kaggle HPAv2 competition provides a cell segmentation (an also nucleus segmentation )model
\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/data
\nhttps://github.com/CellProfiling/HPA-Cell-Segmentation

\n

this is also using unet+watershed approach
\nhttps://github.com/CellProfiling/HPA-Cell-Segmentation/blob/63977be24f67ac483b4ffd6f7f036e6444a5bfeb/hpacellseg/utils.py

", "rawMarkdown": "\" previous competition, we had to segment the nucleus, not the cells, which is different from this time.\"\n\nkaggle HPAv2 competition provides a cell segmentation (an also nucleus segmentation )model\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/data\nhttps://github.com/CellProfiling/HPA-Cell-Segmentation\n\nthis is also using unet+watershed approach\nhttps://github.com/CellProfiling/HPA-Cell-Segmentation/blob/63977be24f67ac483b4ffd6f7f036e6444a5bfeb/hpacellseg/utils.py", "votes": 6 }, { "id": 1567154, "postDate": "2021-11-01T15:52:31.900Z", "content": "

Hi I am not able to understand what is the meaning of training annotations are numbered from left-->right then top-->bottom. I am new to computer vision and deep learning and any help would be appreciated

", "rawMarkdown": "Hi I am not able to understand what is the meaning of training annotations are numbered from left-->right then top-->bottom. I am new to computer vision and deep learning and any help would be appreciated", "votes": 3, "replies": [ { "id": 1569779, "postDate": "2021-11-03T17:55:26.317Z", "content": "

It means that the pixels are numbered in a column-major order. Check out this discussion for more information.

", "rawMarkdown": "It means that the pixels are numbered in a column-major order. Check out [this discussion](https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/102311) for more information.", "votes": 2 } ] }, { "id": 1551847, "postDate": "2021-10-20T22:41:02.980Z", "content": "

Re overlapping: in my opinion this is going to be of utmost importance. See my thoughts here.

", "rawMarkdown": "Re overlapping: in my opinion this is going to be of utmost importance. [See my thoughts here](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/280250).", "votes": 1 }, { "id": 1550411, "postDate": "2021-10-19T17:28:26.990Z", "content": "

I've wasted many subs for this No overlapping thanks for point it out …

", "rawMarkdown": "I've wasted many subs for this `No overlapping` thanks for point it out ...", "votes": 1 }, { "id": 1549918, "postDate": "2021-10-19T09:30:31.290Z", "content": "

Thanks for sharing!
\nDid you exert scipy.ndimage.morphology.binary_fill_holes to deal with broken masks?

", "rawMarkdown": "Thanks for sharing!\nDid you exert *scipy.ndimage.morphology.binary_fill_holes* to deal with broken masks?", "votes": 1, "replies": [ { "id": 1550022, "postDate": "2021-10-19T11:20:36.087Z", "content": "

@solosquad1999
\nNo, I haven't used scipy.ndimage.morphology.binary_fill_holes.

", "rawMarkdown": "@solosquad1999 \nNo, I haven't used *scipy.ndimage.morphology.binary_fill_holes*.", "votes": 2 } ] }, { "id": 1576565, "postDate": "2021-11-09T10:05:37.443Z", "content": "

Evaluation page still shows the pixels are numbered from top to bottom, then left to right which is incorrect.

", "rawMarkdown": "Evaluation page still shows the pixels are numbered from top to bottom, then left to right which is incorrect.", "votes": 2 }, { "id": 1624392, "postDate": "2021-12-20T20:58:43.717Z", "content": "

Thank you! Didn't get what \"no overlapping\" means at first

", "rawMarkdown": "Thank you! Didn't get what \"no overlapping\" means at first" }, { "id": 1600142, "postDate": "2021-11-30T06:41:18.447Z", "content": "

@Inoichan thank you for your post
\nI got stuck in this part.
\nCould you share with me the code you used to decode and encode rle. I am wasting my submissions every day for this reason

\n
\n

but a score with (left -> right, top -> bottom) was over 0.22.
\n How did you implement that part?
\n I will be grateful if you help me
\n Thanks in advance

\n
", "rawMarkdown": "@Inoichan thank you for your post \nI got stuck in this part.\nCould you share with me the code you used to decode and encode rle. I am wasting my submissions every day for this reason\n> but a score with (left -> right, top -> bottom) was over 0.22.\nHow did you implement that part?\nI will be grateful if you help me\nThanks in advance" }, { "id": 1567775, "postDate": "2021-11-02T08:30:49.870Z", "content": "

Does not seem like the evaluation page was corrected. Still says from top to bottom, then left to right. \"The pixels are one-indexed and numbered from top to bottom, then left to right: 1 is pixel (1,1), 2 is pixel (2,1), etc.\"

\n

@christoffersartorius

", "rawMarkdown": "Does not seem like the evaluation page was corrected. Still says from top to bottom, then left to right. \"The pixels are one-indexed and numbered from top to bottom, then left to right: 1 is pixel (1,1), 2 is pixel (2,1), etc.\"\n\n@christoffersartorius " }, { "id": 1556683, "postDate": "2021-10-25T05:00:36.260Z", "content": "

Have you found a way to exploit the semantic models such as Unet in this competition ? I have read Data Science Bowl 2018 solution but did not quite understand.

", "rawMarkdown": "Have you found a way to exploit the semantic models such as Unet in this competition ? I have read Data Science Bowl 2018 solution but did not quite understand.", "replies": [ { "id": 1576570, "postDate": "2021-11-09T10:11:17.347Z", "content": "

I also read that solution and understood that. In that competition, unet model was used for semantic segmentation which gives a single mask. But in this competition we need separate masks for each instances, so we need to separate each instances in output mask of the unet model.

", "rawMarkdown": "I also read that solution and understood that. In that competition, unet model was used for semantic segmentation which gives a single mask. But in this competition we need separate masks for each instances, so we need to separate each instances in output mask of the unet model.", "votes": -2 } ] }, { "id": 1556268, "postDate": "2021-10-24T19:03:48.327Z", "content": "

I have checked my output submission.csv file, it follows the form of sample submission file. Also no overlapping in the prediction as far as I can tell. But still I can not make a single submission. In my output folder submission.csv is not the only output file, would that mess with the submission process in any way?

", "rawMarkdown": "I have checked my output submission.csv file, it follows the form of sample submission file. Also no overlapping in the prediction as far as I can tell. But still I can not make a single submission. In my output folder submission.csv is not the only output file, would that mess with the submission process in any way?", "replies": [ { "id": 1576567, "postDate": "2021-11-09T10:08:04.233Z", "content": "

multiple files in output folder would not mess with the submission process in any way.

", "rawMarkdown": "multiple files in output folder would not mess with the submission process in any way. ", "votes": 1 } ] }, { "id": 1550292, "postDate": "2021-10-19T15:26:39.583Z", "content": "

Thanks for posting. It seems that the order of labelling masks is in play, I'm guessing the evaluation page looks at the row of a mask and compares it to its ground truth mask at that particular row. This seems…silly. Unless I overlooked something in the information provided, the goal isn't to list the masks in particular order?
\nI still haven't had chance to submit or play around with it, but thanks again for the information.

", "rawMarkdown": "Thanks for posting. It seems that the order of labelling masks is in play, I'm guessing the evaluation page looks at the row of a mask and compares it to its ground truth mask at that particular row. This seems...silly. Unless I overlooked something in the information provided, the goal isn't to list the masks in particular order?\nI still haven't had chance to submit or play around with it, but thanks again for the information." }, { "id": 1549675, "postDate": "2021-10-19T06:07:21.393Z", "content": "

This article is very informative as a newbie in this competition. Thanks a lot🙏

", "rawMarkdown": "This article is very informative as a newbie in this competition. Thanks a lot🙏" }, { "id": 1575558, "postDate": "2021-11-08T13:20:14.683Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1567545, "postDate": "2021-11-02T01:48:34.877Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1555056, "postDate": "2021-10-23T14:47:27.697Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1554011, "postDate": "2021-10-22T18:38:23.290Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 1556007, "postDate": "2021-10-24T12:37:04.597Z", "content": "

Thank you for sharing! :D

", "rawMarkdown": "Thank you for sharing! :D", "votes": 1 }, { "id": 1624349, "postDate": "2021-12-20T18:59:00.980Z", "content": "

Thanks for sharing

", "rawMarkdown": "Thanks for sharing " }, { "id": 1604697, "postDate": "2021-12-03T16:05:12.270Z", "content": "

thank you for your sharing

", "rawMarkdown": "thank you for your sharing" }, { "id": 1571958, "postDate": "2021-11-05T10:13:02.387Z", "content": "

Thanks for sharing!!

", "rawMarkdown": "Thanks for sharing!!" }, { "id": 1550991, "postDate": "2021-10-20T07:48:55.920Z", "content": "

very useful, thanks for sharing

", "rawMarkdown": "very useful, thanks for sharing" }, { "id": 1549861, "postDate": "2021-10-19T08:41:25.330Z", "content": "

Thanks for sharing 😁.

", "rawMarkdown": "Thanks for sharing 😁." }, { "id": 1549727, "postDate": "2021-10-19T07:02:51.140Z", "content": "

Informative and Intuitive. Thanks :)

", "rawMarkdown": "Informative and Intuitive. Thanks :)" } ], "comments": [ { "id": 1550270, "author_name": "Slawek Biel", "author_url": "", "post_date": "2021-10-19T15:11:47.413000", "content": "

The third point sure is weird, considering there are overlaps in the ground truth masks. Thanks for the warning

", "votes": 13, "replies": [ { "id": 1550638, "author_name": "Ayush Thakur", "author_url": "", "post_date": "2021-10-19T20:53:54.547000", "content": "

+1 to the fact that there are overlaps in the ground truth masks.

", "votes": 4, "replies": [] }, { "id": 1550653, "author_name": "Gleb", "author_url": "", "post_date": "2021-10-19T21:39:50.317000", "content": "

if masks are broken, its kinda pointless to fight overlap (since in that case we should get another annotation set). But I agree, there should be no overlap in gt masks

", "votes": 0, "replies": [] }, { "id": 1550666, "author_name": "Sohier Dane", "author_url": "", "post_date": "2021-10-19T22:09:36.970000", "content": "

The data description has been updated to clarify that point:

\n

Note: while predictions are not allowed to overlap, the training labels are provided in full (with overlapping portions included). This is to ensure that models are provided the full data for each object. Removing overlap in predictions is a task for the competitor.

", "votes": 7, "replies": [] }, { "id": 1552334, "author_name": "Gleb", "author_url": "", "post_date": "2021-10-21T10:45:01.003000", "content": "

@sohier Im lost on how to score metric then. Is Evaluation tab up to date? Is it possible to score 1 on this map evaluation? Different methods of removing overlap will lead to different map score am I right? On an unrelated note, why kaggle wont share metric code itself?

", "votes": 3, "replies": [] }, { "id": 1556803, "author_name": "A.Demyanchuk", "author_url": "", "post_date": "2021-10-25T06:39:01.980000", "content": "
\n

The pixels are one-indexed
\n and numbered from top to bottom, then left to right

\n
\n

I believe, this one should be also updated. Or maybe just pin this thread to the top?

", "votes": 3, "replies": [] } ] }, { "id": 1551068, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2021-10-20T09:01:27.350000", "content": "

This is my first instance segmentation competition. If I understood correctly, instance segmentation models like Mask RCNN outputs every instance mask separately like an object detection model but semantic segmentation models like U-Net outputs a single mask of logits. In that case, if we can separate every instance in the semantic segmentation model's output, it should also work fine, right?

", "votes": 12, "replies": [ { "id": 1551654, "author_name": "Eren Tekin", "author_url": "", "post_date": "2021-10-20T19:41:31.177000", "content": "

+1 to this, It should not matter at the end if we can separate the Unet's output.

", "votes": 0, "replies": [] }, { "id": 1551853, "author_name": "dr. Konya", "author_url": "", "post_date": "2021-10-20T22:46:01.027000", "content": "

If you take care of the overlaps of the instances, you can then assign every pixel predicted by the unet to one of the instances.

", "votes": 1, "replies": [] }, { "id": 1576561, "author_name": "Karan Yadav", "author_url": "", "post_date": "2021-11-09T10:01:31.883000", "content": "

how to separate every instance in the semantic segmentation model's (unet) output?

", "votes": 2, "replies": [] }, { "id": 1625242, "author_name": "Carlos.gut", "author_url": "", "post_date": "2021-12-21T16:01:31.690000", "content": "

The topic of the nucleus separation on the output is shortly treated here: https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/288376

\n

The following code do it:
\ndef post_process(mask, min_size=180, shape=(520, 704,)):\n num_component, component = cv2.connectedComponents(mask.astype(np.uint8))\n predictions = []\n for c in range(1, num_component):\n p = (component == c)\n if p.sum() > min_size:\n a_prediction = np.zeros(shape, np.float32)\n a_prediction[p] = 1\n predictions.append(a_prediction)\n return predictions

\n

I have included them on a U-Net model and they actually work.
\nNotice that they are image-shape sensitive so be aware of the size of the mask to submit.

", "votes": 0, "replies": [] } ] }, { "id": 1576571, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-09T10:11:56.330000", "content": "

\" previous competition, we had to segment the nucleus, not the cells, which is different from this time.\"

\n

kaggle HPAv2 competition provides a cell segmentation (an also nucleus segmentation )model
\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/data
\nhttps://github.com/CellProfiling/HPA-Cell-Segmentation

\n

this is also using unet+watershed approach
\nhttps://github.com/CellProfiling/HPA-Cell-Segmentation/blob/63977be24f67ac483b4ffd6f7f036e6444a5bfeb/hpacellseg/utils.py

", "votes": 6, "replies": [] }, { "id": 1567154, "author_name": "vishal sharma", "author_url": "", "post_date": "2021-11-01T15:52:31.900000", "content": "

Hi I am not able to understand what is the meaning of training annotations are numbered from left-->right then top-->bottom. I am new to computer vision and deep learning and any help would be appreciated

", "votes": 3, "replies": [ { "id": 1569779, "author_name": "Sarvagya Malaviya", "author_url": "", "post_date": "2021-11-03T17:55:26.317000", "content": "

It means that the pixels are numbered in a column-major order. Check out this discussion for more information.

", "votes": 2, "replies": [] } ] }, { "id": 1551847, "author_name": "dr. Konya", "author_url": "", "post_date": "2021-10-20T22:41:02.980000", "content": "

Re overlapping: in my opinion this is going to be of utmost importance. See my thoughts here.

", "votes": 1, "replies": [] }, { "id": 1550411, "author_name": "Awsaf", "author_url": "", "post_date": "2021-10-19T17:28:26.990000", "content": "

I've wasted many subs for this No overlapping thanks for point it out …

", "votes": 1, "replies": [] }, { "id": 1549918, "author_name": "The fearless", "author_url": "", "post_date": "2021-10-19T09:30:31.290000", "content": "

Thanks for sharing!
\nDid you exert scipy.ndimage.morphology.binary_fill_holes to deal with broken masks?

", "votes": 1, "replies": [ { "id": 1550022, "author_name": "Inoichan", "author_url": "", "post_date": "2021-10-19T11:20:36.087000", "content": "

@solosquad1999
\nNo, I haven't used scipy.ndimage.morphology.binary_fill_holes.

", "votes": 2, "replies": [] } ] }, { "id": 1576565, "author_name": "Karan Yadav", "author_url": "", "post_date": "2021-11-09T10:05:37.443000", "content": "

Evaluation page still shows the pixels are numbered from top to bottom, then left to right which is incorrect.

", "votes": 2, "replies": [] }, { "id": 1624392, "author_name": "Boris Tarovik", "author_url": "", "post_date": "2021-12-20T20:58:43.717000", "content": "

Thank you! Didn't get what \"no overlapping\" means at first

", "votes": 0, "replies": [] }, { "id": 1600142, "author_name": "Fatma Mazen", "author_url": "", "post_date": "2021-11-30T06:41:18.447000", "content": "

@Inoichan thank you for your post
\nI got stuck in this part.
\nCould you share with me the code you used to decode and encode rle. I am wasting my submissions every day for this reason

\n
\n

but a score with (left -> right, top -> bottom) was over 0.22.
\n How did you implement that part?
\n I will be grateful if you help me
\n Thanks in advance

\n
", "votes": 0, "replies": [] }, { "id": 1567775, "author_name": "Yousef Rabi", "author_url": "", "post_date": "2021-11-02T08:30:49.870000", "content": "

Does not seem like the evaluation page was corrected. Still says from top to bottom, then left to right. \"The pixels are one-indexed and numbered from top to bottom, then left to right: 1 is pixel (1,1), 2 is pixel (2,1), etc.\"

\n

@christoffersartorius

", "votes": 0, "replies": [] }, { "id": 1556683, "author_name": "Liam Nguyen", "author_url": "", "post_date": "2021-10-25T05:00:36.260000", "content": "

Have you found a way to exploit the semantic models such as Unet in this competition ? I have read Data Science Bowl 2018 solution but did not quite understand.

", "votes": 0, "replies": [ { "id": 1576570, "author_name": "Karan Yadav", "author_url": "", "post_date": "2021-11-09T10:11:17.347000", "content": "

I also read that solution and understood that. In that competition, unet model was used for semantic segmentation which gives a single mask. But in this competition we need separate masks for each instances, so we need to separate each instances in output mask of the unet model.

", "votes": -2, "replies": [] } ] }, { "id": 1556268, "author_name": "Mes", "author_url": "", "post_date": "2021-10-24T19:03:48.327000", "content": "

I have checked my output submission.csv file, it follows the form of sample submission file. Also no overlapping in the prediction as far as I can tell. But still I can not make a single submission. In my output folder submission.csv is not the only output file, would that mess with the submission process in any way?

", "votes": 0, "replies": [ { "id": 1576567, "author_name": "Karan Yadav", "author_url": "", "post_date": "2021-11-09T10:08:04.233000", "content": "

multiple files in output folder would not mess with the submission process in any way.

", "votes": 1, "replies": [] } ] }, { "id": 1550292, "author_name": "Michael Bolton", "author_url": "", "post_date": "2021-10-19T15:26:39.583000", "content": "

Thanks for posting. It seems that the order of labelling masks is in play, I'm guessing the evaluation page looks at the row of a mask and compares it to its ground truth mask at that particular row. This seems…silly. Unless I overlooked something in the information provided, the goal isn't to list the masks in particular order?
\nI still haven't had chance to submit or play around with it, but thanks again for the information.

", "votes": 0, "replies": [] }, { "id": 1549675, "author_name": "kaggler", "author_url": "", "post_date": "2021-10-19T06:07:21.393000", "content": "

This article is very informative as a newbie in this competition. Thanks a lot🙏

", "votes": 0, "replies": [] }, { "id": 1575558, "author_name": "", "author_url": "", "post_date": "2021-11-08T13:20:14.683000", "content": "", "votes": 0, "replies": [] }, { "id": 1567545, "author_name": "", "author_url": "", "post_date": "2021-11-02T01:48:34.877000", "content": "", "votes": 0, "replies": [] }, { "id": 1555056, "author_name": "", "author_url": "", "post_date": "2021-10-23T14:47:27.697000", "content": "", "votes": 0, "replies": [] }, { "id": 1554011, "author_name": "", "author_url": "", "post_date": "2021-10-22T18:38:23.290000", "content": "", "votes": 1, "replies": [] }, { "id": 1556007, "author_name": "jaehyun park", "author_url": "", "post_date": "2021-10-24T12:37:04.597000", "content": "

Thank you for sharing! :D

", "votes": 1, "replies": [] }, { "id": 1624349, "author_name": "chiba1sonny", "author_url": "", "post_date": "2021-12-20T18:59:00.980000", "content": "

Thanks for sharing

", "votes": 0, "replies": [] }, { "id": 1604697, "author_name": "SaintEgg", "author_url": "", "post_date": "2021-12-03T16:05:12.270000", "content": "

thank you for your sharing

", "votes": 0, "replies": [] }, { "id": 1571958, "author_name": "gyindeu", "author_url": "", "post_date": "2021-11-05T10:13:02.387000", "content": "

Thanks for sharing!!

", "votes": 0, "replies": [] }, { "id": 1550991, "author_name": "NotFound404", "author_url": "", "post_date": "2021-10-20T07:48:55.920000", "content": "

very useful, thanks for sharing

", "votes": 0, "replies": [] }, { "id": 1549861, "author_name": "Saurav Maheshkar ☕️", "author_url": "", "post_date": "2021-10-19T08:41:25.330000", "content": "

Thanks for sharing 😁.

", "votes": 0, "replies": [] }, { "id": 1549727, "author_name": "Srinidhi", "author_url": "", "post_date": "2021-10-19T07:02:51.140000", "content": "

Informative and Intuitive. Thanks :)

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1549585": "Some competitors seems to have troubles in submission.\nI could succeed in submission, so I will share the tips.\n\n## Instance segmentation\nFirst, the task of this competition is \"instance segmentation\", not \"semantic segmentation\". So, we need to predict cell masks separately. My first impression is that it's better to use a instance segmentation model like a Mask RCNN than to use a U-Net or something semantic segmentation models. My base model which scored over 0.22 in LB is a kind of Mask RCNN.\n\nUpdate:\nI read discussions of Data Science Bowl 2018, and found that U-Net based models seems to be also useful. However, in the previous competition, we had to segment the nucleus, not the cells, which is different from this time. It will take some experimentation to see which method is more effective in this competition.\nData Science Bowl 2018 [link](https://www.kaggle.com/c/data-science-bowl-2018/overview)\n\n## Mask encoding\nIn submission, we have to encoding mask. Evaluation page show the pixels are numbered from top to bottom, then left to right. But, this is not correct. As you know, the training annotations provided are numbered from left to right, then top to bottom, and this is also the case with submission.\nAn LB score with encoded mask (top -> bottom, left -> right) was 0.0000, but a score with (left -> right, top -> bottom) was over 0.22.\nUnfortunately, the evaluation page is just a copy from Data Science Bowl 2018. I hope that the competition hosts will update the evaluation page for this competition.\n@christoffersartorius @addisonhoward \n\nUpdate:\nThe evaluation page was updated. Thanks.\n\n## No overlapping\nAt first few submissions, I got Submission scoring error. The model predictions looks good, but the error occurred. The reason is that each predicted mask is not allowed to overlap the other mask in the same image. By removing overlapped region of each mask, the submission was succeeded.\nI think this is one of the important points of this competition. We might need to find better post processing to remove overlapping. In my baseline, I set a threshold of nms=0.0000001, so if even 1 pixel overlaps with other masks, the mask with lower score will be removed. I don't think this is the best way, and further improvements are needed.", "1550270": "The third point sure is weird, considering there are overlaps in the ground truth masks. Thanks for the warning", "1551068": "This is my first instance segmentation competition. If I understood correctly, instance segmentation models like Mask RCNN outputs every instance mask separately like an object detection model but semantic segmentation models like U-Net outputs a single mask of logits. In that case, if we can separate every instance in the semantic segmentation model's output, it should also work fine, right?", "1576571": "\" previous competition, we had to segment the nucleus, not the cells, which is different from this time.\"\n\nkaggle HPAv2 competition provides a cell segmentation (an also nucleus segmentation )model\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/data\nhttps://github.com/CellProfiling/HPA-Cell-Segmentation\n\nthis is also using unet+watershed approach\nhttps://github.com/CellProfiling/HPA-Cell-Segmentation/blob/63977be24f67ac483b4ffd6f7f036e6444a5bfeb/hpacellseg/utils.py", "1567154": "Hi I am not able to understand what is the meaning of training annotations are numbered from left-->right then top-->bottom. I am new to computer vision and deep learning and any help would be appreciated", "1551847": "Re overlapping: in my opinion this is going to be of utmost importance. [See my thoughts here](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/280250).", "1550411": "I've wasted many subs for this `No overlapping` thanks for point it out ...", "1549918": "Thanks for sharing!\nDid you exert *scipy.ndimage.morphology.binary_fill_holes* to deal with broken masks?", "1576565": "Evaluation page still shows the pixels are numbered from top to bottom, then left to right which is incorrect.", "1624392": "Thank you! Didn't get what \"no overlapping\" means at first", "1600142": "@Inoichan thank you for your post \nI got stuck in this part.\nCould you share with me the code you used to decode and encode rle. I am wasting my submissions every day for this reason\n> but a score with (left -> right, top -> bottom) was over 0.22.\nHow did you implement that part?\nI will be grateful if you help me\nThanks in advance", "1567775": "Does not seem like the evaluation page was corrected. Still says from top to bottom, then left to right. \"The pixels are one-indexed and numbered from top to bottom, then left to right: 1 is pixel (1,1), 2 is pixel (2,1), etc.\"\n\n@christoffersartorius ", "1556683": "Have you found a way to exploit the semantic models such as Unet in this competition ? I have read Data Science Bowl 2018 solution but did not quite understand.", "1556268": "I have checked my output submission.csv file, it follows the form of sample submission file. Also no overlapping in the prediction as far as I can tell. But still I can not make a single submission. In my output folder submission.csv is not the only output file, would that mess with the submission process in any way?", "1550292": "Thanks for posting. It seems that the order of labelling masks is in play, I'm guessing the evaluation page looks at the row of a mask and compares it to its ground truth mask at that particular row. This seems...silly. Unless I overlooked something in the information provided, the goal isn't to list the masks in particular order?\nI still haven't had chance to submit or play around with it, but thanks again for the information.", "1549675": "This article is very informative as a newbie in this competition. Thanks a lot🙏", "1575558": "", "1567545": "", "1555056": "", "1554011": "", "1556007": "Thank you for sharing! :D", "1624349": "Thanks for sharing ", "1604697": "thank you for your sharing", "1571958": "Thanks for sharing!!", "1550991": "very useful, thanks for sharing", "1549861": "Thanks for sharing 😁.", "1549727": "Informative and Intuitive. Thanks :)" } }