{ "id": 279995, "title": "No Overlap Issue: Quick Fix", "url": "/competitions/sartorius-cell-instance-segmentation/discussion/279995", "author_name": "", "post_date": "2021-10-19T20:56:07.367992Z", "votes": 61, "comment_count": 5, "views": 0, "content": "

As @inoueu1 mentioned in this thread, each predicted mask is not allowed to overlap the other mask in the same image

\n

Example

\n

\"instances\"

\n

\"remove-mask\"

\n

Check Overlap

\n

You can also check if instances have overlap or not using following function,

\n
 ():\n    msk = msk.astype(np.).astype(np.uint8)\n     np.(np.(msk, axis=-)>)\n
\n

Fix Overlap

\n

Here's a quick fix, Suppose you have a mask with 100 instances, you simply stack them and use the following function to remove overlap pixels. This will take max from overlapping regions instead of taking any random one hence there will be less chance of pixel loss.

\n
 ():\n    \n    msk = np.array(msk)\n    msk = np.pad(msk, [[,],[,],[,]])\n    ins_len = msk.shape[-]\n    msk = np.argmax(msk,axis=-)\n    msk = tf.keras.utils.to_categorical(msk, num_classes=ins_len)\n    msk = msk[...,:]\n    msk = msk[...,np.(msk, axis=(,))]\n     msk\n
\n

You can check out this notebook for the usage of these functions. I've used both in the following notebooks, so far seem fine. Let me know if it works for you or not …

\n

Notebook

\n", "messages": [ { "id": "1550639", "postDate": "10/19/2021 20:56:07", "content": "

As @inoueu1 mentioned in this thread, each predicted mask is not allowed to overlap the other mask in the same image

\n

Example

\n

\"instances\"

\n

\"remove-mask\"

\n

Check Overlap

\n

You can also check if instances have overlap or not using following function,

\n
 ():\n    msk = msk.astype(np.).astype(np.uint8)\n     np.(np.(msk, axis=-)>)\n
\n

Fix Overlap

\n

Here's a quick fix, Suppose you have a mask with 100 instances, you simply stack them and use the following function to remove overlap pixels. This will take max from overlapping regions instead of taking any random one hence there will be less chance of pixel loss.

\n
 ():\n    \n    msk = np.array(msk)\n    msk = np.pad(msk, [[,],[,],[,]])\n    ins_len = msk.shape[-]\n    msk = np.argmax(msk,axis=-)\n    msk = tf.keras.utils.to_categorical(msk, num_classes=ins_len)\n    msk = msk[...,:]\n    msk = msk[...,np.(msk, axis=(,))]\n     msk\n
\n

You can check out this notebook for the usage of these functions. I've used both in the following notebooks, so far seem fine. Let me know if it works for you or not …

\n

Notebook

\n", "rawMarkdown": "As @inoueu1 mentioned in this [thread](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790), `each predicted mask is not allowed to overlap the other mask in the same image`\n\n## Example\n\"instances\"\n\n\"remove-mask\"\n\n## Check Overlap\nYou can also check if instances have overlap or not using following function,\n```py\ndef check_overlap(msk):\n msk = msk.astype(np.bool).astype(np.uint8)\n return np.any(np.sum(msk, axis=-1)>1)\n```\n\n## Fix Overlap\nHere's a quick fix, Suppose you have a mask with `100` instances, you simply stack them and use the following function to remove overlap pixels. This will take max from overlapping regions instead of taking any random one hence there will be less chance of pixel loss.\n```py\ndef fix_overlap(msk):\n \"\"\"\n Args:\n mask: multi-channel mask, each channel is an instance of cell, shape:(520,704,None)\n Returns:\n multi-channel mask with non-overlapping values, shape:(520,704,None)\n \"\"\"\n msk = np.array(msk)\n msk = np.pad(msk, [[0,0],[0,0],[1,0]])\n ins_len = msk.shape[-1]\n msk = np.argmax(msk,axis=-1)\n msk = tf.keras.utils.to_categorical(msk, num_classes=ins_len)\n msk = msk[...,1:]\n msk = msk[...,np.any(msk, axis=(0,1))]\n return msk\n```\n\nYou can check out this [notebook](https://www.kaggle.com/awsaf49/sartorius-remove-overlap) for the usage of these functions. I've used both in the following notebooks, so far seem fine. Let me know if it works for you or not ...\n\n## Notebook\n* Fix-Overlap: [Sartorius: Fix Overlap](https://www.kaggle.com/awsaf49/sartorius-fix-overlap)\n* Train: [Sartorius: MMDetection [Train]](https://www.kaggle.com/awsaf49/sartorius-mmdetection-train)\n* Infer: [Sartorius: MMDetection [Infer]](https://www.kaggle.com/awsaf49/sartorius-mmdetection-infer) `LB: 0.270`", "votes": null }, { "id": "1554359", "postDate": "10/23/2021 02:21:54", "content": "

@Awsaf What is your order of operations with fix_overlap?

\n

Do you first run the training data through fix_overlap, then train, then run the test data through fix_overlap, and finally do inference?

", "rawMarkdown": "Awsaf What is your order of operations with `fix_overlap`? \n\nDo you first run the training data through `fix_overlap`, then train, then run the test data through `fix_overlap`, and finally do inference?", "votes": null }, { "id": "1554441", "postDate": "10/23/2021 05:04:33", "content": "

I only use it for post-processin so for training data I use the masks as it is …

", "rawMarkdown": "I only use it for `post-processin` so for training data I use the masks as it is ...", "votes": null }, { "id": "1579444", "postDate": "11/11/2021 22:35:16", "content": "

Overlap fractions for some images are more than 30 %. I'm still not sure how things work out.

", "rawMarkdown": "Overlap fractions for some images are more than 30 %. I'm still not sure how things work out.", "votes": null }, { "id": "1598279", "postDate": "11/28/2021 10:50:14", "content": "

Could you tell me how to convert my mask shape into this form (520,704,none)
\nMy mask shape is (520,704)
\nmy_mask= cv2.resize(frame1, (704,520), interpolation = cv2.INTER_AREA)
\nWhen I check overlap, the output is True
\nBut when I apply fix_overlap, I have problem with this line
\nmsk = np.pad(msk, [[0,0],[0,0],[1,0]])
\n I will be grateful if you help me to solve this issue
\nThanks in advance

", "rawMarkdown": "Could you tell me how to convert my mask shape into this form (520,704,none)\nMy mask shape is (520,704)\nmy_mask= cv2.resize(frame1, (704,520), interpolation = cv2.INTER_AREA)\nWhen I check overlap, the output is True\nBut when I apply fix_overlap, I have problem with this line\nmsk = np.pad(msk, [[0,0],[0,0],[1,0]])\n I will be grateful if you help me to solve this issue\nThanks in advance", "votes": null }, { "id": "1598282", "postDate": "11/28/2021 10:52:56", "content": "

@awsaf49
\nCould you tell me how to convert my mask shape into this form (520,704,none)
\nMy mask shape is (520,704)
\nmy_mask= cv2.resize(frame1, (704,520), interpolation = cv2.INTER_AREA)
\nWhen I check overlap, the output is True
\nBut when I apply fix_overlap, I have problem with this line
\nmsk = np.pad(msk, [[0,0],[0,0],[1,0]])
\nI will be grateful if you help me to solve this issue
\nThanks in advance

", "rawMarkdown": "awsaf49 \nCould you tell me how to convert my mask shape into this form (520,704,none)\nMy mask shape is (520,704)\nmy_mask= cv2.resize(frame1, (704,520), interpolation = cv2.INTER_AREA)\nWhen I check overlap, the output is True\nBut when I apply fix_overlap, I have problem with this line\nmsk = np.pad(msk, [[0,0],[0,0],[1,0]])\nI will be grateful if you help me to solve this issue\nThanks in advance", "votes": null } ], "comments": [ { "id": 1554359, "author_name": "ohbewise", "author_url": "", "post_date": "10/23/2021 02:21:54", "content": "

@Awsaf What is your order of operations with fix_overlap?

\n

Do you first run the training data through fix_overlap, then train, then run the test data through fix_overlap, and finally do inference?

", "votes": null, "replies": [ { "id": 1554441, "author_name": "awsaf49", "author_url": "", "post_date": "10/23/2021 05:04:33", "content": "

I only use it for post-processin so for training data I use the masks as it is …

", "votes": null, "replies": [] }, { "id": 1579444, "author_name": "tolgadincer", "author_url": "", "post_date": "11/11/2021 22:35:16", "content": "

Overlap fractions for some images are more than 30 %. I'm still not sure how things work out.

", "votes": null, "replies": [] } ] }, { "id": 1598279, "author_name": "fatmamazen", "author_url": "", "post_date": "11/28/2021 10:50:14", "content": "

Could you tell me how to convert my mask shape into this form (520,704,none)
\nMy mask shape is (520,704)
\nmy_mask= cv2.resize(frame1, (704,520), interpolation = cv2.INTER_AREA)
\nWhen I check overlap, the output is True
\nBut when I apply fix_overlap, I have problem with this line
\nmsk = np.pad(msk, [[0,0],[0,0],[1,0]])
\n I will be grateful if you help me to solve this issue
\nThanks in advance

", "votes": null, "replies": [] }, { "id": 1598282, "author_name": "fatmamazen", "author_url": "", "post_date": "11/28/2021 10:52:56", "content": "

@awsaf49
\nCould you tell me how to convert my mask shape into this form (520,704,none)
\nMy mask shape is (520,704)
\nmy_mask= cv2.resize(frame1, (704,520), interpolation = cv2.INTER_AREA)
\nWhen I check overlap, the output is True
\nBut when I apply fix_overlap, I have problem with this line
\nmsk = np.pad(msk, [[0,0],[0,0],[1,0]])
\nI will be grateful if you help me to solve this issue
\nThanks in advance

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1550639": "As @inoueu1 mentioned in this [thread](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790), `each predicted mask is not allowed to overlap the other mask in the same image`\n\n## Example\n\"instances\"\n\n\"remove-mask\"\n\n## Check Overlap\nYou can also check if instances have overlap or not using following function,\n```py\ndef check_overlap(msk):\n msk = msk.astype(np.bool).astype(np.uint8)\n return np.any(np.sum(msk, axis=-1)>1)\n```\n\n## Fix Overlap\nHere's a quick fix, Suppose you have a mask with `100` instances, you simply stack them and use the following function to remove overlap pixels. This will take max from overlapping regions instead of taking any random one hence there will be less chance of pixel loss.\n```py\ndef fix_overlap(msk):\n \"\"\"\n Args:\n mask: multi-channel mask, each channel is an instance of cell, shape:(520,704,None)\n Returns:\n multi-channel mask with non-overlapping values, shape:(520,704,None)\n \"\"\"\n msk = np.array(msk)\n msk = np.pad(msk, [[0,0],[0,0],[1,0]])\n ins_len = msk.shape[-1]\n msk = np.argmax(msk,axis=-1)\n msk = tf.keras.utils.to_categorical(msk, num_classes=ins_len)\n msk = msk[...,1:]\n msk = msk[...,np.any(msk, axis=(0,1))]\n return msk\n```\n\nYou can check out this [notebook](https://www.kaggle.com/awsaf49/sartorius-remove-overlap) for the usage of these functions. I've used both in the following notebooks, so far seem fine. Let me know if it works for you or not ...\n\n## Notebook\n* Fix-Overlap: [Sartorius: Fix Overlap](https://www.kaggle.com/awsaf49/sartorius-fix-overlap)\n* Train: [Sartorius: MMDetection [Train]](https://www.kaggle.com/awsaf49/sartorius-mmdetection-train)\n* Infer: [Sartorius: MMDetection [Infer]](https://www.kaggle.com/awsaf49/sartorius-mmdetection-infer) `LB: 0.270`", "1554359": "Awsaf What is your order of operations with `fix_overlap`? \n\nDo you first run the training data through `fix_overlap`, then train, then run the test data through `fix_overlap`, and finally do inference?", "1554441": "I only use it for `post-processin` so for training data I use the masks as it is ...", "1579444": "Overlap fractions for some images are more than 30 %. I'm still not sure how things work out.", "1598279": "Could you tell me how to convert my mask shape into this form (520,704,none)\nMy mask shape is (520,704)\nmy_mask= cv2.resize(frame1, (704,520), interpolation = cv2.INTER_AREA)\nWhen I check overlap, the output is True\nBut when I apply fix_overlap, I have problem with this line\nmsk = np.pad(msk, [[0,0],[0,0],[1,0]])\n I will be grateful if you help me to solve this issue\nThanks in advance", "1598282": "awsaf49 \nCould you tell me how to convert my mask shape into this form (520,704,none)\nMy mask shape is (520,704)\nmy_mask= cv2.resize(frame1, (704,520), interpolation = cv2.INTER_AREA)\nWhen I check overlap, the output is True\nBut when I apply fix_overlap, I have problem with this line\nmsk = np.pad(msk, [[0,0],[0,0],[1,0]])\nI will be grateful if you help me to solve this issue\nThanks in advance" }, "source": "meta" }