{
  "id": 287988,
  "title": "Pixel Overlap Across instances, how does it affects model ?",
  "url": "/competitions/sartorius-cell-instance-segmentation/discussion/287988",
  "author_name": "Jaideep",
  "post_date": "2021-11-16T08:15:11.269000",
  "votes": 8,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I have not fully started on the competition, just doing some round of analysis. May be I am quite late to arrive at this, <br>\nI find there is pixel overlap across the instances</p>\n<p>eg.</p>\n<pre><code>def rles_to_mask(encs, shape):\n    \"\"\"\n    Decodes a rle.\n\n    Args:\n        encs (list of str): Rles for each class.\n        shape (tuple [2]): Mask size.\n\n    Returns:\n        np array [shape]: Mask.\n    \"\"\"\n    img = np.zeros(shape[0] * shape[1], dtype=np.uint)\n    mask_sum=np.zeros(len(encs))\n    for m, enc in enumerate(encs):\n\n        if isinstance(enc, np.float) and np.isnan(enc):\n            continue\n        enc_split = enc.split()\n        for i in range(len(enc_split) // 2):\n            start = int(enc_split[2 * i]) - 1\n            length = int(enc_split[2 * i + 1])\n            img[start: start + length] = 1 + m\n        ##index=np.where(img==1+m)\n\n        #print(img.sum())     \n        mask_sum[m]=img[np.where(img==1+m)].sum()\n        #break\n    return img.reshape(shape),mask_sum\n</code></pre>\n<pre><code>shape = df1[['height', 'width']].values[0]\n\nmask=rles_to_mask(df1['annotation'][0],shape)\n</code></pre>\n<p>mask[0][np.where(mask[0]==1)].sum() !=mask[1][0] #193 vs 198</p>\n<p>what impact does it has over learning     </p>",
  "messages": [
    {
      "id": 1584032,
      "postDate": "2021-11-16T08:15:11.270Z",
      "content": "<p>I have not fully started on the competition, just doing some round of analysis. May be I am quite late to arrive at this, <br>\nI find there is pixel overlap across the instances</p>\n<p>eg.</p>\n<pre><code>def rles_to_mask(encs, shape):\n    \"\"\"\n    Decodes a rle.\n\n    Args:\n        encs (list of str): Rles for each class.\n        shape (tuple [2]): Mask size.\n\n    Returns:\n        np array [shape]: Mask.\n    \"\"\"\n    img = np.zeros(shape[0] * shape[1], dtype=np.uint)\n    mask_sum=np.zeros(len(encs))\n    for m, enc in enumerate(encs):\n\n        if isinstance(enc, np.float) and np.isnan(enc):\n            continue\n        enc_split = enc.split()\n        for i in range(len(enc_split) // 2):\n            start = int(enc_split[2 * i]) - 1\n            length = int(enc_split[2 * i + 1])\n            img[start: start + length] = 1 + m\n        ##index=np.where(img==1+m)\n\n        #print(img.sum())     \n        mask_sum[m]=img[np.where(img==1+m)].sum()\n        #break\n    return img.reshape(shape),mask_sum\n</code></pre>\n<pre><code>shape = df1[['height', 'width']].values[0]\n\nmask=rles_to_mask(df1['annotation'][0],shape)\n</code></pre>\n<p>mask[0][np.where(mask[0]==1)].sum() !=mask[1][0] #193 vs 198</p>\n<p>what impact does it has over learning     </p>",
      "rawMarkdown": "I have not fully started on the competition, just doing some round of analysis. May be I am quite late to arrive at this, \nI find there is pixel overlap across the instances\n\neg.\n```\ndef rles_to_mask(encs, shape):\n    \"\"\"\n    Decodes a rle.\n\n    Args:\n        encs (list of str): Rles for each class.\n        shape (tuple [2]): Mask size.\n\n    Returns:\n        np array [shape]: Mask.\n    \"\"\"\n    img = np.zeros(shape[0] * shape[1], dtype=np.uint)\n    mask_sum=np.zeros(len(encs))\n    for m, enc in enumerate(encs):\n         \n        if isinstance(enc, np.float) and np.isnan(enc):\n            continue\n        enc_split = enc.split()\n        for i in range(len(enc_split) // 2):\n            start = int(enc_split[2 * i]) - 1\n            length = int(enc_split[2 * i + 1])\n            img[start: start + length] = 1 + m\n        ##index=np.where(img==1+m)\n            \n        #print(img.sum())     \n        mask_sum[m]=img[np.where(img==1+m)].sum()\n        #break\n    return img.reshape(shape),mask_sum\n```\n```\nshape = df1[['height', 'width']].values[0]\n\nmask=rles_to_mask(df1['annotation'][0],shape)\n```\nmask[0][np.where(mask[0]==1)].sum() !=mask[1][0] #193 vs 198\n\nwhat impact does it has over learning     \n",
      "votes": 8
    },
    {
      "id": 1584131,
      "postDate": "2021-11-16T09:43:20.103Z",
      "content": "<p>HI <a href=\"https://www.kaggle.com/jaideepvalani\" target=\"_blank\">@jaideepvalani</a> thats true,</p>\n<p>I guess you are talking about pixel overlaps in the dataset and that is before inference/testing.<br>\nThis topic has been discussed in these threads,</p>\n<ul>\n<li><p><a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790\" target=\"_blank\">Tips in submission and baseline (in the beginning of the competition)</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/280250\" target=\"_blank\"> Overlaps - ambiguous or not?</a></p></li>\n</ul>\n<p><a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790#1550666\" target=\"_blank\">Source</a>, comment by Sohier Dane[kaggle staff]</p>\n<pre><code>The data description has been updated to clarify that point:\n\nNote: while predictions are not allowed to overlap, the training labels are provided in full (with overlapping portions included). This is to ensure that models are provided the full data for each object. Removing overlap in predictions is a task for the competitor.\n</code></pre>\n<ul>\n<li>the overlap problem(overlap at the time of testing) causes submission error,</li>\n</ul>",
      "rawMarkdown": "HI @jaideepvalani thats true,\n\nI guess you are talking about pixel overlaps in the dataset and that is before inference/testing.\nThis topic has been discussed in these threads,\n\n- [Tips in submission and baseline (in the beginning of the competition)](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790)\n\n- [ Overlaps - ambiguous or not?](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/280250)\n \n[Source](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790#1550666), comment by Sohier Dane[kaggle staff]\n```\nThe data description has been updated to clarify that point:\n\nNote: while predictions are not allowed to overlap, the training labels are provided in full (with overlapping portions included). This is to ensure that models are provided the full data for each object. Removing overlap in predictions is a task for the competitor.\n```\n\n- the overlap problem(overlap at the time of testing) causes submission error,",
      "votes": 1
    },
    {
      "id": 1584389,
      "postDate": "2021-11-16T13:55:19.993Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1584131,
      "author_name": "somuSan",
      "author_url": "",
      "post_date": "2021-11-16T09:43:20.103000",
      "content": "<p>HI <a href=\"https://www.kaggle.com/jaideepvalani\" target=\"_blank\">@jaideepvalani</a> thats true,</p>\n<p>I guess you are talking about pixel overlaps in the dataset and that is before inference/testing.<br>\nThis topic has been discussed in these threads,</p>\n<ul>\n<li><p><a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790\" target=\"_blank\">Tips in submission and baseline (in the beginning of the competition)</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/280250\" target=\"_blank\"> Overlaps - ambiguous or not?</a></p></li>\n</ul>\n<p><a href=\"https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790#1550666\" target=\"_blank\">Source</a>, comment by Sohier Dane[kaggle staff]</p>\n<pre><code>The data description has been updated to clarify that point:\n\nNote: while predictions are not allowed to overlap, the training labels are provided in full (with overlapping portions included). This is to ensure that models are provided the full data for each object. Removing overlap in predictions is a task for the competitor.\n</code></pre>\n<ul>\n<li>the overlap problem(overlap at the time of testing) causes submission error,</li>\n</ul>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1584389,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-11-16T13:55:19.993000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1584032": "I have not fully started on the competition, just doing some round of analysis. May be I am quite late to arrive at this, \nI find there is pixel overlap across the instances\n\neg.\n```\ndef rles_to_mask(encs, shape):\n    \"\"\"\n    Decodes a rle.\n\n    Args:\n        encs (list of str): Rles for each class.\n        shape (tuple [2]): Mask size.\n\n    Returns:\n        np array [shape]: Mask.\n    \"\"\"\n    img = np.zeros(shape[0] * shape[1], dtype=np.uint)\n    mask_sum=np.zeros(len(encs))\n    for m, enc in enumerate(encs):\n         \n        if isinstance(enc, np.float) and np.isnan(enc):\n            continue\n        enc_split = enc.split()\n        for i in range(len(enc_split) // 2):\n            start = int(enc_split[2 * i]) - 1\n            length = int(enc_split[2 * i + 1])\n            img[start: start + length] = 1 + m\n        ##index=np.where(img==1+m)\n            \n        #print(img.sum())     \n        mask_sum[m]=img[np.where(img==1+m)].sum()\n        #break\n    return img.reshape(shape),mask_sum\n```\n```\nshape = df1[['height', 'width']].values[0]\n\nmask=rles_to_mask(df1['annotation'][0],shape)\n```\nmask[0][np.where(mask[0]==1)].sum() !=mask[1][0] #193 vs 198\n\nwhat impact does it has over learning     \n",
    "1584131": "HI @jaideepvalani thats true,\n\nI guess you are talking about pixel overlaps in the dataset and that is before inference/testing.\nThis topic has been discussed in these threads,\n\n- [Tips in submission and baseline (in the beginning of the competition)](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790)\n\n- [ Overlaps - ambiguous or not?](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/280250)\n \n[Source](https://www.kaggle.com/c/sartorius-cell-instance-segmentation/discussion/279790#1550666), comment by Sohier Dane[kaggle staff]\n```\nThe data description has been updated to clarify that point:\n\nNote: while predictions are not allowed to overlap, the training labels are provided in full (with overlapping portions included). This is to ensure that models are provided the full data for each object. Removing overlap in predictions is a task for the competitor.\n```\n\n- the overlap problem(overlap at the time of testing) causes submission error,",
    "1584389": ""
  }
}