{
  "id": 339431,
  "title": "RLE Encode fast function",
  "url": "/competitions/hubmap-organ-segmentation/discussion/339431",
  "author_name": "",
  "post_date": "2022-07-24T19:09:59.985608400Z",
  "votes": 7,
  "comment_count": 2,
  "views": 0,
  "content": "<p>In this competition, I wanted to use the encode_rle function written by other users instead of the naive, very slow one, which does not use numpy features.<br>\nHowever, they give an error related to dimensions, and do not allow you to set a threshold of pixel intensity that satisfy the mask.</p>\n<p>Therefore, I wrote a function suitable for images of any dimension and allowing you to set the threshold of pixel intensity that will be included in the mask.</p>\n<p><strong>Here is quick explanation:</strong></p>\n<p>To find a sequence of suitable pixels, we will look for its beginning and end</p>\n<p>Let's add zeros to the beginning and end so that the zero and last pixel are also considered as the beginning and end</p>\n<p>Beginnings:</p>\n<p>A pixel will be the beginning if it is not less than the intensity threshold, and the previous one is strictly less.<br>\nLets shift the pixels one to the right and compare</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2Fc09d982a4d656fbf04bd363d9c8d47cb%2Fexplanation.png?generation=1658685975203166&amp;alt=media\" alt=\"\"></p>\n<p>Ends are made in the same way by shifting to the left.</p>\n<p>Here s code:</p>\n<pre><code>def rle_encode(img, threshold=0.5):\n    img = img.T\n    pixels = np.concatenate((np.array([0]), img.flatten(), np.array([0])))\n    beginings = np.where( (pixels &gt;= threshold) &amp; (np.concatenate((np.array([2]), pixels[:-1])) &lt; threshold) )[0]\n    ends = np.where( (pixels &gt;= threshold) &amp; (np.concatenate((pixels[1:], np.array([2]))) &lt; threshold) )[0]\n\n    rle = ''\n    for begin, end in zip(beginings, ends):\n        rle += str(begin-1) + ' ' + str(end - begin + 1) + ' '\n    return rle[:-1]\n</code></pre>\n<p><strong>Usage</strong></p>\n<p>Predicted mask / After threshold / Target</p>\n<p><strong><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2F185a2f1e8d993084e6dba756678e7c5e%2F1.png?generation=1658689516580785&amp;alt=media\" alt=\"\"></strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2Fdb515cf9b653572a2f217ed510905cb2%2F2.png?generation=1658689586820255&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2F8687b7edf80cefba50de3a47a1f720d0%2F3.png?generation=1658689616953200&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": "1869478",
      "postDate": "07/24/2022 19:09:59",
      "content": "<p>In this competition, I wanted to use the encode_rle function written by other users instead of the naive, very slow one, which does not use numpy features.<br>\nHowever, they give an error related to dimensions, and do not allow you to set a threshold of pixel intensity that satisfy the mask.</p>\n<p>Therefore, I wrote a function suitable for images of any dimension and allowing you to set the threshold of pixel intensity that will be included in the mask.</p>\n<p><strong>Here is quick explanation:</strong></p>\n<p>To find a sequence of suitable pixels, we will look for its beginning and end</p>\n<p>Let's add zeros to the beginning and end so that the zero and last pixel are also considered as the beginning and end</p>\n<p>Beginnings:</p>\n<p>A pixel will be the beginning if it is not less than the intensity threshold, and the previous one is strictly less.<br>\nLets shift the pixels one to the right and compare</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2Fc09d982a4d656fbf04bd363d9c8d47cb%2Fexplanation.png?generation=1658685975203166&amp;alt=media\" alt=\"\"></p>\n<p>Ends are made in the same way by shifting to the left.</p>\n<p>Here s code:</p>\n<pre><code>def rle_encode(img, threshold=0.5):\n    img = img.T\n    pixels = np.concatenate((np.array([0]), img.flatten(), np.array([0])))\n    beginings = np.where( (pixels &gt;= threshold) &amp; (np.concatenate((np.array([2]), pixels[:-1])) &lt; threshold) )[0]\n    ends = np.where( (pixels &gt;= threshold) &amp; (np.concatenate((pixels[1:], np.array([2]))) &lt; threshold) )[0]\n\n    rle = ''\n    for begin, end in zip(beginings, ends):\n        rle += str(begin-1) + ' ' + str(end - begin + 1) + ' '\n    return rle[:-1]\n</code></pre>\n<p><strong>Usage</strong></p>\n<p>Predicted mask / After threshold / Target</p>\n<p><strong><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2F185a2f1e8d993084e6dba756678e7c5e%2F1.png?generation=1658689516580785&amp;alt=media\" alt=\"\"></strong></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2Fdb515cf9b653572a2f217ed510905cb2%2F2.png?generation=1658689586820255&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2F8687b7edf80cefba50de3a47a1f720d0%2F3.png?generation=1658689616953200&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "In this competition, I wanted to use the encode_rle function written by other users instead of the naive, very slow one, which does not use numpy features.\nHowever, they give an error related to dimensions, and do not allow you to set a threshold of pixel intensity that satisfy the mask.\n\nTherefore, I wrote a function suitable for images of any dimension and allowing you to set the threshold of pixel intensity that will be included in the mask.\n\n**Here is quick explanation:**\n\nTo find a sequence of suitable pixels, we will look for its beginning and end\n\nLet's add zeros to the beginning and end so that the zero and last pixel are also considered as the beginning and end\n\nBeginnings:\n\nA pixel will be the beginning if it is not less than the intensity threshold, and the previous one is strictly less.\nLets shift the pixels one to the right and compare\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2Fc09d982a4d656fbf04bd363d9c8d47cb%2Fexplanation.png?generation=1658685975203166&alt=media)\n\nEnds are made in the same way by shifting to the left.\n\nHere s code:\n\n```\ndef rle_encode(img, threshold=0.5):\n    img = img.T\n    pixels = np.concatenate((np.array([0]), img.flatten(), np.array([0])))\n    beginings = np.where( (pixels >= threshold) & (np.concatenate((np.array([2]), pixels[:-1])) < threshold) )[0]\n    ends = np.where( (pixels >= threshold) & (np.concatenate((pixels[1:], np.array([2]))) < threshold) )[0]\n    \n    rle = ''\n    for begin, end in zip(beginings, ends):\n        rle += str(begin-1) + ' ' + str(end - begin + 1) + ' '\n    return rle[:-1]\n```\n\n**Usage**\n\nPredicted mask / After threshold / Target\n\n**![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2F185a2f1e8d993084e6dba756678e7c5e%2F1.png?generation=1658689516580785&alt=media)**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2Fdb515cf9b653572a2f217ed510905cb2%2F2.png?generation=1658689586820255&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2F8687b7edf80cefba50de3a47a1f720d0%2F3.png?generation=1658689616953200&alt=media)",
      "votes": null
    },
    {
      "id": "1870155",
      "postDate": "07/25/2022 10:25:57",
      "content": "<p><a href=\"https://www.kaggle.com/pear2jam\" target=\"_blank\">@pear2jam</a> Thank you for sharing ! That really helps in case of different image shapes &amp; intensity of pixel !</p>",
      "rawMarkdown": "pear2jam Thank you for sharing ! That really helps in case of different image shapes & intensity of pixel !",
      "votes": null
    },
    {
      "id": "1871504",
      "postDate": "07/26/2022 09:58:43",
      "content": "<p>Thank you for sharing your code!<br>\nI think this will be really helpful and might be a quick trick for boosting your score.</p>\n<p>Great work!</p>\n<p>The Devastator.</p>",
      "rawMarkdown": "Thank you for sharing your code!\nI think this will be really helpful and might be a quick trick for boosting your score.\n\nGreat work!\n\nThe Devastator.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1870155,
      "author_name": "arunpurakkatt",
      "author_url": "",
      "post_date": "07/25/2022 10:25:57",
      "content": "<p><a href=\"https://www.kaggle.com/pear2jam\" target=\"_blank\">@pear2jam</a> Thank you for sharing ! That really helps in case of different image shapes &amp; intensity of pixel !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1871504,
      "author_name": "thedevastator",
      "author_url": "",
      "post_date": "07/26/2022 09:58:43",
      "content": "<p>Thank you for sharing your code!<br>\nI think this will be really helpful and might be a quick trick for boosting your score.</p>\n<p>Great work!</p>\n<p>The Devastator.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1869478": "In this competition, I wanted to use the encode_rle function written by other users instead of the naive, very slow one, which does not use numpy features.\nHowever, they give an error related to dimensions, and do not allow you to set a threshold of pixel intensity that satisfy the mask.\n\nTherefore, I wrote a function suitable for images of any dimension and allowing you to set the threshold of pixel intensity that will be included in the mask.\n\n**Here is quick explanation:**\n\nTo find a sequence of suitable pixels, we will look for its beginning and end\n\nLet's add zeros to the beginning and end so that the zero and last pixel are also considered as the beginning and end\n\nBeginnings:\n\nA pixel will be the beginning if it is not less than the intensity threshold, and the previous one is strictly less.\nLets shift the pixels one to the right and compare\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2Fc09d982a4d656fbf04bd363d9c8d47cb%2Fexplanation.png?generation=1658685975203166&alt=media)\n\nEnds are made in the same way by shifting to the left.\n\nHere s code:\n\n```\ndef rle_encode(img, threshold=0.5):\n    img = img.T\n    pixels = np.concatenate((np.array([0]), img.flatten(), np.array([0])))\n    beginings = np.where( (pixels >= threshold) & (np.concatenate((np.array([2]), pixels[:-1])) < threshold) )[0]\n    ends = np.where( (pixels >= threshold) & (np.concatenate((pixels[1:], np.array([2]))) < threshold) )[0]\n    \n    rle = ''\n    for begin, end in zip(beginings, ends):\n        rle += str(begin-1) + ' ' + str(end - begin + 1) + ' '\n    return rle[:-1]\n```\n\n**Usage**\n\nPredicted mask / After threshold / Target\n\n**![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2F185a2f1e8d993084e6dba756678e7c5e%2F1.png?generation=1658689516580785&alt=media)**\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2Fdb515cf9b653572a2f217ed510905cb2%2F2.png?generation=1658689586820255&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6252643%2F8687b7edf80cefba50de3a47a1f720d0%2F3.png?generation=1658689616953200&alt=media)",
    "1870155": "pear2jam Thank you for sharing ! That really helps in case of different image shapes & intensity of pixel !",
    "1871504": "Thank you for sharing your code!\nI think this will be really helpful and might be a quick trick for boosting your score.\n\nGreat work!\n\nThe Devastator."
  },
  "source": "meta"
}