{ "id": 332838, "title": "Understanding run-length encoding (rle) and dice", "url": "/competitions/hubmap-organ-segmentation/discussion/332838", "author_name": "", "post_date": "2022-06-23T14:49:19.000019Z", "votes": 11, "comment_count": 1, "views": 0, "content": "

For this segmentation challenge, it is important to understand run-length encoding (what we are predicting) and dice (how our predictions are being evaluated).

\n

What is rle

\n

Run-length encoding or rle is a compressed way of defining the pixels we are supposed to predict.
\nThe first number is the pixel to start at, and the second number is how many pixels to run (how many after are included).
\nFor a rle with more than two numbers, repeat the pattern described above every 2 numbers

\n

Example 1:
\nrle: 1 4
\nPixels model should predict: {1, 2, 3, 4}

\n

Example 2:
\nrle: 1 4 9 3
\nPixels model should predict: {1, 2, 3, 4, 9, 10, 11}

\n

rle to mask & mask to rle:

\n

The mask is a binary representation of the image you are trying to predict in which the pixels with a value of 1 are the pixels you are trying to predict/segment.
\nThe mask will likely be the form you train and predict your model with, and you will convert to rle before submitting.

\n

Pipeline: rle -> mask -> train_model(imgs, masks) -> predict_mode(imgs) -> masks -> rle

\n

Here are some helper functions from this resource: https://www.kaggle.com/paulorzp/rle-functions-run-lenght-encode-decode

\n
def mask2rle(img):\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated\n    '''\n    pixels= img.T.flatten()\n    pixels = np.concatenate([[0], pixels, [0]])\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n\ndef rle2mask(mask_rle, shape=(1600,256)):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (width,height) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n    for lo, hi in zip(starts, ends):\n        img[lo:hi] = 1\n    return img.reshape(shape).T\n
\n

Dice

\n

The evaluation metric for this competition is dice

\n

Dice = (2 * | x intersection y|) / (|x| + |y|)
\nWhere |x| and |y| are the cardinalities of the two sets (i.e. the number of elements in each set).

\n

When applied to Boolean data, you can also think of it as
\nDice = (2 * TP) / ( 2 * TP + FP + FN)

\n

More info

\n

segmentation_models_pytorch has an implementation that you can import (raw code here)

", "messages": [ { "id": "1830630", "postDate": "06/23/2022 14:49:19", "content": "

For this segmentation challenge, it is important to understand run-length encoding (what we are predicting) and dice (how our predictions are being evaluated).

\n

What is rle

\n

Run-length encoding or rle is a compressed way of defining the pixels we are supposed to predict.
\nThe first number is the pixel to start at, and the second number is how many pixels to run (how many after are included).
\nFor a rle with more than two numbers, repeat the pattern described above every 2 numbers

\n

Example 1:
\nrle: 1 4
\nPixels model should predict: {1, 2, 3, 4}

\n

Example 2:
\nrle: 1 4 9 3
\nPixels model should predict: {1, 2, 3, 4, 9, 10, 11}

\n

rle to mask & mask to rle:

\n

The mask is a binary representation of the image you are trying to predict in which the pixels with a value of 1 are the pixels you are trying to predict/segment.
\nThe mask will likely be the form you train and predict your model with, and you will convert to rle before submitting.

\n

Pipeline: rle -> mask -> train_model(imgs, masks) -> predict_mode(imgs) -> masks -> rle

\n

Here are some helper functions from this resource: https://www.kaggle.com/paulorzp/rle-functions-run-lenght-encode-decode

\n
def mask2rle(img):\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated\n    '''\n    pixels= img.T.flatten()\n    pixels = np.concatenate([[0], pixels, [0]])\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n\ndef rle2mask(mask_rle, shape=(1600,256)):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (width,height) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n    for lo, hi in zip(starts, ends):\n        img[lo:hi] = 1\n    return img.reshape(shape).T\n
\n

Dice

\n

The evaluation metric for this competition is dice

\n

Dice = (2 * | x intersection y|) / (|x| + |y|)
\nWhere |x| and |y| are the cardinalities of the two sets (i.e. the number of elements in each set).

\n

When applied to Boolean data, you can also think of it as
\nDice = (2 * TP) / ( 2 * TP + FP + FN)

\n

More info

\n

segmentation_models_pytorch has an implementation that you can import (raw code here)

", "rawMarkdown": "For this segmentation challenge, it is important to understand run-length encoding (what we are predicting) and dice (how our predictions are being evaluated).\n\n**What is rle**\n\nRun-length encoding or rle is a compressed way of defining the pixels we are supposed to predict. \nThe first number is the pixel to start at, and the second number is how many pixels to run (how many after are included). \nFor a rle with more than two numbers, repeat the pattern described above every 2 numbers\n\nExample 1:\nrle: 1 4\nPixels model should predict: {1, 2, 3, 4}\n \nExample 2:\nrle: 1 4 9 3\nPixels model should predict: {1, 2, 3, 4, 9, 10, 11}\n\n**rle to mask & mask to rle:**\n\nThe mask is a binary representation of the image you are trying to predict in which the pixels with a value of 1 are the pixels you are trying to predict/segment. \nThe mask will likely be the form you train and predict your model with, and you will convert to rle before submitting.\n\nPipeline: rle -> mask -> train_model(imgs, masks) -> predict_mode(imgs) -> masks -> rle\n\nHere are some helper functions from this resource: https://www.kaggle.com/paulorzp/rle-functions-run-lenght-encode-decode \n\n```\ndef mask2rle(img):\n '''\n img: numpy array, 1 - mask, 0 - background\n Returns run length as string formated\n '''\n pixels= img.T.flatten()\n pixels = np.concatenate([[0], pixels, [0]])\n runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n runs[1::2] -= runs[::2]\n return ' '.join(str(x) for x in runs)\n \ndef rle2mask(mask_rle, shape=(1600,256)):\n '''\n mask_rle: run-length as string formated (start length)\n shape: (width,height) of array to return \n Returns numpy array, 1 - mask, 0 - background\n\n '''\n s = mask_rle.split()\n starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n starts -= 1\n ends = starts + lengths\n img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n for lo, hi in zip(starts, ends):\n img[lo:hi] = 1\n return img.reshape(shape).T\n\n```\n\n**Dice**\n\nThe evaluation metric for this competition is dice\n\nDice = (2 * | x intersection y|) / (|x| + |y|)\nWhere |x| and |y| are the cardinalities of the two sets (i.e. the number of elements in each set).\n\nWhen applied to Boolean data, you can also think of it as\nDice = (2 * TP) / ( 2 * TP + FP + FN)\n\n[More info](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient )\n\nsegmentation_models_pytorch has an implementation that you can import ([raw code here](https://github.com/qubvel/segmentation_models.pytorch/blob/master/segmentation_models_pytorch/losses/dice.py))", "votes": null }, { "id": "1835023", "postDate": "06/27/2022 11:42:13", "content": "

Hi, I have done the RLE conversion and export as bitmap in the following notebooks:

\n
\n

https://www.kaggle.com/code/jirkaborovec/ftus-segm-eda-export-rle-mask

\n
", "rawMarkdown": "Hi, I have done the RLE conversion and export as bitmap in the following notebooks:\n> https://www.kaggle.com/code/jirkaborovec/ftus-segm-eda-export-rle-mask", "votes": null } ], "comments": [ { "id": 1835023, "author_name": "jirkaborovec", "author_url": "", "post_date": "06/27/2022 11:42:13", "content": "

Hi, I have done the RLE conversion and export as bitmap in the following notebooks:

\n
\n

https://www.kaggle.com/code/jirkaborovec/ftus-segm-eda-export-rle-mask

\n
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1830630": "For this segmentation challenge, it is important to understand run-length encoding (what we are predicting) and dice (how our predictions are being evaluated).\n\n**What is rle**\n\nRun-length encoding or rle is a compressed way of defining the pixels we are supposed to predict. \nThe first number is the pixel to start at, and the second number is how many pixels to run (how many after are included). \nFor a rle with more than two numbers, repeat the pattern described above every 2 numbers\n\nExample 1:\nrle: 1 4\nPixels model should predict: {1, 2, 3, 4}\n \nExample 2:\nrle: 1 4 9 3\nPixels model should predict: {1, 2, 3, 4, 9, 10, 11}\n\n**rle to mask & mask to rle:**\n\nThe mask is a binary representation of the image you are trying to predict in which the pixels with a value of 1 are the pixels you are trying to predict/segment. \nThe mask will likely be the form you train and predict your model with, and you will convert to rle before submitting.\n\nPipeline: rle -> mask -> train_model(imgs, masks) -> predict_mode(imgs) -> masks -> rle\n\nHere are some helper functions from this resource: https://www.kaggle.com/paulorzp/rle-functions-run-lenght-encode-decode \n\n```\ndef mask2rle(img):\n '''\n img: numpy array, 1 - mask, 0 - background\n Returns run length as string formated\n '''\n pixels= img.T.flatten()\n pixels = np.concatenate([[0], pixels, [0]])\n runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n runs[1::2] -= runs[::2]\n return ' '.join(str(x) for x in runs)\n \ndef rle2mask(mask_rle, shape=(1600,256)):\n '''\n mask_rle: run-length as string formated (start length)\n shape: (width,height) of array to return \n Returns numpy array, 1 - mask, 0 - background\n\n '''\n s = mask_rle.split()\n starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n starts -= 1\n ends = starts + lengths\n img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n for lo, hi in zip(starts, ends):\n img[lo:hi] = 1\n return img.reshape(shape).T\n\n```\n\n**Dice**\n\nThe evaluation metric for this competition is dice\n\nDice = (2 * | x intersection y|) / (|x| + |y|)\nWhere |x| and |y| are the cardinalities of the two sets (i.e. the number of elements in each set).\n\nWhen applied to Boolean data, you can also think of it as\nDice = (2 * TP) / ( 2 * TP + FP + FN)\n\n[More info](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient )\n\nsegmentation_models_pytorch has an implementation that you can import ([raw code here](https://github.com/qubvel/segmentation_models.pytorch/blob/master/segmentation_models_pytorch/losses/dice.py))", "1835023": "Hi, I have done the RLE conversion and export as bitmap in the following notebooks:\n> https://www.kaggle.com/code/jirkaborovec/ftus-segm-eda-export-rle-mask" }, "source": "meta" }