{
  "id": 453986,
  "title": "PNG dataset 40G -> 6G",
  "url": "/competitions/blood-vessel-segmentation/discussion/453986",
  "author_name": "",
  "post_date": "2023-11-08T13:43:29.004481500Z",
  "votes": 29,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I converted the train dataset (kidney images and masks) to 8 bit png. This reduces the dataset size from 40G to ~6G. Original image dimensions remain unchanged.</p>\n<p><a href=\"https://www.kaggle.com/datasets/davidbroberts/sennet-hoa-png\" target=\"_blank\">https://www.kaggle.com/datasets/davidbroberts/sennet-hoa-png</a></p>\n<p>The images are a little more human friendly ..</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4945934%2F55961702d0b44f4c230b568ac7013f33%2F1116.png?generation=1699450830274570&amp;alt=media\" alt=\"\"></p>\n<p>I used simple normalization code on the kidney images. This is a lossy operation.</p>\n<pre><code>image = image - np.(image)\nimage = image / np.(image)\nimage = (image*).astype(np.uint8)\n</code></pre>\n<p>I did not manually normalize the mask/label images. They were just converted to 8 bit grayscale.</p>",
  "messages": [
    {
      "id": "2517448",
      "postDate": "11/08/2023 13:43:29",
      "content": "<p>I converted the train dataset (kidney images and masks) to 8 bit png. This reduces the dataset size from 40G to ~6G. Original image dimensions remain unchanged.</p>\n<p><a href=\"https://www.kaggle.com/datasets/davidbroberts/sennet-hoa-png\" target=\"_blank\">https://www.kaggle.com/datasets/davidbroberts/sennet-hoa-png</a></p>\n<p>The images are a little more human friendly ..</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4945934%2F55961702d0b44f4c230b568ac7013f33%2F1116.png?generation=1699450830274570&amp;alt=media\" alt=\"\"></p>\n<p>I used simple normalization code on the kidney images. This is a lossy operation.</p>\n<pre><code>image = image - np.(image)\nimage = image / np.(image)\nimage = (image*).astype(np.uint8)\n</code></pre>\n<p>I did not manually normalize the mask/label images. They were just converted to 8 bit grayscale.</p>",
      "rawMarkdown": "I converted the train dataset (kidney images and masks) to 8 bit png. This reduces the dataset size from 40G to ~6G. Original image dimensions remain unchanged.\n\nhttps://www.kaggle.com/datasets/davidbroberts/sennet-hoa-png\n\nThe images are a little more human friendly ..\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4945934%2F55961702d0b44f4c230b568ac7013f33%2F1116.png?generation=1699450830274570&alt=media)\n\nI used simple normalization code on the kidney images. This is a lossy operation.\n\n```python\nimage = image - np.min(image)\nimage = image / np.max(image)\nimage = (image*255).astype(np.uint8)\n```\nI did not manually normalize the mask/label images. They were just converted to 8 bit grayscale.",
      "votes": null
    },
    {
      "id": "2517478",
      "postDate": "11/08/2023 14:09:19",
      "content": "<p>Thanks for sharing the dataset. Do you think any windowing make sense for this modality?</p>",
      "rawMarkdown": "Thanks for sharing the dataset. Do you think any windowing make sense for this modality?",
      "votes": null
    },
    {
      "id": "2517490",
      "postDate": "11/08/2023 14:17:39",
      "content": "<p>Yes, I think windowing could help here. The original TIF images are 24 bit, so there's a lot of data available. Some amount of windowing or histogram equalization could maximize the contrast .. especially on tiny vessels</p>",
      "rawMarkdown": "Yes, I think windowing could help here. The original TIF images are 24 bit, so there's a lot of data available. Some amount of windowing or histogram equalization could maximize the contrast .. especially on tiny vessels",
      "votes": null
    },
    {
      "id": "2517653",
      "postDate": "11/08/2023 16:49:20",
      "content": "<p>Thank you for making those images \"a little more human friendly\"</p>\n<p>Amazing work (dataset) as always Roberts. </p>",
      "rawMarkdown": "Thank you for making those images \"a little more human friendly\"\n\nAmazing work (dataset) as always Roberts.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2517478,
      "author_name": "gunesevitan",
      "author_url": "",
      "post_date": "11/08/2023 14:09:19",
      "content": "<p>Thanks for sharing the dataset. Do you think any windowing make sense for this modality?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2517490,
          "author_name": "davidbroberts",
          "author_url": "",
          "post_date": "11/08/2023 14:17:39",
          "content": "<p>Yes, I think windowing could help here. The original TIF images are 24 bit, so there's a lot of data available. Some amount of windowing or histogram equalization could maximize the contrast .. especially on tiny vessels</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2517653,
      "author_name": "mpwolke",
      "author_url": "",
      "post_date": "11/08/2023 16:49:20",
      "content": "<p>Thank you for making those images \"a little more human friendly\"</p>\n<p>Amazing work (dataset) as always Roberts. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2517448": "I converted the train dataset (kidney images and masks) to 8 bit png. This reduces the dataset size from 40G to ~6G. Original image dimensions remain unchanged.\n\nhttps://www.kaggle.com/datasets/davidbroberts/sennet-hoa-png\n\nThe images are a little more human friendly ..\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4945934%2F55961702d0b44f4c230b568ac7013f33%2F1116.png?generation=1699450830274570&alt=media)\n\nI used simple normalization code on the kidney images. This is a lossy operation.\n\n```python\nimage = image - np.min(image)\nimage = image / np.max(image)\nimage = (image*255).astype(np.uint8)\n```\nI did not manually normalize the mask/label images. They were just converted to 8 bit grayscale.",
    "2517478": "Thanks for sharing the dataset. Do you think any windowing make sense for this modality?",
    "2517490": "Yes, I think windowing could help here. The original TIF images are 24 bit, so there's a lot of data available. Some amount of windowing or histogram equalization could maximize the contrast .. especially on tiny vessels",
    "2517653": "Thank you for making those images \"a little more human friendly\"\n\nAmazing work (dataset) as always Roberts."
  },
  "source": "meta"
}