{
  "id": 335064,
  "title": "Image Augementation Dataset Available !!",
  "url": "/competitions/hubmap-organ-segmentation/discussion/335064",
  "author_name": "",
  "post_date": "2022-07-04T14:41:03.379028600Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Here is a Dataset Contain 3159 Images Original 3-Channel Image and Mask with 1024x1024 Resoluation</p>\n<p><a href=\"https://www.kaggle.com/datasets/muki2003/hubmaphpa-augmentation-dataset\" target=\"_blank\">https://www.kaggle.com/datasets/muki2003/hubmaphpa-augmentation-dataset</a></p>",
  "messages": [
    {
      "id": "1843095",
      "postDate": "07/04/2022 14:41:03",
      "content": "<p>Here is a Dataset Contain 3159 Images Original 3-Channel Image and Mask with 1024x1024 Resoluation</p>\n<p><a href=\"https://www.kaggle.com/datasets/muki2003/hubmaphpa-augmentation-dataset\" target=\"_blank\">https://www.kaggle.com/datasets/muki2003/hubmaphpa-augmentation-dataset</a></p>",
      "rawMarkdown": "Here is a Dataset Contain 3159 Images Original 3-Channel Image and Mask with 1024x1024 Resoluation\n\nhttps://www.kaggle.com/datasets/muki2003/hubmaphpa-augmentation-dataset",
      "votes": null
    },
    {
      "id": "1844928",
      "postDate": "07/05/2022 23:30:01",
      "content": "<p>Thanks for sharing this dataset. <br>\nHow was the dataset generated? Can you release a notebook?</p>",
      "rawMarkdown": "Thanks for sharing this dataset. \nHow was the dataset generated? Can you release a notebook?",
      "votes": null
    },
    {
      "id": "1845015",
      "postDate": "07/06/2022 01:53:38",
      "content": "<p><a href=\"https://www.kaggle.com/thedevastator\" target=\"_blank\">@thedevastator</a> </p>\n<p>From Below notebook Data auguentation and Preparation section,<br>\n<a href=\"https://www.kaggle.com/code/muki2003/hubmap-hpa-deep-learning-u-net-cnn\" target=\"_blank\">https://www.kaggle.com/code/muki2003/hubmap-hpa-deep-learning-u-net-cnn</a></p>\n<h1>Data Augumentation &amp; Preparation</h1>\n<h4>Making Temp Directory</h4>\n<pre><code>!mkdir -p /kaggle/temp/images\n!mkdir -p /kaggle/temp/masks\n</code></pre>\n<h4>Image Tile Function</h4>\n<pre><code>def tile_image(p_img, folder, size: int = 1024) -&gt; list:\n    w = h = size\n    im = np.array(Image.open(p_img))\n    tiles = [im[i:(i + h), j:(j + w), ...] for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]\n    idxs = [(i, (i + h), j, (j + w)) for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]\n    name, _ = os.path.splitext(os.path.basename(p_img))\n    files = []\n    for k, tile in enumerate(tiles):\n        if tile.shape[:2] != (h, w):\n            tile_ = tile\n            tile = np.zeros_like(tiles[0])\n            tile[:tile_.shape[0], :tile_.shape[1], ...] = tile_\n        p_img = os.path.join(folder, f\"{name}_{k:02}.png\")\n        Image.fromarray(tile).save(p_img)\n        files.append(p_img)\n    return files, idxs\n</code></pre>\n<h4>Tile every Image in orginal dataset</h4>\n<pre><code>TILE_SIZE = 1024\nfor dir_source, dir_target in [(os.path.join(DATASET_FOLDER, 'train_images'), \"/kaggle/temp/images\"),(os.path.join(ANNOT_DATASET, 'train_masks'), \"/kaggle/temp/masks\")]:\n        ls = glob.glob(os.path.join(dir_source, '*'))\n        _ = Parallel(n_jobs=3)(delayed(tile_image)(p_img, dir_target, size=TILE_SIZE) for p_img in tqdm(ls))\n</code></pre>\n<h4>Make an ZIP of augumented Images</h4>\n<pre><code>!zip -r images.zip \"/kaggle/temp/images/\"\n!zip -r mask.zip \"/kaggle/temp/masks/\"\n</code></pre>",
      "rawMarkdown": "thedevastator \n\nFrom Below notebook Data auguentation and Preparation section,\nhttps://www.kaggle.com/code/muki2003/hubmap-hpa-deep-learning-u-net-cnn\n\n# Data Augumentation & Preparation\n\n#### Making Temp Directory\n```\n!mkdir -p /kaggle/temp/images\n!mkdir -p /kaggle/temp/masks\n```\n\n#### Image Tile Function\n```\ndef tile_image(p_img, folder, size: int = 1024) -> list:\n    w = h = size\n    im = np.array(Image.open(p_img))\n    tiles = [im[i:(i + h), j:(j + w), ...] for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]\n    idxs = [(i, (i + h), j, (j + w)) for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]\n    name, _ = os.path.splitext(os.path.basename(p_img))\n    files = []\n    for k, tile in enumerate(tiles):\n        if tile.shape[:2] != (h, w):\n            tile_ = tile\n            tile = np.zeros_like(tiles[0])\n            tile[:tile_.shape[0], :tile_.shape[1], ...] = tile_\n        p_img = os.path.join(folder, f\"{name}_{k:02}.png\")\n        Image.fromarray(tile).save(p_img)\n        files.append(p_img)\n    return files, idxs\n```\n\n#### Tile every Image in orginal dataset\n```\nTILE_SIZE = 1024\nfor dir_source, dir_target in [(os.path.join(DATASET_FOLDER, 'train_images'), \"/kaggle/temp/images\"),(os.path.join(ANNOT_DATASET, 'train_masks'), \"/kaggle/temp/masks\")]:\n        ls = glob.glob(os.path.join(dir_source, '*'))\n        _ = Parallel(n_jobs=3)(delayed(tile_image)(p_img, dir_target, size=TILE_SIZE) for p_img in tqdm(ls))\n```\n \n#### Make an ZIP of augumented Images\n```\n!zip -r images.zip \"/kaggle/temp/images/\"\n!zip -r mask.zip \"/kaggle/temp/masks/\"\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1844928,
      "author_name": "thedevastator",
      "author_url": "",
      "post_date": "07/05/2022 23:30:01",
      "content": "<p>Thanks for sharing this dataset. <br>\nHow was the dataset generated? Can you release a notebook?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1845015,
          "author_name": "muki2003",
          "author_url": "",
          "post_date": "07/06/2022 01:53:38",
          "content": "<p><a href=\"https://www.kaggle.com/thedevastator\" target=\"_blank\">@thedevastator</a> </p>\n<p>From Below notebook Data auguentation and Preparation section,<br>\n<a href=\"https://www.kaggle.com/code/muki2003/hubmap-hpa-deep-learning-u-net-cnn\" target=\"_blank\">https://www.kaggle.com/code/muki2003/hubmap-hpa-deep-learning-u-net-cnn</a></p>\n<h1>Data Augumentation &amp; Preparation</h1>\n<h4>Making Temp Directory</h4>\n<pre><code>!mkdir -p /kaggle/temp/images\n!mkdir -p /kaggle/temp/masks\n</code></pre>\n<h4>Image Tile Function</h4>\n<pre><code>def tile_image(p_img, folder, size: int = 1024) -&gt; list:\n    w = h = size\n    im = np.array(Image.open(p_img))\n    tiles = [im[i:(i + h), j:(j + w), ...] for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]\n    idxs = [(i, (i + h), j, (j + w)) for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]\n    name, _ = os.path.splitext(os.path.basename(p_img))\n    files = []\n    for k, tile in enumerate(tiles):\n        if tile.shape[:2] != (h, w):\n            tile_ = tile\n            tile = np.zeros_like(tiles[0])\n            tile[:tile_.shape[0], :tile_.shape[1], ...] = tile_\n        p_img = os.path.join(folder, f\"{name}_{k:02}.png\")\n        Image.fromarray(tile).save(p_img)\n        files.append(p_img)\n    return files, idxs\n</code></pre>\n<h4>Tile every Image in orginal dataset</h4>\n<pre><code>TILE_SIZE = 1024\nfor dir_source, dir_target in [(os.path.join(DATASET_FOLDER, 'train_images'), \"/kaggle/temp/images\"),(os.path.join(ANNOT_DATASET, 'train_masks'), \"/kaggle/temp/masks\")]:\n        ls = glob.glob(os.path.join(dir_source, '*'))\n        _ = Parallel(n_jobs=3)(delayed(tile_image)(p_img, dir_target, size=TILE_SIZE) for p_img in tqdm(ls))\n</code></pre>\n<h4>Make an ZIP of augumented Images</h4>\n<pre><code>!zip -r images.zip \"/kaggle/temp/images/\"\n!zip -r mask.zip \"/kaggle/temp/masks/\"\n</code></pre>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1843095": "Here is a Dataset Contain 3159 Images Original 3-Channel Image and Mask with 1024x1024 Resoluation\n\nhttps://www.kaggle.com/datasets/muki2003/hubmaphpa-augmentation-dataset",
    "1844928": "Thanks for sharing this dataset. \nHow was the dataset generated? Can you release a notebook?",
    "1845015": "thedevastator \n\nFrom Below notebook Data auguentation and Preparation section,\nhttps://www.kaggle.com/code/muki2003/hubmap-hpa-deep-learning-u-net-cnn\n\n# Data Augumentation & Preparation\n\n#### Making Temp Directory\n```\n!mkdir -p /kaggle/temp/images\n!mkdir -p /kaggle/temp/masks\n```\n\n#### Image Tile Function\n```\ndef tile_image(p_img, folder, size: int = 1024) -> list:\n    w = h = size\n    im = np.array(Image.open(p_img))\n    tiles = [im[i:(i + h), j:(j + w), ...] for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]\n    idxs = [(i, (i + h), j, (j + w)) for i in range(0, im.shape[0], h) for j in range(0, im.shape[1], w)]\n    name, _ = os.path.splitext(os.path.basename(p_img))\n    files = []\n    for k, tile in enumerate(tiles):\n        if tile.shape[:2] != (h, w):\n            tile_ = tile\n            tile = np.zeros_like(tiles[0])\n            tile[:tile_.shape[0], :tile_.shape[1], ...] = tile_\n        p_img = os.path.join(folder, f\"{name}_{k:02}.png\")\n        Image.fromarray(tile).save(p_img)\n        files.append(p_img)\n    return files, idxs\n```\n\n#### Tile every Image in orginal dataset\n```\nTILE_SIZE = 1024\nfor dir_source, dir_target in [(os.path.join(DATASET_FOLDER, 'train_images'), \"/kaggle/temp/images\"),(os.path.join(ANNOT_DATASET, 'train_masks'), \"/kaggle/temp/masks\")]:\n        ls = glob.glob(os.path.join(dir_source, '*'))\n        _ = Parallel(n_jobs=3)(delayed(tile_image)(p_img, dir_target, size=TILE_SIZE) for p_img in tqdm(ls))\n```\n \n#### Make an ZIP of augumented Images\n```\n!zip -r images.zip \"/kaggle/temp/images/\"\n!zip -r mask.zip \"/kaggle/temp/masks/\"\n```"
  },
  "source": "meta"
}