{ "id": 210111, "title": "Reading the TIFF images", "url": "/competitions/hubmap-kidney-segmentation/discussion/210111", "author_name": "", "post_date": "2021-01-09T17:56:30.389083Z", "votes": 5, "comment_count": 6, "views": 0, "content": "

Since the images are very large, it seems impossible to use them as is.

\n

As one possible solution, @iafoss shares a notebook to extract 256 by 256 tiles from the original images: https://www.kaggle.com/iafoss/256x256-images.

\n

For exploration, I also suggest to use gimp to open the TIFF original images: you need enough RAM otherwise it won't open unfortunately.

\n

Indeed, the 0486052bb.tiff image is 4.2GB on disk and thus more once opened. It is around 35k x 26k in size with RGB 8-bit gamma integer colors.

\n

Here is how it looks like once opened:\"\"

\n

You can also most likely use the HuBMAP data portal dataset exploration tool. Here is an example:

\n

\"\"

", "messages": [ { "id": "1146344", "postDate": "01/09/2021 17:56:30", "content": "

Since the images are very large, it seems impossible to use them as is.

\n

As one possible solution, @iafoss shares a notebook to extract 256 by 256 tiles from the original images: https://www.kaggle.com/iafoss/256x256-images.

\n

For exploration, I also suggest to use gimp to open the TIFF original images: you need enough RAM otherwise it won't open unfortunately.

\n

Indeed, the 0486052bb.tiff image is 4.2GB on disk and thus more once opened. It is around 35k x 26k in size with RGB 8-bit gamma integer colors.

\n

Here is how it looks like once opened:\"\"

\n

You can also most likely use the HuBMAP data portal dataset exploration tool. Here is an example:

\n

\"\"

", "rawMarkdown": "Since the **images are very large**, it seems impossible to use them as is. \n\nAs one possible solution, @iafoss shares a notebook to extract **256 by 256** tiles from the original images: https://www.kaggle.com/iafoss/256x256-images.\n\nFor exploration, I also suggest to use gimp to open the [**TIFF**](https://en.wikipedia.org/wiki/TIFF) original images: you need enough RAM otherwise it won't open unfortunately. \n\nIndeed, the **0486052bb.tiff** image is **4.2GB** on disk and thus more once opened. It is around **35k x 26k** in size with RGB 8-bit gamma integer colors. \n\nHere is how it looks like once opened:![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F0dc07ac5d657b268f94ff992f61ed804%2Ftiff_sample.png?generation=1610214920485924&alt=media)\n\nYou can also most likely use the HuBMAP data portal dataset exploration tool. Here is an example: \n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F5548adb3e616fcba18dbb2773df38d15%2Fhubmap_portal.png?generation=1610283730349174&alt=media)", "votes": null }, { "id": "1146830", "postDate": "01/10/2021 05:04:07", "content": "

@yassinealouini This was explained nicely. Thanks for sharing

", "rawMarkdown": "yassinealouini This was explained nicely. Thanks for sharing", "votes": null }, { "id": "1146927", "postDate": "01/10/2021 07:11:49", "content": "

I am glad it helps at least another person than me. :D

", "rawMarkdown": "I am glad it helps at least another person than me. :D", "votes": null }, { "id": "1147620", "postDate": "01/10/2021 16:03:14", "content": "

Another good solution is to use gdal (to install it, the easiest way is conda install gdal):

\n
from osgeo import gdal\nimport numpy as np\nimport matplotlib.pylab as plt\n\nsample_tiff = gdal.Open(\"2f6ecfcdf.tiff\")\nimage = np.array(sample_tiff.ReadAsArray())\n\n# Need to transpose since channels should be last when plotting\nimage = image.transpose((1,2,0))\nplt.imshow(image)\n
\n

Here is what to expect:

\n

\"\"

", "rawMarkdown": "Another good solution is to use `gdal` (to install it, the easiest way is conda install gdal): \n\n``` \nfrom osgeo import gdal\nimport numpy as np\nimport matplotlib.pylab as plt\n\nsample_tiff = gdal.Open(\"2f6ecfcdf.tiff\")\nimage = np.array(sample_tiff.ReadAsArray())\n\n# Need to transpose since channels should be last when plotting\nimage = image.transpose((1,2,0))\nplt.imshow(image)\n```\n\n\nHere is what to expect:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2Fed8228558d3db35b82d9b9fc50cc73c5%2Ftiff_gdal_sample.png?generation=1610294585802506&alt=media)", "votes": null }, { "id": "1148353", "postDate": "01/11/2021 04:57:33", "content": "

I learn a lot from your post

", "rawMarkdown": "I learn a lot from your post", "votes": null }, { "id": "1149294", "postDate": "01/11/2021 18:37:07", "content": "

That's the intent, thanks!

", "rawMarkdown": "That's the intent, thanks!", "votes": null }, { "id": "1157011", "postDate": "01/17/2021 15:27:08", "content": "

Yet another solution, using rasterio:

\n
import rasterio\nimport numpy as np\nfrom rasterio.plot import show\n\nimg_id = \"1e2425f28\"\nraster = rasterio.open(f\"/hdd/hubmap-kidney-segmentation/train/{img_id}.tiff\")\nfig, ax = plt.subplots(1, 1, figsize=(50, 50))\nshow(raster, ax=ax)\n
", "rawMarkdown": "Yet another solution, using [rasterio](https://rasterio.readthedocs.io/en/latest/): \n\n\n```\nimport rasterio\nimport numpy as np\nfrom rasterio.plot import show\n\nimg_id = \"1e2425f28\"\nraster = rasterio.open(f\"/hdd/hubmap-kidney-segmentation/train/{img_id}.tiff\")\nfig, ax = plt.subplots(1, 1, figsize=(50, 50))\nshow(raster, ax=ax)\n```", "votes": null } ], "comments": [ { "id": 1146830, "author_name": "balaramk", "author_url": "", "post_date": "01/10/2021 05:04:07", "content": "

@yassinealouini This was explained nicely. Thanks for sharing

", "votes": null, "replies": [ { "id": 1146927, "author_name": "yassinealouini", "author_url": "", "post_date": "01/10/2021 07:11:49", "content": "

I am glad it helps at least another person than me. :D

", "votes": null, "replies": [] } ] }, { "id": 1147620, "author_name": "yassinealouini", "author_url": "", "post_date": "01/10/2021 16:03:14", "content": "

Another good solution is to use gdal (to install it, the easiest way is conda install gdal):

\n
from osgeo import gdal\nimport numpy as np\nimport matplotlib.pylab as plt\n\nsample_tiff = gdal.Open(\"2f6ecfcdf.tiff\")\nimage = np.array(sample_tiff.ReadAsArray())\n\n# Need to transpose since channels should be last when plotting\nimage = image.transpose((1,2,0))\nplt.imshow(image)\n
\n

Here is what to expect:

\n

\"\"

", "votes": null, "replies": [] }, { "id": 1148353, "author_name": "datawarriors", "author_url": "", "post_date": "01/11/2021 04:57:33", "content": "

I learn a lot from your post

", "votes": null, "replies": [ { "id": 1149294, "author_name": "yassinealouini", "author_url": "", "post_date": "01/11/2021 18:37:07", "content": "

That's the intent, thanks!

", "votes": null, "replies": [] } ] }, { "id": 1157011, "author_name": "yassinealouini", "author_url": "", "post_date": "01/17/2021 15:27:08", "content": "

Yet another solution, using rasterio:

\n
import rasterio\nimport numpy as np\nfrom rasterio.plot import show\n\nimg_id = \"1e2425f28\"\nraster = rasterio.open(f\"/hdd/hubmap-kidney-segmentation/train/{img_id}.tiff\")\nfig, ax = plt.subplots(1, 1, figsize=(50, 50))\nshow(raster, ax=ax)\n
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1146344": "Since the **images are very large**, it seems impossible to use them as is. \n\nAs one possible solution, @iafoss shares a notebook to extract **256 by 256** tiles from the original images: https://www.kaggle.com/iafoss/256x256-images.\n\nFor exploration, I also suggest to use gimp to open the [**TIFF**](https://en.wikipedia.org/wiki/TIFF) original images: you need enough RAM otherwise it won't open unfortunately. \n\nIndeed, the **0486052bb.tiff** image is **4.2GB** on disk and thus more once opened. It is around **35k x 26k** in size with RGB 8-bit gamma integer colors. \n\nHere is how it looks like once opened:![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F0dc07ac5d657b268f94ff992f61ed804%2Ftiff_sample.png?generation=1610214920485924&alt=media)\n\nYou can also most likely use the HuBMAP data portal dataset exploration tool. Here is an example: \n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F5548adb3e616fcba18dbb2773df38d15%2Fhubmap_portal.png?generation=1610283730349174&alt=media)", "1146830": "yassinealouini This was explained nicely. Thanks for sharing", "1146927": "I am glad it helps at least another person than me. :D", "1147620": "Another good solution is to use `gdal` (to install it, the easiest way is conda install gdal): \n\n``` \nfrom osgeo import gdal\nimport numpy as np\nimport matplotlib.pylab as plt\n\nsample_tiff = gdal.Open(\"2f6ecfcdf.tiff\")\nimage = np.array(sample_tiff.ReadAsArray())\n\n# Need to transpose since channels should be last when plotting\nimage = image.transpose((1,2,0))\nplt.imshow(image)\n```\n\n\nHere is what to expect:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2Fed8228558d3db35b82d9b9fc50cc73c5%2Ftiff_gdal_sample.png?generation=1610294585802506&alt=media)", "1148353": "I learn a lot from your post", "1149294": "That's the intent, thanks!", "1157011": "Yet another solution, using [rasterio](https://rasterio.readthedocs.io/en/latest/): \n\n\n```\nimport rasterio\nimport numpy as np\nfrom rasterio.plot import show\n\nimg_id = \"1e2425f28\"\nraster = rasterio.open(f\"/hdd/hubmap-kidney-segmentation/train/{img_id}.tiff\")\nfig, ax = plt.subplots(1, 1, figsize=(50, 50))\nshow(raster, ax=ax)\n```" }, "source": "meta" }