![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2605845%2Fd081228f7cd5a1ba5618be82ceaaf306%2FQGIS_intro.png?generation=1611392127994602&alt=media\")
Hello!
\nI've joined this competition after the initial first wave of notebooks with submissions, and I was quite surprised to see people using rasterio of all to handle the images. It's a great library, but it is mostly used for geospatial data (understand satellite / drone / airborne imagery). That's why I do like Kaggle so much, you get to see tools be used in a different way!
Anyways this got me thinking, why not use more of the tools I use daily to work with large geopsatial data?
\nOne of the best visualization tools out there is called QGIS, and it's basically a GUI for exploring any type of geospatial data. It's a free and open source project, and works very nicely for this competition.
\nYou can open the software (works like a charm on any of the big 3 OS) and simply drag and drop the images in their (this things reads a lot of formats, and is pretty much built for .tiff files). It takes a few seconds to load the image (we all know these images are quite big), but then you can pan around, zoom in and out very easily:
You can also then save your predictions to a .tif, also drag and drop then, and change the transparency of that layer by Right Clicking on the name of the Layer -> Properties -> Transparency and sliding the transparency down.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2605845%2Fdfb29c9b6490be21da301a750cd2e60d%2Fpreds_overlay.png?generation=1611392091171926&alt=media\")
This was a very quick way to evaluate that my prediction on 256x256 tiles suffered in tiles that only contained a little bit of the area needed to be assessed, but not covering enough of the tiles so my model doesn't see them, it becomes very visible in a few seconds by panning through the image:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2605845%2Fe57be497ee118c12c6ab909a29aa97cb%2Fzoom_pred.png?generation=1611392400877736&alt=media\")
You could do all of this in Python, but arguably this is much quicker to work with than slicing an array multiple times, especially since these files can be massive.
\nThere are many more things this software can do, I just want to show some quick basics :)
\nI know a lot of people here are particularly good at finding niche features and using them at their advantage ;)
\nNote that you could probably do all of that in Photoshop or any other image manipulation software, but: 1. Photoshop doesn't work on Linux; 2. I'm a remote sensing engineer, not an artist :P and 3. I thought this could be something most of you don't know about
Hope this helps!
", "messages": [ { "id": "1165818", "postDate": "01/23/2021 09:01:16", "content": "Hello!
\nI've joined this competition after the initial first wave of notebooks with submissions, and I was quite surprised to see people using rasterio of all to handle the images. It's a great library, but it is mostly used for geospatial data (understand satellite / drone / airborne imagery). That's why I do like Kaggle so much, you get to see tools be used in a different way!
Anyways this got me thinking, why not use more of the tools I use daily to work with large geopsatial data?
\nOne of the best visualization tools out there is called QGIS, and it's basically a GUI for exploring any type of geospatial data. It's a free and open source project, and works very nicely for this competition.
\nYou can open the software (works like a charm on any of the big 3 OS) and simply drag and drop the images in their (this things reads a lot of formats, and is pretty much built for .tiff files). It takes a few seconds to load the image (we all know these images are quite big), but then you can pan around, zoom in and out very easily:
You can also then save your predictions to a .tif, also drag and drop then, and change the transparency of that layer by Right Clicking on the name of the Layer -> Properties -> Transparency and sliding the transparency down.
This was a very quick way to evaluate that my prediction on 256x256 tiles suffered in tiles that only contained a little bit of the area needed to be assessed, but not covering enough of the tiles so my model doesn't see them, it becomes very visible in a few seconds by panning through the image:
\n
You could do all of this in Python, but arguably this is much quicker to work with than slicing an array multiple times, especially since these files can be massive.
\nThere are many more things this software can do, I just want to show some quick basics :)
\nI know a lot of people here are particularly good at finding niche features and using them at their advantage ;)
\nNote that you could probably do all of that in Photoshop or any other image manipulation software, but: 1. Photoshop doesn't work on Linux; 2. I'm a remote sensing engineer, not an artist :P and 3. I thought this could be something most of you don't know about
Hope this helps!
", "rawMarkdown": "Hello!\n\nI've joined this competition after the initial first wave of notebooks with submissions, and I was quite surprised to see people using `rasterio` of all to handle the images. It's a great library, but it is mostly used for geospatial data (understand satellite / drone / airborne imagery). That's why I do like Kaggle so much, you get to see tools be used in a different way!\n\nAnyways this got me thinking, why not use more of the tools I use daily to work with large geopsatial data?\n\nOne of the best visualization tools out there is called [QGIS](https://www.qgis.org/en/site/), and it's basically a GUI for exploring any type of geospatial data. It's a free and open source project, and works very nicely for this competition.\n\nYou can open the software (works like a charm on any of the big 3 OS) and simply drag and drop the images in their (this things reads a lot of formats, and is pretty much built for `.tiff` files). It takes a few seconds to load the image (we all know these images are quite big), but then you can pan around, zoom in and out very easily:\n\n\n\nYou can also then save your predictions to a `.tif`, also drag and drop then, and change the transparency of that layer by Right Clicking on the name of the Layer -> Properties -> Transparency and sliding the transparency down.\n\n\n\nThis was a very quick way to evaluate that my prediction on 256x256 tiles suffered in tiles that only contained a little bit of the area needed to be assessed, but not covering enough of the tiles so my model doesn't see them, it becomes very visible in a few seconds by panning through the image:\n\n\n\nYou could do all of this in Python, but arguably this is much quicker to work with than slicing an array multiple times, especially since these files can be massive.\n\nThere are many more things this software can do, I just want to show some quick basics :)\n\nI know a lot of people here are particularly good at finding niche features and using them at their advantage ;)\nNote that you could probably do all of that in Photoshop or any other image manipulation software, but: 1. Photoshop doesn't work on Linux; 2. I'm a remote sensing engineer, not an artist :P and 3. I thought this could be something most of you _don't_ know about\n\nHope this helps!", "votes": null }, { "id": "1167298", "postDate": "01/24/2021 06:55:47", "content": "Thanks for sharing! I am in third category, not aware till now :)
", "rawMarkdown": "Thanks for sharing! I am in third category, not aware till now :)", "votes": null }, { "id": "1215711", "postDate": "02/23/2021 22:04:33", "content": "@maxlenormand , this is very handy. Thanks for sharing! The tiff loads initially in a sec (compared to 20+ sec with Gimp). It also instantly gets the binary mask right (instead of 256 grey scale).
\nNow I only need to get more RAM ;-)
Thanks for sharing! I am in third category, not aware till now :)
", "votes": null, "replies": [] }, { "id": 1215711, "author_name": "joatom", "author_url": "", "post_date": "02/23/2021 22:04:33", "content": "@maxlenormand , this is very handy. Thanks for sharing! The tiff loads initially in a sec (compared to 20+ sec with Gimp). It also instantly gets the binary mask right (instead of 256 grey scale).
\nNow I only need to get more RAM ;-)