{ "id": 333081, "title": "Clarification Regarding Pixel Size", "url": "/competitions/hubmap-organ-segmentation/discussion/333081", "author_name": "", "post_date": "2022-06-24T15:01:57.907434500Z", "votes": 24, "comment_count": 15, "views": 0, "content": "

Hi there, I just want to confirm my understanding of the Pixel Size feature and propose how we might be able to leverage this to create more representative training data.

\n
\n



\n

\n

First let me confirm that I understand pixel size properly:

\n

My understanding of this can be best shown with a metaphor. Imagine instead of a biologist/engineer we were a painter. We can further imagine that the only method of painting we know how to do is pointillism (i.e. using dots of paint to create an image).

\n

Depending upon what tool we use we can create dots that are 1x1cm, 2x2cm, or 4x4cm (in this world we use SQUARE paintbrushes I guess).

\n\n

This metaphor demonstrates that pixel size is telling us, in real terms, the actual resolution of the tool used to paint the picture. Note I use the term visible detail to describe smoothness/resolution as interpreted by our eyes/models.

\n\n

Note that in our dataset, the pixel size for the SAME ORGAN (prostate) varies from 0.4µm in the training data up to 6.263µm in the test data. That means are pixels represent almost 16 times more tissue in the test data than in the training data. The result, as explained before, is that the images will show the same amount of tissue but be much less visibly detailed, or the images will be very zoomed-in and will show the same amount of visible detail, or something in-between!

\n

In our painting example above this difference is equivalent to the difference between the 1x1 tool and that represented by a 16x16cm tool allowing for only 39 dots when creating a 100x100cm painting (tree would be ~4 dots).*

\n
\n



\n

\n

Next we can understand the various Pixel Sizes and how they are distributed throughout the Training and Testing datasets.

\n
\n

The data section of the competition page has this to say:

\n
\n

\"The height/width of a single pixel from this image in micrometers. All HPA images have a pixel size of 0.4 µm. For Hubmap imagery the pixel size is 0.5 µm for kidney, 0.2290 µm for large intestine, 0.7562 µm for lung, 0.4945 µm for spleen, and 6.263 µm for prostate.\"

\n
\n

Here's that information as a table for easy viewing.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Pixel SizeTraining DatasetTesting Dataset
0.2290µmn/alarge intestine
0.4000µmprostate, lung, large intestine, spleen, kidneyn/a
0.4945µmn/aspleen
0.5000µmn/akidney
0.7562µmn/alung
6.2630µmn/aprostate
\n
\n

I will limit my observations here regarding the significance of these values and instead direct you to my Exploratory Data Analysis where I will be diving into this in greater detail over the coming days/weeks --> Exploratory Data Analysis.

\n
\n

For now, all I will point out is the obvious:

\n\n
\n



\n

\n

Let's next call out the problem here and possible solution

\n
\n

The problem is that the images in the test set, regardless of organ type, will have different pixel sizes compared to the provided training data (there are other inconsistencies (staining, slice thickness, etc) but for this discussion, I am focusing on pixel size only).

\n

If we train a model on the training data and try to perform inference on the test data, the model may/will fail because this new data is different than the data it was trained on. i.e. imagine training data on RGB data and trying to do inference on BGR data.

\n

Two possible solutions I can see are:

\n
    \n
  1. Convert all tissue slices to a consistent pixel size for each organ type (data engineering allowing for fixed pixel size model inference)
    2. \n
  2. Generate augmented data that converts images from their original pixel size to many different pixel sizes (data engineering allowing for pixel size robust model inference)
    3. \n
\n
\n

Hopefully, this helps… not sure if it will. But it certainly helped me to try and write it all out! Thanks for reading :)

", "messages": [ { "id": "1831977", "postDate": "06/24/2022 15:01:57", "content": "

Hi there, I just want to confirm my understanding of the Pixel Size feature and propose how we might be able to leverage this to create more representative training data.

\n
\n



\n

\n

First let me confirm that I understand pixel size properly:

\n

My understanding of this can be best shown with a metaphor. Imagine instead of a biologist/engineer we were a painter. We can further imagine that the only method of painting we know how to do is pointillism (i.e. using dots of paint to create an image).

\n

Depending upon what tool we use we can create dots that are 1x1cm, 2x2cm, or 4x4cm (in this world we use SQUARE paintbrushes I guess).

\n\n

This metaphor demonstrates that pixel size is telling us, in real terms, the actual resolution of the tool used to paint the picture. Note I use the term visible detail to describe smoothness/resolution as interpreted by our eyes/models.

\n\n

Note that in our dataset, the pixel size for the SAME ORGAN (prostate) varies from 0.4µm in the training data up to 6.263µm in the test data. That means are pixels represent almost 16 times more tissue in the test data than in the training data. The result, as explained before, is that the images will show the same amount of tissue but be much less visibly detailed, or the images will be very zoomed-in and will show the same amount of visible detail, or something in-between!

\n

In our painting example above this difference is equivalent to the difference between the 1x1 tool and that represented by a 16x16cm tool allowing for only 39 dots when creating a 100x100cm painting (tree would be ~4 dots).*

\n
\n



\n

\n

Next we can understand the various Pixel Sizes and how they are distributed throughout the Training and Testing datasets.

\n
\n

The data section of the competition page has this to say:

\n
\n

\"The height/width of a single pixel from this image in micrometers. All HPA images have a pixel size of 0.4 µm. For Hubmap imagery the pixel size is 0.5 µm for kidney, 0.2290 µm for large intestine, 0.7562 µm for lung, 0.4945 µm for spleen, and 6.263 µm for prostate.\"

\n
\n

Here's that information as a table for easy viewing.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Pixel SizeTraining DatasetTesting Dataset
0.2290µmn/alarge intestine
0.4000µmprostate, lung, large intestine, spleen, kidneyn/a
0.4945µmn/aspleen
0.5000µmn/akidney
0.7562µmn/alung
6.2630µmn/aprostate
\n
\n

I will limit my observations here regarding the significance of these values and instead direct you to my Exploratory Data Analysis where I will be diving into this in greater detail over the coming days/weeks --> Exploratory Data Analysis.

\n
\n

For now, all I will point out is the obvious:

\n\n
\n



\n

\n

Let's next call out the problem here and possible solution

\n
\n

The problem is that the images in the test set, regardless of organ type, will have different pixel sizes compared to the provided training data (there are other inconsistencies (staining, slice thickness, etc) but for this discussion, I am focusing on pixel size only).

\n

If we train a model on the training data and try to perform inference on the test data, the model may/will fail because this new data is different than the data it was trained on. i.e. imagine training data on RGB data and trying to do inference on BGR data.

\n

Two possible solutions I can see are:

\n
    \n
  1. Convert all tissue slices to a consistent pixel size for each organ type (data engineering allowing for fixed pixel size model inference)
    2. \n
  2. Generate augmented data that converts images from their original pixel size to many different pixel sizes (data engineering allowing for pixel size robust model inference)
    3. \n
\n
\n

Hopefully, this helps… not sure if it will. But it certainly helped me to try and write it all out! Thanks for reading :)

", "rawMarkdown": "Hi there, I just want to confirm my understanding of the Pixel Size feature and propose how we might be able to leverage this to create more representative training data.\n\n---\n\n
\n
\n\n**First let me confirm that I understand pixel size properly:**\n\nMy understanding of this can be best shown with a metaphor. Imagine instead of a biologist/engineer we were a painter. We can further imagine that the only method of painting we know how to do is pointillism (i.e. using dots of paint to create an image).\n\nDepending upon what tool we use we can create dots that are 1x1cm, 2x2cm, or 4x4cm (in this world we use SQUARE paintbrushes I guess).\n* If we use the 1x1cm tool and create a 100x100cm painting, our painting can have a whopping 10,000 dots *(pixel size of 1)*\n * At this scale, a tree might be made up of 1000 dots (tree is kinda like FTU)\n* If we use the 2x2cm tool and create a 100x100cm painting, our painting can have up to 2,500 dots *(pixel size of 2)*\n * At this scale, a tree might be made up of 2500 dots (tree is kinda like FTU)\n* If we use the 2x2cm tool and wish to have 10,000 dots than we would need to create a 200x200cm painting\n * At this scale, a tree would still be made of only 2500 dots (tree is kinda like FTU)\n* If we use the 4x4cm tool and create a 100x100cm painting, our painting can only have 625 dots *(pixel size of 4)*\n * At this scale, a tree would might be made of only 62 dots (tree is kinda like FTU)\n* If we use the 4x4cm tool and wish to have 10,000 dots than we would need to create a 400x400cm painting\n * At this scale, a tree would still be made of only 62 dots (tree is kinda like FTU)\n\nThis metaphor demonstrates that pixel size is telling us, in real terms, the actual resolution of the tool used to paint the picture. Note I use the term ***visible detail*** to describe smoothness/resolution as interpreted by our eyes/models.\n* If pixel size goes up, either we have less visible detail over the same amount of tissue or the same visible detail over a larger amount of tissue\n* If pixel size goes down, either we have more visible detail over the same amount of tissue or the same visible detail over a smaller amount of tissue\n\n**Note that in our dataset, the pixel size for the SAME ORGAN (prostate) varies from 0.4µm in the training data up to 6.263µm in the test data. That means are pixels represent almost 16 times more tissue in the test data than in the training data. The result, as explained before, is that the images will show the same amount of tissue but be much less visibly detailed, or the images will be very zoomed-in and will show the same amount of visible detail, or something in-between!**\n\n**In our painting example above this difference is equivalent to the difference between the 1x1 tool and that represented by a 16x16cm tool allowing for only 39 dots when creating a 100x100cm painting (tree would be ~4 dots).***\n\n---\n\n
\n
\n\n**Next we can understand the various Pixel Sizes and how they are distributed throughout the Training and Testing datasets.**\n\n---\n\nThe **data** section of the competition page has this to say:\n\n> *\"The height/width of a single pixel from this image in micrometers. All HPA images have a pixel size of 0.4 µm. For Hubmap imagery the pixel size is 0.5 µm for kidney, 0.2290 µm for large intestine, 0.7562 µm for lung, 0.4945 µm for spleen, and 6.263 µm for prostate.\"*\n\nHere's that information as a table for easy viewing.\n\n| **Pixel Size** | **Training Dataset** | **Testing Dataset** |\n|----------------|-------------------------------------------------|---------------------|\n| **0.2290µm** | n/a | large intestine |\n| **0.4000µm** | prostate, lung, large intestine, spleen, kidney | n/a |\n| **0.4945µm** | n/a | spleen |\n| **0.5000µm** | n/a | kidney |\n| **0.7562µm** | n/a | lung |\n| **6.2630µm** | n/a | prostate |\n\n---\n\nI will limit my observations here regarding the significance of these values and instead direct you to my [**Exploratory Data Analysis**](https://www.kaggle.com/code/dschettler8845/eda-hubmap-hpa-organ-segmentation) where I will be diving into this in greater detail over the coming days/weeks --> [**Exploratory Data Analysis**](https://www.kaggle.com/code/dschettler8845/eda-hubmap-hpa-organ-segmentation). \n\n---\n\nFor now, all I will point out is the obvious:\n* There are various pixel sizes ranging from 0.2290 to 6.263µm contained within the test dataset (one pixel size to one organ type respectively).\n* There is only the 0.4945µm pixel size available in the training dataset (for all organs).\n\n---\n\n
\n
\n\n**Let's next call out the problem here and possible solution**\n\n---\n\nThe problem is that the images in the test set, regardless of organ type, will have different pixel sizes compared to the provided training data (there are other inconsistencies (staining, slice thickness, etc) but for this discussion, I am focusing on pixel size only).\n\nIf we train a model on the training data and try to perform inference on the test data, the model may/will fail because this new data is different than the data it was trained on. i.e. imagine training data on RGB data and trying to do inference on BGR data.\n\nTwo possible solutions I can see are:\n1. Convert all tissue slices to a consistent pixel size for each organ type (data engineering allowing for fixed pixel size model inference)\n2. Generate augmented data that converts images from their original pixel size to many different pixel sizes (data engineering allowing for pixel size robust model inference)\n\n---\n\nHopefully, this helps... not sure if it will. But it certainly helped me to try and write it all out! Thanks for reading :)", "votes": null }, { "id": "1835032", "postDate": "06/27/2022 11:52:39", "content": "

Hi, I believe I have done this px size normalization for the inference in the following notebook:

\n
\n

https://www.kaggle.com/jirkaborovec/ftus-segm-baseline-flash-unet-on-tiled-images

\n
\n

See following sample code and pls correct me if I made mistake :)

\n
from skimage.transform import rescale, resize\nscale = row[\"pixel_size\"] / 0.4\nimg = plt.imread(test_img)\n# scale image by pixel size\nim = rescale(img, scale, order=1, anti_aliasing=True)\n
\n

so later on also the resulted predictions need to be scaled back to original image size
\nseg = resize(seg * 255, img.shape[:2], order=0) / 255

", "rawMarkdown": "Hi, I believe I have done this px size normalization for the inference in the following notebook:\n> https://www.kaggle.com/jirkaborovec/ftus-segm-baseline-flash-unet-on-tiled-images\n\nSee following sample code and pls correct me if I made mistake :)\n```py\nfrom skimage.transform import rescale, resize\nscale = row[\"pixel_size\"] / 0.4\nimg = plt.imread(test_img)\n# scale image by pixel size\nim = rescale(img, scale, order=1, anti_aliasing=True)\n```\nso later on also the resulted predictions need to be scaled back to original image size\n`seg = resize(seg * 255, img.shape[:2], order=0) / 255`", "votes": null }, { "id": "1840600", "postDate": "07/02/2022 11:49:17", "content": "

Thank you this makes sense!

", "rawMarkdown": "Thank you this makes sense!", "votes": null }, { "id": "1882286", "postDate": "08/03/2022 07:29:54", "content": "

@jirkaborovec @dschettler8845 I'm puzzled for this example code. For example, the Prostate class has 6.263 in HuBMAP vs. 0.4 in HPA. The scale = row[\"pixel_size\"] / 0.4 ~ 16, why not resize img by the factor of sqrt(scale) = 4? If you scale down 16x, it will lead a pixel size 16^2 larger than 0.4 in HPA images?

", "rawMarkdown": "jirkaborovec @dschettler8845 I'm puzzled for this example code. For example, the Prostate class has 6.263 in HuBMAP vs. 0.4 in HPA. The `scale = row[\"pixel_size\"] / 0.4 ~ 16`, why not resize `img` by the factor of `sqrt(scale) = 4`? If you scale down 16x, it will lead a pixel size 16^2 larger than 0.4 in HPA images?", "votes": null }, { "id": "1882424", "postDate": "08/03/2022 09:13:06", "content": "

The 0.4um/pixel and 6.263 um/pixel are for each side of a square pixel and not for its area. So the pixels are 0.4 x 0.4 and 6.263 x 6.263 and thus you need to scale each side by 6.263/0.4. By scaling the picture down 4 times you'll reduce the area of the image by 16, when you need to reduce each side of the image 16x

", "rawMarkdown": "The 0.4um/pixel and 6.263 um/pixel are for each side of a square pixel and not for its area. So the pixels are 0.4 x 0.4 and 6.263 x 6.263 and thus you need to scale each side by 6.263/0.4. By scaling the picture down 4 times you'll reduce the area of the image by 16, when you need to reduce each side of the image 16x", "votes": null }, { "id": "1882505", "postDate": "08/03/2022 10:04:35", "content": "

for pixel size, i always have this question. which is better
\n1) train=use common 0.4 um images. test = nomalised to 0.4 um (i.e. test normalised to train)
\n2) train = resize train images to different um according to organ (i.e. train normalised to test). test = just feed in images without normalisation.

\n

of course you can ensemble both models.

", "rawMarkdown": "for pixel size, i always have this question. which is better\n1) train=use common 0.4 um images. test = nomalised to 0.4 um (i.e. test normalised to train)\n2) train = resize train images to different um according to organ (i.e. train normalised to test). test = just feed in images without normalisation.\n\nof course you can ensemble both models.", "votes": null }, { "id": "1882530", "postDate": "08/03/2022 10:24:28", "content": "

@sakvaua Oh, I missed it. I've been thinking the \"scaler\" is for the area of a pixel. Great thanks for your help!

", "rawMarkdown": "sakvaua Oh, I missed it. I've been thinking the \"scaler\" is for the area of a pixel. Great thanks for your help!", "votes": null }, { "id": "1882545", "postDate": "08/03/2022 10:31:27", "content": "

Hi @hengck23 I'm wondering how to achieve your 2nd point. As clarified above, the pixel size of HubMap images is 16x of that of HPA images. Assume I slice the original HPA images with 1024x1024 windows, correct me if I'm wrong, there're 2 options available.

\n
    \n
  1. Scale up the original tiff_image by 16 along both h & w directions, then crop 1024x1024 tiles;
    2. \n
  2. Slice tiles on the original tiff_image, but with 1024 / 16 windows;
    3. \n
\n

I'm puzzled such resizing working can align pixel size at exactly the same level.

", "rawMarkdown": "Hi @hengck23 I'm wondering how to achieve your 2nd point. As clarified above, the pixel size of HubMap images is 16x of that of HPA images. Assume I slice the original HPA images with 1024x1024 windows, correct me if I'm wrong, there're 2 options available. \n1. Scale up the original tiff_image by 16 along both h & w directions, then crop 1024x1024 tiles; \n2. Slice tiles on the original tiff_image, but with `1024 / 16` windows;\n\nI'm puzzled such resizing working can align pixel size at exactly the same level.", "votes": null }, { "id": "1882590", "postDate": "08/03/2022 10:53:07", "content": "

\"HubMap images is 16x of that of HPA images. \"
\nthat is only for prostate.

\n
\n

you may want to read my post https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/332941
\nsearch for

\n
\n

\"what is the normalized size of hidden test images? in order to apply transformer segmentation, i need to have a rough ideal of the max and min input size of the test image. (i need to decide absolute/relative positional encoding, token patch size ….)
\n hence i do a probe.\"

\n
\n

from my post, once you normalised test images to the same 0.4um as the train, the train and test images are about the same width/height.

\n

in the test big ruler goes with big images. small ruler goes with small images. there is no big rulers with smaller images nor vice versa.

\n

there is the issue of rounding, padding to multiple of 32, etc. Hence the simplest way is to resize to normalised scale first, then crop if you use sliding windows (actually image are not very big after resize and I would recommend to discard sliding windows and use whole image as input)

\n

\"https://i.ibb.co/sJLcRhf/Selection-173.png\"

", "rawMarkdown": "\"HubMap images is 16x of that of HPA images. \"\nthat is only for prostate.\n\n---\n\nyou may want to read my post https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/332941\nsearch for \n\n> \"what is the normalized size of hidden test images? in order to apply transformer segmentation, i need to have a rough ideal of the max and min input size of the test image. (i need to decide absolute/relative positional encoding, token patch size ….)\nhence i do a probe.\"\n\n\n\n from my post, once you normalised test images to the same 0.4um as the train, the train and test images are about the same width/height. \n\nin the test big ruler goes with big images. small ruler goes with small images. there is no big rulers with smaller images nor vice versa. \n\nthere is the issue of rounding, padding to multiple of 32, etc. Hence the simplest way is to resize to normalised scale first, then crop if you use sliding windows (actually image are not very big after resize and I would recommend to discard sliding windows and use whole image as input)\n\n ![https://i.ibb.co/sJLcRhf/Selection-173.png](https://i.ibb.co/sJLcRhf/Selection-173.png)", "votes": null }, { "id": "1882633", "postDate": "08/03/2022 11:09:47", "content": "

reagrding my second point, i was thinking why the slides are stored in different magnification in the first place.
\ni think these magnification are probably best at identifying certain structure?

\n

Hence, if we use point 2, we are giving the model implicit information about the organ through the image magnification.
\nwe are also giving the model information about \"best scale for discriminative features.\"

\n

however, there is also the possibility that the magnification of the sldies have nothing to do with best scale of discriminative tissue features. It is simply because some lab use more expensive or better equipment then others.

", "rawMarkdown": "reagrding my second point, i was thinking why the slides are stored in different magnification in the first place.\ni think these magnification are probably best at identifying certain structure?\n\nHence, if we use point 2, we are giving the model implicit information about the organ through the image magnification.\nwe are also giving the model information about \"best scale for discriminative features.\"\n\nhowever, there is also the possibility that the magnification of the sldies have nothing to do with best scale of discriminative tissue features. It is simply because some lab use more expensive or better equipment then others.", "votes": null }, { "id": "1883191", "postDate": "08/03/2022 16:29:56", "content": "

@hengck23 sorry for my late reply, it took me a while to understand the mechanism behind it 😨. According to my understanding of your probe findings, if we want to mimic HubMap images via HPA images, I should resize the tiff_image into size of [0.8 * t, 1.2 * t], where t = (0.4 * 3000) / hubmap_pixel_size, accordingly, and then slice each window or directly resize expected shapes and feed into the model. As a result, the \"simulated\" Hubmap prostate images will be so tiny as to be scaled up again. This does not sound reasonable to me.

\n

I also agree the magnification of tiffs simply results from different equipments.

", "rawMarkdown": "hengck23 sorry for my late reply, it took me a while to understand the mechanism behind it 😨. According to my understanding of your probe findings, if we want to mimic HubMap images via HPA images, I should resize the tiff_image into size of `[0.8 * t, 1.2 * t], where t = (0.4 * 3000) / hubmap_pixel_size, accordingly`, and then slice each window or directly resize expected shapes and feed into the model. As a result, the \"simulated\" Hubmap prostate images will be so tiny as to be scaled up again. This does not sound reasonable to me. \n\nI also agree the magnification of tiffs simply results from different equipments.", "votes": null }, { "id": "1883221", "postDate": "08/03/2022 17:02:30", "content": "

\"As a result, the \"simulated\" Hubmap prostate images will be so tiny as to be scaled up again. This does not sound reasonable to me. \"

\n

see this post:
\nhttps://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/333552

", "rawMarkdown": "\"As a result, the \"simulated\" Hubmap prostate images will be so tiny as to be scaled up again. This does not sound reasonable to me. \"\n\nsee this post:\nhttps://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/333552", "votes": null }, { "id": "1883235", "postDate": "08/03/2022 17:11:57", "content": "

\" should resize the tiff_image into size of [0.8 * t, 1.2 * t], where t = (0.4 * 3000) / hubmap_pixel_size,\"

\n

No. just resize using scale = 0.4 / hubmap_pixel_size. Then you find that the resulting image size is [0.8 * 3000, 1.2 * 3000]

", "rawMarkdown": "\" should resize the tiff_image into size of [0.8 * t, 1.2 * t], where t = (0.4 * 3000) / hubmap_pixel_size,\"\n\nNo. just resize using scale = 0.4 / hubmap_pixel_size. Then you find that the resulting image size is [0.8 * 3000, 1.2 * 3000]", "votes": null }, { "id": "1883667", "postDate": "08/04/2022 02:11:53", "content": "

@hengck23 Thanks for your patient explanation.

\n
\n

resize using scale = 0.4 / hubmap_pixel_size.

\n
\n

That's what I've adopted, and it indeed helps to predict a maks more reasonable. See the following pictures

\n

\"\"

\n

\"\"

\n

However, what I didn't quite get the gist of is your metaphor,

\n
\n

in the test big ruler goes with big images. small ruler goes with small images. there is no big rulers with smaller images nor vice versa.

\n
\n

does \"ruler\" refer to the \"tile_size\" of windows?
\nThanks!

", "rawMarkdown": "hengck23 Thanks for your patient explanation. \n> resize using scale = 0.4 / hubmap_pixel_size.\n\nThat's what I've adopted, and it indeed helps to predict a maks more reasonable. See the following pictures\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5552698%2F63e6fa397b9a7d12be34d1b709f351df%2F111.png?generation=1659578964747792&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5552698%2Fdbeabe03f340db7c616c7bf7d38f806e%2F222.png?generation=1659578999963810&alt=media)\n\nHowever, what I didn't quite get the gist of is your metaphor, \n> in the test big ruler goes with big images. small ruler goes with small images. there is no big rulers with smaller images nor vice versa.\n\ndoes \"ruler\" refer to the \"tile_size\" of windows?\nThanks!", "votes": null }, { "id": "1883674", "postDate": "08/04/2022 02:19:27", "content": "

does \"ruler\" refer to the \"tile_size\" of windows?
\nruler is related pixel_size.

\n

in the test if pixel_size is large (e.g. prostate), image is small(e.g. 160x160). you won't find large pixel_size and large image.

", "rawMarkdown": "does \"ruler\" refer to the \"tile_size\" of windows?\nruler is related pixel\\_size.\n\nin the test if pixel\\_size is large (e.g. prostate), image is small(e.g. 160x160). you won't find large pixel\\_size and large image.", "votes": null }, { "id": "1883872", "postDate": "08/04/2022 06:42:30", "content": "

Okay, now I fully understood your metaphor. Thanks heng!

", "rawMarkdown": "Okay, now I fully understood your metaphor. Thanks heng!", "votes": null } ], "comments": [ { "id": 1835032, "author_name": "jirkaborovec", "author_url": "", "post_date": "06/27/2022 11:52:39", "content": "

Hi, I believe I have done this px size normalization for the inference in the following notebook:

\n
\n

https://www.kaggle.com/jirkaborovec/ftus-segm-baseline-flash-unet-on-tiled-images

\n
\n

See following sample code and pls correct me if I made mistake :)

\n
from skimage.transform import rescale, resize\nscale = row[\"pixel_size\"] / 0.4\nimg = plt.imread(test_img)\n# scale image by pixel size\nim = rescale(img, scale, order=1, anti_aliasing=True)\n
\n

so later on also the resulted predictions need to be scaled back to original image size
\nseg = resize(seg * 255, img.shape[:2], order=0) / 255

", "votes": null, "replies": [ { "id": 1840600, "author_name": "dschettler8845", "author_url": "", "post_date": "07/02/2022 11:49:17", "content": "

Thank you this makes sense!

", "votes": null, "replies": [] }, { "id": 1882286, "author_name": "fuckvenkatraman", "author_url": "", "post_date": "08/03/2022 07:29:54", "content": "

@jirkaborovec @dschettler8845 I'm puzzled for this example code. For example, the Prostate class has 6.263 in HuBMAP vs. 0.4 in HPA. The scale = row[\"pixel_size\"] / 0.4 ~ 16, why not resize img by the factor of sqrt(scale) = 4? If you scale down 16x, it will lead a pixel size 16^2 larger than 0.4 in HPA images?

", "votes": null, "replies": [] }, { "id": 1882424, "author_name": "sakvaua", "author_url": "", "post_date": "08/03/2022 09:13:06", "content": "

The 0.4um/pixel and 6.263 um/pixel are for each side of a square pixel and not for its area. So the pixels are 0.4 x 0.4 and 6.263 x 6.263 and thus you need to scale each side by 6.263/0.4. By scaling the picture down 4 times you'll reduce the area of the image by 16, when you need to reduce each side of the image 16x

", "votes": null, "replies": [] }, { "id": 1882530, "author_name": "fuckvenkatraman", "author_url": "", "post_date": "08/03/2022 10:24:28", "content": "

@sakvaua Oh, I missed it. I've been thinking the \"scaler\" is for the area of a pixel. Great thanks for your help!

", "votes": null, "replies": [] } ] }, { "id": 1882505, "author_name": "hengck23", "author_url": "", "post_date": "08/03/2022 10:04:35", "content": "

for pixel size, i always have this question. which is better
\n1) train=use common 0.4 um images. test = nomalised to 0.4 um (i.e. test normalised to train)
\n2) train = resize train images to different um according to organ (i.e. train normalised to test). test = just feed in images without normalisation.

\n

of course you can ensemble both models.

", "votes": null, "replies": [ { "id": 1882545, "author_name": "fuckvenkatraman", "author_url": "", "post_date": "08/03/2022 10:31:27", "content": "

Hi @hengck23 I'm wondering how to achieve your 2nd point. As clarified above, the pixel size of HubMap images is 16x of that of HPA images. Assume I slice the original HPA images with 1024x1024 windows, correct me if I'm wrong, there're 2 options available.

\n
    \n
  1. Scale up the original tiff_image by 16 along both h & w directions, then crop 1024x1024 tiles;
    2. \n
  2. Slice tiles on the original tiff_image, but with 1024 / 16 windows;
    3. \n
\n

I'm puzzled such resizing working can align pixel size at exactly the same level.

", "votes": null, "replies": [] }, { "id": 1882590, "author_name": "hengck23", "author_url": "", "post_date": "08/03/2022 10:53:07", "content": "

\"HubMap images is 16x of that of HPA images. \"
\nthat is only for prostate.

\n
\n

you may want to read my post https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/332941
\nsearch for

\n
\n

\"what is the normalized size of hidden test images? in order to apply transformer segmentation, i need to have a rough ideal of the max and min input size of the test image. (i need to decide absolute/relative positional encoding, token patch size ….)
\n hence i do a probe.\"

\n
\n

from my post, once you normalised test images to the same 0.4um as the train, the train and test images are about the same width/height.

\n

in the test big ruler goes with big images. small ruler goes with small images. there is no big rulers with smaller images nor vice versa.

\n

there is the issue of rounding, padding to multiple of 32, etc. Hence the simplest way is to resize to normalised scale first, then crop if you use sliding windows (actually image are not very big after resize and I would recommend to discard sliding windows and use whole image as input)

\n

\"https://i.ibb.co/sJLcRhf/Selection-173.png\"

", "votes": null, "replies": [] }, { "id": 1882633, "author_name": "hengck23", "author_url": "", "post_date": "08/03/2022 11:09:47", "content": "

reagrding my second point, i was thinking why the slides are stored in different magnification in the first place.
\ni think these magnification are probably best at identifying certain structure?

\n

Hence, if we use point 2, we are giving the model implicit information about the organ through the image magnification.
\nwe are also giving the model information about \"best scale for discriminative features.\"

\n

however, there is also the possibility that the magnification of the sldies have nothing to do with best scale of discriminative tissue features. It is simply because some lab use more expensive or better equipment then others.

", "votes": null, "replies": [] }, { "id": 1883191, "author_name": "fuckvenkatraman", "author_url": "", "post_date": "08/03/2022 16:29:56", "content": "

@hengck23 sorry for my late reply, it took me a while to understand the mechanism behind it 😨. According to my understanding of your probe findings, if we want to mimic HubMap images via HPA images, I should resize the tiff_image into size of [0.8 * t, 1.2 * t], where t = (0.4 * 3000) / hubmap_pixel_size, accordingly, and then slice each window or directly resize expected shapes and feed into the model. As a result, the \"simulated\" Hubmap prostate images will be so tiny as to be scaled up again. This does not sound reasonable to me.

\n

I also agree the magnification of tiffs simply results from different equipments.

", "votes": null, "replies": [] }, { "id": 1883221, "author_name": "hengck23", "author_url": "", "post_date": "08/03/2022 17:02:30", "content": "

\"As a result, the \"simulated\" Hubmap prostate images will be so tiny as to be scaled up again. This does not sound reasonable to me. \"

\n

see this post:
\nhttps://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/333552

", "votes": null, "replies": [] }, { "id": 1883235, "author_name": "hengck23", "author_url": "", "post_date": "08/03/2022 17:11:57", "content": "

\" should resize the tiff_image into size of [0.8 * t, 1.2 * t], where t = (0.4 * 3000) / hubmap_pixel_size,\"

\n

No. just resize using scale = 0.4 / hubmap_pixel_size. Then you find that the resulting image size is [0.8 * 3000, 1.2 * 3000]

", "votes": null, "replies": [] }, { "id": 1883667, "author_name": "fuckvenkatraman", "author_url": "", "post_date": "08/04/2022 02:11:53", "content": "

@hengck23 Thanks for your patient explanation.

\n
\n

resize using scale = 0.4 / hubmap_pixel_size.

\n
\n

That's what I've adopted, and it indeed helps to predict a maks more reasonable. See the following pictures

\n

\"\"

\n

\"\"

\n

However, what I didn't quite get the gist of is your metaphor,

\n
\n

in the test big ruler goes with big images. small ruler goes with small images. there is no big rulers with smaller images nor vice versa.

\n
\n

does \"ruler\" refer to the \"tile_size\" of windows?
\nThanks!

", "votes": null, "replies": [] }, { "id": 1883674, "author_name": "hengck23", "author_url": "", "post_date": "08/04/2022 02:19:27", "content": "

does \"ruler\" refer to the \"tile_size\" of windows?
\nruler is related pixel_size.

\n

in the test if pixel_size is large (e.g. prostate), image is small(e.g. 160x160). you won't find large pixel_size and large image.

", "votes": null, "replies": [] }, { "id": 1883872, "author_name": "fuckvenkatraman", "author_url": "", "post_date": "08/04/2022 06:42:30", "content": "

Okay, now I fully understood your metaphor. Thanks heng!

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1831977": "Hi there, I just want to confirm my understanding of the Pixel Size feature and propose how we might be able to leverage this to create more representative training data.\n\n---\n\n
\n
\n\n**First let me confirm that I understand pixel size properly:**\n\nMy understanding of this can be best shown with a metaphor. Imagine instead of a biologist/engineer we were a painter. We can further imagine that the only method of painting we know how to do is pointillism (i.e. using dots of paint to create an image).\n\nDepending upon what tool we use we can create dots that are 1x1cm, 2x2cm, or 4x4cm (in this world we use SQUARE paintbrushes I guess).\n* If we use the 1x1cm tool and create a 100x100cm painting, our painting can have a whopping 10,000 dots *(pixel size of 1)*\n * At this scale, a tree might be made up of 1000 dots (tree is kinda like FTU)\n* If we use the 2x2cm tool and create a 100x100cm painting, our painting can have up to 2,500 dots *(pixel size of 2)*\n * At this scale, a tree might be made up of 2500 dots (tree is kinda like FTU)\n* If we use the 2x2cm tool and wish to have 10,000 dots than we would need to create a 200x200cm painting\n * At this scale, a tree would still be made of only 2500 dots (tree is kinda like FTU)\n* If we use the 4x4cm tool and create a 100x100cm painting, our painting can only have 625 dots *(pixel size of 4)*\n * At this scale, a tree would might be made of only 62 dots (tree is kinda like FTU)\n* If we use the 4x4cm tool and wish to have 10,000 dots than we would need to create a 400x400cm painting\n * At this scale, a tree would still be made of only 62 dots (tree is kinda like FTU)\n\nThis metaphor demonstrates that pixel size is telling us, in real terms, the actual resolution of the tool used to paint the picture. Note I use the term ***visible detail*** to describe smoothness/resolution as interpreted by our eyes/models.\n* If pixel size goes up, either we have less visible detail over the same amount of tissue or the same visible detail over a larger amount of tissue\n* If pixel size goes down, either we have more visible detail over the same amount of tissue or the same visible detail over a smaller amount of tissue\n\n**Note that in our dataset, the pixel size for the SAME ORGAN (prostate) varies from 0.4µm in the training data up to 6.263µm in the test data. That means are pixels represent almost 16 times more tissue in the test data than in the training data. The result, as explained before, is that the images will show the same amount of tissue but be much less visibly detailed, or the images will be very zoomed-in and will show the same amount of visible detail, or something in-between!**\n\n**In our painting example above this difference is equivalent to the difference between the 1x1 tool and that represented by a 16x16cm tool allowing for only 39 dots when creating a 100x100cm painting (tree would be ~4 dots).***\n\n---\n\n
\n
\n\n**Next we can understand the various Pixel Sizes and how they are distributed throughout the Training and Testing datasets.**\n\n---\n\nThe **data** section of the competition page has this to say:\n\n> *\"The height/width of a single pixel from this image in micrometers. All HPA images have a pixel size of 0.4 µm. For Hubmap imagery the pixel size is 0.5 µm for kidney, 0.2290 µm for large intestine, 0.7562 µm for lung, 0.4945 µm for spleen, and 6.263 µm for prostate.\"*\n\nHere's that information as a table for easy viewing.\n\n| **Pixel Size** | **Training Dataset** | **Testing Dataset** |\n|----------------|-------------------------------------------------|---------------------|\n| **0.2290µm** | n/a | large intestine |\n| **0.4000µm** | prostate, lung, large intestine, spleen, kidney | n/a |\n| **0.4945µm** | n/a | spleen |\n| **0.5000µm** | n/a | kidney |\n| **0.7562µm** | n/a | lung |\n| **6.2630µm** | n/a | prostate |\n\n---\n\nI will limit my observations here regarding the significance of these values and instead direct you to my [**Exploratory Data Analysis**](https://www.kaggle.com/code/dschettler8845/eda-hubmap-hpa-organ-segmentation) where I will be diving into this in greater detail over the coming days/weeks --> [**Exploratory Data Analysis**](https://www.kaggle.com/code/dschettler8845/eda-hubmap-hpa-organ-segmentation). \n\n---\n\nFor now, all I will point out is the obvious:\n* There are various pixel sizes ranging from 0.2290 to 6.263µm contained within the test dataset (one pixel size to one organ type respectively).\n* There is only the 0.4945µm pixel size available in the training dataset (for all organs).\n\n---\n\n
\n
\n\n**Let's next call out the problem here and possible solution**\n\n---\n\nThe problem is that the images in the test set, regardless of organ type, will have different pixel sizes compared to the provided training data (there are other inconsistencies (staining, slice thickness, etc) but for this discussion, I am focusing on pixel size only).\n\nIf we train a model on the training data and try to perform inference on the test data, the model may/will fail because this new data is different than the data it was trained on. i.e. imagine training data on RGB data and trying to do inference on BGR data.\n\nTwo possible solutions I can see are:\n1. Convert all tissue slices to a consistent pixel size for each organ type (data engineering allowing for fixed pixel size model inference)\n2. Generate augmented data that converts images from their original pixel size to many different pixel sizes (data engineering allowing for pixel size robust model inference)\n\n---\n\nHopefully, this helps... not sure if it will. But it certainly helped me to try and write it all out! Thanks for reading :)", "1835032": "Hi, I believe I have done this px size normalization for the inference in the following notebook:\n> https://www.kaggle.com/jirkaborovec/ftus-segm-baseline-flash-unet-on-tiled-images\n\nSee following sample code and pls correct me if I made mistake :)\n```py\nfrom skimage.transform import rescale, resize\nscale = row[\"pixel_size\"] / 0.4\nimg = plt.imread(test_img)\n# scale image by pixel size\nim = rescale(img, scale, order=1, anti_aliasing=True)\n```\nso later on also the resulted predictions need to be scaled back to original image size\n`seg = resize(seg * 255, img.shape[:2], order=0) / 255`", "1840600": "Thank you this makes sense!", "1882286": "jirkaborovec @dschettler8845 I'm puzzled for this example code. For example, the Prostate class has 6.263 in HuBMAP vs. 0.4 in HPA. The `scale = row[\"pixel_size\"] / 0.4 ~ 16`, why not resize `img` by the factor of `sqrt(scale) = 4`? If you scale down 16x, it will lead a pixel size 16^2 larger than 0.4 in HPA images?", "1882424": "The 0.4um/pixel and 6.263 um/pixel are for each side of a square pixel and not for its area. So the pixels are 0.4 x 0.4 and 6.263 x 6.263 and thus you need to scale each side by 6.263/0.4. By scaling the picture down 4 times you'll reduce the area of the image by 16, when you need to reduce each side of the image 16x", "1882505": "for pixel size, i always have this question. which is better\n1) train=use common 0.4 um images. test = nomalised to 0.4 um (i.e. test normalised to train)\n2) train = resize train images to different um according to organ (i.e. train normalised to test). test = just feed in images without normalisation.\n\nof course you can ensemble both models.", "1882530": "sakvaua Oh, I missed it. I've been thinking the \"scaler\" is for the area of a pixel. Great thanks for your help!", "1882545": "Hi @hengck23 I'm wondering how to achieve your 2nd point. As clarified above, the pixel size of HubMap images is 16x of that of HPA images. Assume I slice the original HPA images with 1024x1024 windows, correct me if I'm wrong, there're 2 options available. \n1. Scale up the original tiff_image by 16 along both h & w directions, then crop 1024x1024 tiles; \n2. Slice tiles on the original tiff_image, but with `1024 / 16` windows;\n\nI'm puzzled such resizing working can align pixel size at exactly the same level.", "1882590": "\"HubMap images is 16x of that of HPA images. \"\nthat is only for prostate.\n\n---\n\nyou may want to read my post https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/332941\nsearch for \n\n> \"what is the normalized size of hidden test images? in order to apply transformer segmentation, i need to have a rough ideal of the max and min input size of the test image. (i need to decide absolute/relative positional encoding, token patch size ….)\nhence i do a probe.\"\n\n\n\n from my post, once you normalised test images to the same 0.4um as the train, the train and test images are about the same width/height. \n\nin the test big ruler goes with big images. small ruler goes with small images. there is no big rulers with smaller images nor vice versa. \n\nthere is the issue of rounding, padding to multiple of 32, etc. Hence the simplest way is to resize to normalised scale first, then crop if you use sliding windows (actually image are not very big after resize and I would recommend to discard sliding windows and use whole image as input)\n\n ![https://i.ibb.co/sJLcRhf/Selection-173.png](https://i.ibb.co/sJLcRhf/Selection-173.png)", "1882633": "reagrding my second point, i was thinking why the slides are stored in different magnification in the first place.\ni think these magnification are probably best at identifying certain structure?\n\nHence, if we use point 2, we are giving the model implicit information about the organ through the image magnification.\nwe are also giving the model information about \"best scale for discriminative features.\"\n\nhowever, there is also the possibility that the magnification of the sldies have nothing to do with best scale of discriminative tissue features. It is simply because some lab use more expensive or better equipment then others.", "1883191": "hengck23 sorry for my late reply, it took me a while to understand the mechanism behind it 😨. According to my understanding of your probe findings, if we want to mimic HubMap images via HPA images, I should resize the tiff_image into size of `[0.8 * t, 1.2 * t], where t = (0.4 * 3000) / hubmap_pixel_size, accordingly`, and then slice each window or directly resize expected shapes and feed into the model. As a result, the \"simulated\" Hubmap prostate images will be so tiny as to be scaled up again. This does not sound reasonable to me. \n\nI also agree the magnification of tiffs simply results from different equipments.", "1883221": "\"As a result, the \"simulated\" Hubmap prostate images will be so tiny as to be scaled up again. This does not sound reasonable to me. \"\n\nsee this post:\nhttps://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/333552", "1883235": "\" should resize the tiff_image into size of [0.8 * t, 1.2 * t], where t = (0.4 * 3000) / hubmap_pixel_size,\"\n\nNo. just resize using scale = 0.4 / hubmap_pixel_size. Then you find that the resulting image size is [0.8 * 3000, 1.2 * 3000]", "1883667": "hengck23 Thanks for your patient explanation. \n> resize using scale = 0.4 / hubmap_pixel_size.\n\nThat's what I've adopted, and it indeed helps to predict a maks more reasonable. See the following pictures\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5552698%2F63e6fa397b9a7d12be34d1b709f351df%2F111.png?generation=1659578964747792&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5552698%2Fdbeabe03f340db7c616c7bf7d38f806e%2F222.png?generation=1659578999963810&alt=media)\n\nHowever, what I didn't quite get the gist of is your metaphor, \n> in the test big ruler goes with big images. small ruler goes with small images. there is no big rulers with smaller images nor vice versa.\n\ndoes \"ruler\" refer to the \"tile_size\" of windows?\nThanks!", "1883674": "does \"ruler\" refer to the \"tile_size\" of windows?\nruler is related pixel\\_size.\n\nin the test if pixel\\_size is large (e.g. prostate), image is small(e.g. 160x160). you won't find large pixel\\_size and large image.", "1883872": "Okay, now I fully understood your metaphor. Thanks heng!" }, "source": "meta" }