{ "id": 416901, "title": "Dilation increases the score, who understands why?", "url": "/competitions/hubmap-hacking-the-human-vasculature/discussion/416901", "author_name": "Kostiantyn Maksymov", "post_date": "2023-06-13T11:56:51.856000", "votes": 48, "comment_count": 30, "views": 0, "content": "

Hi everyone!
\nSo I was stuck at some point that all my models where showing 0.26-0.27 on the LB - and I decided to add postprocessing to masks - and after I did it (I have used dilation) score improved from 0.239 to 0.345 - that is the huge jump - so my question to participatnts is next - someone understands why this happens? maybe it makes sense to train on already dilated masks?

", "messages": [ { "id": 2300734, "postDate": "2023-06-13T11:56:51.857Z", "content": "

Hi everyone!
\nSo I was stuck at some point that all my models where showing 0.26-0.27 on the LB - and I decided to add postprocessing to masks - and after I did it (I have used dilation) score improved from 0.239 to 0.345 - that is the huge jump - so my question to participatnts is next - someone understands why this happens? maybe it makes sense to train on already dilated masks?

", "rawMarkdown": "Hi everyone!\nSo I was stuck at some point that all my models where showing 0.26-0.27 on the LB - and I decided to add postprocessing to masks - and after I did it (I have used dilation) score improved from 0.239 to 0.345 - that is the huge jump - so my question to participatnts is next - someone understands why this happens? maybe it makes sense to train on already dilated masks?", "votes": 48 }, { "id": 2328654, "postDate": "2023-07-03T18:52:02.393Z", "content": "

Consider the pixel-level softmax output for a blood vessel region. Near the center of the blood vessel, the outputs will be close to 1.0. Near the edges of the blood vessel, the outputs will be smaller, maybe even ~0.1-0.3 right at the edges. If your model has a condition such as \"mask = softmax > 0.5\", then your binary mask will only include the center of the blood vessels, and dilation will help you include the edges of the blood vessel. But if you used a different condition, such as \"mask = softmax > 0.1\", you are likely including the whole blood vessel already (plus maybe some background around it), and dilation will not help. That is what I think is happening.

", "rawMarkdown": "Consider the pixel-level softmax output for a blood vessel region. Near the center of the blood vessel, the outputs will be close to 1.0. Near the edges of the blood vessel, the outputs will be smaller, maybe even ~0.1-0.3 right at the edges. If your model has a condition such as \"mask = softmax > 0.5\", then your binary mask will only include the center of the blood vessels, and dilation will help you include the edges of the blood vessel. But if you used a different condition, such as \"mask = softmax > 0.1\", you are likely including the whole blood vessel already (plus maybe some background around it), and dilation will not help. That is what I think is happening.", "votes": 6, "replies": [ { "id": 2328927, "postDate": "2023-07-04T01:15:38.640Z", "content": "

nice point, need to check it

", "rawMarkdown": "nice point, need to check it" } ] }, { "id": 2301847, "postDate": "2023-06-14T07:34:37.250Z", "content": "

same here… 0.297 to 0.397, nicely found bro.

", "rawMarkdown": "same here... 0.297 to 0.397, nicely found bro.", "votes": 5 }, { "id": 2322028, "postDate": "2023-06-29T03:34:52.917Z", "content": "

using opencv dilation:

\n", "rawMarkdown": "using opencv dilation:\n- cv2.dilate(m,kernel=(3x3 box), iteration=3) is optimal\n- iteration=2 has slight lower lb\n- iteration=4 drops significantly\n", "votes": 3, "replies": [ { "id": 2322522, "postDate": "2023-06-29T10:21:55.200Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] }, { "id": 2300809, "postDate": "2023-06-13T13:12:35.073Z", "content": "

UPD - for my local CV I have the same behaviour - if I train on dilated masks - score inreases on local validation as well

", "rawMarkdown": "UPD - for my local CV I have the same behaviour - if I train on dilated masks - score inreases on local validation as well", "votes": 3, "replies": [ { "id": 2301094, "postDate": "2023-06-13T16:02:21.067Z", "content": "

Very interesting. Have you observed a difference between dataset 1/dataset 2?
\nAs they were annotated differently, and we the test set is purely from dataset 1, it might be that the model learns something in-between dataset 1's annotations and dataset 2's annotations, but the LB favours 1.

\n

Pure speculation, btw.

", "rawMarkdown": "Very interesting. Have you observed a difference between dataset 1/dataset 2? \nAs they were annotated differently, and we the test set is purely from dataset 1, it might be that the model learns something in-between dataset 1's annotations and dataset 2's annotations, but the LB favours 1. \n\nPure speculation, btw. \n", "replies": [ { "id": 2301415, "postDate": "2023-06-13T22:37:51.217Z", "content": "

need to try, now I have split that mixes everything

", "rawMarkdown": "need to try, now I have split that mixes everything" }, { "id": 2346112, "postDate": "2023-07-16T03:25:20.790Z", "content": "

Is the train on dilated masks effective?

", "rawMarkdown": "Is the train on dilated masks effective?" } ] } ] }, { "id": 2300785, "postDate": "2023-06-13T12:46:40.850Z", "content": "

The key is whether your CV has such an improvement. LB is not always reliable. Like here, https://www.kaggle.com/competitions/tensorflow-great-barrier-reef/discussion/307605. By modifying bounding boxes for no reason, he can easily increase LB to the 2nd place.

", "rawMarkdown": "The key is whether your CV has such an improvement. LB is not always reliable. Like here, https://www.kaggle.com/competitions/tensorflow-great-barrier-reef/discussion/307605. By modifying bounding boxes for no reason, he can easily increase LB to the 2nd place.", "votes": 3, "replies": [ { "id": 2300801, "postDate": "2023-06-13T13:04:20.107Z", "content": "

in your case does dilation also help for public score?

", "rawMarkdown": "in your case does dilation also help for public score?", "replies": [ { "id": 2300947, "postDate": "2023-06-13T14:26:37.770Z", "content": "

I haven't tried it.

", "rawMarkdown": "I haven't tried it." } ] } ] }, { "id": 2314195, "postDate": "2023-06-23T08:48:28.847Z", "content": "

if the truth masks are large and has many pixels, effects of dilation is less.

\n

so maybe public test has more smaller maskes?

\n
\n

another way to prove if the test mask maskes are small is to try input size 512,640,1024 at submission.
\nThe lb score should imporve and the effects of dilation decreases.

", "rawMarkdown": "if the truth masks are large and has many pixels, effects of dilation is less.\n\nso maybe public test has more smaller maskes?\n\n---\n\nanother way to prove if the test mask maskes are small is to try input size 512,640,1024 at submission.\nThe lb score should imporve and the effects of dilation decreases.", "votes": 4, "replies": [ { "id": 2315314, "postDate": "2023-06-24T03:46:36.643Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 2303240, "postDate": "2023-06-15T07:23:42.177Z", "content": "

Same. 0.250 -> 0.332
\nA bit scary, would like to figure out why it works before just blindly using it.

", "rawMarkdown": "Same. 0.250 -> 0.332\nA bit scary, would like to figure out why it works before just blindly using it. \n", "votes": 1 }, { "id": 2301156, "postDate": "2023-06-13T16:51:01.820Z", "content": "

Just a guess, but maybe our models are quite conservative when generating the masks.

\n

So some of the edges of the image that were manually annotated as masks are ignored by our models. If so, dilating the masks trains the model to overshoot a bit and gives it a push to reach the IoU threshold of 0.6

\n

Just my interpretation, you may need to inspect the generated masks with and without to confirm.

", "rawMarkdown": "Just a guess, but maybe our models are quite conservative when generating the masks.\n\nSo some of the edges of the image that were manually annotated as masks are ignored by our models. If so, dilating the masks trains the model to overshoot a bit and gives it a push to reach the IoU threshold of 0.6\n\nJust my interpretation, you may need to inspect the generated masks with and without to confirm.", "votes": 2, "replies": [ { "id": 2314100, "postDate": "2023-06-23T06:43:52.360Z", "content": "

any progress or justice for this method？

", "rawMarkdown": "any progress or justice for this method？", "replies": [ { "id": 2321756, "postDate": "2023-06-28T21:00:41.423Z", "content": "

I haven't tried it yet, at this point I'm only testing the base methods without dilation.

", "rawMarkdown": "I haven't tried it yet, at this point I'm only testing the base methods without dilation." } ] } ] }, { "id": 2314098, "postDate": "2023-06-23T06:36:16.113Z", "content": "

how to modify the convolution in yoloV7 model to use the dilated convolution？

", "rawMarkdown": "how to modify the convolution in yoloV7 model to use the dilated convolution？" }, { "id": 2302208, "postDate": "2023-06-14T12:08:01.737Z", "content": "

@maksimovka for sharing. Could you explain what is dilated mask?

", "rawMarkdown": "@maksimovka for sharing. Could you explain what is dilated mask?", "replies": [ { "id": 2302288, "postDate": "2023-06-14T13:00:37.467Z", "content": "

I used https://scikit-image.org/docs/stable/api/skimage.morphology.html#skimage.morphology.binary_dilation and here you can read what is dilation https://en.wikipedia.org/wiki/Dilation_(morphology)

", "rawMarkdown": "I used https://scikit-image.org/docs/stable/api/skimage.morphology.html#skimage.morphology.binary_dilation and here you can read what is dilation https://en.wikipedia.org/wiki/Dilation_(morphology)", "votes": 4 } ] }, { "id": 2301512, "postDate": "2023-06-14T02:06:57.423Z", "content": "

what is dilation pp?

", "rawMarkdown": "what is dilation pp?", "replies": [ { "id": 2301524, "postDate": "2023-06-14T02:21:26.827Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2302279, "postDate": "2023-06-14T12:51:37.210Z", "content": "
img = cv2.dilate(img, kernel, iterations)\n
", "rawMarkdown": "```python\nimg = cv2.dilate(img, kernel, iterations)\n\n```", "votes": 3 }, { "id": 2302287, "postDate": "2023-06-14T12:59:52.160Z", "content": "

I used https://scikit-image.org/docs/stable/api/skimage.morphology.html#skimage.morphology.binary_dilation

", "rawMarkdown": "I used https://scikit-image.org/docs/stable/api/skimage.morphology.html#skimage.morphology.binary_dilation", "votes": 2, "replies": [ { "id": 2302863, "postDate": "2023-06-14T22:55:29.540Z", "content": "

thanks for sharing!
\nUnfortunately for me this PP lowered LB. It may be because ds2 is not used

", "rawMarkdown": "thanks for sharing!\nUnfortunately for me this PP lowered LB. It may be because ds2 is not used\n", "votes": 3 }, { "id": 2321022, "postDate": "2023-06-28T08:28:39.200Z", "content": "

train: ds1+2(WSI1~4) val:ds1
\npp:cv2 dialate 3*3
\nLB349->483

\n

If we use dataset 2 ,we can use dialate PP.

", "rawMarkdown": "train: ds1+2(WSI1~4) val:ds1\npp:cv2 dialate 3*3\nLB349->483\n\nIf we use dataset 2 ,we can use dialate PP.", "votes": 2 }, { "id": 2321617, "postDate": "2023-06-28T17:40:57.407Z", "content": "

It is really interesting. Why dataset 2 with the dilate can make better LB score?

", "rawMarkdown": "It is really interesting. Why dataset 2 with the **dilate** can make better LB score?" }, { "id": 2322851, "postDate": "2023-06-29T14:27:22.727Z", "content": "

I am speculating that annotations in dataset 2 are sparse means lesser number of pixels are annotated and not lesser segments of the image. So, for every segment, they have not annotated the entire pixels falling under that segment. and when we dilate around a mask, we are compensating for the lack of annotation in the neighbourhood of a mask. This also means, that dilating the masks of the dataset 2 during training should improve the results

", "rawMarkdown": "I am speculating that annotations in dataset 2 are sparse means lesser number of pixels are annotated and not lesser segments of the image. So, for every segment, they have not annotated the entire pixels falling under that segment. and when we dilate around a mask, we are compensating for the lack of annotation in the neighbourhood of a mask. This also means, that dilating the masks of the dataset 2 during training should improve the results", "votes": 2 } ] } ] }, { "id": 2315311, "postDate": "2023-06-24T03:44:17.660Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2328654, "author_name": "Sol Erika Boman", "author_url": "", "post_date": "2023-07-03T18:52:02.393000", "content": "

Consider the pixel-level softmax output for a blood vessel region. Near the center of the blood vessel, the outputs will be close to 1.0. Near the edges of the blood vessel, the outputs will be smaller, maybe even ~0.1-0.3 right at the edges. If your model has a condition such as \"mask = softmax > 0.5\", then your binary mask will only include the center of the blood vessels, and dilation will help you include the edges of the blood vessel. But if you used a different condition, such as \"mask = softmax > 0.1\", you are likely including the whole blood vessel already (plus maybe some background around it), and dilation will not help. That is what I think is happening.

", "votes": 6, "replies": [ { "id": 2328927, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2023-07-04T01:15:38.640000", "content": "

nice point, need to check it

", "votes": 0, "replies": [] } ] }, { "id": 2301847, "author_name": "Chenglu", "author_url": "", "post_date": "2023-06-14T07:34:37.250000", "content": "

same here… 0.297 to 0.397, nicely found bro.

", "votes": 5, "replies": [] }, { "id": 2322028, "author_name": "hengck23", "author_url": "", "post_date": "2023-06-29T03:34:52.917000", "content": "

using opencv dilation:

\n", "votes": 3, "replies": [ { "id": 2322522, "author_name": "", "author_url": "", "post_date": "2023-06-29T10:21:55.200000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 2300809, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2023-06-13T13:12:35.073000", "content": "

UPD - for my local CV I have the same behaviour - if I train on dilated masks - score inreases on local validation as well

", "votes": 3, "replies": [ { "id": 2301094, "author_name": "fnands", "author_url": "", "post_date": "2023-06-13T16:02:21.067000", "content": "

Very interesting. Have you observed a difference between dataset 1/dataset 2?
\nAs they were annotated differently, and we the test set is purely from dataset 1, it might be that the model learns something in-between dataset 1's annotations and dataset 2's annotations, but the LB favours 1.

\n

Pure speculation, btw.

", "votes": 0, "replies": [ { "id": 2301415, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2023-06-13T22:37:51.217000", "content": "

need to try, now I have split that mixes everything

", "votes": 0, "replies": [] }, { "id": 2346112, "author_name": "bent1e", "author_url": "", "post_date": "2023-07-16T03:25:20.790000", "content": "

Is the train on dilated masks effective?

", "votes": 0, "replies": [] } ] } ] }, { "id": 2300785, "author_name": "Leon", "author_url": "", "post_date": "2023-06-13T12:46:40.850000", "content": "

The key is whether your CV has such an improvement. LB is not always reliable. Like here, https://www.kaggle.com/competitions/tensorflow-great-barrier-reef/discussion/307605. By modifying bounding boxes for no reason, he can easily increase LB to the 2nd place.

", "votes": 3, "replies": [ { "id": 2300801, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2023-06-13T13:04:20.107000", "content": "

in your case does dilation also help for public score?

", "votes": 0, "replies": [ { "id": 2300947, "author_name": "Leon", "author_url": "", "post_date": "2023-06-13T14:26:37.770000", "content": "

I haven't tried it.

", "votes": 0, "replies": [] } ] } ] }, { "id": 2314195, "author_name": "hengck23", "author_url": "", "post_date": "2023-06-23T08:48:28.847000", "content": "

if the truth masks are large and has many pixels, effects of dilation is less.

\n

so maybe public test has more smaller maskes?

\n
\n

another way to prove if the test mask maskes are small is to try input size 512,640,1024 at submission.
\nThe lb score should imporve and the effects of dilation decreases.

", "votes": 4, "replies": [ { "id": 2315314, "author_name": "", "author_url": "", "post_date": "2023-06-24T03:46:36.643000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2303240, "author_name": "fnands", "author_url": "", "post_date": "2023-06-15T07:23:42.177000", "content": "

Same. 0.250 -> 0.332
\nA bit scary, would like to figure out why it works before just blindly using it.

", "votes": 1, "replies": [] }, { "id": 2301156, "author_name": "Abdelfattah Toulaoui", "author_url": "", "post_date": "2023-06-13T16:51:01.820000", "content": "

Just a guess, but maybe our models are quite conservative when generating the masks.

\n

So some of the edges of the image that were manually annotated as masks are ignored by our models. If so, dilating the masks trains the model to overshoot a bit and gives it a push to reach the IoU threshold of 0.6

\n

Just my interpretation, you may need to inspect the generated masks with and without to confirm.

", "votes": 2, "replies": [ { "id": 2314100, "author_name": "HongCheng", "author_url": "", "post_date": "2023-06-23T06:43:52.360000", "content": "

any progress or justice for this method？

", "votes": 0, "replies": [ { "id": 2321756, "author_name": "Abdelfattah Toulaoui", "author_url": "", "post_date": "2023-06-28T21:00:41.423000", "content": "

I haven't tried it yet, at this point I'm only testing the base methods without dilation.

", "votes": 0, "replies": [] } ] } ] }, { "id": 2314098, "author_name": "HongCheng", "author_url": "", "post_date": "2023-06-23T06:36:16.113000", "content": "

how to modify the convolution in yoloV7 model to use the dilated convolution？

", "votes": 0, "replies": [] }, { "id": 2302208, "author_name": "Thang Vu", "author_url": "", "post_date": "2023-06-14T12:08:01.737000", "content": "

@maksimovka for sharing. Could you explain what is dilated mask?

", "votes": 0, "replies": [ { "id": 2302288, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2023-06-14T13:00:37.467000", "content": "

I used https://scikit-image.org/docs/stable/api/skimage.morphology.html#skimage.morphology.binary_dilation and here you can read what is dilation https://en.wikipedia.org/wiki/Dilation_(morphology)

", "votes": 4, "replies": [] } ] }, { "id": 2301512, "author_name": "patriot", "author_url": "", "post_date": "2023-06-14T02:06:57.423000", "content": "

what is dilation pp?

", "votes": 0, "replies": [ { "id": 2301524, "author_name": "", "author_url": "", "post_date": "2023-06-14T02:21:26.827000", "content": "", "votes": 0, "replies": [] }, { "id": 2302279, "author_name": "Igor Krashenyi", "author_url": "", "post_date": "2023-06-14T12:51:37.210000", "content": "
img = cv2.dilate(img, kernel, iterations)\n
", "votes": 3, "replies": [] }, { "id": 2302287, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2023-06-14T12:59:52.160000", "content": "

I used https://scikit-image.org/docs/stable/api/skimage.morphology.html#skimage.morphology.binary_dilation

", "votes": 2, "replies": [ { "id": 2302863, "author_name": "patriot", "author_url": "", "post_date": "2023-06-14T22:55:29.540000", "content": "

thanks for sharing!
\nUnfortunately for me this PP lowered LB. It may be because ds2 is not used

", "votes": 3, "replies": [] }, { "id": 2321022, "author_name": "patriot", "author_url": "", "post_date": "2023-06-28T08:28:39.200000", "content": "

train: ds1+2(WSI1~4) val:ds1
\npp:cv2 dialate 3*3
\nLB349->483

\n

If we use dataset 2 ,we can use dialate PP.

", "votes": 2, "replies": [] }, { "id": 2321617, "author_name": "HongCheng", "author_url": "", "post_date": "2023-06-28T17:40:57.407000", "content": "

It is really interesting. Why dataset 2 with the dilate can make better LB score?

", "votes": 0, "replies": [] }, { "id": 2322851, "author_name": "Sathishkumartheta", "author_url": "", "post_date": "2023-06-29T14:27:22.727000", "content": "

I am speculating that annotations in dataset 2 are sparse means lesser number of pixels are annotated and not lesser segments of the image. So, for every segment, they have not annotated the entire pixels falling under that segment. and when we dilate around a mask, we are compensating for the lack of annotation in the neighbourhood of a mask. This also means, that dilating the masks of the dataset 2 during training should improve the results

", "votes": 2, "replies": [] } ] } ] }, { "id": 2315311, "author_name": "", "author_url": "", "post_date": "2023-06-24T03:44:17.660000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2300734": "Hi everyone!\nSo I was stuck at some point that all my models where showing 0.26-0.27 on the LB - and I decided to add postprocessing to masks - and after I did it (I have used dilation) score improved from 0.239 to 0.345 - that is the huge jump - so my question to participatnts is next - someone understands why this happens? maybe it makes sense to train on already dilated masks?", "2328654": "Consider the pixel-level softmax output for a blood vessel region. Near the center of the blood vessel, the outputs will be close to 1.0. Near the edges of the blood vessel, the outputs will be smaller, maybe even ~0.1-0.3 right at the edges. If your model has a condition such as \"mask = softmax > 0.5\", then your binary mask will only include the center of the blood vessels, and dilation will help you include the edges of the blood vessel. But if you used a different condition, such as \"mask = softmax > 0.1\", you are likely including the whole blood vessel already (plus maybe some background around it), and dilation will not help. That is what I think is happening.", "2301847": "same here... 0.297 to 0.397, nicely found bro.", "2322028": "using opencv dilation:\n- cv2.dilate(m,kernel=(3x3 box), iteration=3) is optimal\n- iteration=2 has slight lower lb\n- iteration=4 drops significantly\n", "2300809": "UPD - for my local CV I have the same behaviour - if I train on dilated masks - score inreases on local validation as well", "2300785": "The key is whether your CV has such an improvement. LB is not always reliable. Like here, https://www.kaggle.com/competitions/tensorflow-great-barrier-reef/discussion/307605. By modifying bounding boxes for no reason, he can easily increase LB to the 2nd place.", "2314195": "if the truth masks are large and has many pixels, effects of dilation is less.\n\nso maybe public test has more smaller maskes?\n\n---\n\nanother way to prove if the test mask maskes are small is to try input size 512,640,1024 at submission.\nThe lb score should imporve and the effects of dilation decreases.", "2303240": "Same. 0.250 -> 0.332\nA bit scary, would like to figure out why it works before just blindly using it. \n", "2301156": "Just a guess, but maybe our models are quite conservative when generating the masks.\n\nSo some of the edges of the image that were manually annotated as masks are ignored by our models. If so, dilating the masks trains the model to overshoot a bit and gives it a push to reach the IoU threshold of 0.6\n\nJust my interpretation, you may need to inspect the generated masks with and without to confirm.", "2314098": "how to modify the convolution in yoloV7 model to use the dilated convolution？", "2302208": "@maksimovka for sharing. Could you explain what is dilated mask?", "2301512": "what is dilation pp?", "2315311": "" } }