{ "id": 475657, "title": "2nd place solution", "url": "/competitions/blood-vessel-segmentation/writeups/ryo-2nd-place-solution", "author_name": "", "post_date": "2024-02-13T03:54:58.030Z", "votes": 33, "comment_count": 12, "views": 0, "content": "

What's happened? My name is written at 2nd place? I remember my public score was 0.43 and my place was 1052…

\n

Overview

\n

My solution consists of U-Net3D (128x128x32), threshold adjustment, and post-processing to remove unconnected vessels as they are false positives.

\n\n

https://github.com/tail-island/blood-vessel-segmentation
\nhttps://www.kaggle.com/code/ojimaryoji/sennet-hoa-2nd-place-solution?scriptVersionId=159388443

\n

Data

\n

To make it easier cutting out the data, I created a 3D Numpy array and adjusted the scale. I created all and dense data because I was planning to do curriculum learning in the order of all to dense. However, since it took a long time to learn in my PC, I only trained on sparse data this time. Also, I did not normalize or clipping the data because I thought there should not be a big difference since the data is visible to the human eye.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/create_volumetric_images.py

\n

Train

\n

I generate data from random positions and rotations in each n epochs. To reduce data generation time, I used multiple processes.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/dataset.py

\n

The neural network is U-Net3D.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/model.py

\n

I used binary-focal loss. Optimizer is AdamW and learning rate is scheduled by cosine-decay.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/train_0.py

\n

Submit

\n

Prediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit.py

\n

Searching the big blood vessel chunk (and clip) version, private score is 0.756793 and public score is 0.000000…

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit_.py

\n

Other scores…

\n

\n

What's happend?

\n

What's happened? My name is written at 2nd place? I remember my public score was 0.43 and my place was 1052…

\n

Overview

\n

My solution consists of U-Net3D (128x128x32), threshold adjustment, and post-processing to remove unconnected vessels as they are false positives.

\n\n

https://github.com/tail-island/blood-vessel-segmentation

\n

Data

\n

To make it easier cutting out the data, I created a 3D Numpy array and adjusted the scale. I created all and dense data because I was planning to do curriculum learning in the order of all to dense. However, since it took a long time to learn in my PC, I only trained on sparse data this time. Also, I did not normalize or clipping the data because I thought there should not be a big difference since the data is visible to the human eye.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/create_volumetric_images.py

\n

Train

\n

I generate data from random positions and rotations in each n epochs. To reduce data generation time, I used multiple processes.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/dataset.py

\n

The neural network is U-Net3D.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/model.py

\n

I used binary-focal loss. Optimizer is AdamW and learning rate is scheduled by cosine-decay.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/train_0.py

\n

Submit

\n

Prediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit.py

\n

Searching the big blood vessel chunk (and clip) version, private score is 0.756793 and public score is 0.000000…

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit_.py

\n

Other scores…

\n

\n

What's happend?

\n

\n

\n

\n

What's happened?

", "messages": [ { "id": "2644091", "postDate": "02/09/2024 09:29:50", "content": "

What's happened? My name is written at 2nd place? I remember my public score was 0.43 and my place was 1052…

\n

Overview

\n

My solution consists of U-Net3D (128x128x32), threshold adjustment, and post-processing to remove unconnected vessels as they are false positives.

\n\n

https://github.com/tail-island/blood-vessel-segmentation
\nhttps://www.kaggle.com/code/ojimaryoji/sennet-hoa-2nd-place-solution?scriptVersionId=159388443

\n

Data

\n

To make it easier cutting out the data, I created a 3D Numpy array and adjusted the scale. I created all and dense data because I was planning to do curriculum learning in the order of all to dense. However, since it took a long time to learn in my PC, I only trained on sparse data this time. Also, I did not normalize or clipping the data because I thought there should not be a big difference since the data is visible to the human eye.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/create_volumetric_images.py

\n

Train

\n

I generate data from random positions and rotations in each n epochs. To reduce data generation time, I used multiple processes.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/dataset.py

\n

The neural network is U-Net3D.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/model.py

\n

I used binary-focal loss. Optimizer is AdamW and learning rate is scheduled by cosine-decay.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/train_0.py

\n

Submit

\n

Prediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit.py

\n

Searching the big blood vessel chunk (and clip) version, private score is 0.756793 and public score is 0.000000…

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit_.py

\n

Other scores…

\n

\n

What's happend?

\n

What's happened? My name is written at 2nd place? I remember my public score was 0.43 and my place was 1052…

\n

Overview

\n

My solution consists of U-Net3D (128x128x32), threshold adjustment, and post-processing to remove unconnected vessels as they are false positives.

\n\n

https://github.com/tail-island/blood-vessel-segmentation

\n

Data

\n

To make it easier cutting out the data, I created a 3D Numpy array and adjusted the scale. I created all and dense data because I was planning to do curriculum learning in the order of all to dense. However, since it took a long time to learn in my PC, I only trained on sparse data this time. Also, I did not normalize or clipping the data because I thought there should not be a big difference since the data is visible to the human eye.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/create_volumetric_images.py

\n

Train

\n

I generate data from random positions and rotations in each n epochs. To reduce data generation time, I used multiple processes.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/dataset.py

\n

The neural network is U-Net3D.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/model.py

\n

I used binary-focal loss. Optimizer is AdamW and learning rate is scheduled by cosine-decay.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/train_0.py

\n

Submit

\n

Prediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks.

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit.py

\n

Searching the big blood vessel chunk (and clip) version, private score is 0.756793 and public score is 0.000000…

\n

https://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit_.py

\n

Other scores…

\n

\n

What's happend?

\n

\n

\n

\n

What's happened?

", "rawMarkdown": "What's happened? My name is written at 2nd place? I remember my public score was 0.43 and my place was 1052...\n\n# Overview\n\nMy solution consists of U-Net3D (128x128x32), threshold adjustment, and post-processing to remove unconnected vessels as they are false positives.\n\n* data augmentation using random rotation (and position), same as 1st place solution.\n* U-Net3D. I assumed that 3D would be more accurate because it provides more information. I think this assumption might be wrong, since the 1st place solution uses 2.5D, .\n* Binary-focal loss. Since there are a few positive data.\n* Adjusting threshold. Since the volume ratio of blood vessels are not so different betwwen persons, the threshold is set according to the ratio.\n* Post-processing. Since blood vessels are supposed to be connected, extract small chunks with depth-first-search and remove them.\n\nhttps://github.com/tail-island/blood-vessel-segmentation\nhttps://www.kaggle.com/code/ojimaryoji/sennet-hoa-2nd-place-solution?scriptVersionId=159388443\n\n# Data\n\nTo make it easier cutting out the data, I created a 3D Numpy array and adjusted the scale. I created *all* and *dense* data because I was planning to do curriculum learning in the order of *all* to *dense*. However, since it took a long time to learn in my PC, I only trained on sparse data this time. Also, I did not normalize or clipping the data because I thought there should not be a big difference since the data is visible to the human eye.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/create_volumetric_images.py\n\n# Train\n\nI generate data from random positions and rotations in each *n* epochs. To reduce data generation time, I used multiple processes.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/dataset.py\n\nThe neural network is U-Net3D.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/model.py\n\nI used binary-focal loss. Optimizer is AdamW and learning rate is scheduled by cosine-decay.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/train_0.py\n\n# Submit\n\nPrediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit.py\n\nSearching the big blood vessel chunk (and clip) version, private score is 0.756793 and public score is 0.000000...\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit_.py\n\nOther scores...\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5895718%2Ff7be4975a186ddc226fd463f32632d0a%2Fscores.png?generation=1707479299719710&alt=media)\n\n# What's happend?\n\nWhat's happened? My name is written at 2nd place? I remember my public score was 0.43 and my place was 1052...\n\n# Overview\n\nMy solution consists of U-Net3D (128x128x32), threshold adjustment, and post-processing to remove unconnected vessels as they are false positives.\n\n* data augmentation using random rotation (and position), same as 1st place solution.\n* U-Net3D. I assumed that 3D would be more accurate because it provides more information. I think this assumption might be wrong, since the 1st place solution uses 2.5D, .\n* Binary-focal loss. Since there are a few positive data.\n* Adjusting threshold. Since the volume ratio of blood vessels are not so different betwwen persons, the threshold is set according to the ratio.\n* Post-processing. Since blood vessels are supposed to be connected, extract small chunks with depth-first-search and remove them.\n\nhttps://github.com/tail-island/blood-vessel-segmentation\n\n# Data\n\nTo make it easier cutting out the data, I created a 3D Numpy array and adjusted the scale. I created *all* and *dense* data because I was planning to do curriculum learning in the order of *all* to *dense*. However, since it took a long time to learn in my PC, I only trained on sparse data this time. Also, I did not normalize or clipping the data because I thought there should not be a big difference since the data is visible to the human eye.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/create_volumetric_images.py\n\n# Train\n\nI generate data from random positions and rotations in each *n* epochs. To reduce data generation time, I used multiple processes.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/dataset.py\n\nThe neural network is U-Net3D.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/model.py\n\nI used binary-focal loss. Optimizer is AdamW and learning rate is scheduled by cosine-decay.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/train_0.py\n\n# Submit\n\nPrediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit.py\n\nSearching the big blood vessel chunk (and clip) version, private score is 0.756793 and public score is 0.000000...\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit_.py\n\nOther scores...\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5895718%2Ff7be4975a186ddc226fd463f32632d0a%2Fscores.png?generation=1707479299719710&alt=media)\n\n# What's happend?\n\n~~Maybe public data is for the first *x*% of the images, I think. It contains only the end part of the blood vessels; the root part is not included. So, it seems that my post-processing would have resulted in a lower score.~~\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5895718%2Fc3db76cc66c0f14a28e86393c735b925%2Fpublic-private.png?generation=1707470652290411&alt=media)\n\n~~I gave up to improve my program quite early because my public score was too low. And some other Kagglers continued to improve their programs for the end part of the blood vessels in public data. I suspect that their improvements were not very effective in the private data that contains the root of the blood vessels. In other words, my 2nd place would be caused by LUCK...~~\n\nWhat's happened?", "votes": null }, { "id": "2644355", "postDate": "02/09/2024 12:14:42", "content": "

Congratulations!
\nIt shows that surface dice score is a poor choice for the final metric.

", "rawMarkdown": "Congratulations! \nIt shows that surface dice score is a poor choice for the final metric.", "votes": null }, { "id": "2644356", "postDate": "02/09/2024 12:14:49", "content": "

\"Maybe public data is for the first x% of the images, I think. It contains only the end part of the blood vessels; the root part is not included. So, it seems that my post-processing would have resulted in a lower score.\"

\n

NO there are 2 different kidney for private and public, you can check the updated data description page

\n
\n

good work! sometimes, the simplest solution works for the most uncertain cases!

", "rawMarkdown": "\"Maybe public data is for the first x% of the images, I think. It contains only the end part of the blood vessels; the root part is not included. So, it seems that my post-processing would have resulted in a lower score.\"\n\nNO there are 2 different kidney for private and public, you can check the updated data description page\n\n---\n\ngood work! sometimes, the simplest solution works for the most uncertain cases!", "votes": null }, { "id": "2644375", "postDate": "02/09/2024 12:32:31", "content": "

Thanks! Please give me time for checking.

", "rawMarkdown": "Thanks! Please give me time for checking.", "votes": null }, { "id": "2644379", "postDate": "02/09/2024 12:35:37", "content": "

Top1 solution also rotates in 3D space while training. So maybe training with 3d rotation augment is the key of this challenge. Hence, even your public score is low, your solution is insightful.

\n

Also, removing small unconnected chunks is what I planned to do. But I joined this competition late and don't have time to implement it(Another reason is that I didn't expect it cause such a huge improvement).

\n

In words, I think although your solution only have 2 two key points, both points are very important and valuable.
\nCongratulations for solo top 2. @ojimaryoji .

", "rawMarkdown": "Top1 solution also rotates in 3D space while training. So maybe training with 3d rotation augment is the key of this challenge. Hence, even your public score is low, your solution is insightful. \n\nAlso, removing small unconnected chunks is what I planned to do. But I joined this competition late and don't have time to implement it(Another reason is that I didn't expect it cause such a huge improvement).\n\nIn words, I think although your solution only have 2 two key points, both points are very important and valuable.\nCongratulations for solo top 2. @ojimaryoji .", "votes": null }, { "id": "2644385", "postDate": "02/09/2024 12:43:11", "content": "

Thank you very much. I had updated my post.

\n
\n

Public Test:
\n Continuous 3D part of a whole human kidney…
\n Private Test:
\n Continuous 3D part of a whole human kidney…

\n
\n

What's happened?

\n

", "rawMarkdown": "Thank you very much. I had updated my post.\n\n>Public Test:\n>Continuous 3D part of a whole human kidney...\n>Private Test:\n>Continuous 3D part of a whole human kidney...\n\nWhat's happened?\n\n~~\"part of\"?~~", "votes": null }, { "id": "2644556", "postDate": "02/09/2024 15:14:05", "content": "

I checked my submissions.

\n

Version 12: clip + search large size chunk (>20% of candidates) -> private score: 0.756793, public score: 0.

\n\n

Version 32: no clip + delete size >= 100 chunks -> private score: 0.759959, public score: 0.043141.

\n

Version 33: no clip + delete size ≥ 1000 chunks → private score: 0.742442, public score: 0.041478.

\n\n

What happened? I think any of the following is happened.

\n
    \n
  1. My program luckily overfit the public data.
    2. \n
  2. My program overfit the training data + the private data and the training data are very similar + the public data are too different.
    \nAnd fitting to the public data will reduce accuracy on the private data?
    3. \n
  3. Or, surface-dice-metrics is the source of this confusion…
    4. \n
\n

Please post what do you think, great Kagglers.

", "rawMarkdown": "I checked my submissions.\n\nVersion 12: clip + search large size chunk (>20% of candidates) -> private score: 0.756793, public score: 0.\n\n* My program was able to find a large chunk of blood vessels in the private data, but could not find a large chunk in the public data.\n* If the large chunk is not found, my program uses prediction. -> My program could not find the blood vessels from the public data.\n\nVersion 32: no clip + delete size >= 100 chunks -> private score: 0.759959, public score: 0.043141.\n\nVersion 33: no clip + delete size ≥ 1000 chunks → private score: 0.742442, public score: 0.041478.\n\n* Version 33 has a lower score, so the 100-1000 size chunks are blood vessels. -> Without clipping, the large chunk are split into small unconnected chunks.\n* However, the score does not change much. -> My program found very few blood vessels in the private data.\n\nWhat happened? I think any of the following is happened.\n\n1. My program luckily overfit the public data.\n2. My program overfit the training data + the private data and the training data are very similar + the public data are too different.\n And fitting to the public data will reduce accuracy on the private data?\n3. Or, surface-dice-metrics is the source of this confusion...\n\nPlease post what do you think, great Kagglers.", "votes": null }, { "id": "2644907", "postDate": "02/09/2024 19:02:44", "content": "

Truly incredible, I am astounded how this works great strategy with the post-processing, I wish I had thought of that one!

", "rawMarkdown": "Truly incredible, I am astounded how this works great strategy with the post-processing, I wish I had thought of that one!", "votes": null }, { "id": "2644989", "postDate": "02/09/2024 20:12:37", "content": "

And you, my friend, you are a true genius.

", "rawMarkdown": "And you, my friend, you are a true genius.", "votes": null }, { "id": "2645075", "postDate": "02/09/2024 23:24:31", "content": "

Congratulations

", "rawMarkdown": "Congratulations", "votes": null }, { "id": "2645537", "postDate": "02/10/2024 10:10:00", "content": "

Congratulations @ojimaryoji !! It is good to see you made a 2nd place with 3D. There was discussion here around results of 3D during the competition and seemed it would not be promising but you found a way to do it. And probably quite useful for the hosts - indicated as one of the key features of their imaging technique.

\n
\n

Prediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks

\n
\n

This may have been key to eliminating false positives - but the differences in resolution for public and private perhaps affected how each scored? As @hengck23 posted

\n
\n

your solution need to:
\n 1) either remove false positive
\n 2) limit the number of 3d connected component objects

\n
\n

Certainly using connected-components-3d cc3d.dust post processing (also from hengck23) gave improvements to private LB. (not that I selected of course!)

\n

It might be good if the hosts released the public and private kidney images to analyse why what worked for one did not for the other.

\n

How did you select your 2 final submissions or were they autoselected? Pretty insane all the 0s !! You must be happy with the end result.

", "rawMarkdown": "Congratulations @ojimaryoji !! It is good to see you made a 2nd place with 3D. There was discussion [here](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/461213) around results of 3D during the competition and seemed it would not be promising but you found a way to do it. And probably quite useful for the hosts - indicated as one of the key features of their imaging technique. \n\n>Prediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks\n\nThis may have been key to eliminating false positives - but the differences in resolution for public and private perhaps affected how each scored? As @hengck23 [posted](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/457702)\n\n>your solution need to:\n1) either remove false positive\n2) limit the number of 3d connected component objects \n\nCertainly using connected-components-3d cc3d.dust post processing (also from hengck23) gave improvements to private LB. (not that I selected of course!)\n\nIt might be good if the hosts released the public and private kidney images to analyse why what worked for one did not for the other.\n\nHow did you select your 2 final submissions or were they autoselected? Pretty insane all the 0s !! You must be happy with the end result.", "votes": null }, { "id": "2645659", "postDate": "02/10/2024 11:51:32", "content": "

It might be good if the hosts released the public and private kidney images to analyse why what worked for one did not for the other.

\n

Yes, I also think that it would be useful not only for us, the participants, but also for the host to read the consequent comments.

\n

Now we can only wonder and speculate why some solutions (nearly \"traditionally\" for some participants as me unselected) worked so well on private dataset though bringing very bad results on public dataset.

\n

Please, @jonathanmcwilliams , @clairewalsh , would the release be possible?

", "rawMarkdown": "`It might be good if the hosts released the public and private kidney images to analyse why what worked for one did not for the other.`\n\nYes, I also think that it would be useful not only for us, the participants, but also for the host to read the consequent comments.\n\nNow we can only wonder and speculate why some solutions (nearly \"traditionally\" for some participants as me unselected) worked so well on private dataset though bringing very bad results on public dataset.\n\nPlease, @jonathanmcwilliams , @clairewalsh , would the release be possible?", "votes": null }, { "id": "2647944", "postDate": "02/11/2024 22:59:03", "content": "

Thank you very much. I will study \"connected-components-3d cc3d.dust\" as you suggested. It's beautiful.

\n

Final submissions were auto-selected. It's too difficult to choice from 0, 0, 0.043, 0.002…

", "rawMarkdown": "Thank you very much. I will study \"connected-components-3d cc3d.dust\" as you suggested. It's beautiful.\n\nFinal submissions were auto-selected. It's too difficult to choice from 0, 0, 0.043, 0.002...", "votes": null } ], "comments": [ { "id": 2644355, "author_name": "jankowalski2000", "author_url": "", "post_date": "02/09/2024 12:14:42", "content": "

Congratulations!
\nIt shows that surface dice score is a poor choice for the final metric.

", "votes": null, "replies": [] }, { "id": 2644356, "author_name": "hengck23", "author_url": "", "post_date": "02/09/2024 12:14:49", "content": "

\"Maybe public data is for the first x% of the images, I think. It contains only the end part of the blood vessels; the root part is not included. So, it seems that my post-processing would have resulted in a lower score.\"

\n

NO there are 2 different kidney for private and public, you can check the updated data description page

\n
\n

good work! sometimes, the simplest solution works for the most uncertain cases!

", "votes": null, "replies": [ { "id": 2644375, "author_name": "ojimaryoji", "author_url": "", "post_date": "02/09/2024 12:32:31", "content": "

Thanks! Please give me time for checking.

", "votes": null, "replies": [ { "id": 2644385, "author_name": "ojimaryoji", "author_url": "", "post_date": "02/09/2024 12:43:11", "content": "

Thank you very much. I had updated my post.

\n
\n

Public Test:
\n Continuous 3D part of a whole human kidney…
\n Private Test:
\n Continuous 3D part of a whole human kidney…

\n
\n

What's happened?

\n

", "votes": null, "replies": [] } ] } ] }, { "id": 2644379, "author_name": "forcewithme", "author_url": "", "post_date": "02/09/2024 12:35:37", "content": "

Top1 solution also rotates in 3D space while training. So maybe training with 3d rotation augment is the key of this challenge. Hence, even your public score is low, your solution is insightful.

\n

Also, removing small unconnected chunks is what I planned to do. But I joined this competition late and don't have time to implement it(Another reason is that I didn't expect it cause such a huge improvement).

\n

In words, I think although your solution only have 2 two key points, both points are very important and valuable.
\nCongratulations for solo top 2. @ojimaryoji .

", "votes": null, "replies": [] }, { "id": 2644556, "author_name": "ojimaryoji", "author_url": "", "post_date": "02/09/2024 15:14:05", "content": "

I checked my submissions.

\n

Version 12: clip + search large size chunk (>20% of candidates) -> private score: 0.756793, public score: 0.

\n\n

Version 32: no clip + delete size >= 100 chunks -> private score: 0.759959, public score: 0.043141.

\n

Version 33: no clip + delete size ≥ 1000 chunks → private score: 0.742442, public score: 0.041478.

\n\n

What happened? I think any of the following is happened.

\n
    \n
  1. My program luckily overfit the public data.
    2. \n
  2. My program overfit the training data + the private data and the training data are very similar + the public data are too different.
    \nAnd fitting to the public data will reduce accuracy on the private data?
    3. \n
  3. Or, surface-dice-metrics is the source of this confusion…
    4. \n
\n

Please post what do you think, great Kagglers.

", "votes": null, "replies": [] }, { "id": 2644907, "author_name": "cody11null", "author_url": "", "post_date": "02/09/2024 19:02:44", "content": "

Truly incredible, I am astounded how this works great strategy with the post-processing, I wish I had thought of that one!

", "votes": null, "replies": [] }, { "id": 2644989, "author_name": "pkuszboi", "author_url": "", "post_date": "02/09/2024 20:12:37", "content": "

And you, my friend, you are a true genius.

", "votes": null, "replies": [] }, { "id": 2645075, "author_name": "humaperveen", "author_url": "", "post_date": "02/09/2024 23:24:31", "content": "

Congratulations

", "votes": null, "replies": [] }, { "id": 2645537, "author_name": "something4kag", "author_url": "", "post_date": "02/10/2024 10:10:00", "content": "

Congratulations @ojimaryoji !! It is good to see you made a 2nd place with 3D. There was discussion here around results of 3D during the competition and seemed it would not be promising but you found a way to do it. And probably quite useful for the hosts - indicated as one of the key features of their imaging technique.

\n
\n

Prediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks

\n
\n

This may have been key to eliminating false positives - but the differences in resolution for public and private perhaps affected how each scored? As @hengck23 posted

\n
\n

your solution need to:
\n 1) either remove false positive
\n 2) limit the number of 3d connected component objects

\n
\n

Certainly using connected-components-3d cc3d.dust post processing (also from hengck23) gave improvements to private LB. (not that I selected of course!)

\n

It might be good if the hosts released the public and private kidney images to analyse why what worked for one did not for the other.

\n

How did you select your 2 final submissions or were they autoselected? Pretty insane all the 0s !! You must be happy with the end result.

", "votes": null, "replies": [ { "id": 2645659, "author_name": "blankaf", "author_url": "", "post_date": "02/10/2024 11:51:32", "content": "

It might be good if the hosts released the public and private kidney images to analyse why what worked for one did not for the other.

\n

Yes, I also think that it would be useful not only for us, the participants, but also for the host to read the consequent comments.

\n

Now we can only wonder and speculate why some solutions (nearly \"traditionally\" for some participants as me unselected) worked so well on private dataset though bringing very bad results on public dataset.

\n

Please, @jonathanmcwilliams , @clairewalsh , would the release be possible?

", "votes": null, "replies": [] }, { "id": 2647944, "author_name": "ojimaryoji", "author_url": "", "post_date": "02/11/2024 22:59:03", "content": "

Thank you very much. I will study \"connected-components-3d cc3d.dust\" as you suggested. It's beautiful.

\n

Final submissions were auto-selected. It's too difficult to choice from 0, 0, 0.043, 0.002…

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2644091": "What's happened? My name is written at 2nd place? I remember my public score was 0.43 and my place was 1052...\n\n# Overview\n\nMy solution consists of U-Net3D (128x128x32), threshold adjustment, and post-processing to remove unconnected vessels as they are false positives.\n\n* data augmentation using random rotation (and position), same as 1st place solution.\n* U-Net3D. I assumed that 3D would be more accurate because it provides more information. I think this assumption might be wrong, since the 1st place solution uses 2.5D, .\n* Binary-focal loss. Since there are a few positive data.\n* Adjusting threshold. Since the volume ratio of blood vessels are not so different betwwen persons, the threshold is set according to the ratio.\n* Post-processing. Since blood vessels are supposed to be connected, extract small chunks with depth-first-search and remove them.\n\nhttps://github.com/tail-island/blood-vessel-segmentation\nhttps://www.kaggle.com/code/ojimaryoji/sennet-hoa-2nd-place-solution?scriptVersionId=159388443\n\n# Data\n\nTo make it easier cutting out the data, I created a 3D Numpy array and adjusted the scale. I created *all* and *dense* data because I was planning to do curriculum learning in the order of *all* to *dense*. However, since it took a long time to learn in my PC, I only trained on sparse data this time. Also, I did not normalize or clipping the data because I thought there should not be a big difference since the data is visible to the human eye.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/create_volumetric_images.py\n\n# Train\n\nI generate data from random positions and rotations in each *n* epochs. To reduce data generation time, I used multiple processes.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/dataset.py\n\nThe neural network is U-Net3D.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/model.py\n\nI used binary-focal loss. Optimizer is AdamW and learning rate is scheduled by cosine-decay.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/train_0.py\n\n# Submit\n\nPrediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit.py\n\nSearching the big blood vessel chunk (and clip) version, private score is 0.756793 and public score is 0.000000...\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit_.py\n\nOther scores...\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5895718%2Ff7be4975a186ddc226fd463f32632d0a%2Fscores.png?generation=1707479299719710&alt=media)\n\n# What's happend?\n\nWhat's happened? My name is written at 2nd place? I remember my public score was 0.43 and my place was 1052...\n\n# Overview\n\nMy solution consists of U-Net3D (128x128x32), threshold adjustment, and post-processing to remove unconnected vessels as they are false positives.\n\n* data augmentation using random rotation (and position), same as 1st place solution.\n* U-Net3D. I assumed that 3D would be more accurate because it provides more information. I think this assumption might be wrong, since the 1st place solution uses 2.5D, .\n* Binary-focal loss. Since there are a few positive data.\n* Adjusting threshold. Since the volume ratio of blood vessels are not so different betwwen persons, the threshold is set according to the ratio.\n* Post-processing. Since blood vessels are supposed to be connected, extract small chunks with depth-first-search and remove them.\n\nhttps://github.com/tail-island/blood-vessel-segmentation\n\n# Data\n\nTo make it easier cutting out the data, I created a 3D Numpy array and adjusted the scale. I created *all* and *dense* data because I was planning to do curriculum learning in the order of *all* to *dense*. However, since it took a long time to learn in my PC, I only trained on sparse data this time. Also, I did not normalize or clipping the data because I thought there should not be a big difference since the data is visible to the human eye.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/create_volumetric_images.py\n\n# Train\n\nI generate data from random positions and rotations in each *n* epochs. To reduce data generation time, I used multiple processes.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/dataset.py\n\nThe neural network is U-Net3D.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/model.py\n\nI used binary-focal loss. Optimizer is AdamW and learning rate is scheduled by cosine-decay.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/train_0.py\n\n# Submit\n\nPrediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks.\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit.py\n\nSearching the big blood vessel chunk (and clip) version, private score is 0.756793 and public score is 0.000000...\n\nhttps://github.com/tail-island/blood-vessel-segmentation/blob/main/src/submit_.py\n\nOther scores...\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5895718%2Ff7be4975a186ddc226fd463f32632d0a%2Fscores.png?generation=1707479299719710&alt=media)\n\n# What's happend?\n\n~~Maybe public data is for the first *x*% of the images, I think. It contains only the end part of the blood vessels; the root part is not included. So, it seems that my post-processing would have resulted in a lower score.~~\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5895718%2Fc3db76cc66c0f14a28e86393c735b925%2Fpublic-private.png?generation=1707470652290411&alt=media)\n\n~~I gave up to improve my program quite early because my public score was too low. And some other Kagglers continued to improve their programs for the end part of the blood vessels in public data. I suspect that their improvements were not very effective in the private data that contains the root of the blood vessels. In other words, my 2nd place would be caused by LUCK...~~\n\nWhat's happened?", "2644355": "Congratulations! \nIt shows that surface dice score is a poor choice for the final metric.", "2644356": "\"Maybe public data is for the first x% of the images, I think. It contains only the end part of the blood vessels; the root part is not included. So, it seems that my post-processing would have resulted in a lower score.\"\n\nNO there are 2 different kidney for private and public, you can check the updated data description page\n\n---\n\ngood work! sometimes, the simplest solution works for the most uncertain cases!", "2644375": "Thanks! Please give me time for checking.", "2644379": "Top1 solution also rotates in 3D space while training. So maybe training with 3d rotation augment is the key of this challenge. Hence, even your public score is low, your solution is insightful. \n\nAlso, removing small unconnected chunks is what I planned to do. But I joined this competition late and don't have time to implement it(Another reason is that I didn't expect it cause such a huge improvement).\n\nIn words, I think although your solution only have 2 two key points, both points are very important and valuable.\nCongratulations for solo top 2. @ojimaryoji .", "2644385": "Thank you very much. I had updated my post.\n\n>Public Test:\n>Continuous 3D part of a whole human kidney...\n>Private Test:\n>Continuous 3D part of a whole human kidney...\n\nWhat's happened?\n\n~~\"part of\"?~~", "2644556": "I checked my submissions.\n\nVersion 12: clip + search large size chunk (>20% of candidates) -> private score: 0.756793, public score: 0.\n\n* My program was able to find a large chunk of blood vessels in the private data, but could not find a large chunk in the public data.\n* If the large chunk is not found, my program uses prediction. -> My program could not find the blood vessels from the public data.\n\nVersion 32: no clip + delete size >= 100 chunks -> private score: 0.759959, public score: 0.043141.\n\nVersion 33: no clip + delete size ≥ 1000 chunks → private score: 0.742442, public score: 0.041478.\n\n* Version 33 has a lower score, so the 100-1000 size chunks are blood vessels. -> Without clipping, the large chunk are split into small unconnected chunks.\n* However, the score does not change much. -> My program found very few blood vessels in the private data.\n\nWhat happened? I think any of the following is happened.\n\n1. My program luckily overfit the public data.\n2. My program overfit the training data + the private data and the training data are very similar + the public data are too different.\n And fitting to the public data will reduce accuracy on the private data?\n3. Or, surface-dice-metrics is the source of this confusion...\n\nPlease post what do you think, great Kagglers.", "2644907": "Truly incredible, I am astounded how this works great strategy with the post-processing, I wish I had thought of that one!", "2644989": "And you, my friend, you are a true genius.", "2645075": "Congratulations", "2645537": "Congratulations @ojimaryoji !! It is good to see you made a 2nd place with 3D. There was discussion [here](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/461213) around results of 3D during the competition and seemed it would not be promising but you found a way to do it. And probably quite useful for the hosts - indicated as one of the key features of their imaging technique. \n\n>Prediction is made by tiling. get_candidate() finds candidates with a given ratio and get_blood_vessels() removes small unconnected chunks\n\nThis may have been key to eliminating false positives - but the differences in resolution for public and private perhaps affected how each scored? As @hengck23 [posted](https://www.kaggle.com/competitions/blood-vessel-segmentation/discussion/457702)\n\n>your solution need to:\n1) either remove false positive\n2) limit the number of 3d connected component objects \n\nCertainly using connected-components-3d cc3d.dust post processing (also from hengck23) gave improvements to private LB. (not that I selected of course!)\n\nIt might be good if the hosts released the public and private kidney images to analyse why what worked for one did not for the other.\n\nHow did you select your 2 final submissions or were they autoselected? Pretty insane all the 0s !! You must be happy with the end result.", "2645659": "`It might be good if the hosts released the public and private kidney images to analyse why what worked for one did not for the other.`\n\nYes, I also think that it would be useful not only for us, the participants, but also for the host to read the consequent comments.\n\nNow we can only wonder and speculate why some solutions (nearly \"traditionally\" for some participants as me unselected) worked so well on private dataset though bringing very bad results on public dataset.\n\nPlease, @jonathanmcwilliams , @clairewalsh , would the release be possible?", "2647944": "Thank you very much. I will study \"connected-components-3d cc3d.dust\" as you suggested. It's beautiful.\n\nFinal submissions were auto-selected. It's too difficult to choice from 0, 0, 0.043, 0.002..." }, "source": "meta" }