{ "id": 286195, "title": "0.000 submission score", "url": "/competitions/sartorius-cell-instance-segmentation/discussion/286195", "author_name": "", "post_date": "2021-11-08T11:44:27.594270400Z", "votes": 4, "comment_count": 20, "views": 0, "content": "

I trained an unet model with 512, 512 image size and masks with horizontal and vertical flip for about 60 epochs and got binary_cross_entropy loss around 0.11, and predicted masks were also looking similar but not exactly same. But after submission, I got 0.000 submission score.
\nCan someone please review my notebook and explain me the reason for such a low score even predicted mask are looking similar.

\n

This is my notebook https://www.kaggle.com/karan23258/cell-instance-segmentation-unetfromscratch

", "messages": [ { "id": "1575426", "postDate": "11/08/2021 11:44:27", "content": "

I trained an unet model with 512, 512 image size and masks with horizontal and vertical flip for about 60 epochs and got binary_cross_entropy loss around 0.11, and predicted masks were also looking similar but not exactly same. But after submission, I got 0.000 submission score.
\nCan someone please review my notebook and explain me the reason for such a low score even predicted mask are looking similar.

\n

This is my notebook https://www.kaggle.com/karan23258/cell-instance-segmentation-unetfromscratch

", "rawMarkdown": "I trained an unet model with 512, 512 image size and masks with horizontal and vertical flip for about 60 epochs and got binary_cross_entropy loss around 0.11, and predicted masks were also looking similar but not exactly same. But after submission, I got 0.000 submission score. \nCan someone please review my notebook and explain me the reason for such a low score even predicted mask are looking similar.\n\nThis is my notebook [https://www.kaggle.com/karan23258/cell-instance-segmentation-unetfromscratch](url)", "votes": null }, { "id": "1575504", "postDate": "11/08/2021 12:39:42", "content": "

This is a segmentation problem, you need to return a mask for each cell separately, not a single mask with all the cells in the image.

", "rawMarkdown": "This is a segmentation problem, you need to return a mask for each cell separately, not a single mask with all the cells in the image.", "votes": null }, { "id": "1576387", "postDate": "11/09/2021 06:13:43", "content": "

Thank you… So, that means we can't use unet for this task? Because it outputs a single mask. I saw in code section that the notebook with significant cv score has used either mask rcnn or detectron…

", "rawMarkdown": "Thank you... So, that means we can't use unet for this task? Because it outputs a single mask. I saw in code section that the notebook with significant cv score has used either mask rcnn or detectron...", "votes": null }, { "id": "1576474", "postDate": "11/09/2021 08:01:52", "content": "

There are several public notebooks using u-net, some talk of breaking it down into individual masks. But I haven’t seen an example of it successfully done. So it’s either a closely guarded secret, or very hard to do.

", "rawMarkdown": "There are several public notebooks using u-net, some talk of breaking it down into individual masks. But I haven’t seen an example of it successfully done. So it’s either a closely guarded secret, or very hard to do.", "votes": null }, { "id": "1576713", "postDate": "11/09/2021 12:08:12", "content": "

Yeah, I too have not seen in public notebooks, will search outside kaggle for this…Thanks

", "rawMarkdown": "Yeah, I too have not seen in public notebooks, will search outside kaggle for this...Thanks", "votes": null }, { "id": "1577341", "postDate": "11/10/2021 02:14:16", "content": "

I add a post processing step to clip masks to not overlap. Here’s a link. I took the code from @julian3833.

", "rawMarkdown": "I add a post processing step to clip masks to not overlap. Here’s a link. I took the code from @julian3833.", "votes": null }, { "id": "1578508", "postDate": "11/11/2021 06:01:27", "content": "

Hej Karan,

\n

you are generating only one mask per image, but, as already mentioned you should generate a mask per cell in each image.

\n

I used watersheding for that

\n

best
\nivo

\n

.

", "rawMarkdown": "Hej Karan, \n\nyou are generating only one mask per image, but, as already mentioned you should generate a mask per cell in each image.\n\nI used watersheding for that\n\nbest\nivo\n\n\n.", "votes": null }, { "id": "1578836", "postDate": "11/11/2021 11:30:18", "content": "

Thanks Ivo

", "rawMarkdown": "Thanks Ivo", "votes": null }, { "id": "1580099", "postDate": "11/12/2021 13:25:44", "content": "

Can you explain about watersheding, how can I use it for getting the desired output?

", "rawMarkdown": "Can you explain about watersheding, how can I use it for getting the desired output?", "votes": null }, { "id": "1580101", "postDate": "11/12/2021 13:26:04", "content": "

Where is the link?

", "rawMarkdown": "Where is the link?", "votes": null }, { "id": "1580162", "postDate": "11/12/2021 14:32:40", "content": "

Haha that’s a great question! I’m wondering the same thing myself!

\n

Here you go: https://www.kaggle.com/evancofsky/sartorius-torch-lightning-mask-r-cnn

", "rawMarkdown": "Haha that’s a great question! I’m wondering the same thing myself!\n\nHere you go: https://www.kaggle.com/evancofsky/sartorius-torch-lightning-mask-r-cnn", "votes": null }, { "id": "1580175", "postDate": "11/12/2021 14:48:52", "content": "

Hej Karan,

\n

unfortunately I am a complete newbie to deep learning, thus I can not give you any deep insight.
\nHowevwe, please find my post proceysing on your model here: https://www.kaggle.com/ivoflorinscheiber/unetfromscratch-mypostpro/edit/run/79425057

\n

Apparently watersheding works well only for the cortical neuron cultures that display very well seperated cell bodies. For the SH-SY5Y neuroblastoma cell line and for the Astros, in particular(see last figure in the notebook).

\n

it is not working very well, since watersheding needs clear borders (like a cup to fill with water).
\nCurrently my best guess is to somehow combine U-Net with Mask R-CNN (I like to keep the U-Net since it is doing already a great job in semantic segmentation of the cells)

\n

Curious how your team achieved a score of almost 0.300 on the public leaderboard!

\n

Sorry for not being a great help,
\nbest ivo

", "rawMarkdown": "Hej Karan,\n\nunfortunately I am a complete newbie to deep learning, thus I can not give you any deep insight. \nHowevwe, please find my post proceysing on your model here: https://www.kaggle.com/ivoflorinscheiber/unetfromscratch-mypostpro/edit/run/79425057\n\nApparently watersheding works well only for the cortical neuron cultures that display very well seperated cell bodies. For the SH-SY5Y neuroblastoma cell line and for the Astros, in particular(see last figure in the notebook).\n\n it is not working very well, since watersheding needs clear borders (like a cup to fill with water). \nCurrently my best guess is to somehow combine U-Net with Mask R-CNN (I like to keep the U-Net since it is doing already a great job in semantic segmentation of the cells)\n\nCurious how your team achieved a score of almost 0.300 on the public leaderboard!\n\nSorry for not being a great help,\nbest ivo", "votes": null }, { "id": "1580243", "postDate": "11/12/2021 15:44:47", "content": "

\"Curious how your team achieved a score of almost 0.300 on the public leaderboard!\"
\nIt is obvious. Their team may have used the popular method of many kagglers - \"copy and paste\" the highest scoring public kernel.
\nWelcome to Kaggle, Ivo🙁.

", "rawMarkdown": "\"Curious how your team achieved a score of almost 0.300 on the public leaderboard!\"\n\nIt is obvious. Their team may have used the popular method of many kagglers - \"copy and paste\" the highest scoring public kernel.\nWelcome to Kaggle, Ivo🙁.", "votes": null }, { "id": "1583949", "postDate": "11/16/2021 06:48:06", "content": "

yeah, I forked the best public kernel and change some parameters also to see if there is any improvements… Then I also try to understand the implementation… by forking these kernels I m sure that I mtrying to understand the correct implementation…

", "rawMarkdown": "yeah, I forked the best public kernel and change some parameters also to see if there is any improvements... Then I also try to understand the implementation... by forking these kernels I m sure that I mtrying to understand the correct implementation...", "votes": null }, { "id": "1584235", "postDate": "11/16/2021 11:11:28", "content": "

Changing some (relevant) parameters and getting the very, very same result…… It must have been a miracle! And everyone can run a kernel to understand the implementation without submitting (someone else's work).

", "rawMarkdown": "Changing some (relevant) parameters and getting the very, very same result...... It must have been a miracle! And everyone can run a kernel to understand the implementation without submitting (someone else's work).", "votes": null }, { "id": "1584346", "postDate": "11/16/2021 13:31:10", "content": "

Hey @karan23258! I'm the author of the public notebook, and just wanted to say it's totally fine from my point of view. When I make things public it's with full awareness that people will run with it - improve it, ensamble it, or just submit for the sake of stepping onto the leaderboard.

\n

It now has 450 forks, which I find cool and it motivates me to continue working and sharing stuff.

", "rawMarkdown": "Hey @karan23258! I'm the author of the public notebook, and just wanted to say it's totally fine from my point of view. When I make things public it's with full awareness that people will run with it - improve it, ensamble it, or just submit for the sake of stepping onto the leaderboard.\n\nIt now has 450 forks, which I find cool and it motivates me to continue working and sharing stuff.", "votes": null }, { "id": "1584429", "postDate": "11/16/2021 14:24:07", "content": "

@slawekbiel you shared your notebooks, that's great. My critique is not about your point of view but about the approach of their \"users\". It is great to take inspiration, improve it (more than changing random_state) - it supports creativity and brings a new value. But literally copying it - it suppresses creativity not only of the \"user\" but also of other new to the leaderboard and it brings no new value.
\nMy long time opinion is that ideas and fragments of code should be shared but results should not.

", "rawMarkdown": "slawekbiel you shared your notebooks, that's great. My critique is not about your point of view but about the approach of their \"users\". It is great to take inspiration, improve it (more than changing random_state) - it supports creativity and brings a new value. But literally copying it - it suppresses creativity not only of the \"user\" but also of other new to the leaderboard and it brings no new value. \nMy long time opinion is that ideas and fragments of code should be shared but results should not.", "votes": null }, { "id": "1585040", "postDate": "11/17/2021 02:47:49", "content": "

Thanks @slawekbie, It really motivates us to be in the competition…

", "rawMarkdown": "Thanks @slawekbie, It really motivates us to be in the competition…", "votes": null }, { "id": "1585046", "postDate": "11/17/2021 02:56:18", "content": "

Hi Allie, everyone has a different approach and pov and I respect yours… but please don’t be upset, even the author is happy 😊 with this, then why are you getting so sad🙁

", "rawMarkdown": "Hi Allie, everyone has a different approach and pov and I respect yours… but please don’t be upset, even the author is happy 😊 with this, then why are you getting so sad🙁", "votes": null }, { "id": "1585585", "postDate": "11/17/2021 11:07:53", "content": "

Hi @karan23258, it's funny, I am neither upset nor sad. After earning significant experience and 3 well deserved solo medals for my own solutions in last 3 months, I learned how to only smile leniently at some 200 teams in the leaderboard with the very same score from the public notebook. But I feel sorry and empathic for novices who are honestly trying to create their own solutions but are jumped over in the leaderboard by those \"copiers\".

", "rawMarkdown": "Hi @karan23258, it's funny, I am neither upset nor sad. After earning significant experience and 3 well deserved solo medals for my own solutions in last 3 months, I learned how to only smile leniently at some 200 teams in the leaderboard with the very same score from the public notebook. But I feel sorry and empathic for novices who are honestly trying to create their own solutions but are jumped over in the leaderboard by those \"copiers\".", "votes": null }, { "id": "1585754", "postDate": "11/17/2021 13:56:14", "content": "

Congrats Allie for your solo medals…

", "rawMarkdown": "Congrats Allie for your solo medals…", "votes": null } ], "comments": [ { "id": 1575504, "author_name": "slawekbiel", "author_url": "", "post_date": "11/08/2021 12:39:42", "content": "

This is a segmentation problem, you need to return a mask for each cell separately, not a single mask with all the cells in the image.

", "votes": null, "replies": [ { "id": 1576387, "author_name": "karan23258", "author_url": "", "post_date": "11/09/2021 06:13:43", "content": "

Thank you… So, that means we can't use unet for this task? Because it outputs a single mask. I saw in code section that the notebook with significant cv score has used either mask rcnn or detectron…

", "votes": null, "replies": [] }, { "id": 1576474, "author_name": "slawekbiel", "author_url": "", "post_date": "11/09/2021 08:01:52", "content": "

There are several public notebooks using u-net, some talk of breaking it down into individual masks. But I haven’t seen an example of it successfully done. So it’s either a closely guarded secret, or very hard to do.

", "votes": null, "replies": [] }, { "id": 1576713, "author_name": "karan23258", "author_url": "", "post_date": "11/09/2021 12:08:12", "content": "

Yeah, I too have not seen in public notebooks, will search outside kaggle for this…Thanks

", "votes": null, "replies": [] } ] }, { "id": 1577341, "author_name": "evancofsky", "author_url": "", "post_date": "11/10/2021 02:14:16", "content": "

I add a post processing step to clip masks to not overlap. Here’s a link. I took the code from @julian3833.

", "votes": null, "replies": [ { "id": 1580101, "author_name": "karan23258", "author_url": "", "post_date": "11/12/2021 13:26:04", "content": "

Where is the link?

", "votes": null, "replies": [] }, { "id": 1580162, "author_name": "evancofsky", "author_url": "", "post_date": "11/12/2021 14:32:40", "content": "

Haha that’s a great question! I’m wondering the same thing myself!

\n

Here you go: https://www.kaggle.com/evancofsky/sartorius-torch-lightning-mask-r-cnn

", "votes": null, "replies": [] } ] }, { "id": 1578508, "author_name": "ivoflorinscheiber", "author_url": "", "post_date": "11/11/2021 06:01:27", "content": "

Hej Karan,

\n

you are generating only one mask per image, but, as already mentioned you should generate a mask per cell in each image.

\n

I used watersheding for that

\n

best
\nivo

\n

.

", "votes": null, "replies": [ { "id": 1578836, "author_name": "karan23258", "author_url": "", "post_date": "11/11/2021 11:30:18", "content": "

Thanks Ivo

", "votes": null, "replies": [] }, { "id": 1580099, "author_name": "karan23258", "author_url": "", "post_date": "11/12/2021 13:25:44", "content": "

Can you explain about watersheding, how can I use it for getting the desired output?

", "votes": null, "replies": [] }, { "id": 1580175, "author_name": "ivoflorinscheiber", "author_url": "", "post_date": "11/12/2021 14:48:52", "content": "

Hej Karan,

\n

unfortunately I am a complete newbie to deep learning, thus I can not give you any deep insight.
\nHowevwe, please find my post proceysing on your model here: https://www.kaggle.com/ivoflorinscheiber/unetfromscratch-mypostpro/edit/run/79425057

\n

Apparently watersheding works well only for the cortical neuron cultures that display very well seperated cell bodies. For the SH-SY5Y neuroblastoma cell line and for the Astros, in particular(see last figure in the notebook).

\n

it is not working very well, since watersheding needs clear borders (like a cup to fill with water).
\nCurrently my best guess is to somehow combine U-Net with Mask R-CNN (I like to keep the U-Net since it is doing already a great job in semantic segmentation of the cells)

\n

Curious how your team achieved a score of almost 0.300 on the public leaderboard!

\n

Sorry for not being a great help,
\nbest ivo

", "votes": null, "replies": [] }, { "id": 1580243, "author_name": "blankaf", "author_url": "", "post_date": "11/12/2021 15:44:47", "content": "

\"Curious how your team achieved a score of almost 0.300 on the public leaderboard!\"
\nIt is obvious. Their team may have used the popular method of many kagglers - \"copy and paste\" the highest scoring public kernel.
\nWelcome to Kaggle, Ivo🙁.

", "votes": null, "replies": [] }, { "id": 1583949, "author_name": "karan23258", "author_url": "", "post_date": "11/16/2021 06:48:06", "content": "

yeah, I forked the best public kernel and change some parameters also to see if there is any improvements… Then I also try to understand the implementation… by forking these kernels I m sure that I mtrying to understand the correct implementation…

", "votes": null, "replies": [] }, { "id": 1584235, "author_name": "blankaf", "author_url": "", "post_date": "11/16/2021 11:11:28", "content": "

Changing some (relevant) parameters and getting the very, very same result…… It must have been a miracle! And everyone can run a kernel to understand the implementation without submitting (someone else's work).

", "votes": null, "replies": [] }, { "id": 1584346, "author_name": "slawekbiel", "author_url": "", "post_date": "11/16/2021 13:31:10", "content": "

Hey @karan23258! I'm the author of the public notebook, and just wanted to say it's totally fine from my point of view. When I make things public it's with full awareness that people will run with it - improve it, ensamble it, or just submit for the sake of stepping onto the leaderboard.

\n

It now has 450 forks, which I find cool and it motivates me to continue working and sharing stuff.

", "votes": null, "replies": [] }, { "id": 1584429, "author_name": "blankaf", "author_url": "", "post_date": "11/16/2021 14:24:07", "content": "

@slawekbiel you shared your notebooks, that's great. My critique is not about your point of view but about the approach of their \"users\". It is great to take inspiration, improve it (more than changing random_state) - it supports creativity and brings a new value. But literally copying it - it suppresses creativity not only of the \"user\" but also of other new to the leaderboard and it brings no new value.
\nMy long time opinion is that ideas and fragments of code should be shared but results should not.

", "votes": null, "replies": [] }, { "id": 1585040, "author_name": "karan23258", "author_url": "", "post_date": "11/17/2021 02:47:49", "content": "

Thanks @slawekbie, It really motivates us to be in the competition…

", "votes": null, "replies": [] }, { "id": 1585046, "author_name": "karan23258", "author_url": "", "post_date": "11/17/2021 02:56:18", "content": "

Hi Allie, everyone has a different approach and pov and I respect yours… but please don’t be upset, even the author is happy 😊 with this, then why are you getting so sad🙁

", "votes": null, "replies": [] }, { "id": 1585585, "author_name": "blankaf", "author_url": "", "post_date": "11/17/2021 11:07:53", "content": "

Hi @karan23258, it's funny, I am neither upset nor sad. After earning significant experience and 3 well deserved solo medals for my own solutions in last 3 months, I learned how to only smile leniently at some 200 teams in the leaderboard with the very same score from the public notebook. But I feel sorry and empathic for novices who are honestly trying to create their own solutions but are jumped over in the leaderboard by those \"copiers\".

", "votes": null, "replies": [] }, { "id": 1585754, "author_name": "karan23258", "author_url": "", "post_date": "11/17/2021 13:56:14", "content": "

Congrats Allie for your solo medals…

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1575426": "I trained an unet model with 512, 512 image size and masks with horizontal and vertical flip for about 60 epochs and got binary_cross_entropy loss around 0.11, and predicted masks were also looking similar but not exactly same. But after submission, I got 0.000 submission score. \nCan someone please review my notebook and explain me the reason for such a low score even predicted mask are looking similar.\n\nThis is my notebook [https://www.kaggle.com/karan23258/cell-instance-segmentation-unetfromscratch](url)", "1575504": "This is a segmentation problem, you need to return a mask for each cell separately, not a single mask with all the cells in the image.", "1576387": "Thank you... So, that means we can't use unet for this task? Because it outputs a single mask. I saw in code section that the notebook with significant cv score has used either mask rcnn or detectron...", "1576474": "There are several public notebooks using u-net, some talk of breaking it down into individual masks. But I haven’t seen an example of it successfully done. So it’s either a closely guarded secret, or very hard to do.", "1576713": "Yeah, I too have not seen in public notebooks, will search outside kaggle for this...Thanks", "1577341": "I add a post processing step to clip masks to not overlap. Here’s a link. I took the code from @julian3833.", "1578508": "Hej Karan, \n\nyou are generating only one mask per image, but, as already mentioned you should generate a mask per cell in each image.\n\nI used watersheding for that\n\nbest\nivo\n\n\n.", "1578836": "Thanks Ivo", "1580099": "Can you explain about watersheding, how can I use it for getting the desired output?", "1580101": "Where is the link?", "1580162": "Haha that’s a great question! I’m wondering the same thing myself!\n\nHere you go: https://www.kaggle.com/evancofsky/sartorius-torch-lightning-mask-r-cnn", "1580175": "Hej Karan,\n\nunfortunately I am a complete newbie to deep learning, thus I can not give you any deep insight. \nHowevwe, please find my post proceysing on your model here: https://www.kaggle.com/ivoflorinscheiber/unetfromscratch-mypostpro/edit/run/79425057\n\nApparently watersheding works well only for the cortical neuron cultures that display very well seperated cell bodies. For the SH-SY5Y neuroblastoma cell line and for the Astros, in particular(see last figure in the notebook).\n\n it is not working very well, since watersheding needs clear borders (like a cup to fill with water). \nCurrently my best guess is to somehow combine U-Net with Mask R-CNN (I like to keep the U-Net since it is doing already a great job in semantic segmentation of the cells)\n\nCurious how your team achieved a score of almost 0.300 on the public leaderboard!\n\nSorry for not being a great help,\nbest ivo", "1580243": "\"Curious how your team achieved a score of almost 0.300 on the public leaderboard!\"\n\nIt is obvious. Their team may have used the popular method of many kagglers - \"copy and paste\" the highest scoring public kernel.\nWelcome to Kaggle, Ivo🙁.", "1583949": "yeah, I forked the best public kernel and change some parameters also to see if there is any improvements... Then I also try to understand the implementation... by forking these kernels I m sure that I mtrying to understand the correct implementation...", "1584235": "Changing some (relevant) parameters and getting the very, very same result...... It must have been a miracle! And everyone can run a kernel to understand the implementation without submitting (someone else's work).", "1584346": "Hey @karan23258! I'm the author of the public notebook, and just wanted to say it's totally fine from my point of view. When I make things public it's with full awareness that people will run with it - improve it, ensamble it, or just submit for the sake of stepping onto the leaderboard.\n\nIt now has 450 forks, which I find cool and it motivates me to continue working and sharing stuff.", "1584429": "slawekbiel you shared your notebooks, that's great. My critique is not about your point of view but about the approach of their \"users\". It is great to take inspiration, improve it (more than changing random_state) - it supports creativity and brings a new value. But literally copying it - it suppresses creativity not only of the \"user\" but also of other new to the leaderboard and it brings no new value. \nMy long time opinion is that ideas and fragments of code should be shared but results should not.", "1585040": "Thanks @slawekbie, It really motivates us to be in the competition…", "1585046": "Hi Allie, everyone has a different approach and pov and I respect yours… but please don’t be upset, even the author is happy 😊 with this, then why are you getting so sad🙁", "1585585": "Hi @karan23258, it's funny, I am neither upset nor sad. After earning significant experience and 3 well deserved solo medals for my own solutions in last 3 months, I learned how to only smile leniently at some 200 teams in the leaderboard with the very same score from the public notebook. But I feel sorry and empathic for novices who are honestly trying to create their own solutions but are jumped over in the leaderboard by those \"copiers\".", "1585754": "Congrats Allie for your solo medals…" }, "source": "meta" }