{
  "id": 297984,
  "title": "Go with the flow (3rd place)",
  "url": "/competitions/sartorius-cell-instance-segmentation/discussion/297984",
  "author_name": "",
  "post_date": "2021-12-31T00:07:29.391181Z",
  "votes": 124,
  "comment_count": 38,
  "views": 0,
  "content": "<p><em>This post describes part of the 3rd place solution that's based on <strong>cellpose</strong></em></p>\n<h1>Backstory</h1>\n<p>Going into this competition I had no experience with image segmentation. I knew a library called Detectron exists so I started with reading tutorials and learning to use that and it gave me good early LB results. Taking the pretrained model from the LiveCell repo and finetuning it on the competition data was good enough to put me in the top 10 for a while during the first month. Meanwhile I've seen many people trying U-net in public notebooks and on the forum but all the shared results were far bellow mask-rcnn scores so I discarded it as an inferior approach. That is until start of December when I stumbled upon a mention of <a href=\"http://www.cellpose.org/\" target=\"_blank\">cellpose</a> on the forum - the example images they show on their site looked promissing and the paper claims <em>\"significantly outperforming Stardist and Mask R-CNN\"</em> so I've decided to try to run it myself.</p>\n<p>Imagine my excitement when the very first model I trained scored 0.307 on the LB. No training code written, no postprocessing, just saved annotations into png files and run the train script. Comparable out of the box detectron model, without any tuning, was scoring around 0.28 so I started to realize that U-net might not be an inferior architecture after all. I battled with myself whether to share the finding. On the one hand I found it very cool, but I also felt I could be giving a powerful tool into hands of my competitors with only few weeks go. In the end I shared the initial experiment and then conciously refrained from any more comments or mentions of it :)</p>\n<p>From there I followed the same game plan that worked with mask rcnn</p>\n<ol>\n<li>pretrain on livecell</li>\n<li>tune postprocessing on different cell types</li>\n<li>ensemble mutliple models</li>\n<li>add unlabled data annotated by a larger ensamble (inconclusive)</li>\n</ol>\n<p>With all that I managed to push it into the public score with single model: 0.333, and best 5 models ensemble: 0.335. This didn't end up as the strongest model in our team submissions (@alexandrecc mask r-cnn claimed that) but I think it provided helpful diversity in the ensemble.</p>\n<h1>How it works</h1>\n<p>The key idea is that rather than train U-net directly on the binary masks it first creates an intermediate representation of \"flow mask\". To generate that first it calculates the center of mass of each mask. And then all the other points represent an angle to flow from that point towards the center. This makes it easier to disentangle touching or overlapping masks - which is the main issue a regular U-Net faces.<br>\n<img src=\"https://raw.githubusercontent.com/slawekslex/random/main/flow_masks.png\" alt=\"\"></p>\n<p>The model itself is fairly simple, 20 conv layers on both down and up paths. Total of 6 million paramters which is less than resnet18. After the training the model output on the same image looks like this:<br>\n<img src=\"https://raw.githubusercontent.com/slawekslex/random/main/flow_output.png\" alt=\"\"><br>\nYou can see many of the centers identified correctly and how it's able to separate even tightly packed cells. (this image still only scores 0.155 showing how difficult a high astro score is)</p>\n<p>From this output they follow the flows from each pixel to it's center to go back to binary masks. I must admit I don't understand the exact details of this. The paper says <em>\"the predicted flow fields are used to construct a dynamical system with fixed points whose basins of attraction represent the predicted masks.\"</em></p>\n<h1>The gory details</h1>\n<p>What I said earlier about how easy it was to get a baseline solution unfortunately didn't translate into ease of further refinement. I have spent a lot of time on experimenting and digging through their code in the last four weeks.</p>\n<p>I hit the first wall when trying to train it on LiveCell. The library insists on loading all the images and masks into the memory before the training even starts -this is not a good idea when working with ten thousand of 520x704 images. I ended up pulling the relevant bits out of the library and writing a fast.ai dataloader and training script.</p>\n<p>Even though the model itself is small the training utilzes heavy augmentations and high zooms on small areas of an image. So you need to train for a lot of epochs to fully cover all the data. (I was using 300-500 epochs). When I started training LiveCell with my new dataloaders I've noticed that now the disk IO is the bottleneck - spinning at 100% while GPU waits idly. I had to rearange stuff on my system to free enough SSD space and only then I got a smooth training fully utilizing the GPU.</p>\n<p>A key detail in cellpose training is that it tries to maintain the pixel size of cells within a fixed range. At train time it's fairly simple - just look at the ground truth sizes and resize the image accordingly. At inference that's harder as you need to guess what the cell sizes are. That's the dreaded <strong>diameter</strong> discussed in a couple of threads here on the forum. Setting this correctly has a huge impact on models performance and I've spent so much time fiddling with it. What I have tried:</p>\n<ul>\n<li>setting a constant value per cell type</li>\n<li>setting a range of values and use that as TTA, </li>\n<li>using cellpose's \"size models\" - which are linear regressions run on the U-net innermost activations</li>\n<li>using output of a detectron model and taking sizes from there. </li>\n</ul>\n<p>In my final submission I have an ensamble of 10 models, use the first one with the \"size model\" and then pass the diamater to the following models. At each step recalculating it on the predicted masks to improve the estimate. This works ok, but I feel like this a weakness of this model that would require more exploring.</p>",
  "messages": [
    {
      "id": "1633609",
      "postDate": "12/31/2021 00:07:29",
      "content": "<p><em>This post describes part of the 3rd place solution that's based on <strong>cellpose</strong></em></p>\n<h1>Backstory</h1>\n<p>Going into this competition I had no experience with image segmentation. I knew a library called Detectron exists so I started with reading tutorials and learning to use that and it gave me good early LB results. Taking the pretrained model from the LiveCell repo and finetuning it on the competition data was good enough to put me in the top 10 for a while during the first month. Meanwhile I've seen many people trying U-net in public notebooks and on the forum but all the shared results were far bellow mask-rcnn scores so I discarded it as an inferior approach. That is until start of December when I stumbled upon a mention of <a href=\"http://www.cellpose.org/\" target=\"_blank\">cellpose</a> on the forum - the example images they show on their site looked promissing and the paper claims <em>\"significantly outperforming Stardist and Mask R-CNN\"</em> so I've decided to try to run it myself.</p>\n<p>Imagine my excitement when the very first model I trained scored 0.307 on the LB. No training code written, no postprocessing, just saved annotations into png files and run the train script. Comparable out of the box detectron model, without any tuning, was scoring around 0.28 so I started to realize that U-net might not be an inferior architecture after all. I battled with myself whether to share the finding. On the one hand I found it very cool, but I also felt I could be giving a powerful tool into hands of my competitors with only few weeks go. In the end I shared the initial experiment and then conciously refrained from any more comments or mentions of it :)</p>\n<p>From there I followed the same game plan that worked with mask rcnn</p>\n<ol>\n<li>pretrain on livecell</li>\n<li>tune postprocessing on different cell types</li>\n<li>ensemble mutliple models</li>\n<li>add unlabled data annotated by a larger ensamble (inconclusive)</li>\n</ol>\n<p>With all that I managed to push it into the public score with single model: 0.333, and best 5 models ensemble: 0.335. This didn't end up as the strongest model in our team submissions (@alexandrecc mask r-cnn claimed that) but I think it provided helpful diversity in the ensemble.</p>\n<h1>How it works</h1>\n<p>The key idea is that rather than train U-net directly on the binary masks it first creates an intermediate representation of \"flow mask\". To generate that first it calculates the center of mass of each mask. And then all the other points represent an angle to flow from that point towards the center. This makes it easier to disentangle touching or overlapping masks - which is the main issue a regular U-Net faces.<br>\n<img src=\"https://raw.githubusercontent.com/slawekslex/random/main/flow_masks.png\" alt=\"\"></p>\n<p>The model itself is fairly simple, 20 conv layers on both down and up paths. Total of 6 million paramters which is less than resnet18. After the training the model output on the same image looks like this:<br>\n<img src=\"https://raw.githubusercontent.com/slawekslex/random/main/flow_output.png\" alt=\"\"><br>\nYou can see many of the centers identified correctly and how it's able to separate even tightly packed cells. (this image still only scores 0.155 showing how difficult a high astro score is)</p>\n<p>From this output they follow the flows from each pixel to it's center to go back to binary masks. I must admit I don't understand the exact details of this. The paper says <em>\"the predicted flow fields are used to construct a dynamical system with fixed points whose basins of attraction represent the predicted masks.\"</em></p>\n<h1>The gory details</h1>\n<p>What I said earlier about how easy it was to get a baseline solution unfortunately didn't translate into ease of further refinement. I have spent a lot of time on experimenting and digging through their code in the last four weeks.</p>\n<p>I hit the first wall when trying to train it on LiveCell. The library insists on loading all the images and masks into the memory before the training even starts -this is not a good idea when working with ten thousand of 520x704 images. I ended up pulling the relevant bits out of the library and writing a fast.ai dataloader and training script.</p>\n<p>Even though the model itself is small the training utilzes heavy augmentations and high zooms on small areas of an image. So you need to train for a lot of epochs to fully cover all the data. (I was using 300-500 epochs). When I started training LiveCell with my new dataloaders I've noticed that now the disk IO is the bottleneck - spinning at 100% while GPU waits idly. I had to rearange stuff on my system to free enough SSD space and only then I got a smooth training fully utilizing the GPU.</p>\n<p>A key detail in cellpose training is that it tries to maintain the pixel size of cells within a fixed range. At train time it's fairly simple - just look at the ground truth sizes and resize the image accordingly. At inference that's harder as you need to guess what the cell sizes are. That's the dreaded <strong>diameter</strong> discussed in a couple of threads here on the forum. Setting this correctly has a huge impact on models performance and I've spent so much time fiddling with it. What I have tried:</p>\n<ul>\n<li>setting a constant value per cell type</li>\n<li>setting a range of values and use that as TTA, </li>\n<li>using cellpose's \"size models\" - which are linear regressions run on the U-net innermost activations</li>\n<li>using output of a detectron model and taking sizes from there. </li>\n</ul>\n<p>In my final submission I have an ensamble of 10 models, use the first one with the \"size model\" and then pass the diamater to the following models. At each step recalculating it on the predicted masks to improve the estimate. This works ok, but I feel like this a weakness of this model that would require more exploring.</p>",
      "rawMarkdown": "*This post describes part of the 3rd place solution that's based on **cellpose***\n\n# Backstory\nGoing into this competition I had no experience with image segmentation. I knew a library called Detectron exists so I started with reading tutorials and learning to use that and it gave me good early LB results. Taking the pretrained model from the LiveCell repo and finetuning it on the competition data was good enough to put me in the top 10 for a while during the first month. Meanwhile I've seen many people trying U-net in public notebooks and on the forum but all the shared results were far bellow mask-rcnn scores so I discarded it as an inferior approach. That is until start of December when I stumbled upon a mention of [cellpose](http://www.cellpose.org/) on the forum - the example images they show on their site looked promissing and the paper claims *\"significantly outperforming Stardist and Mask R-CNN\"* so I've decided to try to run it myself.\n\nImagine my excitement when the very first model I trained scored 0.307 on the LB. No training code written, no postprocessing, just saved annotations into png files and run the train script. Comparable out of the box detectron model, without any tuning, was scoring around 0.28 so I started to realize that U-net might not be an inferior architecture after all. I battled with myself whether to share the finding. On the one hand I found it very cool, but I also felt I could be giving a powerful tool into hands of my competitors with only few weeks go. In the end I shared the initial experiment and then conciously refrained from any more comments or mentions of it :)\n\nFrom there I followed the same game plan that worked with mask rcnn\n1. pretrain on livecell\n2. tune postprocessing on different cell types\n3. ensemble mutliple models\n4. add unlabled data annotated by a larger ensamble (inconclusive)\n\nWith all that I managed to push it into the public score with single model: 0.333, and best 5 models ensemble: 0.335. This didn't end up as the strongest model in our team submissions (@alexandrecc mask r-cnn claimed that) but I think it provided helpful diversity in the ensemble.\n\n# How it works\nThe key idea is that rather than train U-net directly on the binary masks it first creates an intermediate representation of \"flow mask\". To generate that first it calculates the center of mass of each mask. And then all the other points represent an angle to flow from that point towards the center. This makes it easier to disentangle touching or overlapping masks - which is the main issue a regular U-Net faces.\n![](https://raw.githubusercontent.com/slawekslex/random/main/flow_masks.png)\n\nThe model itself is fairly simple, 20 conv layers on both down and up paths. Total of 6 million paramters which is less than resnet18. After the training the model output on the same image looks like this:\n![](https://raw.githubusercontent.com/slawekslex/random/main/flow_output.png)\nYou can see many of the centers identified correctly and how it's able to separate even tightly packed cells. (this image still only scores 0.155 showing how difficult a high astro score is)\n\nFrom this output they follow the flows from each pixel to it's center to go back to binary masks. I must admit I don't understand the exact details of this. The paper says *\"the predicted flow fields are used to construct a dynamical system with fixed points whose basins of attraction represent the predicted masks.\"*\n\n#The gory details\nWhat I said earlier about how easy it was to get a baseline solution unfortunately didn't translate into ease of further refinement. I have spent a lot of time on experimenting and digging through their code in the last four weeks.\n\nI hit the first wall when trying to train it on LiveCell. The library insists on loading all the images and masks into the memory before the training even starts -this is not a good idea when working with ten thousand of 520x704 images. I ended up pulling the relevant bits out of the library and writing a fast.ai dataloader and training script.\n\nEven though the model itself is small the training utilzes heavy augmentations and high zooms on small areas of an image. So you need to train for a lot of epochs to fully cover all the data. (I was using 300-500 epochs). When I started training LiveCell with my new dataloaders I've noticed that now the disk IO is the bottleneck - spinning at 100% while GPU waits idly. I had to rearange stuff on my system to free enough SSD space and only then I got a smooth training fully utilizing the GPU.\n\nA key detail in cellpose training is that it tries to maintain the pixel size of cells within a fixed range. At train time it's fairly simple - just look at the ground truth sizes and resize the image accordingly. At inference that's harder as you need to guess what the cell sizes are. That's the dreaded **diameter** discussed in a couple of threads here on the forum. Setting this correctly has a huge impact on models performance and I've spent so much time fiddling with it. What I have tried:\n- setting a constant value per cell type\n- setting a range of values and use that as TTA, \n- using cellpose's \"size models\" - which are linear regressions run on the U-net innermost activations\n- using output of a detectron model and taking sizes from there. \n\nIn my final submission I have an ensamble of 10 models, use the first one with the \"size model\" and then pass the diamater to the following models. At each step recalculating it on the predicted masks to improve the estimate. This works ok, but I feel like this a weakness of this model that would require more exploring.",
      "votes": null
    },
    {
      "id": "1633613",
      "postDate": "12/31/2021 00:16:57",
      "content": "<p>Great job 🙌</p>",
      "rawMarkdown": "Great job 🙌",
      "votes": null
    },
    {
      "id": "1633616",
      "postDate": "12/31/2021 00:22:25",
      "content": "<p>Congratulations Slawek and thanks for sharing. </p>",
      "rawMarkdown": "Congratulations Slawek and thanks for sharing.",
      "votes": null
    },
    {
      "id": "1633617",
      "postDate": "12/31/2021 00:22:30",
      "content": "<p>Great work and congratulations 🎉 </p>\n<p>This is my first time also with image processing , 90% of the keywords ( roughly) I never heard about before the competition 😅 plus i spent more time fixing python related issues ( I am new to python also) </p>\n<p>I ended up with detectron and cellpose and this competition brought me my first competition based medal (bronze) 🎉</p>",
      "rawMarkdown": "Great work and congratulations 🎉 \n\nThis is my first time also with image processing , 90% of the keywords ( roughly) I never heard about before the competition 😅 plus i spent more time fixing python related issues ( I am new to python also) \n\nI ended up with detectron and cellpose and this competition brought me my first competition based medal (bronze) 🎉",
      "votes": null
    },
    {
      "id": "1633618",
      "postDate": "12/31/2021 00:23:42",
      "content": "<p>I have to say thank you<br>\n<a href=\"https://www.kaggle.com/slawekbiel\" target=\"_blank\">@slawekbiel</a> </p>\n<p>You are the star of this competition.</p>\n<p>It is impressive that you had no experience with image segmentation before this competition.</p>\n<p>This is really motivating.</p>\n<p>Wishing you happy new year.</p>",
      "rawMarkdown": "I have to say thank you\n@slawekbiel \n\nYou are the star of this competition.\n\nIt is impressive that you had no experience with image segmentation before this competition.\n\nThis is really motivating.\n\nWishing you happy new year.",
      "votes": null
    },
    {
      "id": "1633619",
      "postDate": "12/31/2021 00:25:23",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/slawekbiel\" target=\"_blank\">@slawekbiel</a> on an awesome finish and thanks for your sharing throughout the competition.</p>\n<p>I have some questions regarding finetuning the model from the LIVECell repo.</p>\n<ol>\n<li>Did you successfully finetune the resnest200 cascade model on this comp's data?</li>\n<li>Did you use the exact same hyperparameters provided by the authors?</li>\n<li>What was the compute used?</li>\n</ol>",
      "rawMarkdown": "Congratulations @slawekbiel on an awesome finish and thanks for your sharing throughout the competition.\n\nI have some questions regarding finetuning the model from the LIVECell repo.\n\n1. Did you successfully finetune the resnest200 cascade model on this comp's data?\n2.  Did you use the exact same hyperparameters provided by the authors?\n3. What was the compute used?",
      "votes": null
    },
    {
      "id": "1633620",
      "postDate": "12/31/2021 00:25:24",
      "content": "<p>Congrats and thank you for all the great insight you've provided along the way! A very well deserved prize.</p>",
      "rawMarkdown": "Congrats and thank you for all the great insight you've provided along the way! A very well deserved prize.",
      "votes": null
    },
    {
      "id": "1633622",
      "postDate": "12/31/2021 00:25:56",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/faisalalsrheed\" target=\"_blank\">@faisalalsrheed</a>! That's very kind</p>",
      "rawMarkdown": "Thank you @faisalalsrheed! That's very kind",
      "votes": null
    },
    {
      "id": "1633623",
      "postDate": "12/31/2021 00:27:20",
      "content": "<p>Your approach is very interesting. \"size prediction\" is great idea.  Wonder if you can write \"size prediction branch\" into the cellpose model instead of running as a separate model.   Thanks for writing up the summary, I difinitely learned a lot.  I saw your post a few days ago.  Didn't have time to experiment with cellpose.  I will go through your notebooks and the original paper in next few days to learn more about cellpose.  </p>",
      "rawMarkdown": "Your approach is very interesting. \"size prediction\" is great idea.  Wonder if you can write \"size prediction branch\" into the cellpose model instead of running as a separate model.   Thanks for writing up the summary, I difinitely learned a lot.  I saw your post a few days ago.  Didn't have time to experiment with cellpose.  I will go through your notebooks and the original paper in next few days to learn more about cellpose.",
      "votes": null
    },
    {
      "id": "1633625",
      "postDate": "12/31/2021 00:27:33",
      "content": "<p>Congrats! Thanks for your amazing solution and meaningful notebooks published in this competition!</p>",
      "rawMarkdown": "Congrats! Thanks for your amazing solution and meaningful notebooks published in this competition!",
      "votes": null
    },
    {
      "id": "1633627",
      "postDate": "12/31/2021 00:30:20",
      "content": "<p>Yes I did. The best score I got with it was .324 public so not phenomenal but less effort and time than training on LiveCell from scratch.<br>\nI kept mostly the config from the repo, set BS to 2 for a single GPU and did some trial and error on lr scheduling.</p>\n<p>The training takes around 2.5hrs on a RTX3090</p>",
      "rawMarkdown": "Yes I did. The best score I got with it was .324 public so not phenomenal but less effort and time than training on LiveCell from scratch.\nI kept mostly the config from the repo, set BS to 2 for a single GPU and did some trial and error on lr scheduling.\n\nThe training takes around 2.5hrs on a RTX3090",
      "votes": null
    },
    {
      "id": "1633632",
      "postDate": "12/31/2021 00:33:53",
      "content": "<p>Congratulations! Can I ask more about how you did the pretraining? E.g. Is there a library that you used or did you design the pretraining tasks yourself?</p>",
      "rawMarkdown": "Congratulations! Can I ask more about how you did the pretraining? E.g. Is there a library that you used or did you design the pretraining tasks yourself?",
      "votes": null
    },
    {
      "id": "1633633",
      "postDate": "12/31/2021 00:34:17",
      "content": "<p>Honestly I did think about adding an extra head to the model for just that - but the issue is we resize the image to it's desired resolution before passing it to the model. So it would have to be some two stage approach that takes the original image. Too complicated to attempt in the limited time I had.</p>",
      "rawMarkdown": "Honestly I did think about adding an extra head to the model for just that - but the issue is we resize the image to it's desired resolution before passing it to the model. So it would have to be some two stage approach that takes the original image. Too complicated to attempt in the limited time I had.",
      "votes": null
    },
    {
      "id": "1633639",
      "postDate": "12/31/2021 00:41:18",
      "content": "<p>I used the same code to first train on the LiveCell data and then finetune on the competition data. </p>",
      "rawMarkdown": "I used the same code to first train on the LiveCell data and then finetune on the competition data.",
      "votes": null
    },
    {
      "id": "1633643",
      "postDate": "12/31/2021 00:45:53",
      "content": "<p>Thanks for the replay.  Two stages make more sense.  Otherwise, it would be too complicated like you said.  </p>",
      "rawMarkdown": "Thanks for the replay.  Two stages make more sense.  Otherwise, it would be too complicated like you said.",
      "votes": null
    },
    {
      "id": "1633700",
      "postDate": "12/31/2021 01:47:04",
      "content": "<p>Well deserved win and thanks for your community contributions. I learned about Cellpose for the first time here. Didn't use it because of the computational resources needed.</p>",
      "rawMarkdown": "Well deserved win and thanks for your community contributions. I learned about Cellpose for the first time here. Didn't use it because of the computational resources needed.",
      "votes": null
    },
    {
      "id": "1633746",
      "postDate": "12/31/2021 02:52:15",
      "content": "<p>Congrats and thank you for sharing about cellpose. I was stuck at maskrcnn(mmdetection) for a long time and then get better result with cellpose instantly. I got only a 100+ bronze in the end but there are some similar exprience with you.<br>\nIt's weird cellpose trys load all images into memory. Perhaps it was desighed for much smaller dataset. I'm on win pc and train with setting 128gb virtual memory at second ssd. I train only 150 epoch until the loss seems to stop decreasing.<br>\nI also pick some livecell cell type for base model train with regard of sartorius cell type. But using all dataset give better performance.<br>\nFine tuning is not hard but it's difficult to find the right diameter for interfence. The sartorius cells sizes are way too various. I use a simple decision tree model to classify the image into cell types and use fixed diameter for 3 type of images. However I think the idle performance  should utilize various diameters in every portion of a image. An additionl test diameter is the drawback of cellpose though. <br>\nIMO the building size predicted model  performance poor. I tried to predict the size with my own resnet model it performance just the same with the cellpose one. I was too stupid not using maskrcnn as a good size predicter though it's not a decent segmentation model here. Maybe the mmdeteciton is not friendly to use that I don't want to use it laterly.</p>",
      "rawMarkdown": "Congrats and thank you for sharing about cellpose. I was stuck at maskrcnn(mmdetection) for a long time and then get better result with cellpose instantly. I got only a 100+ bronze in the end but there are some similar exprience with you.\nIt's weird cellpose trys load all images into memory. Perhaps it was desighed for much smaller dataset. I'm on win pc and train with setting 128gb virtual memory at second ssd. I train only 150 epoch until the loss seems to stop decreasing.\nI also pick some livecell cell type for base model train with regard of sartorius cell type. But using all dataset give better performance.\nFine tuning is not hard but it's difficult to find the right diameter for interfence. The sartorius cells sizes are way too various. I use a simple decision tree model to classify the image into cell types and use fixed diameter for 3 type of images. However I think the idle performance  should utilize various diameters in every portion of a image. An additionl test diameter is the drawback of cellpose though. \nIMO the building size predicted model  performance poor. I tried to predict the size with my own resnet model it performance just the same with the cellpose one. I was too stupid not using maskrcnn as a good size predicter though it's not a decent segmentation model here. Maybe the mmdeteciton is not friendly to use that I don't want to use it laterly.",
      "votes": null
    },
    {
      "id": "1633749",
      "postDate": "12/31/2021 02:59:53",
      "content": "<p>Congrats <a href=\"https://www.kaggle.com/slawekbiel\" target=\"_blank\">@slawekbiel</a> and team on 3rd place and thanks for sharing solution. Our Cellpose model can get 0.332 LB score but we not use for ensemble final.</p>",
      "rawMarkdown": "Congrats @slawekbiel and team on 3rd place and thanks for sharing solution. Our Cellpose model can get 0.332 LB score but we not use for ensemble final.",
      "votes": null
    },
    {
      "id": "1633786",
      "postDate": "12/31/2021 04:08:53",
      "content": "<p>congrats for gold!!! I learned a lot from your notebooks and discussions … thanks</p>",
      "rawMarkdown": "congrats for gold!!! I learned a lot from your notebooks and discussions ... thanks",
      "votes": null
    },
    {
      "id": "1633794",
      "postDate": "12/31/2021 04:21:08",
      "content": "<p>my notebook is cellpose and 0.318 ，thanks for your work </p>",
      "rawMarkdown": "my notebook is cellpose and 0.318 ，thanks for your work",
      "votes": null
    },
    {
      "id": "1633905",
      "postDate": "12/31/2021 06:38:33",
      "content": "<p><code>disk IO is the bottleneck - spinning at 100% while GPU waits idly. I had to rearange stuff on my system to free enough SSD space and only then I got a smooth training fully utilizing the GPU.</code></p>\n<p>Does it mean we just need to copy file that place model and dataset on SSD rather than mech hard disk ?<br>\nthen traning will be faster ???</p>",
      "rawMarkdown": "`disk IO is the bottleneck - spinning at 100% while GPU waits idly. I had to rearange stuff on my system to free enough SSD space and only then I got a smooth training fully utilizing the GPU.`\n\n\nDoes it mean we just need to copy file that place model and dataset on SSD rather than mech hard disk ?\nthen traning will be faster ???",
      "votes": null
    },
    {
      "id": "1633909",
      "postDate": "12/31/2021 06:40:29",
      "content": "<p>Congratulations! Thanks for sharing your solution and all the great work you put out during the competition, I learned so much from you and I am very happy that you get rewarded for your hard work. Cheers!</p>",
      "rawMarkdown": "Congratulations! Thanks for sharing your solution and all the great work you put out during the competition, I learned so much from you and I am very happy that you get rewarded for your hard work. Cheers!",
      "votes": null
    },
    {
      "id": "1633943",
      "postDate": "12/31/2021 07:45:30",
      "content": "<p>Thanks for sharing!</p>",
      "rawMarkdown": "Thanks for sharing!",
      "votes": null
    },
    {
      "id": "1633961",
      "postDate": "12/31/2021 08:12:57",
      "content": "<p>You always need to look at the resources consumption in your specific setup. In this particular case where the model is small and there are more files disk cache can handle - Yes, moving files to SSD helped me a lot with my training time.</p>",
      "rawMarkdown": "You always need to look at the resources consumption in your specific setup. In this particular case where the model is small and there are more files disk cache can handle - Yes, moving files to SSD helped me a lot with my training time.",
      "votes": null
    },
    {
      "id": "1634016",
      "postDate": "12/31/2021 09:14:14",
      "content": "<p>\"Going into this competition I had no experience with image segmentation\"</p>\n<p><a href=\"https://www.kaggle.com/slawekbiel\" target=\"_blank\">@slawekbiel</a>: this is an inspiration for everyone wanting to take a challenge in a competition but they think that they don't have the necessary previous experience.</p>\n<p>Great work! All my compliments for your achievement!</p>",
      "rawMarkdown": "\"Going into this competition I had no experience with image segmentation\"\n\n@slawekbiel: this is an inspiration for everyone wanting to take a challenge in a competition but they think that they don't have the necessary previous experience.\n\nGreat work! All my compliments for your achievement!",
      "votes": null
    },
    {
      "id": "1634068",
      "postDate": "12/31/2021 10:34:13",
      "content": "<p>would u like to share you notebook？ i apply cellpose too，but<br>\nMy PB is 0.318 ,I want learn something from your netobook,thx,thanks for your work and ideas </p>",
      "rawMarkdown": "would u like to share you notebook？ i apply cellpose too，but\nMy PB is 0.318 ,I want learn something from your netobook,thx,thanks for your work and ideas",
      "votes": null
    },
    {
      "id": "1634155",
      "postDate": "12/31/2021 11:33:21",
      "content": "<p>Thanks Luca! My experience with kaggle so far has been that you can learn a lot in three months if you fully commit to it. </p>\n<p>Also while competing in various competitions you acquire meta skills that translate from one to another, even if the problem domains are completely different. I definitely feel I’ve done much better than I would have a year ago by experience in approaching competition, using tools, having my work environment streamlined, taking advantage of forum and public notebooks, etc.</p>",
      "rawMarkdown": "Thanks Luca! My experience with kaggle so far has been that you can learn a lot in three months if you fully commit to it. \n\nAlso while competing in various competitions you acquire meta skills that translate from one to another, even if the problem domains are completely different. I definitely feel I’ve done much better than I would have a year ago by experience in approaching competition, using tools, having my work environment streamlined, taking advantage of forum and public notebooks, etc.",
      "votes": null
    },
    {
      "id": "1634161",
      "postDate": "12/31/2021 11:35:59",
      "content": "<p>I need some time to clean it up and make presentable.</p>",
      "rawMarkdown": "I need some time to clean it up and make presentable.",
      "votes": null
    },
    {
      "id": "1634240",
      "postDate": "12/31/2021 12:57:36",
      "content": "<p>Thanks for Sharing!</p>",
      "rawMarkdown": "Thanks for Sharing!",
      "votes": null
    },
    {
      "id": "1634381",
      "postDate": "12/31/2021 16:48:18",
      "content": "<p>Great work and congrats on 3rd place🎉</p>",
      "rawMarkdown": "Great work and congrats on 3rd place🎉",
      "votes": null
    },
    {
      "id": "1634393",
      "postDate": "12/31/2021 16:57:05",
      "content": "<p>You are right that loading all the images into memory is weird. Maybe they did work only on smaller datasets but I've also found code like this:</p>\n<pre><code>#again, attempt to deal with memory overuse\nyf, style = None, None\ngc.collect()\ntorch.cuda.empty_cache()\n</code></pre>\n<p>So they were aware of high memory consumption, just didn't think \"Hey lets not keep all of the dataset in RAM all the time\" 🤔</p>",
      "rawMarkdown": "You are right that loading all the images into memory is weird. Maybe they did work only on smaller datasets but I've also found code like this:\n```\n#again, attempt to deal with memory overuse\nyf, style = None, None\ngc.collect()\ntorch.cuda.empty_cache()\n```\nSo they were aware of high memory consumption, just didn't think \"Hey lets not keep all of the dataset in RAM all the time\" 🤔",
      "votes": null
    },
    {
      "id": "1634652",
      "postDate": "01/01/2022 00:29:21",
      "content": "<p>Inspirational and fascinating, congrats on 3rd place! 🙌</p>",
      "rawMarkdown": "Inspirational and fascinating, congrats on 3rd place! 🙌",
      "votes": null
    },
    {
      "id": "1635036",
      "postDate": "01/01/2022 12:34:28",
      "content": "<p>I really want to thank you for the wonderful codes that you have shared during the competition. It gave me a lot of ideas. Congrats on 3rd place. </p>",
      "rawMarkdown": "I really want to thank you for the wonderful codes that you have shared during the competition. It gave me a lot of ideas. Congrats on 3rd place.",
      "votes": null
    },
    {
      "id": "1635070",
      "postDate": "01/01/2022 13:02:05",
      "content": "<p>I appreciate you saying that <a href=\"https://www.kaggle.com/hsadeghian\" target=\"_blank\">@hsadeghian</a> feedback like that is what motivates me to continue trying to improve and sharing things I’ve learned along the way.</p>",
      "rawMarkdown": "I appreciate you saying that @hsadeghian feedback like that is what motivates me to continue trying to improve and sharing things I’ve learned along the way.",
      "votes": null
    },
    {
      "id": "1636762",
      "postDate": "01/03/2022 08:50:53",
      "content": "<p>congratulations and thanks for sharing the code</p>",
      "rawMarkdown": "congratulations and thanks for sharing the code",
      "votes": null
    },
    {
      "id": "1637466",
      "postDate": "01/03/2022 23:01:26",
      "content": "<p>Slawek, excellent job! <br>\nOne night towards the end, I also tried to take pieces of the interesting part of the cellpose repo: it trained but the predictions were an awful fail…</p>\n<p>I have a question: <br>\nwhen you generate the \"flows\", are you generating on the fly in the batch generator or are you generating first on disk and then you load them  in your batch generator? </p>",
      "rawMarkdown": "Slawek, excellent job! \nOne night towards the end, I also tried to take pieces of the interesting part of the cellpose repo: it trained but the predictions were an awful fail...\n\nI have a question: \nwhen you generate the \"flows\", are you generating on the fly in the batch generator or are you generating first on disk and then you load them  in your batch generator?",
      "votes": null
    },
    {
      "id": "1637713",
      "postDate": "01/04/2022 06:21:42",
      "content": "<p>I generate it before I start training and read tiff files in my DataLoader.</p>",
      "rawMarkdown": "I generate it before I start training and read tiff files in my DataLoader.",
      "votes": null
    },
    {
      "id": "1637902",
      "postDate": "01/04/2022 09:33:25",
      "content": "<p>nice sir, excellent job! </p>",
      "rawMarkdown": "nice sir, excellent job!",
      "votes": null
    },
    {
      "id": "1638373",
      "postDate": "01/04/2022 16:56:13",
      "content": "<p>Amazing work!! Congratulations! Thank you so much for sharing this!!</p>",
      "rawMarkdown": "Amazing work!! Congratulations! Thank you so much for sharing this!!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1633613,
      "author_name": "eslamomar",
      "author_url": "",
      "post_date": "12/31/2021 00:16:57",
      "content": "<p>Great job 🙌</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633616,
      "author_name": "drpatrickchan",
      "author_url": "",
      "post_date": "12/31/2021 00:22:25",
      "content": "<p>Congratulations Slawek and thanks for sharing. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633617,
      "author_name": "asalhi",
      "author_url": "",
      "post_date": "12/31/2021 00:22:30",
      "content": "<p>Great work and congratulations 🎉 </p>\n<p>This is my first time also with image processing , 90% of the keywords ( roughly) I never heard about before the competition 😅 plus i spent more time fixing python related issues ( I am new to python also) </p>\n<p>I ended up with detectron and cellpose and this competition brought me my first competition based medal (bronze) 🎉</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633618,
      "author_name": "faisalalsrheed",
      "author_url": "",
      "post_date": "12/31/2021 00:23:42",
      "content": "<p>I have to say thank you<br>\n<a href=\"https://www.kaggle.com/slawekbiel\" target=\"_blank\">@slawekbiel</a> </p>\n<p>You are the star of this competition.</p>\n<p>It is impressive that you had no experience with image segmentation before this competition.</p>\n<p>This is really motivating.</p>\n<p>Wishing you happy new year.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1633622,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "12/31/2021 00:25:56",
          "content": "<p>Thank you <a href=\"https://www.kaggle.com/faisalalsrheed\" target=\"_blank\">@faisalalsrheed</a>! That's very kind</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1633619,
      "author_name": "yousof9",
      "author_url": "",
      "post_date": "12/31/2021 00:25:23",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/slawekbiel\" target=\"_blank\">@slawekbiel</a> on an awesome finish and thanks for your sharing throughout the competition.</p>\n<p>I have some questions regarding finetuning the model from the LIVECell repo.</p>\n<ol>\n<li>Did you successfully finetune the resnest200 cascade model on this comp's data?</li>\n<li>Did you use the exact same hyperparameters provided by the authors?</li>\n<li>What was the compute used?</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 1633627,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "12/31/2021 00:30:20",
          "content": "<p>Yes I did. The best score I got with it was .324 public so not phenomenal but less effort and time than training on LiveCell from scratch.<br>\nI kept mostly the config from the repo, set BS to 2 for a single GPU and did some trial and error on lr scheduling.</p>\n<p>The training takes around 2.5hrs on a RTX3090</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1633620,
      "author_name": "woprime",
      "author_url": "",
      "post_date": "12/31/2021 00:25:24",
      "content": "<p>Congrats and thank you for all the great insight you've provided along the way! A very well deserved prize.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633623,
      "author_name": "joejeo1",
      "author_url": "",
      "post_date": "12/31/2021 00:27:20",
      "content": "<p>Your approach is very interesting. \"size prediction\" is great idea.  Wonder if you can write \"size prediction branch\" into the cellpose model instead of running as a separate model.   Thanks for writing up the summary, I difinitely learned a lot.  I saw your post a few days ago.  Didn't have time to experiment with cellpose.  I will go through your notebooks and the original paper in next few days to learn more about cellpose.  </p>",
      "votes": null,
      "replies": [
        {
          "id": 1633633,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "12/31/2021 00:34:17",
          "content": "<p>Honestly I did think about adding an extra head to the model for just that - but the issue is we resize the image to it's desired resolution before passing it to the model. So it would have to be some two stage approach that takes the original image. Too complicated to attempt in the limited time I had.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1633643,
          "author_name": "joejeo1",
          "author_url": "",
          "post_date": "12/31/2021 00:45:53",
          "content": "<p>Thanks for the replay.  Two stages make more sense.  Otherwise, it would be too complicated like you said.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1633625,
      "author_name": "charonwangg",
      "author_url": "",
      "post_date": "12/31/2021 00:27:33",
      "content": "<p>Congrats! Thanks for your amazing solution and meaningful notebooks published in this competition!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633632,
      "author_name": "kagglerz",
      "author_url": "",
      "post_date": "12/31/2021 00:33:53",
      "content": "<p>Congratulations! Can I ask more about how you did the pretraining? E.g. Is there a library that you used or did you design the pretraining tasks yourself?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1633639,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "12/31/2021 00:41:18",
          "content": "<p>I used the same code to first train on the LiveCell data and then finetune on the competition data. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1633700,
      "author_name": "shir0mani",
      "author_url": "",
      "post_date": "12/31/2021 01:47:04",
      "content": "<p>Well deserved win and thanks for your community contributions. I learned about Cellpose for the first time here. Didn't use it because of the computational resources needed.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633746,
      "author_name": "yichenwang1988",
      "author_url": "",
      "post_date": "12/31/2021 02:52:15",
      "content": "<p>Congrats and thank you for sharing about cellpose. I was stuck at maskrcnn(mmdetection) for a long time and then get better result with cellpose instantly. I got only a 100+ bronze in the end but there are some similar exprience with you.<br>\nIt's weird cellpose trys load all images into memory. Perhaps it was desighed for much smaller dataset. I'm on win pc and train with setting 128gb virtual memory at second ssd. I train only 150 epoch until the loss seems to stop decreasing.<br>\nI also pick some livecell cell type for base model train with regard of sartorius cell type. But using all dataset give better performance.<br>\nFine tuning is not hard but it's difficult to find the right diameter for interfence. The sartorius cells sizes are way too various. I use a simple decision tree model to classify the image into cell types and use fixed diameter for 3 type of images. However I think the idle performance  should utilize various diameters in every portion of a image. An additionl test diameter is the drawback of cellpose though. <br>\nIMO the building size predicted model  performance poor. I tried to predict the size with my own resnet model it performance just the same with the cellpose one. I was too stupid not using maskrcnn as a good size predicter though it's not a decent segmentation model here. Maybe the mmdeteciton is not friendly to use that I don't want to use it laterly.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1634393,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "12/31/2021 16:57:05",
          "content": "<p>You are right that loading all the images into memory is weird. Maybe they did work only on smaller datasets but I've also found code like this:</p>\n<pre><code>#again, attempt to deal with memory overuse\nyf, style = None, None\ngc.collect()\ntorch.cuda.empty_cache()\n</code></pre>\n<p>So they were aware of high memory consumption, just didn't think \"Hey lets not keep all of the dataset in RAM all the time\" 🤔</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1633749,
      "author_name": "duykhanh99",
      "author_url": "",
      "post_date": "12/31/2021 02:59:53",
      "content": "<p>Congrats <a href=\"https://www.kaggle.com/slawekbiel\" target=\"_blank\">@slawekbiel</a> and team on 3rd place and thanks for sharing solution. Our Cellpose model can get 0.332 LB score but we not use for ensemble final.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633786,
      "author_name": "raghaw",
      "author_url": "",
      "post_date": "12/31/2021 04:08:53",
      "content": "<p>congrats for gold!!! I learned a lot from your notebooks and discussions … thanks</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633794,
      "author_name": "clavinharris",
      "author_url": "",
      "post_date": "12/31/2021 04:21:08",
      "content": "<p>my notebook is cellpose and 0.318 ，thanks for your work </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633905,
      "author_name": "drzhuzhe",
      "author_url": "",
      "post_date": "12/31/2021 06:38:33",
      "content": "<p><code>disk IO is the bottleneck - spinning at 100% while GPU waits idly. I had to rearange stuff on my system to free enough SSD space and only then I got a smooth training fully utilizing the GPU.</code></p>\n<p>Does it mean we just need to copy file that place model and dataset on SSD rather than mech hard disk ?<br>\nthen traning will be faster ???</p>",
      "votes": null,
      "replies": [
        {
          "id": 1633961,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "12/31/2021 08:12:57",
          "content": "<p>You always need to look at the resources consumption in your specific setup. In this particular case where the model is small and there are more files disk cache can handle - Yes, moving files to SSD helped me a lot with my training time.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1633909,
      "author_name": "ollibolli",
      "author_url": "",
      "post_date": "12/31/2021 06:40:29",
      "content": "<p>Congratulations! Thanks for sharing your solution and all the great work you put out during the competition, I learned so much from you and I am very happy that you get rewarded for your hard work. Cheers!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1633943,
      "author_name": "rnakao",
      "author_url": "",
      "post_date": "12/31/2021 07:45:30",
      "content": "<p>Thanks for sharing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1634016,
      "author_name": "lucamassaron",
      "author_url": "",
      "post_date": "12/31/2021 09:14:14",
      "content": "<p>\"Going into this competition I had no experience with image segmentation\"</p>\n<p><a href=\"https://www.kaggle.com/slawekbiel\" target=\"_blank\">@slawekbiel</a>: this is an inspiration for everyone wanting to take a challenge in a competition but they think that they don't have the necessary previous experience.</p>\n<p>Great work! All my compliments for your achievement!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1634155,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "12/31/2021 11:33:21",
          "content": "<p>Thanks Luca! My experience with kaggle so far has been that you can learn a lot in three months if you fully commit to it. </p>\n<p>Also while competing in various competitions you acquire meta skills that translate from one to another, even if the problem domains are completely different. I definitely feel I’ve done much better than I would have a year ago by experience in approaching competition, using tools, having my work environment streamlined, taking advantage of forum and public notebooks, etc.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1634068,
      "author_name": "clavinharris",
      "author_url": "",
      "post_date": "12/31/2021 10:34:13",
      "content": "<p>would u like to share you notebook？ i apply cellpose too，but<br>\nMy PB is 0.318 ,I want learn something from your netobook,thx,thanks for your work and ideas </p>",
      "votes": null,
      "replies": [
        {
          "id": 1634161,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "12/31/2021 11:35:59",
          "content": "<p>I need some time to clean it up and make presentable.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1634240,
      "author_name": "balavashan",
      "author_url": "",
      "post_date": "12/31/2021 12:57:36",
      "content": "<p>Thanks for Sharing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1634381,
      "author_name": "ichimarugin",
      "author_url": "",
      "post_date": "12/31/2021 16:48:18",
      "content": "<p>Great work and congrats on 3rd place🎉</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1634652,
      "author_name": "niekvanderzwaag",
      "author_url": "",
      "post_date": "01/01/2022 00:29:21",
      "content": "<p>Inspirational and fascinating, congrats on 3rd place! 🙌</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1635036,
      "author_name": "hsadeghian",
      "author_url": "",
      "post_date": "01/01/2022 12:34:28",
      "content": "<p>I really want to thank you for the wonderful codes that you have shared during the competition. It gave me a lot of ideas. Congrats on 3rd place. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1635070,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "01/01/2022 13:02:05",
          "content": "<p>I appreciate you saying that <a href=\"https://www.kaggle.com/hsadeghian\" target=\"_blank\">@hsadeghian</a> feedback like that is what motivates me to continue trying to improve and sharing things I’ve learned along the way.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1636762,
      "author_name": "aiswaryasivakumar",
      "author_url": "",
      "post_date": "01/03/2022 08:50:53",
      "content": "<p>congratulations and thanks for sharing the code</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1637466,
      "author_name": "chabir",
      "author_url": "",
      "post_date": "01/03/2022 23:01:26",
      "content": "<p>Slawek, excellent job! <br>\nOne night towards the end, I also tried to take pieces of the interesting part of the cellpose repo: it trained but the predictions were an awful fail…</p>\n<p>I have a question: <br>\nwhen you generate the \"flows\", are you generating on the fly in the batch generator or are you generating first on disk and then you load them  in your batch generator? </p>",
      "votes": null,
      "replies": [
        {
          "id": 1637713,
          "author_name": "slawekbiel",
          "author_url": "",
          "post_date": "01/04/2022 06:21:42",
          "content": "<p>I generate it before I start training and read tiff files in my DataLoader.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1637902,
      "author_name": "alamirobbi7121",
      "author_url": "",
      "post_date": "01/04/2022 09:33:25",
      "content": "<p>nice sir, excellent job! </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1638373,
      "author_name": "lililycai",
      "author_url": "",
      "post_date": "01/04/2022 16:56:13",
      "content": "<p>Amazing work!! Congratulations! Thank you so much for sharing this!!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1633609": "*This post describes part of the 3rd place solution that's based on **cellpose***\n\n# Backstory\nGoing into this competition I had no experience with image segmentation. I knew a library called Detectron exists so I started with reading tutorials and learning to use that and it gave me good early LB results. Taking the pretrained model from the LiveCell repo and finetuning it on the competition data was good enough to put me in the top 10 for a while during the first month. Meanwhile I've seen many people trying U-net in public notebooks and on the forum but all the shared results were far bellow mask-rcnn scores so I discarded it as an inferior approach. That is until start of December when I stumbled upon a mention of [cellpose](http://www.cellpose.org/) on the forum - the example images they show on their site looked promissing and the paper claims *\"significantly outperforming Stardist and Mask R-CNN\"* so I've decided to try to run it myself.\n\nImagine my excitement when the very first model I trained scored 0.307 on the LB. No training code written, no postprocessing, just saved annotations into png files and run the train script. Comparable out of the box detectron model, without any tuning, was scoring around 0.28 so I started to realize that U-net might not be an inferior architecture after all. I battled with myself whether to share the finding. On the one hand I found it very cool, but I also felt I could be giving a powerful tool into hands of my competitors with only few weeks go. In the end I shared the initial experiment and then conciously refrained from any more comments or mentions of it :)\n\nFrom there I followed the same game plan that worked with mask rcnn\n1. pretrain on livecell\n2. tune postprocessing on different cell types\n3. ensemble mutliple models\n4. add unlabled data annotated by a larger ensamble (inconclusive)\n\nWith all that I managed to push it into the public score with single model: 0.333, and best 5 models ensemble: 0.335. This didn't end up as the strongest model in our team submissions (@alexandrecc mask r-cnn claimed that) but I think it provided helpful diversity in the ensemble.\n\n# How it works\nThe key idea is that rather than train U-net directly on the binary masks it first creates an intermediate representation of \"flow mask\". To generate that first it calculates the center of mass of each mask. And then all the other points represent an angle to flow from that point towards the center. This makes it easier to disentangle touching or overlapping masks - which is the main issue a regular U-Net faces.\n![](https://raw.githubusercontent.com/slawekslex/random/main/flow_masks.png)\n\nThe model itself is fairly simple, 20 conv layers on both down and up paths. Total of 6 million paramters which is less than resnet18. After the training the model output on the same image looks like this:\n![](https://raw.githubusercontent.com/slawekslex/random/main/flow_output.png)\nYou can see many of the centers identified correctly and how it's able to separate even tightly packed cells. (this image still only scores 0.155 showing how difficult a high astro score is)\n\nFrom this output they follow the flows from each pixel to it's center to go back to binary masks. I must admit I don't understand the exact details of this. The paper says *\"the predicted flow fields are used to construct a dynamical system with fixed points whose basins of attraction represent the predicted masks.\"*\n\n#The gory details\nWhat I said earlier about how easy it was to get a baseline solution unfortunately didn't translate into ease of further refinement. I have spent a lot of time on experimenting and digging through their code in the last four weeks.\n\nI hit the first wall when trying to train it on LiveCell. The library insists on loading all the images and masks into the memory before the training even starts -this is not a good idea when working with ten thousand of 520x704 images. I ended up pulling the relevant bits out of the library and writing a fast.ai dataloader and training script.\n\nEven though the model itself is small the training utilzes heavy augmentations and high zooms on small areas of an image. So you need to train for a lot of epochs to fully cover all the data. (I was using 300-500 epochs). When I started training LiveCell with my new dataloaders I've noticed that now the disk IO is the bottleneck - spinning at 100% while GPU waits idly. I had to rearange stuff on my system to free enough SSD space and only then I got a smooth training fully utilizing the GPU.\n\nA key detail in cellpose training is that it tries to maintain the pixel size of cells within a fixed range. At train time it's fairly simple - just look at the ground truth sizes and resize the image accordingly. At inference that's harder as you need to guess what the cell sizes are. That's the dreaded **diameter** discussed in a couple of threads here on the forum. Setting this correctly has a huge impact on models performance and I've spent so much time fiddling with it. What I have tried:\n- setting a constant value per cell type\n- setting a range of values and use that as TTA, \n- using cellpose's \"size models\" - which are linear regressions run on the U-net innermost activations\n- using output of a detectron model and taking sizes from there. \n\nIn my final submission I have an ensamble of 10 models, use the first one with the \"size model\" and then pass the diamater to the following models. At each step recalculating it on the predicted masks to improve the estimate. This works ok, but I feel like this a weakness of this model that would require more exploring.",
    "1633613": "Great job 🙌",
    "1633616": "Congratulations Slawek and thanks for sharing.",
    "1633617": "Great work and congratulations 🎉 \n\nThis is my first time also with image processing , 90% of the keywords ( roughly) I never heard about before the competition 😅 plus i spent more time fixing python related issues ( I am new to python also) \n\nI ended up with detectron and cellpose and this competition brought me my first competition based medal (bronze) 🎉",
    "1633618": "I have to say thank you\n@slawekbiel \n\nYou are the star of this competition.\n\nIt is impressive that you had no experience with image segmentation before this competition.\n\nThis is really motivating.\n\nWishing you happy new year.",
    "1633619": "Congratulations @slawekbiel on an awesome finish and thanks for your sharing throughout the competition.\n\nI have some questions regarding finetuning the model from the LIVECell repo.\n\n1. Did you successfully finetune the resnest200 cascade model on this comp's data?\n2.  Did you use the exact same hyperparameters provided by the authors?\n3. What was the compute used?",
    "1633620": "Congrats and thank you for all the great insight you've provided along the way! A very well deserved prize.",
    "1633622": "Thank you @faisalalsrheed! That's very kind",
    "1633623": "Your approach is very interesting. \"size prediction\" is great idea.  Wonder if you can write \"size prediction branch\" into the cellpose model instead of running as a separate model.   Thanks for writing up the summary, I difinitely learned a lot.  I saw your post a few days ago.  Didn't have time to experiment with cellpose.  I will go through your notebooks and the original paper in next few days to learn more about cellpose.",
    "1633625": "Congrats! Thanks for your amazing solution and meaningful notebooks published in this competition!",
    "1633627": "Yes I did. The best score I got with it was .324 public so not phenomenal but less effort and time than training on LiveCell from scratch.\nI kept mostly the config from the repo, set BS to 2 for a single GPU and did some trial and error on lr scheduling.\n\nThe training takes around 2.5hrs on a RTX3090",
    "1633632": "Congratulations! Can I ask more about how you did the pretraining? E.g. Is there a library that you used or did you design the pretraining tasks yourself?",
    "1633633": "Honestly I did think about adding an extra head to the model for just that - but the issue is we resize the image to it's desired resolution before passing it to the model. So it would have to be some two stage approach that takes the original image. Too complicated to attempt in the limited time I had.",
    "1633639": "I used the same code to first train on the LiveCell data and then finetune on the competition data.",
    "1633643": "Thanks for the replay.  Two stages make more sense.  Otherwise, it would be too complicated like you said.",
    "1633700": "Well deserved win and thanks for your community contributions. I learned about Cellpose for the first time here. Didn't use it because of the computational resources needed.",
    "1633746": "Congrats and thank you for sharing about cellpose. I was stuck at maskrcnn(mmdetection) for a long time and then get better result with cellpose instantly. I got only a 100+ bronze in the end but there are some similar exprience with you.\nIt's weird cellpose trys load all images into memory. Perhaps it was desighed for much smaller dataset. I'm on win pc and train with setting 128gb virtual memory at second ssd. I train only 150 epoch until the loss seems to stop decreasing.\nI also pick some livecell cell type for base model train with regard of sartorius cell type. But using all dataset give better performance.\nFine tuning is not hard but it's difficult to find the right diameter for interfence. The sartorius cells sizes are way too various. I use a simple decision tree model to classify the image into cell types and use fixed diameter for 3 type of images. However I think the idle performance  should utilize various diameters in every portion of a image. An additionl test diameter is the drawback of cellpose though. \nIMO the building size predicted model  performance poor. I tried to predict the size with my own resnet model it performance just the same with the cellpose one. I was too stupid not using maskrcnn as a good size predicter though it's not a decent segmentation model here. Maybe the mmdeteciton is not friendly to use that I don't want to use it laterly.",
    "1633749": "Congrats @slawekbiel and team on 3rd place and thanks for sharing solution. Our Cellpose model can get 0.332 LB score but we not use for ensemble final.",
    "1633786": "congrats for gold!!! I learned a lot from your notebooks and discussions ... thanks",
    "1633794": "my notebook is cellpose and 0.318 ，thanks for your work",
    "1633905": "`disk IO is the bottleneck - spinning at 100% while GPU waits idly. I had to rearange stuff on my system to free enough SSD space and only then I got a smooth training fully utilizing the GPU.`\n\n\nDoes it mean we just need to copy file that place model and dataset on SSD rather than mech hard disk ?\nthen traning will be faster ???",
    "1633909": "Congratulations! Thanks for sharing your solution and all the great work you put out during the competition, I learned so much from you and I am very happy that you get rewarded for your hard work. Cheers!",
    "1633943": "Thanks for sharing!",
    "1633961": "You always need to look at the resources consumption in your specific setup. In this particular case where the model is small and there are more files disk cache can handle - Yes, moving files to SSD helped me a lot with my training time.",
    "1634016": "\"Going into this competition I had no experience with image segmentation\"\n\n@slawekbiel: this is an inspiration for everyone wanting to take a challenge in a competition but they think that they don't have the necessary previous experience.\n\nGreat work! All my compliments for your achievement!",
    "1634068": "would u like to share you notebook？ i apply cellpose too，but\nMy PB is 0.318 ,I want learn something from your netobook,thx,thanks for your work and ideas",
    "1634155": "Thanks Luca! My experience with kaggle so far has been that you can learn a lot in three months if you fully commit to it. \n\nAlso while competing in various competitions you acquire meta skills that translate from one to another, even if the problem domains are completely different. I definitely feel I’ve done much better than I would have a year ago by experience in approaching competition, using tools, having my work environment streamlined, taking advantage of forum and public notebooks, etc.",
    "1634161": "I need some time to clean it up and make presentable.",
    "1634240": "Thanks for Sharing!",
    "1634381": "Great work and congrats on 3rd place🎉",
    "1634393": "You are right that loading all the images into memory is weird. Maybe they did work only on smaller datasets but I've also found code like this:\n```\n#again, attempt to deal with memory overuse\nyf, style = None, None\ngc.collect()\ntorch.cuda.empty_cache()\n```\nSo they were aware of high memory consumption, just didn't think \"Hey lets not keep all of the dataset in RAM all the time\" 🤔",
    "1634652": "Inspirational and fascinating, congrats on 3rd place! 🙌",
    "1635036": "I really want to thank you for the wonderful codes that you have shared during the competition. It gave me a lot of ideas. Congrats on 3rd place.",
    "1635070": "I appreciate you saying that @hsadeghian feedback like that is what motivates me to continue trying to improve and sharing things I’ve learned along the way.",
    "1636762": "congratulations and thanks for sharing the code",
    "1637466": "Slawek, excellent job! \nOne night towards the end, I also tried to take pieces of the interesting part of the cellpose repo: it trained but the predictions were an awful fail...\n\nI have a question: \nwhen you generate the \"flows\", are you generating on the fly in the batch generator or are you generating first on disk and then you load them  in your batch generator?",
    "1637713": "I generate it before I start training and read tiff files in my DataLoader.",
    "1637902": "nice sir, excellent job!",
    "1638373": "Amazing work!! Congratulations! Thank you so much for sharing this!!"
  },
  "source": "meta"
}