![\"Selection-999-379\"](\"https://i.ibb.co/MC860H0/Selection-999-379.png\")
this post will document my approach and results for the competition submission.
\ni will update as I work along.
summary of approach:
\nstep 1:
step 2:
\ni suspect with the additional pixel label, anchor-free/box-free instance segmentation model may also work
\nto get high lb score, it is important that
\nthis post will document my approach and results for the competition submission.
\ni will update as I work along.
summary of approach:
\nstep 1:
step 2:
\ni suspect with the additional pixel label, anchor-free/box-free instance segmentation model may also work
\nto get high lb score, it is important that
\nfirst validation results!
\n*no post-processing/filtering of results (e.g. remove small mask)
https://ibb.co/Bf9sgqr
\nhttps://ibb.co/8x5tXwK
\nhttps://ibb.co/GH5FPPS
I train one model for single cell type for now to check pre/post processing parameters. i may combine them into single model later. Here is validation results for other cell type:
\ncort 0.359497
\n\nhttps://ibb.co/Bf9sgqr\nhttps://ibb.co/8x5tXwK\nhttps://ibb.co/GH5FPPS\n\n\nI train one model for single cell type for now to check pre/post processing parameters. i may combine them into single model later. Here is validation results for other cell type:\ncort 0.359497",
"votes": 7
},
{
"id": 1575237,
"postDate": "2021-11-08T09:08:47.943Z",
"content": "update of results:
\ni think this is promising
![\"Selection-999-287\"](\"https://i.ibb.co/F4D0K10/Selection-999-287.png\")
\n![\"Selection-999-286\"](\"https://i.ibb.co/mDL5BZT/Selection-999-286.png\")
\n![\"Selection-999-285\"](\"https://i.ibb.co/v4n53QC/Selection-999-285.png\")
\n![\"Selection-999-284\"](\"https://i.ibb.co/P42B9RG/Selection-999-284.png\")
\n![\"Selection-999-283\"](\"https://i.ibb.co/yNn2VLN/Selection-999-283.png\")
\n![\"Selection-999-282\"](\"https://i.ibb.co/vzwSXPL/Selection-999-282.png\")
deduplicate list online
\n\n\n\n\nother reference:
\nTextField: Learning A Deep Direction Field for Irregular Scene Text Detection
\nhttps://github.com/YukangWang/TextField
Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation
\nhttps://github.com/MouseLand/cellpose
![\"Selection-999-288\"](\"https://i.ibb.co/LQdfRZ3/Selection-999-288.png\")
Applying Deep Watershed Transform to Kaggle Data Science Bowl 2018 (dockerized solution)
\nhttps://spark-in.me/post/playing-with-dwt-and-ds-bowl-2018
Learn to segment single cells with deep distance estimator and deep cell detector
", "rawMarkdown": "other reference:\n\nTextField: Learning A Deep Direction Field for Irregular Scene Text Detection\nhttps://github.com/YukangWang/TextField\n\nOmnipose: a high-precision morphology-independent solution for bacterial cell segmentation\nhttps://github.com/MouseLand/cellpose\n\n\n\nApplying Deep Watershed Transform to Kaggle Data Science Bowl 2018 (dockerized solution)\nhttps://spark-in.me/post/playing-with-dwt-and-ds-bowl-2018\n\nLearn to segment single cells with deep distance estimator and deep cell detector",
"votes": 3
},
{
"id": 1576564,
"postDate": "2021-11-09T10:04:55.357Z",
"content": "![\"Selection-999-296\"](\"https://i.ibb.co/pnz8YWh/Selection-999-296.png\")
be sure to read the top solution and code from @selimsef
\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54741
\n\nbe sure to read the top solution and code from @selimsef \nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54741"
},
{
"id": 1577370,
"postDate": "2021-11-10T03:07:59.140Z",
"content": "intermediate dirty code to generate the barrier:
\nhttps://drive.google.com/drive/folders/1KKx88H-CxpU_mzn555jymLrg0VE4cvZ2?usp=sharing
\n![\"https://i.ibb.co/Cmr5yqf/Selection-999-306.png\"](\"https://i.ibb.co/Cmr5yqf/Selection-999-306.png\")
In section 4.3 of the Deep Watershed Transform paper, the authors point out that, after training the Direction Net and Watershed Transform Net separately, they cascade the 2 networks and fine-tune the this, using the distance transform as the target.
\n\n\nEnd-to-end fine-tuning: We cascaded the pre-trained
\n
\n models for the DN and WTN and fine-tuned the complete
\n model for 20 epochs using the RGB image and semantic
\n segmentation output of PSPNet as input, and the ground
\n truth distance transforms as the training target. We use a
\n batch size of 3, constant learning rate of 5e-6, and a L2
\n weight penalty of 1e-6.
I don't think they mention which loss function is used for this fine-tuning. Any idea?
", "rawMarkdown": "In section 4.3 of the [Deep Watershed Transform paper](https://arxiv.org/pdf/1611.08303.pdf), the authors point out that, after training the Direction Net and Watershed Transform Net separately, they cascade the 2 networks and fine-tune the this, using the *distance transform* as the target. \n\n> End-to-end fine-tuning: We cascaded the pre-trained\nmodels for the DN and WTN and fine-tuned the complete\nmodel for 20 epochs using the RGB image and semantic\nsegmentation output of PSPNet as input, and the ground\ntruth distance transforms as the training target. We use a\nbatch size of 3, constant learning rate of 5e-6, and a L2\nweight penalty of 1e-6.\n\nI don't think they mention which loss function is used for this fine-tuning. Any idea?" } ] }, { "id": 1612343, "postDate": "2021-12-08T19:22:43.053Z", "content": "yet another good method:
\nhttps://github.com/ruotianluo/adaptis.pytorch
![\"toy-v2-comparison\"](\"https://i.ibb.co/1GcFgGg/toy-v2-comparison.jpg\")
",
"votes": 3
},
{
"id": 1583696,
"postDate": "2021-11-16T02:15:33.907Z",
"content": "solo instance encode:
\n![\"Selection-999-392\"](\"https://i.ibb.co/MfqdMJb/Selection-999-392.png\")
to convert from instance encoding to label image, use max pooling:
\nhttps://github.com/kornia/kornia/pull/1184
the whole process from the input image to the connected component label (CCL) to now fully differentiable.
\nbut I haven't figure out how to write the differentiable loss function between two CCL images.
in training you can modify kornia code to include seeding value. this ensures the CCL labels corresponds to ground truth CCL in loss
", "rawMarkdown": "solo instance encode:\n\n![\"Selection-999-410\"](\"https://i.ibb.co/7pk5p8k/Selection-999-410.png\")
\n![\"Selection-999-409\"](\"https://i.ibb.co/0mgcjzt/Selection-999-409.png\")
\n"
},
{
"id": 1586294,
"postDate": "2021-11-18T01:08:04.807Z",
"content": "Do you really need CCL though? the whole point of SOLO is to assign a mask to location. So you already have instance segmentation info in the outputs…
", "rawMarkdown": "Do you really need CCL though? the whole point of SOLO is to assign a mask to location. So you already have instance segmentation info in the outputs..." }, { "id": 1586315, "postDate": "2021-11-18T01:47:14.970Z", "content": "it depends on how you divide the grid for solo.
\nif the target object is small and overlapping, your grid is dense. I work out that SxS in solo is too large for efficient convolution if you use the full image resolution.
some cell are large and some are small. some needs large context, etc
\nit seems to me that instance-FCN paper is a more efficient solution
", "rawMarkdown": "it depends on how you divide the grid for solo.\nif the target object is small and overlapping, your grid is dense. I work out that SxS in solo is too large for efficient convolution if you use the full image resolution.\n\nsome cell are large and some are small. some needs large context, etc\n\nit seems to me that instance-FCN paper is a more efficient solution" }, { "id": 1586317, "postDate": "2021-11-18T01:52:08.027Z", "content": "Have you tried SOLOv2? The idea behind it is quite neat, and doesn't require SxS segmentation maps. So in theory you can have finer grid (like, S=50) and still be able to produce some results.
\nPS: thanks for trying out all sorts of ideas btw :) I find this topic quite helpful in broadening my knowledge about segmentation models :)
", "rawMarkdown": "Have you tried SOLOv2? The idea behind it is quite neat, and doesn't require SxS segmentation maps. So in theory you can have finer grid (like, S=50) and still be able to produce some results.\n\nPS: thanks for trying out all sorts of ideas btw :) I find this topic quite helpful in broadening my knowledge about segmentation models :)" } ] }, { "id": 1634355, "postDate": "2021-12-31T16:10:38.293Z", "content": "@hengck23 do you mind if I ask what happened, have you never finished your solution? I was fully expecting you to jump to the top at any time, based on the intermediate results you were sharing.
", "rawMarkdown": "@hengck23 do you mind if I ask what happened, have you never finished your solution? I was fully expecting you to jump to the top at any time, based on the intermediate results you were sharing.", "votes": 1, "replies": [ { "id": 1634755, "postDate": "2022-01-01T05:59:48.640Z", "content": "\nthanks for the concern. unfortunately, i had a fall a few weeks ago and fractured my kneecap.
\nmy doctor advised me to rest more in bed and hence I haven't completed my solution yet.
\nmaybe i will make a notebook when i am well and free later.
lastly, congrats for finishing 3rd place in the ranking. it is very good work in this very short period of time.👍
", "rawMarkdown": "@slawekbiel \n\nthanks for the concern. unfortunately, i had a fall a few weeks ago and fractured my kneecap.\nmy doctor advised me to rest more in bed and hence I haven't completed my solution yet. \nmaybe i will make a notebook when i am well and free later.\n\nlastly, congrats for finishing 3rd place in the ranking. it is very good work in this very short period of time.👍", "votes": 3 }, { "id": 1634986, "postDate": "2022-01-01T11:17:25.937Z", "content": "Ouch! Get well and thanks for sharing your work and insights.
", "rawMarkdown": "Ouch! Get well and thanks for sharing your work and insights." } ] }, { "id": 1600592, "postDate": "2021-11-30T14:51:34.277Z", "content": "the astro class is the most difficult. it has complex cell shape. some are very long and thin.
\nIt is difficult to use watershed for complex shape.
\nAfter some work, I manage to find a feasible solution below.
Hint: this method is also applicable to any other method like mask rcnn. results of your model are used as seed.
\nthe coloring net is a refinement network at full resolution.
![\"Selection-999-520\"](\"https://i.ibb.co/pLQgtm7/Selection-999-520.png\")
\n![\"Selection-999-519\"](\"https://i.ibb.co/cL8bd73/Selection-999-519.png\")
\n![\"Selection-999-518\"](\"https://i.ibb.co/s90sWjm/Selection-999-518.png\")
only top results of the top seeds are shown. i need to train a mask iou predicted to rank the results
", "rawMarkdown": "the astro class is the most difficult. it has complex cell shape. some are very long and thin.\nIt is difficult to use watershed for complex shape.\nAfter some work, I manage to find a feasible solution below.\n\n\nHint: this method is also applicable to any other method like mask rcnn. results of your model are used as seed.\nthe coloring net is a refinement network at full resolution.\n\n\n\n\n\n\n\nonly top results of the top seeds are shown. i need to train a mask iou predicted to rank the results",
"votes": 1,
"replies": [
{
"id": 1603963,
"postDate": "2021-12-03T01:23:55.333Z",
"content": "code to split cells into subset of non touching ones:
\nhttps://www.kaggle.com/hengck23/split-adjoining-cell-into-subsets-of-non-touching
example code for coloring network is up
\nhttps://www.kaggle.com/hengck23/coloring-network-for-instance-segmentation
Thank you. great job. how did you train the model? can there be a repository?
", "rawMarkdown": "Thank you. great job. how did you train the model? can there be a repository?" } ] }, { "id": 1583587, "postDate": "2021-11-16T00:38:03.783Z", "content": "in theory, we could output mask RLE encode directly
\n![\"https://i.ibb.co/gZK3BKQ/Selection-999-382.png\"](\"https://i.ibb.co/gZK3BKQ/Selection-999-382.png\")
I also see papers that uses pca to compress the mask. they predict pca coefficient per pixel to encode mask
\n(hmm … how about VAE as a non linear version of pca)
i just find a paper that does this:
\nhttp://www.cs.toronto.edu/~fidler/papers/sgn_iccv17.pdf
\nSequential Grouping Networks for Instance Segmentation (SGN，ICCV 2017)
![\"https://miro.medium.com/max/700/1*3AdI0hce3YVv_fwe1JqWeQ.png\"](\"https://miro.medium.com/max/700/1*3AdI0hce3YVv_fwe1JqWeQ.png\")
https://github.com/JunMa11/SegWithDistMap
\nMotivation: How Distance Transform Maps Boost Segmentation CNNs (MIDL 2020)
I tried deep watershed transform for a bit. The model seems to over-smooth the energy landscape (compared to normal watershed technique which over segments). Logical next step seems to be using boundary information as suggested.
\nP.S. Trained with ground truth image gradient.
", "rawMarkdown": "I tried deep watershed transform for a bit. The model seems to over-smooth the energy landscape (compared to normal watershed technique which over segments). Logical next step seems to be using boundary information as suggested.\n\nP.S. Trained with ground truth image gradient." } ] }, { "id": 1577987, "postDate": "2021-11-10T14:43:09.080Z", "content": "Inpainting Networks Learn to Separate Cells in Microscopy Images
\nhttps://www.bmvc2020-conference.com/conference/papers/paper_0528.html
\nhttps://arxiv.org/pdf/2003.00891.pdf
\nhttps://archiv.ub.uni-heidelberg.de/volltextserver/28353/1/steffen_wolf_thesis_compressed.pdf
\n\nhttps://www.bmvc2020-conference.com/conference/papers/paper_0528.html
\nA current limitation of our
\n
\n method is the runtime: INPAINTAFF requires around 48h to process a 700x1100 image on
\n a single GPU.
This might not work :)
", "rawMarkdown": "> https://www.bmvc2020-conference.com/conference/papers/paper_0528.html\n\n>A current limitation of our\nmethod is the runtime: INPAINTAFF requires around 48h to process a 700x1100 image on\na single GPU. \n\nThis might not work :)" } ] }, { "id": 1577082, "postDate": "2021-11-09T18:34:12.800Z", "content": "i came across this interesting paper:
\n![\"Selection-999-304\"](\"https://i.ibb.co/wp8Br6j/Selection-999-304.png\")
",
"votes": 1
},
{
"id": 1572633,
"postDate": "2021-11-05T20:10:37.337Z",
"content": "some tricks
\nimportant resource:
\n\n… more to come …
", "rawMarkdown": "some tricks\n1. upsize image for input to give better iou at prediction\n\n\n---\nimportant resource:\n- https://github.com/sartorius-research/LIVECell/tree/main/model\n\n... more to come ...", "votes": 1 }, { "id": 1578015, "postDate": "2021-11-10T15:06:10.960Z", "content": "a good writeup:
\nhttps://towardsdatascience.com/single-stage-instance-segmentation-a-review-1eeb66e0cc49
also here:
\nhttps://www.youtube.com/watch?v=LMZI8DDyltQ&t=3699s
after reading the article, the conclusion is that you don't need to crop roi if the target object don't overlap. we can convert mask-rcnn (instance segmentation) into multiple unet problem (semantic):
\n![\"Selection-999-344\"](\"https://i.ibb.co/BZH87wF/Selection-999-344.png\")
there are multiple channels corresponding to different object location and sizes. if we can predict that the output channel would be empty, we can skip it.
\nactually, in mask rcnn, we have the predicted size of the object prior to segmentation. we can use this information to group segmentation such that they don't overlap. then we can output a set of semantic segmentation sub problems. For each sub problem, we need to specify which object to segment, e.g. using seed or marker as input.
\nimproved rcnn (get rid of the roi cropping!!!):\nimage -->[rpn]-->proposal -->[rcnn head] --> location,size of object -->[assemble] --> group of non-overlap objects -->[semantic segmnatation head] --->output masks\n![\"Selection-999-352\"](\"https://i.ibb.co/4YYL8Jt/Selection-999-352.png\")
tip: perturb seed slightly for TTA
", "rawMarkdown": "after reading the article, the conclusion is that you don't need to crop roi if the target object don't overlap. we can convert mask-rcnn (instance segmentation) into multiple unet problem (semantic):\n\n\n\n\ntip: perturb seed slightly for TTA",
"votes": 1
},
{
"id": 1579034,
"postDate": "2021-11-11T13:27:57.850Z",
"content": "will the output channel number effect performance?
", "rawMarkdown": "will the output channel number effect performance?" } ] }, { "id": 1577868, "postDate": "2021-11-10T13:26:50.013Z", "content": "i try on a few validation images, it seems to work.
\nbut I have a slight problem assigning random colors to the seed … i need to make sure no two adjacent seed have the same color
\n![\"Selection-999-342\"](\"https://i.ibb.co/3B8QwnD/Selection-999-342.png\")
\n![\"Selection-999-341\"](\"https://i.ibb.co/WtytKpb/Selection-999-341.png\")
\n",
"votes": 2
},
{
"id": 1576530,
"postDate": "2021-11-09T09:27:36.343Z",
"content": "I'm following you again (and I still have to debug my ventilator code from your baseline 😤)
", "rawMarkdown": "I'm following you again (and I still have to debug my ventilator code from your baseline 😤)", "votes": 2 }, { "id": 1571932, "postDate": "2021-11-05T09:36:06.920Z", "content": "@hengck23 welcome to this competition! After finishing GB Ventilator I was considering whether to join this competition. After seeing you here it' clear. Thank you for all your absolutely great analyses in both my previous competition - GB Ventilator and G2Net. They are an extremely valuable source of knowledge for many others.
", "rawMarkdown": "@hengck23 welcome to this competition! After finishing GB Ventilator I was considering whether to join this competition. After seeing you here it' clear. Thank you for all your absolutely great analyses in both my previous competition - GB Ventilator and G2Net. They are an extremely valuable source of knowledge for many others.", "votes": 2 }, { "id": 1602155, "postDate": "2021-12-01T19:51:48.330Z", "content": "i find an interesting paper:
\n2d instance segmentation is just semantic segmentation in 3d
![\"Selection-999-529\"](\"https://i.ibb.co/ZJ2rdpB/Selection-999-529.png\")
"
},
{
"id": 1601500,
"postDate": "2021-12-01T11:02:35.180Z",
"content": "demo results from open source
\n![\"Selection-999-527\"](\"https://i.ibb.co/cDRjqJD/Selection-999-527.png\")
"
},
{
"id": 1587504,
"postDate": "2021-11-18T18:43:37.703Z",
"content": "i have an interesting bug.
\nresults of coloring network?
\nhttps://i.postimg.cc/1t8GJgDG/coloring.gif
Nice work!
", "rawMarkdown": "Nice work!" }, { "id": 1580807, "postDate": "2021-11-13T06:11:48.233Z", "content": "https://github.com/constantinpape/torch-em
\nhttps://openaccess.thecvf.com/content_ECCV_2018/html/Steffen_Wolf_The_Mutex_Watershed_ECCV_2018_paper.html
one way to encode instance segmentation
\n![\"Selection-999-365\"](\"https://i.ibb.co/VmjkZZ5/Selection-999-365.png\")
i came to realise that this is what the \"assembling based\" methods are doing(e.g. rfcn, yoloact,solov2)
\nsee also: https://arxiv.org/pdf/2101.10913.pdf
\n\ni came to realise that this is what the \"assembling based\" methods are doing(e.g. rfcn, yoloact,solov2)\nsee also: https://arxiv.org/pdf/2101.10913.pdf"
},
{
"id": 1580655,
"postDate": "2021-11-13T01:50:44.717Z",
"content": "https://www.youtube.com/watch?v=MVDUnbVXMGM
\nDeep Learning for Cell Imaging Segmentation - Lecture 20 - MIT ML in Life Sciences (Spring 2021)
", "rawMarkdown": "https://www.youtube.com/watch?v=MVDUnbVXMGM\n\nDeep Learning for Cell Imaging Segmentation - Lecture 20 - MIT ML in Life Sciences (Spring 2021)" }, { "id": 1580649, "postDate": "2021-11-13T01:27:43.223Z", "content": "Self-supervised pretraining for transferable quantitative phase image cell segmentation
\nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC8547997/
![\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8547997/bin/boe-12-10-6514-g001.jpg\"](\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8547997/bin/boe-12-10-6514-g001.jpg\")
![\"Selection-999-363\"](\"https://i.ibb.co/ZHbw8w2/Selection-999-363.png\")
![\"boe-12-10-6514-g001\"](\"https://i.ibb.co/6b1C8cD/boe-12-10-6514-g001.jpg\")
\n\n"
},
{
"id": 1578721,
"postDate": "2021-11-11T09:39:07.363Z",
"content": "👍good work
", "rawMarkdown": "👍good work" }, { "id": 1577965, "postDate": "2021-11-10T14:27:38.027Z", "content": "https://github.com/juglab/EmbedSeg
\nembedding-based Instance Segmentation of Microscopy Images.
![\"https://github.com/yijingru/ObjGuided-Instance-Segmentation/raw/main/imgs/figure.png\"](\"https://github.com/yijingru/ObjGuided-Instance-Segmentation/raw/main/imgs/figure.png\")
https://github.com/yijingru/ObjGuided-Instance-Segmentation
", "rawMarkdown": "\n\nhttps://github.com/yijingru/ObjGuided-Instance-Segmentation" }, { "id": 1577088, "postDate": "2021-11-09T18:46:08.197Z", "content": "[place holder]
\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/239001
", "rawMarkdown": "[place holder] \n\n1. how to adapt puzzle-cam for semi-supervised learning\n2. are the semi-supervised learning images related by time? if so can cell tracking be used?\n\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/239001 " }, { "id": 1576942, "postDate": "2021-11-09T15:23:57.027Z", "content": "following to learn
", "rawMarkdown": "following to learn " }, { "id": 1576478, "postDate": "2021-11-09T08:07:25.883Z", "content": "Good job! from the Ventilator Pressure Prediction, i learn many from you, this competition, i will expect your works!
", "rawMarkdown": "Good job! from the Ventilator Pressure Prediction, i learn many from you, this competition, i will expect your works!" }, { "id": 1573897, "postDate": "2021-11-07T03:21:13.967Z", "content": "I am really looking forward to learning some surprises with you in this game.😄
", "rawMarkdown": "I am really looking forward to learning some surprises with you in this game.😄" }, { "id": 1571905, "postDate": "2021-11-05T09:13:24.060Z", "content": "I've really enjoyed your insightful experiment write-ups in ventilator competition, so looking forward to this one!
", "rawMarkdown": "I've really enjoyed your insightful experiment write-ups in ventilator competition, so looking forward to this one!" }, { "id": 1571870, "postDate": "2021-11-05T08:37:51.617Z", "content": "new journey begins. haha~~~
", "rawMarkdown": "new journey begins. haha~~~" }, { "id": 1582675, "postDate": "2021-11-15T06:31:17.547Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 1582745, "postDate": "2021-11-15T08:22:51.387Z", "content": "i suggest you can use the code from data sciencebowl 2018
", "rawMarkdown": "i suggest you can use the code from data sciencebowl 2018" }, { "id": 1582761, "postDate": "2021-11-15T08:38:56.187Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1583285, "postDate": "2021-11-15T17:25:03.270Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1579678, "postDate": "2021-11-12T06:01:13.883Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1582746, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-15T08:23:16.937000", "content": "first validation results!
\n*no post-processing/filtering of results (e.g. remove small mask)
https://ibb.co/Bf9sgqr
\nhttps://ibb.co/8x5tXwK
\nhttps://ibb.co/GH5FPPS
I train one model for single cell type for now to check pre/post processing parameters. i may combine them into single model later. Here is validation results for other cell type:
\ncort 0.359497
update of results:
\ni think this is promising
\n
\n
\n
\n
\n
deduplicate list online
other reference:
\nTextField: Learning A Deep Direction Field for Irregular Scene Text Detection
\nhttps://github.com/YukangWang/TextField
Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation
\nhttps://github.com/MouseLand/cellpose
Applying Deep Watershed Transform to Kaggle Data Science Bowl 2018 (dockerized solution)
\nhttps://spark-in.me/post/playing-with-dwt-and-ds-bowl-2018
Learn to segment single cells with deep distance estimator and deep cell detector
", "votes": 3, "replies": [] }, { "id": 1576564, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-09T10:04:55.357000", "content": "
be sure to read the top solution and code from @selimsef
\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54741
intermediate dirty code to generate the barrier:
\nhttps://drive.google.com/drive/folders/1KKx88H-CxpU_mzn555jymLrg0VE4cvZ2?usp=sharing
\n
In section 4.3 of the Deep Watershed Transform paper, the authors point out that, after training the Direction Net and Watershed Transform Net separately, they cascade the 2 networks and fine-tune the this, using the distance transform as the target.
\n\n\nEnd-to-end fine-tuning: We cascaded the pre-trained
\n
\n models for the DN and WTN and fine-tuned the complete
\n model for 20 epochs using the RGB image and semantic
\n segmentation output of PSPNet as input, and the ground
\n truth distance transforms as the training target. We use a
\n batch size of 3, constant learning rate of 5e-6, and a L2
\n weight penalty of 1e-6.
I don't think they mention which loss function is used for this fine-tuning. Any idea?
", "votes": 0, "replies": [] } ] }, { "id": 1612343, "author_name": "hengck23", "author_url": "", "post_date": "2021-12-08T19:22:43.053000", "content": "yet another good method:
\nhttps://github.com/ruotianluo/adaptis.pytorch
solo instance encode:
\n
to convert from instance encoding to label image, use max pooling:
\nhttps://github.com/kornia/kornia/pull/1184
the whole process from the input image to the connected component label (CCL) to now fully differentiable.
\nbut I haven't figure out how to write the differentiable loss function between two CCL images.
in training you can modify kornia code to include seeding value. this ensures the CCL labels corresponds to ground truth CCL in loss
", "votes": 3, "replies": [ { "id": 1583824, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-16T05:06:48.190000", "content": "
\n
Do you really need CCL though? the whole point of SOLO is to assign a mask to location. So you already have instance segmentation info in the outputs…
", "votes": 0, "replies": [] }, { "id": 1586315, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-18T01:47:14.970000", "content": "it depends on how you divide the grid for solo.
\nif the target object is small and overlapping, your grid is dense. I work out that SxS in solo is too large for efficient convolution if you use the full image resolution.
some cell are large and some are small. some needs large context, etc
\nit seems to me that instance-FCN paper is a more efficient solution
", "votes": 0, "replies": [] }, { "id": 1586317, "author_name": "Chan Kha Vu", "author_url": "", "post_date": "2021-11-18T01:52:08.027000", "content": "Have you tried SOLOv2? The idea behind it is quite neat, and doesn't require SxS segmentation maps. So in theory you can have finer grid (like, S=50) and still be able to produce some results.
\nPS: thanks for trying out all sorts of ideas btw :) I find this topic quite helpful in broadening my knowledge about segmentation models :)
", "votes": 0, "replies": [] } ] }, { "id": 1634355, "author_name": "Slawek Biel", "author_url": "", "post_date": "2021-12-31T16:10:38.293000", "content": "@hengck23 do you mind if I ask what happened, have you never finished your solution? I was fully expecting you to jump to the top at any time, based on the intermediate results you were sharing.
", "votes": 1, "replies": [ { "id": 1634755, "author_name": "hengck23", "author_url": "", "post_date": "2022-01-01T05:59:48.640000", "content": "\nthanks for the concern. unfortunately, i had a fall a few weeks ago and fractured my kneecap.
\nmy doctor advised me to rest more in bed and hence I haven't completed my solution yet.
\nmaybe i will make a notebook when i am well and free later.
lastly, congrats for finishing 3rd place in the ranking. it is very good work in this very short period of time.👍
", "votes": 3, "replies": [] }, { "id": 1634986, "author_name": "Slawek Biel", "author_url": "", "post_date": "2022-01-01T11:17:25.937000", "content": "Ouch! Get well and thanks for sharing your work and insights.
", "votes": 0, "replies": [] } ] }, { "id": 1600592, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-30T14:51:34.277000", "content": "the astro class is the most difficult. it has complex cell shape. some are very long and thin.
\nIt is difficult to use watershed for complex shape.
\nAfter some work, I manage to find a feasible solution below.
Hint: this method is also applicable to any other method like mask rcnn. results of your model are used as seed.
\nthe coloring net is a refinement network at full resolution.
\n
\n
only top results of the top seeds are shown. i need to train a mask iou predicted to rank the results
", "votes": 1, "replies": [ { "id": 1603963, "author_name": "hengck23", "author_url": "", "post_date": "2021-12-03T01:23:55.333000", "content": "code to split cells into subset of non touching ones:
\nhttps://www.kaggle.com/hengck23/split-adjoining-cell-into-subsets-of-non-touching
example code for coloring network is up
\nhttps://www.kaggle.com/hengck23/coloring-network-for-instance-segmentation
Thank you. great job. how did you train the model? can there be a repository?
", "votes": 0, "replies": [] } ] }, { "id": 1583587, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-16T00:38:03.783000", "content": "in theory, we could output mask RLE encode directly
\n
I also see papers that uses pca to compress the mask. they predict pca coefficient per pixel to encode mask
\n(hmm … how about VAE as a non linear version of pca)
i just find a paper that does this:
\nhttp://www.cs.toronto.edu/~fidler/papers/sgn_iccv17.pdf
\nSequential Grouping Networks for Instance Segmentation (SGN，ICCV 2017)
https://github.com/JunMa11/SegWithDistMap
\nMotivation: How Distance Transform Maps Boost Segmentation CNNs (MIDL 2020)
I tried deep watershed transform for a bit. The model seems to over-smooth the energy landscape (compared to normal watershed technique which over segments). Logical next step seems to be using boundary information as suggested.
\nP.S. Trained with ground truth image gradient.
", "votes": 0, "replies": [] } ] }, { "id": 1577987, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-10T14:43:09.080000", "content": "Inpainting Networks Learn to Separate Cells in Microscopy Images
\nhttps://www.bmvc2020-conference.com/conference/papers/paper_0528.html
\nhttps://arxiv.org/pdf/2003.00891.pdf
\nhttps://archiv.ub.uni-heidelberg.de/volltextserver/28353/1/steffen_wolf_thesis_compressed.pdf
\n\nhttps://www.bmvc2020-conference.com/conference/papers/paper_0528.html
\nA current limitation of our
\n
\n method is the runtime: INPAINTAFF requires around 48h to process a 700x1100 image on
\n a single GPU.
This might not work :)
", "votes": 0, "replies": [] } ] }, { "id": 1577082, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-09T18:34:12.800000", "content": "i came across this interesting paper:
\n
some tricks
\nimportant resource:
\n\n… more to come …
", "votes": 1, "replies": [] }, { "id": 1578015, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-10T15:06:10.960000", "content": "a good writeup:
\nhttps://towardsdatascience.com/single-stage-instance-segmentation-a-review-1eeb66e0cc49
also here:
\nhttps://www.youtube.com/watch?v=LMZI8DDyltQ&t=3699s
after reading the article, the conclusion is that you don't need to crop roi if the target object don't overlap. we can convert mask-rcnn (instance segmentation) into multiple unet problem (semantic):
\n
there are multiple channels corresponding to different object location and sizes. if we can predict that the output channel would be empty, we can skip it.
\nactually, in mask rcnn, we have the predicted size of the object prior to segmentation. we can use this information to group segmentation such that they don't overlap. then we can output a set of semantic segmentation sub problems. For each sub problem, we need to specify which object to segment, e.g. using seed or marker as input.
\nimproved rcnn (get rid of the roi cropping!!!):\nimage -->[rpn]-->proposal -->[rcnn head] --> location,size of object -->[assemble] --> group of non-overlap objects -->[semantic segmnatation head] --->output masks\n
tip: perturb seed slightly for TTA
", "votes": 1, "replies": [] }, { "id": 1579034, "author_name": "dragon zhang", "author_url": "", "post_date": "2021-11-11T13:27:57.850000", "content": "will the output channel number effect performance?
", "votes": 0, "replies": [] } ] }, { "id": 1577868, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-10T13:26:50.013000", "content": "i try on a few validation images, it seems to work.
\nbut I have a slight problem assigning random colors to the seed … i need to make sure no two adjacent seed have the same color
\n
\n
I'm following you again (and I still have to debug my ventilator code from your baseline 😤)
", "votes": 2, "replies": [] }, { "id": 1571932, "author_name": "Allie K.", "author_url": "", "post_date": "2021-11-05T09:36:06.920000", "content": "@hengck23 welcome to this competition! After finishing GB Ventilator I was considering whether to join this competition. After seeing you here it' clear. Thank you for all your absolutely great analyses in both my previous competition - GB Ventilator and G2Net. They are an extremely valuable source of knowledge for many others.
", "votes": 2, "replies": [] }, { "id": 1602155, "author_name": "hengck23", "author_url": "", "post_date": "2021-12-01T19:51:48.330000", "content": "i find an interesting paper:
\n2d instance segmentation is just semantic segmentation in 3d
demo results from open source
\n
i have an interesting bug.
\nresults of coloring network?
\nhttps://i.postimg.cc/1t8GJgDG/coloring.gif
Nice work!
", "votes": 0, "replies": [] }, { "id": 1580807, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-13T06:11:48.233000", "content": "https://github.com/constantinpape/torch-em
\nhttps://openaccess.thecvf.com/content_ECCV_2018/html/Steffen_Wolf_The_Mutex_Watershed_ECCV_2018_paper.html
one way to encode instance segmentation
\n
i came to realise that this is what the \"assembling based\" methods are doing(e.g. rfcn, yoloact,solov2)
\nsee also: https://arxiv.org/pdf/2101.10913.pdf
https://www.youtube.com/watch?v=MVDUnbVXMGM
\nDeep Learning for Cell Imaging Segmentation - Lecture 20 - MIT ML in Life Sciences (Spring 2021)
", "votes": 0, "replies": [] }, { "id": 1580649, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-13T01:27:43.223000", "content": "Self-supervised pretraining for transferable quantitative phase image cell segmentation
\nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC8547997/
👍good work
", "votes": 0, "replies": [] }, { "id": 1577965, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-10T14:27:38.027000", "content": "https://github.com/juglab/EmbedSeg
\nembedding-based Instance Segmentation of Microscopy Images.
https://github.com/yijingru/ObjGuided-Instance-Segmentation
", "votes": 0, "replies": [] }, { "id": 1577088, "author_name": "hengck23", "author_url": "", "post_date": "2021-11-09T18:46:08.197000", "content": "[place holder]
\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/239001
", "votes": 0, "replies": [] }, { "id": 1576942, "author_name": "Haw Keat", "author_url": "", "post_date": "2021-11-09T15:23:57.027000", "content": "following to learn
", "votes": 0, "replies": [] }, { "id": 1576478, "author_name": "Ctrl_CV", "author_url": "", "post_date": "2021-11-09T08:07:25.883000", "content": "Good job! from the Ventilator Pressure Prediction, i learn many from you, this competition, i will expect your works!
", "votes": 0, "replies": [] }, { "id": 1573897, "author_name": "", "author_url": "", "post_date": "2021-11-07T03:21:13.967000", "content": "", "votes": 0, "replies": [] }, { "id": 1571905, "author_name": "", "author_url": "", "post_date": "2021-11-05T09:13:24.060000", "content": "", "votes": 0, "replies": [] }, { "id": 1571870, "author_name": "", "author_url": "", "post_date": "2021-11-05T08:37:51.617000", "content": "", "votes": 0, "replies": [] }, { "id": 1582675, "author_name": "", "author_url": "", "post_date": "2021-11-15T06:31:17.547000", "content": "", "votes": 0, "replies": [ { "id": 1582745, "author_name": "", "author_url": "", "post_date": "2021-11-15T08:22:51.387000", "content": "", "votes": 0, "replies": [] }, { "id": 1582761, "author_name": "", "author_url": "", "post_date": "2021-11-15T08:38:56.187000", "content": "", "votes": 0, "replies": [] }, { "id": 1583285, "author_name": "", "author_url": "", "post_date": "2021-11-15T17:25:03.270000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1579678, "author_name": "", "author_url": "", "post_date": "2021-11-12T06:01:13.883000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1571625": "this post will document my approach and results for the competition submission.\ni will update as I work along.\n\nsummary of approach:\nstep 1:\n- use unet,etc method to translate input image to pixel per label segmentation (e.g. background, cell, cell boundary, cell center)\n\nstep 2:\n- input per label segmentation (and maybe also original image) into instance segmentation model like mask-rcnn.\n\ni suspect with the additional pixel label, anchor-free/box-free instance segmentation model may also work\n\nto get high lb score, it is important that\n- the center detection rate mask be high in step.1\n- if we can do without the use of original image in step.2, we can have effective transfer learning. I am also thinking of using GAN to generate more train samples\n- it may be possible to combine step.1 and step2 in single end-to-end model\n- to find the upper limit of the approach or as a proof of concept, one can feed in ground truth of step1 into mask rcnn/anchor free model, etc and train that model first.\n- a simple way to do self-supervised learning:\n - from an image, create an augmented version, e.g. flip or scale or rotate\n - the pixel label from unet in step.1 should be consistent. hence you can use consistency loss or contrastive loss", "1582746": "first validation results!\n*no post-processing/filtering of results (e.g. remove small mask)\n\n\n\n\nhttps://ibb.co/Bf9sgqr\nhttps://ibb.co/8x5tXwK\nhttps://ibb.co/GH5FPPS\n\n\nI train one model for single cell type for now to check pre/post processing parameters. i may combine them into single model later. Here is validation results for other cell type:\ncort 0.359497",
"1575237": "update of results:\ni think this is promising\n\n\n\n\n\n\n",
"1583696": "solo instance encode:\n\n\n\n\n\nonly top results of the top seeds are shown. i need to train a mask iou predicted to rank the results",
"1583587": "in theory, we could output mask RLE encode directly\n\n\nI also see papers that uses pca to compress the mask. they predict pca coefficient per pixel to encode mask\n(hmm ... how about VAE as a non linear version of pca)",
"1581659": "https://github.com/JunMa11/SegWithDistMap\nMotivation: How Distance Transform Maps Boost Segmentation CNNs (MIDL 2020)",
"1577987": "Inpainting Networks Learn to Separate Cells in Microscopy Images\nhttps://www.bmvc2020-conference.com/conference/papers/paper_0528.html\nhttps://arxiv.org/pdf/2003.00891.pdf\nhttps://archiv.ub.uni-heidelberg.de/volltextserver/28353/1/steffen_wolf_thesis_compressed.pdf",
"1577082": "i came across this interesting paper:\n\n- perform cell vs non-cell 2 class semantic segmentation\n- for each cell pixel, use a voting network to vote its center location. this creates a heatmap.\n- the detect the center, threshold on the heatmap. though voting backprojection recover the instance segmentation.\n\n\n\n",
"1572633": "some tricks\n1. upsize image for input to give better iou at prediction\n\n\n---\nimportant resource:\n- https://github.com/sartorius-research/LIVECell/tree/main/model\n\n... more to come ...",
"1578015": "a good writeup:\nhttps://towardsdatascience.com/single-stage-instance-segmentation-a-review-1eeb66e0cc49\n\nalso here:\nhttps://www.youtube.com/watch?v=LMZI8DDyltQ&t=3699s",
"1577868": "i try on a few validation images, it seems to work.\n\nbut I have a slight problem assigning random colors to the seed ... i need to make sure no two adjacent seed have the same color\n\n\n",
"1576530": "I'm following you again (and I still have to debug my ventilator code from your baseline 😤)",
"1571932": "@hengck23 welcome to this competition! After finishing GB Ventilator I was considering whether to join this competition. After seeing you here it' clear. Thank you for all your absolutely great analyses in both my previous competition - GB Ventilator and G2Net. They are an extremely valuable source of knowledge for many others.",
"1602155": "i find an interesting paper:\n2d instance segmentation is just semantic segmentation in 3d\n\n\n",
"1601500": "demo results from open source\n",
"1587504": "i have an interesting bug.\n\nresults of coloring network?\nhttps://i.postimg.cc/1t8GJgDG/coloring.gif\n\n ",
"1581557": "Nice work!",
"1580807": "https://github.com/constantinpape/torch-em\nhttps://openaccess.thecvf.com/content_ECCV_2018/html/Steffen_Wolf_The_Mutex_Watershed_ECCV_2018_paper.html\n",
"1580675": "one way to encode instance segmentation\n\n\ni came to realise that this is what the \"assembling based\" methods are doing(e.g. rfcn, yoloact,solov2)\nsee also: https://arxiv.org/pdf/2101.10913.pdf",
"1580655": "https://www.youtube.com/watch?v=MVDUnbVXMGM\n\nDeep Learning for Cell Imaging Segmentation - Lecture 20 - MIT ML in Life Sciences (Spring 2021)",
"1580649": "Self-supervised pretraining for transferable quantitative phase image cell segmentation\nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC8547997/\n\n\n\n\n\n",
"1578721": "👍good work",
"1577965": "https://github.com/juglab/EmbedSeg\nembedding-based Instance Segmentation of Microscopy Images.",
"1577533": "\n\nhttps://github.com/yijingru/ObjGuided-Instance-Segmentation",
"1577088": "[place holder] \n\n1. how to adapt puzzle-cam for semi-supervised learning\n2. are the semi-supervised learning images related by time? if so can cell tracking be used?\n\nhttps://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/239001 ",
"1576942": "following to learn ",
"1576478": "Good job! from the Ventilator Pressure Prediction, i learn many from you, this competition, i will expect your works!",
"1573897": "I am really looking forward to learning some surprises with you in this game.😄",
"1571905": "I've really enjoyed your insightful experiment write-ups in ventilator competition, so looking forward to this one!",
"1571870": "new journey begins. haha~~~",
"1582675": "",
"1579678": ""
}
}