\n
![\"1\"](\"https://i.ibb.co/QXZRSVb/10078-spleen-fold1-epoch1-predictions.png\")
\"The training dataset consists of data from public HPA data, the public test set is a combination of private HPA data and HuBMAP data, and the private test set contains only HuBMAP data.\"
\nIn this competition we have:
\nAnd finally, we have only one test image which provides a sneak peek into how different the test set is. Apparently, it seems that without external data, local validation is entirely impossible and we have to resort to something like predicting only HuMBAP images on the public LB as a sort of surrogate validation.
\nWhat do you think?
\nP.S.
\nOh, well. I'll probably go and do a logistic regression of prostate vs sex to get my dose of dopamine.
\"The training dataset consists of data from public HPA data, the public test set is a combination of private HPA data and HuBMAP data, and the private test set contains only HuBMAP data.\"
\nIn this competition we have:
\nAnd finally, we have only one test image which provides a sneak peek into how different the test set is. Apparently, it seems that without external data, local validation is entirely impossible and we have to resort to something like predicting only HuMBAP images on the public LB as a sort of surrogate validation.
\nWhat do you think?
\nP.S.
\nOh, well. I'll probably go and do a logistic regression of prostate vs sex to get my dose of dopamine.
refer to my discussion at: https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/333631
\nroughly 550 test images \nthere are exactly 529 test images, of which Hubmap=448, HPA=81 \n\npublic test : 55% of the test data() \n291 --> Hubmap=210 (0.7216), HPA=81 (0.2783) \n\nprivate test = 45% \n238--> Hubmap= 238 \nLocal validation is like comparing apples to oranges. It is really hard to do something significant only with competition data because of the reasons you mentioned. As my training progress further, predictions on test image become worse.
\nEpoch 1
\n
Epoch 20 (early stopping by validation set)
\n![\"20\"](\"https://i.ibb.co/x1SPmC7/10078-spleen-fold1-epoch20-predictions.png\")
Yes local validation is hard! I am doing many tests just to find out how to validate in a smart way.
\nCurrent best pipeline gives 0.54 on HuBMAP data alone ( HPA rle set to \"\") and I am predicting that this value holds the best PB value for my tests so far.
", "rawMarkdown": "Yes local validation is hard! I am doing many tests just to find out how to validate in a smart way. \n\nCurrent best pipeline gives 0.54 on HuBMAP data alone ( HPA rle set to \"\") and I am predicting that this value holds the best PB value for my tests so far.", "votes": null }, { "id": "1857883", "postDate": "07/16/2022 13:19:01", "content": "ultimately, you need to prepare external data. and you may need to submit for reach organ separately.
\nfortunately, we do have some samples from HuBMAP for 'kidney', 'largeintestine', 'spleen'.
\n'prostate' stains are common and will not be a problem. Hence for these 3+1 cases, we can have a good estimates of the private lb.
the only problem is 'lung' (problem because the results are no predictable)
", "rawMarkdown": "ultimately, you need to prepare external data. and you may need to submit for reach organ separately.\n\nfortunately, we do have some samples from HuBMAP for 'kidney', 'largeintestine', 'spleen'.\n'prostate' stains are common and will not be a problem. Hence for these 3+1 cases, we can have a good estimates of the private lb.\n\n\nthe only problem is 'lung' (problem because the results are no predictable)", "votes": null }, { "id": "1858625", "postDate": "07/17/2022 04:47:18", "content": "![\"https://i.ibb.co/dkxh7zs/Selection-061.png\"](\"https://i.ibb.co/dkxh7zs/Selection-061.png\")
the first step is to think of input domain space.
\nthink of hubmap and HPA images as data points. how to plot them on the input space?
\n(e.g. use probing, information like um per pixel, stain color given, examples on HPA/Humap website, train a classfiier to predict image characteristic)
then in the second step, for a subset in the input domain space, what is the dice score?
\nyou model should be stable loss landscape (constant low valley) in the target domain space.
", "rawMarkdown": "\n\nthe first step is to think of input domain space.\nthink of hubmap and HPA images as data points. how to plot them on the input space?\n(e.g. use probing, information like um per pixel, stain color given, examples on HPA/Humap website, train a classfiier to predict image characteristic)\n\nthen in the second step, for a subset in the input domain space, what is the dice score?\n\nyou model should be stable loss landscape (constant low valley) in the target domain space.", "votes": null }, { "id": "1858634", "postDate": "07/17/2022 05:00:24", "content": "aka: how to ensemble for domain shift
\n![\"https://i.ibb.co/wdsk5z5/Selection-068.png\"](\"https://i.ibb.co/wdsk5z5/Selection-068.png\")
Are you sure the white parts are spleen FTU?
", "rawMarkdown": "Are you sure the white parts are spleen FTU?", "votes": null }, { "id": "1861499", "postDate": "07/19/2022 04:36:15", "content": "The input data that we were been provided with, is probably created from their own segmentation algorithm, but this task should be done
\nby hand, with precision of human inteligence. Buuuuuuuuuuuuuuuuuuuuuuuuuut…………………
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2386017%2F3561dfe32246b2bda01e79a26ec23ce6%2F200w.gif?generation=1658205348620548&alt=media\")
I'm not sure. It was gut feeling.
", "rawMarkdown": "I'm not sure. It was gut feeling.", "votes": null }, { "id": "1861539", "postDate": "07/19/2022 05:06:05", "content": "example of using PCA/umap to map domain space:
\nhttps://github.com/j-sripad/colon_crypt_segmentation
in summary, divide your image into patches. use some feature extraction (can be trained CNN or simply PCA) methods to turn patch into feature vector. use tsne, umap etc for visualisation. some mapping can shown in the papers below
\nhttps://www.diva-portal.org/smash/get/diva2:1478702/FULLTEXT02.pdf
\nMeasuring Domain Shift for Deep Learning in Histopathology
I don't think white parts are FTUs. But how do we even know?
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3436753%2F64d8886fdafbd5214265f66ee7eb0550%2F__results___25_20.png?generation=1658208656712795&alt=media\")
I dont think so, it seems to be annotated by multiple people but with different ways to annotate.
\nFor example are some FTUs sometimes only segmented, if they are completly surrounded by tissiue.
https://www.proteinatlas.org/learn/dictionary/normal/spleen
\nYou are right. According to wikipedia, there are two tissues on spleen.
\nThe spleen contains two different tissues, white pulp (A) and red pulp (B). The white pulp functions in producing and growing immune and blood cells. The red pulp functions in filtering blood of antigens, microorganisms, and defective or worn-out red blood cells.
This seems consistent with labels.
", "rawMarkdown": "https://www.proteinatlas.org/learn/dictionary/normal/spleen\n\nYou are right. According to wikipedia, there are two tissues on spleen.\n\n`The spleen contains two different tissues, white pulp (A) and red pulp (B). The white pulp functions in producing and growing immune and blood cells. The red pulp functions in filtering blood of antigens, microorganisms, and defective or worn-out red blood cells.`\n\nThis seems consistent with labels.", "votes": null } ], "comments": [ { "id": 1857780, "author_name": "hengck23", "author_url": "", "post_date": "07/16/2022 11:32:35", "content": "refer to my discussion at: https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/333631
\nroughly 550 test images \nthere are exactly 529 test images, of which Hubmap=448, HPA=81 \n\npublic test : 55% of the test data() \n291 --> Hubmap=210 (0.7216), HPA=81 (0.2783) \n\nprivate test = 45% \n238--> Hubmap= 238 \nLocal validation is like comparing apples to oranges. It is really hard to do something significant only with competition data because of the reasons you mentioned. As my training progress further, predictions on test image become worse.
\nEpoch 1
\n
Epoch 20 (early stopping by validation set)
\n
Are you sure the white parts are spleen FTU?
", "votes": null, "replies": [] }, { "id": 1861522, "author_name": "gunesevitan", "author_url": "", "post_date": "07/19/2022 04:57:36", "content": "I'm not sure. It was gut feeling.
", "votes": null, "replies": [] }, { "id": 1861587, "author_name": "quandapro", "author_url": "", "post_date": "07/19/2022 05:31:30", "content": "I don't think white parts are FTUs. But how do we even know?
\n
https://www.proteinatlas.org/learn/dictionary/normal/spleen
\nYou are right. According to wikipedia, there are two tissues on spleen.
\nThe spleen contains two different tissues, white pulp (A) and red pulp (B). The white pulp functions in producing and growing immune and blood cells. The red pulp functions in filtering blood of antigens, microorganisms, and defective or worn-out red blood cells.
This seems consistent with labels.
", "votes": null, "replies": [] } ] }, { "id": 1857818, "author_name": "asalhi", "author_url": "", "post_date": "07/16/2022 12:35:42", "content": "Yes local validation is hard! I am doing many tests just to find out how to validate in a smart way.
\nCurrent best pipeline gives 0.54 on HuBMAP data alone ( HPA rle set to \"\") and I am predicting that this value holds the best PB value for my tests so far.
", "votes": null, "replies": [ { "id": 1857883, "author_name": "hengck23", "author_url": "", "post_date": "07/16/2022 13:19:01", "content": "ultimately, you need to prepare external data. and you may need to submit for reach organ separately.
\nfortunately, we do have some samples from HuBMAP for 'kidney', 'largeintestine', 'spleen'.
\n'prostate' stains are common and will not be a problem. Hence for these 3+1 cases, we can have a good estimates of the private lb.
the only problem is 'lung' (problem because the results are no predictable)
", "votes": null, "replies": [] } ] }, { "id": 1858625, "author_name": "hengck23", "author_url": "", "post_date": "07/17/2022 04:47:18", "content": "
the first step is to think of input domain space.
\nthink of hubmap and HPA images as data points. how to plot them on the input space?
\n(e.g. use probing, information like um per pixel, stain color given, examples on HPA/Humap website, train a classfiier to predict image characteristic)
then in the second step, for a subset in the input domain space, what is the dice score?
\nyou model should be stable loss landscape (constant low valley) in the target domain space.
", "votes": null, "replies": [ { "id": 1858634, "author_name": "hengck23", "author_url": "", "post_date": "07/17/2022 05:00:24", "content": "aka: how to ensemble for domain shift
\n
example of using PCA/umap to map domain space:
\nhttps://github.com/j-sripad/colon_crypt_segmentation
in summary, divide your image into patches. use some feature extraction (can be trained CNN or simply PCA) methods to turn patch into feature vector. use tsne, umap etc for visualisation. some mapping can shown in the papers below
\nhttps://www.diva-portal.org/smash/get/diva2:1478702/FULLTEXT02.pdf
\nMeasuring Domain Shift for Deep Learning in Histopathology
The input data that we were been provided with, is probably created from their own segmentation algorithm, but this task should be done
\nby hand, with precision of human inteligence. Buuuuuuuuuuuuuuuuuuuuuuuuuut…………………
I dont think so, it seems to be annotated by multiple people but with different ways to annotate.
\nFor example are some FTUs sometimes only segmented, if they are completly surrounded by tissiue.