![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F223160%2F7b3dfec8bed76e219f8735f5ab783c99%2Fscheme_1280.png?generation=1610718781393321&alt=media\")
I am using the annotations as a segmentation problem to inference the catheters. It is just a binary prediction, it does not classify the output into labels.
\nAnd I'm using the lung masks (thanks @raddar !!!) to have a segmentation position of the lungs.
\nThese segmentations are used in conjunction with xrays as input.
\nAt first, I tried to split this information using the three RGB channels (one channel per info layer) as the input image and fine-tune a previously trained resnet200d in Imagenet.
\nI also gave it a second try adding the segmentation information to the G (catheter) and B (lungs) channels. I thought it must be easily understandable by a Imagenet pre-trained network
\nBut neither of these approaches is giving me better results:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F223160%2Fe162963b7bf731fd7ca93cff199219fb%2Fresults.png?generation=1610718825446302&alt=media\")
I used the same kfold split for lungs sementation, catheter segmentation, and finally combined image classification.
\nThe segmentation approach seemed to overfitting a bit and didn't give me any improvement in public LB score.
\nI thought the resolution of the input image was so important to this image classification task, to allow the network to find the position of the catheters.
\nSince I am giving you the catheter masks as input, I thought high resolution was no longer important, and I tried other CNN architectures with lower resolutions and very high batch sizes (32 and 64) with no success.
Has anyone improved their results using image segmentation as input?
", "messages": [ { "id": 1154226, "postDate": "2021-01-15T13:56:46.730Z", "content": "
I am using the annotations as a segmentation problem to inference the catheters. It is just a binary prediction, it does not classify the output into labels.
\nAnd I'm using the lung masks (thanks @raddar !!!) to have a segmentation position of the lungs.
\nThese segmentations are used in conjunction with xrays as input.
\nAt first, I tried to split this information using the three RGB channels (one channel per info layer) as the input image and fine-tune a previously trained resnet200d in Imagenet.
\nI also gave it a second try adding the segmentation information to the G (catheter) and B (lungs) channels. I thought it must be easily understandable by a Imagenet pre-trained network
\nBut neither of these approaches is giving me better results:
\n
I used the same kfold split for lungs sementation, catheter segmentation, and finally combined image classification.
\nThe segmentation approach seemed to overfitting a bit and didn't give me any improvement in public LB score.
\nI thought the resolution of the input image was so important to this image classification task, to allow the network to find the position of the catheters.
\nSince I am giving you the catheter masks as input, I thought high resolution was no longer important, and I tried other CNN architectures with lower resolutions and very high batch sizes (32 and 64) with no success.
Has anyone improved their results using image segmentation as input?
", "rawMarkdown": "\n\nI am using the annotations as a segmentation problem to inference the catheters. It is just a binary prediction, it does not classify the output into labels.\n\nAnd I'm using the [lung masks](https://www.kaggle.com/raddar/ranzcr-clip-lung-contours) (thanks @raddar !!!) to have a segmentation position of the lungs.\n\nThese segmentations are used in conjunction with xrays as input.\n\nAt first, I tried to split this information using the three RGB channels (one channel per info layer) as the input image and fine-tune a previously trained resnet200d in Imagenet.\n\nI also gave it a second try adding the segmentation information to the G (catheter) and B (lungs) channels. I thought it must be easily understandable by a Imagenet pre-trained network\n\nBut neither of these approaches is giving me better results:\n\n\n\nI used the same kfold split for lungs sementation, catheter segmentation, and finally combined image classification.\n\nThe segmentation approach seemed to overfitting a bit and didn't give me any improvement in public LB score.\n\nI thought the resolution of the input image was so important to this image classification task, to allow the network to find the position of the catheters.\nSince I am giving you the catheter masks as input, I thought high resolution was no longer important, and I tried other CNN architectures with lower resolutions and very high batch sizes (32 and 64) with no success.\n\nHas anyone improved their results using image segmentation as input?", "votes": 35 }, { "id": 1155918, "postDate": "2021-01-16T19:17:46.427Z", "content": "I ve tried this too. It does not make so big diference as i thought at first. We must understand that the CNN models are Black Box models, meaning that we are totally unaware of knowing what actually the look into the image and what patterns too. For examble one letter drawn into the image maybe is a better pattern than actually the catheter. We think as what a human would look or a doctor, but the NN look at whatever it likes based on the dataset's instances. Here is where explainability matters in which we can vizualize on what areas of the images the models mainly look. I have tried some vizualizations using some post hoc explainability tools and saw that the models mainly look at the chest rather the catheter or the lungs.
", "rawMarkdown": "I ve tried this too. It does not make so big diference as i thought at first. We must understand that the CNN models are Black Box models, meaning that we are totally unaware of knowing what actually the look into the image and what patterns too. For examble one letter drawn into the image maybe is a better pattern than actually the catheter. We think as what a human would look or a doctor, but the NN look at whatever it likes based on the dataset's instances. Here is where explainability matters in which we can vizualize on what areas of the images the models mainly look. I have tried some vizualizations using some post hoc explainability tools and saw that the models mainly look at the chest rather the catheter or the lungs.", "votes": 8 }, { "id": 1154838, "postDate": "2021-01-16T01:36:00.293Z", "content": "The authors of this paper used similar segmentation (of line/catheter and anatomy) and found that random forest was better than DNNs for classification. Would be interested to hear if anyone has tried this out
\nAutomated Detection and Type Classification of
\n Central Venous Catheters in Chest X-rays
\nhttps://arxiv.org/pdf/1907.01656.pdf
Is there a reason you want to segment the lungs, particularly? Unfortunately, none of the lines or catheters terminate in the lungs, so I'm not sure how much valuable information it would add. ETTs are always between the clavicles, for instance, so I imagine segmenting them could help
", "rawMarkdown": "Is there a reason you want to segment the lungs, particularly? Unfortunately, none of the lines or catheters terminate in the lungs, so I'm not sure how much valuable information it would add. ETTs are always between the clavicles, for instance, so I imagine segmenting them could help", "votes": 1, "replies": [ { "id": 1156445, "postDate": "2021-01-17T06:52:46.647Z", "content": "I segmented the lung in an attempt to \"focus\" the CNN's attention to the catheters around/near the lungs. It doesn't quite work that way with NN as @manolispintelas pointed out, \"the NN looks at whatever it likes based on the dataset's instances\".
\nThe lungs segmentation is a way to provide normalized coordinates for downstream modelling task. For example, if combined with tip detection, the tip's coordinates can be normalized with respect to the lungs segmentation and then be provided to a classifier. (You can swap lung segmentation with clavicles segmentation in the example.) There are probably other use than this, but my mere mortal mind cannot think of any >.<
", "rawMarkdown": "I segmented the lung in an attempt to \"focus\" the CNN's attention to the catheters around/near the lungs. It doesn't quite work that way with NN as @manolispintelas pointed out, \"the NN looks at whatever it likes based on the dataset's instances\".\n\nThe lungs segmentation is a way to provide normalized coordinates for downstream modelling task. For example, if combined with tip detection, the tip's coordinates can be normalized with respect to the lungs segmentation and then be provided to a classifier. (You can swap lung segmentation with clavicles segmentation in the example.) There are probably other use than this, but my mere mortal mind cannot think of any >.<", "votes": 4 } ] }, { "id": 1155559, "postDate": "2021-01-16T13:13:55.610Z", "content": "Segmenting the image to concentrate on the focus areas is a good idea.
\nBut don't you think CNN based models themselves have quite the capability to filter out structural components(depending on the tasks on which they are trained) in an image. That's the whole reason why they are used in UNet.
I tried something similar using Efficientnet-b5 and Xception at 512x512. I also did not get much or any improvement to both my local CV and LB.
\nHowever, I did find the model converge/overfit much faster to the training set, so I may try adding more augmentation/regularization.
", "rawMarkdown": "I tried something similar using Efficientnet-b5 and Xception at 512x512. I also did not get much or any improvement to both my local CV and LB.\n\nHowever, I did find the model converge/overfit much faster to the training set, so I may try adding more augmentation/regularization.", "votes": 1 }, { "id": 1156940, "postDate": "2021-01-17T14:38:22.540Z", "content": "Even I was wondering to use the mask. Good you are attempting this approach.
", "rawMarkdown": "Even I was wondering to use the mask. Good you are attempting this approach. ", "votes": 2 }, { "id": 1154809, "postDate": "2021-01-15T23:32:50.107Z", "content": "Did you use thresholded masks (lung/tubes) or raw output of your segmentation models?
", "rawMarkdown": "Did you use thresholded masks (lung/tubes) or raw output of your segmentation models?", "votes": 2, "replies": [ { "id": 1154811, "postDate": "2021-01-15T23:48:22.167Z", "content": "My attempt was with raw segmentation output. CV 0.935 (with segmentation), 0.945 (without segmentation).
\nModel with thresholded mask is in the oven; seems to be progressing better.
\nInterested to see @virilo's approach.
UPDATE: model with thresholded mask turns out to be similar with local CV 0.937
", "rawMarkdown": "My attempt was with raw segmentation output. CV 0.935 (with segmentation), 0.945 (without segmentation). \n\nModel with thresholded mask is in the oven; seems to be progressing better. \nInterested to see @virilo's approach.\n\nUPDATE: model with thresholded mask turns out to be similar with local CV 0.937", "votes": 3 }, { "id": 1154833, "postDate": "2021-01-16T01:16:31.210Z", "content": "@raddar I used the raw output, without any threshold
", "rawMarkdown": "@raddar I used the raw output, without any threshold", "votes": 1 } ] }, { "id": 1224541, "postDate": "2021-03-02T20:39:51.803Z", "content": "\nI wouldn't like to discourage you from using this kind of schema.
\nI stopped using it at the same time I asked in this forum
\nBut I'm waiting the moment to have time to test it again. Because it should be possible to take advantage of these annotations. Even better adding hand-labels to the trainset.
\nMy intuition is that there must be a room of improvement here. Don't give up!
", "rawMarkdown": "@CBR and @all\n\nI wouldn't like to discourage you from using this kind of schema.\n\nI stopped using it at the same time I asked in this forum\n\nBut I'm waiting the moment to have time to test it again. Because it should be possible to take advantage of these annotations. Even better adding hand-labels to the trainset.\n\nMy intuition is that there must be a room of improvement here. Don't give up!" }, { "id": 1156268, "postDate": "2021-01-17T03:56:04.353Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1155918, "author_name": "", "author_url": "", "post_date": "2021-01-16T19:17:46.427000", "content": "I ve tried this too. It does not make so big diference as i thought at first. We must understand that the CNN models are Black Box models, meaning that we are totally unaware of knowing what actually the look into the image and what patterns too. For examble one letter drawn into the image maybe is a better pattern than actually the catheter. We think as what a human would look or a doctor, but the NN look at whatever it likes based on the dataset's instances. Here is where explainability matters in which we can vizualize on what areas of the images the models mainly look. I have tried some vizualizations using some post hoc explainability tools and saw that the models mainly look at the chest rather the catheter or the lungs.
", "votes": 8, "replies": [] }, { "id": 1154838, "author_name": "Reuben Schmidt", "author_url": "", "post_date": "2021-01-16T01:36:00.293000", "content": "The authors of this paper used similar segmentation (of line/catheter and anatomy) and found that random forest was better than DNNs for classification. Would be interested to hear if anyone has tried this out
\nAutomated Detection and Type Classification of
\n Central Venous Catheters in Chest X-rays
\nhttps://arxiv.org/pdf/1907.01656.pdf
Is there a reason you want to segment the lungs, particularly? Unfortunately, none of the lines or catheters terminate in the lungs, so I'm not sure how much valuable information it would add. ETTs are always between the clavicles, for instance, so I imagine segmenting them could help
", "votes": 1, "replies": [ { "id": 1156445, "author_name": "JunYong Tong", "author_url": "", "post_date": "2021-01-17T06:52:46.647000", "content": "I segmented the lung in an attempt to \"focus\" the CNN's attention to the catheters around/near the lungs. It doesn't quite work that way with NN as @manolispintelas pointed out, \"the NN looks at whatever it likes based on the dataset's instances\".
\nThe lungs segmentation is a way to provide normalized coordinates for downstream modelling task. For example, if combined with tip detection, the tip's coordinates can be normalized with respect to the lungs segmentation and then be provided to a classifier. (You can swap lung segmentation with clavicles segmentation in the example.) There are probably other use than this, but my mere mortal mind cannot think of any >.<
", "votes": 4, "replies": [] } ] }, { "id": 1155559, "author_name": "Izzy Adesanya", "author_url": "", "post_date": "2021-01-16T13:13:55.610000", "content": "Segmenting the image to concentrate on the focus areas is a good idea.
\nBut don't you think CNN based models themselves have quite the capability to filter out structural components(depending on the tasks on which they are trained) in an image. That's the whole reason why they are used in UNet.
I tried something similar using Efficientnet-b5 and Xception at 512x512. I also did not get much or any improvement to both my local CV and LB.
\nHowever, I did find the model converge/overfit much faster to the training set, so I may try adding more augmentation/regularization.
", "votes": 1, "replies": [] }, { "id": 1156940, "author_name": "CBR", "author_url": "", "post_date": "2021-01-17T14:38:22.540000", "content": "Even I was wondering to use the mask. Good you are attempting this approach.
", "votes": 2, "replies": [] }, { "id": 1154809, "author_name": "raddar", "author_url": "", "post_date": "2021-01-15T23:32:50.107000", "content": "Did you use thresholded masks (lung/tubes) or raw output of your segmentation models?
", "votes": 2, "replies": [ { "id": 1154811, "author_name": "JunYong Tong", "author_url": "", "post_date": "2021-01-15T23:48:22.167000", "content": "My attempt was with raw segmentation output. CV 0.935 (with segmentation), 0.945 (without segmentation).
\nModel with thresholded mask is in the oven; seems to be progressing better.
\nInterested to see @virilo's approach.
UPDATE: model with thresholded mask turns out to be similar with local CV 0.937
", "votes": 3, "replies": [] }, { "id": 1154833, "author_name": "Virilo Tejedor Aguilera", "author_url": "", "post_date": "2021-01-16T01:16:31.210000", "content": "@raddar I used the raw output, without any threshold
", "votes": 1, "replies": [] } ] }, { "id": 1224541, "author_name": "Virilo Tejedor Aguilera", "author_url": "", "post_date": "2021-03-02T20:39:51.803000", "content": "\nI wouldn't like to discourage you from using this kind of schema.
\nI stopped using it at the same time I asked in this forum
\nBut I'm waiting the moment to have time to test it again. Because it should be possible to take advantage of these annotations. Even better adding hand-labels to the trainset.
\nMy intuition is that there must be a room of improvement here. Don't give up!
", "votes": 0, "replies": [] }, { "id": 1156268, "author_name": "", "author_url": "", "post_date": "2021-01-17T03:56:04.353000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1154226": "\n\nI am using the annotations as a segmentation problem to inference the catheters. It is just a binary prediction, it does not classify the output into labels.\n\nAnd I'm using the [lung masks](https://www.kaggle.com/raddar/ranzcr-clip-lung-contours) (thanks @raddar !!!) to have a segmentation position of the lungs.\n\nThese segmentations are used in conjunction with xrays as input.\n\nAt first, I tried to split this information using the three RGB channels (one channel per info layer) as the input image and fine-tune a previously trained resnet200d in Imagenet.\n\nI also gave it a second try adding the segmentation information to the G (catheter) and B (lungs) channels. I thought it must be easily understandable by a Imagenet pre-trained network\n\nBut neither of these approaches is giving me better results:\n\n\n\nI used the same kfold split for lungs sementation, catheter segmentation, and finally combined image classification.\n\nThe segmentation approach seemed to overfitting a bit and didn't give me any improvement in public LB score.\n\nI thought the resolution of the input image was so important to this image classification task, to allow the network to find the position of the catheters.\nSince I am giving you the catheter masks as input, I thought high resolution was no longer important, and I tried other CNN architectures with lower resolutions and very high batch sizes (32 and 64) with no success.\n\nHas anyone improved their results using image segmentation as input?", "1155918": "I ve tried this too. It does not make so big diference as i thought at first. We must understand that the CNN models are Black Box models, meaning that we are totally unaware of knowing what actually the look into the image and what patterns too. For examble one letter drawn into the image maybe is a better pattern than actually the catheter. We think as what a human would look or a doctor, but the NN look at whatever it likes based on the dataset's instances. Here is where explainability matters in which we can vizualize on what areas of the images the models mainly look. I have tried some vizualizations using some post hoc explainability tools and saw that the models mainly look at the chest rather the catheter or the lungs.", "1154838": "The authors of this paper used similar segmentation (of line/catheter and anatomy) and found that random forest was better than DNNs for classification. Would be interested to hear if anyone has tried this out\n\nAutomated Detection and Type Classification of\r Central Venous Catheters in Chest X-rays\nhttps://arxiv.org/pdf/1907.01656.pdf\n", "1156267": "Is there a reason you want to segment the lungs, particularly? Unfortunately, none of the lines or catheters terminate in the lungs, so I'm not sure how much valuable information it would add. ETTs are always between the clavicles, for instance, so I imagine segmenting them could help", "1155559": "Segmenting the image to concentrate on the focus areas is a good idea. \nBut don't you think CNN based models themselves have quite the capability to filter out structural components(depending on the tasks on which they are trained) in an image. That's the whole reason why they are used in UNet.", "1154541": "I tried something similar using Efficientnet-b5 and Xception at 512x512. I also did not get much or any improvement to both my local CV and LB.\n\nHowever, I did find the model converge/overfit much faster to the training set, so I may try adding more augmentation/regularization.", "1156940": "Even I was wondering to use the mask. Good you are attempting this approach. ", "1154809": "Did you use thresholded masks (lung/tubes) or raw output of your segmentation models?", "1224541": "@CBR and @all\n\nI wouldn't like to discourage you from using this kind of schema.\n\nI stopped using it at the same time I asked in this forum\n\nBut I'm waiting the moment to have time to test it again. Because it should be possible to take advantage of these annotations. Even better adding hand-labels to the trainset.\n\nMy intuition is that there must be a room of improvement here. Don't give up!", "1156268": "" } }