![\"Lung\"](\"http://drkonya.com/projects/kaggle/lung3ds.jpg\")
\n3D Lung
Dear fellow Kagglers,
\nmy team & I have tried various experiments with newly derived features from Image data, like mentioned in Laura's Notebook: pulmonary-dicom-preprocessing.
\nSome of those features were:
\nOur current finding is, that those features sadly DO NOT improve CV/LB.
\nDo you have share the same experience & get similar results, or are any of those features working for you?
\nDid I miss important features?
To be more precise:
\nThe above mentioned features do NOT have any causal relationship with the actual medical condition or prognosis of the patients. It's basically a test if there is a correlation: we couldn't find any. Even if there is a correlation, this would only help in the competition, but obviously not in the real-world examples.
Using features mentioned by @aadhavvignesh and @abhishekgbhat might have a causal relationship with the actual prognosis and the FVC:
\nThank you all for adding some insights on fighting Pulmonary Fibrosis!
\n@tanulsingh077 feel free to add!
", "messages": [ { "id": "1017848", "postDate": "09/19/2020 09:44:48", "content": "Dear fellow Kagglers,
\nmy team & I have tried various experiments with newly derived features from Image data, like mentioned in Laura's Notebook: pulmonary-dicom-preprocessing.
\nSome of those features were:
\nOur current finding is, that those features sadly DO NOT improve CV/LB.
\nDo you have share the same experience & get similar results, or are any of those features working for you?
\nDid I miss important features?
To be more precise:
\nThe above mentioned features do NOT have any causal relationship with the actual medical condition or prognosis of the patients. It's basically a test if there is a correlation: we couldn't find any. Even if there is a correlation, this would only help in the competition, but obviously not in the real-world examples.
Using features mentioned by @aadhavvignesh and @abhishekgbhat might have a causal relationship with the actual prognosis and the FVC:
\nThank you all for adding some insights on fighting Pulmonary Fibrosis!
\n@tanulsingh077 feel free to add!
", "rawMarkdown": "Dear fellow Kagglers,\n\nmy team & I have tried various experiments with **newly derived features from Image data**, like mentioned in Laura's [Notebook: pulmonary-dicom-preprocessing.](https://www.kaggle.com/allunia/pulmonary-dicom-preprocessing)\n\nSome of those features were:\n\n- img resolution\n- slice-thickness\n- pixelspacing_r\n- pixelspacing_c\n- pixelspacing_area\n- window_width\n- image_area_cm2\n- slice_volume_cm3\n\nOur current finding is, that those features sadly DO NOT improve CV/LB.\n\nDo you have share the same experience & get similar results, or are any of those features working for you?\nDid I miss important features?\n\nTo be more precise:\nThe above mentioned features do NOT have any causal relationship with the actual medical condition or prognosis of the patients. It's basically a test if there is a correlation: we couldn't find any. Even if there is a correlation, this would only help in the competition, but obviously not in the real-world examples.\n\nUsing features mentioned by @aadhavvignesh and @abhishekgbhat **might** have a causal relationship with the actual prognosis and the FVC:\n- Lung Volume in cm^3\n- Lung Area in cm^2\n- Average of (Tissue area)/(Lung Area) across all images for a particular patient and FVC\n\n\nThank you all for adding some insights on fighting Pulmonary Fibrosis!\n\n@tanulsingh077 feel free to add!", "votes": null }, { "id": "1017994", "postDate": "09/19/2020 11:28:17", "content": "I have tried including the following features you mentioned:
\nBut I did observe some good correlation between:
\nYou can consider including some of the above features in your experiment.
\nAlso how do you define - image_area_cm2, slice_volume_cm3?
i agree these features didnt helped me to get a better CV/LB .\nbtw would you like to merge team with me as i dont have a team ,
", "rawMarkdown": "i agree these features didnt helped me to get a better CV/LB .\nbtw would you like to merge team with me as i dont have a team ,", "votes": null }, { "id": "1018277", "postDate": "09/19/2020 15:15:39", "content": "What I got to know from Image Data:
\nI had already mentioned that these features didn't work:
\nin cm^3in cm^2Segmented lung images didn't work as expected. Poor CV/LB still persists.
\nThis has led our team to put less focus on image data, and more on the available tabular data. I unfortunately can't reveal anything beyond this, because every team would love to keep their sauce 'secret' :P
\nAbout LB and rankings:
\nI personally feel that the LB is NOT a good indicator of your team's results. I guess that I can safely say that our team has a stable model (my instinct says so :P), and our rankings aren't good. A shake-up is imminent, so I do feel teams should not be looking at the LB scores, but indeed should have a good CV set up.
", "rawMarkdown": "**What I got to know from Image Data:**\n\nI had already mentioned that these features didn't work:\n\n- Lung Volume `in cm^3`\n- Lung Area `in cm^2`\n- Image Resolution/Dimensions\n\nSegmented lung images didn't work as expected. Poor CV/LB still persists. \n\nThis has led our team to put less focus on image data, and more on the available tabular data. I unfortunately can't reveal anything beyond this, because every team would love to keep their sauce 'secret' :P\n\n**About LB and rankings:**\n\nI personally feel that the LB is **NOT** a good indicator of your team's results. I guess that I can safely say that our team has a stable model (my instinct says so :P), and our rankings aren't good. A shake-up is imminent, so I do feel teams should not be looking at the LB scores, but indeed should have a good CV set up.", "votes": null }, { "id": "1018303", "postDate": "09/19/2020 15:32:43", "content": "it doesn't help for CV/LB.
", "rawMarkdown": "it doesn't help for CV/LB.", "votes": null }, { "id": "1018351", "postDate": "09/19/2020 16:18:50", "content": "hey adhav does your team needs a member ? , i am currently seraching for a team , my current rank is 26
", "rawMarkdown": "hey adhav does your team needs a member ? , i am currently seraching for a team , my current rank is 26", "votes": null }, { "id": "1018991", "postDate": "09/20/2020 06:02:46", "content": "Tried image resolution , augmentation while training , DenseNet and Efficientnet ensembling but all in vain
", "rawMarkdown": "Tried image resolution , augmentation while training , DenseNet and Efficientnet ensembling but all in vain", "votes": null }, { "id": "1019284", "postDate": "09/20/2020 10:28:07", "content": "Thanks for sharing your findings. Even if they improve cv score, I wouldn't use them anyway. Since FVC is not dependent to them, it would be huge gamble to use them in your final submissions.
", "rawMarkdown": "Thanks for sharing your findings. Even if they improve cv score, I wouldn't use them anyway. Since FVC is not dependent to them, it would be huge gamble to use them in your final submissions.", "votes": null }, { "id": "1019757", "postDate": "09/20/2020 16:45:34", "content": "These features contain diagnostic information for the CT scanner setup, any correlation they have with lung function will be completely incidental anyway.
", "rawMarkdown": "These features contain diagnostic information for the CT scanner setup, any correlation they have with lung function will be completely incidental anyway.", "votes": null }, { "id": "1020565", "postDate": "09/21/2020 09:07:22", "content": "The holdout predictions for private LB is 85% where as public LB is scored on 15% of the predictions, although features related pixel spacing, slice thickness are useless, others like lung volume, lung are useful.
\nWait for a huge LB shakeup once private LB is announced.
", "rawMarkdown": "The holdout predictions for private LB is 85% where as public LB is scored on 15% of the predictions, although features related pixel spacing, slice thickness are useless, others like lung volume, lung are useful. \n\nWait for a huge LB shakeup once private LB is announced.", "votes": null }, { "id": "1022098", "postDate": "09/22/2020 10:07:01", "content": "Hi @abhishekgbhat, I just saw your post yesterday night, sorry for the delay.
\nThe feautes are derived from Laura's Notebook pulmonary-dicom-preprocessing. E.g. img_area_cm2 is derived by the resolution.
How do you derive \"Average of (Tissue area)/(Lung Area)\"?
\nIs there a notebook available? Sounds like a lot of work using Hounsfield values & segmentation.
Thank you for the feedback, that supports our findings.
", "rawMarkdown": "Thank you for the feedback, that supports our findings.", "votes": null }, { "id": "1022140", "postDate": "09/22/2020 10:37:55", "content": "I'm really sorry @subzeroop, our team is full. I hope you can find a team soon :)
", "rawMarkdown": "I'm really sorry @subzeroop, our team is full. I hope you can find a team soon :)", "votes": null }, { "id": "1022812", "postDate": "09/22/2020 18:25:35", "content": "Hey @ChristianDenich, thanks for the clarification.
\n\n\nHow do you derive \"Average of (Tissue area)/(Lung Area)\"?
\n
No, I haven't seen any notebook with this feature. From the limited research I have done, the fiberosis tissue plays an important role in the severity of the disease. So I found out a way to segment the tissues within the lung. From this segmented image I obtained the number of tissue pixels and multiplied it with Pixel Spacing R and C. This gives the tissue area in each image. Assuming you have the lung area for each image, you can get this ratio for each image. Finally, if you have this ratio for all images of a particular patient you can simply take the average. The tough part is to segment the tissues effectively.
", "rawMarkdown": "Hey @ChristianDenich, thanks for the clarification. \n\n> How do you derive \"Average of (Tissue area)/(Lung Area)\"?\n\nNo, I haven't seen any notebook with this feature. From the limited research I have done, the fiberosis tissue plays an important role in the severity of the disease. So I found out a way to segment the tissues within the lung. From this segmented image I obtained the number of tissue pixels and multiplied it with Pixel Spacing R and C. This gives the tissue area in each image. Assuming you have the lung area for each image, you can get this ratio for each image. Finally, if you have this ratio for all images of a particular patient you can simply take the average. The tough part is to segment the tissues effectively.", "votes": null }, { "id": "1023338", "postDate": "09/23/2020 06:25:40", "content": "So from a maths point of view, this should be the same as considering (Tissue volume)/(Lung volume) I guess.
\nI considered this feature and thought it gave me a slight improvement, but after changing some hyperparameters etc. I tried again without the feature and got a higher score than with the feature.
\nMaybe it works better with a better segmentation technique though. I just used the ranges of the HU scale for segmentation.
", "rawMarkdown": "So from a maths point of view, this should be the same as considering (Tissue volume)/(Lung volume) I guess. \n\nI considered this feature and thought it gave me a slight improvement, but after changing some hyperparameters etc. I tried again without the feature and got a higher score than with the feature. \n\nMaybe it works better with a better segmentation technique though. I just used the ranges of the HU scale for segmentation.", "votes": null }, { "id": "1024017", "postDate": "09/23/2020 15:35:13", "content": "This is not exactly the volume because im not considering the third dimension(i.e slice thickness) here. This feature tells us the following: For a particular patient, on an average what percent of the lung area does the tissue take up
", "rawMarkdown": "This is not exactly the volume because im not considering the third dimension(i.e slice thickness) here. This feature tells us the following: For a particular patient, on an average what percent of the lung area does the tissue take up", "votes": null }, { "id": "1024030", "postDate": "09/23/2020 15:46:04", "content": "Yes, but to get the volume of lung, you would have to multiply the average lung area by the length of the lung (3rd dimension). Analoguously, you get the volume of tissue, when you multiply the average tissue area by the length of the lung. When you divide both things, the length cancels out, such that
\nvolume of lung / volume of tissue = average area of lung / average area of tissue.
\nDo you agree?
We tried:
\nCenterCrop, Resampled to Pixel Spacing and SliceThickness equal to 1, Mask the Lung and added the following
\nPixel Voxel Statistics: Mean, STD, Kurtosis & Skew Distribution
\nPercentage in Lung Window
\nChest Circunference
\nLung Height
\nLung Volume in cm3 or liters (dm3)
\nHU Bins Distribution from 2 to 14 bins ( Just adding the HU distribution in bins ).
Any addition or combination of those items just made it worse in the LB, sometimes better CV but always worse LB.
", "rawMarkdown": "We tried:\n\nCenterCrop, Resampled to Pixel Spacing and SliceThickness equal to 1, Mask the Lung and added the following\n\nPixel Voxel Statistics: Mean, STD, Kurtosis & Skew Distribution\nPercentage in Lung Window\nChest Circunference\nLung Height\nLung Volume in cm3 or liters (dm3)\nHU Bins Distribution from 2 to 14 bins ( Just adding the HU distribution in bins ).\n\nAny addition or combination of those items just made it worse in the LB, sometimes better CV but always worse LB.", "votes": null }, { "id": "1024639", "postDate": "09/24/2020 03:33:35", "content": "I agree with this. We've tried all possible combinations and on integrating them with the model gave us worse LB and worse CV in almost every case.
\nDoes your team know what exactly the problem is, because at this point making use of scans seems pointless. (not meant to be conclusive, but including them is making CV/LB worse)
\nDo you also think that the winners' solution would help the hosts if they don't use image data for the final model, as I guess the primary objective of the hosts would be to make use of the scans for determining pulmonary fibrosis?
", "rawMarkdown": "I agree with this. We've tried all possible combinations and on integrating them with the model gave us worse LB and worse CV in almost every case.\n\nDoes your team know what exactly the problem is, because at this point making use of scans seems pointless. (not meant to be conclusive, but including them is making CV/LB worse)\n\nDo you also think that the winners' solution would help the hosts if they don't use image data for the final model, as I guess the primary objective of the hosts would be to make use of the scans for determining pulmonary fibrosis?", "votes": null }, { "id": "1024676", "postDate": "09/24/2020 04:15:49", "content": "We are not sure what is wrong, but we think that the release of just the baseline week of the images and also the test tabular data was a fundamental flaw in this competition. We are supposed to predict 146 weeks - time series data - and we don't have a time series data to train on.
\nMedical data is already extremely heterogenous, it seems we have different distributions of patients in the training set ( this notebook by @gannadolinska shows we can consider at least 3 different distributions of patients, those which decrease lung capacity, those steady and those which get better. )
\nThe tabular data has the percent feature, another controversial addition given that is highly correlated with the target, and we also don't have it in the test data.
\nIn an ideal case the image in the baseline would have enough information to guide the model to the different distributions of patients and the evolution of them but we failed to extract any
\nWe pretty much gave up using the images entirely, it was a very frustrating couple of months.
\nEither someone has a bag of tricks no one else has implemented using the images or the organizers are going to pay for a baseline tabular model, overfitted and completely useless medically.
Yes, we had initially planned to extract some useful features using image data, and I even saw some teams trying to manually honeycomb lungs and trying to make it work. Eventually, everyone who had used images became frustrated due to poor results.
\nI too fear that the organizers are going to end up with a baseline tabular model, rendering the objective of the competition to be useless.
\nHope we get to see a novel approach from the winners, but for now I guess most of the LB is saturated with overfitting models.
", "rawMarkdown": "Yes, we had initially planned to extract some useful features using image data, and I even saw some teams trying to manually honeycomb lungs and trying to make it work. Eventually, everyone who had used images became frustrated due to poor results.\n\nI too fear that the organizers are going to end up with a baseline tabular model, rendering the objective of the competition to be useless.\n\nHope we get to see a novel approach from the winners, but for now I guess most of the LB is saturated with overfitting models.", "votes": null }, { "id": "1025632", "postDate": "09/24/2020 17:21:10", "content": "Hi, I've been noticing too that the images are frustratingly not helping the public LB position but I'm going to keep focussing on them anyway. My hope is that the CT scans must have some value and that a big shakeup will occur where those models that make good use of the scans perform better on the private LB. I'm not looking at the technial aspects of the ct-scan (like pixel size, slice thickness) but I'm confident that the shapes, areas and textures of the slices themselves can be useful.
\nI must admit though that this hope is more an ensemble of sunk-cost fallacy + dumb-optimism + \"what is point of this competition if not\"…. :)
", "rawMarkdown": "Hi, I've been noticing too that the images are frustratingly not helping the public LB position but I'm going to keep focussing on them anyway. My hope is that the CT scans must have some value and that a big shakeup will occur where those models that make good use of the scans perform better on the private LB. I'm not looking at the technial aspects of the ct-scan (like pixel size, slice thickness) but I'm confident that the shapes, areas and textures of the slices themselves can be useful.\n\nI must admit though that this hope is more an ensemble of sunk-cost fallacy + dumb-optimism + \"what is point of this competition if not\".... :)", "votes": null }, { "id": "1025705", "postDate": "09/24/2020 18:07:31", "content": "I understand the logic though: If a Radiologist can see and characterize the prognosis of decline by just looking into a baseline CT, then a properly designed model would be able to do it too because the information is there. Either we failed to extract it properly or the images are problematic and is very difficult to do it or the info is not really there.
", "rawMarkdown": "I understand the logic though: If a Radiologist can see and characterize the prognosis of decline by just looking into a baseline CT, then a properly designed model would be able to do it too because the information is there. Either we failed to extract it properly or the images are problematic and is very difficult to do it or the info is not really there.", "votes": null }, { "id": "1025706", "postDate": "09/24/2020 18:08:22", "content": "I've insisted on the previous 2 months with the same hope…
", "rawMarkdown": "I've insisted on the previous 2 months with the same hope...", "votes": null }, { "id": "1025914", "postDate": "09/24/2020 21:46:13", "content": "In general, my impression is that this task is quite specific.
\nFirst of all, fighting for picture technical characteristics (pixels, slices etc) has no correlation with actual image content. And really cool that the experiment showed that no correlation has been found. If some correlation would have been found, I would bet on some bias in the data sample, which one definitely should not rely on as there would be no guarantee that the test data contains the same bias.
But what actually resulted the features like Lung Volume and Lung Area?
\nTo be honest, I think the task is quite complicated and has many internal interdependencies (and for that too few data ;) ).
\nAs I have mentioned before, there are clients with different dynamics of the disease --> not a good idea to mix them up in creating a model.
\nMoreover, I see even deeper patterns in the data. There is even more variety in the dynamics of the disease caused by patient individual characteristics. Like two clients might be recovering, but one much faster than the other, because this patient has better \"input\" - higher lung volume and non-smoker and a female. Thus, I would first try to split the data to the meaningful groups with \"similar\" dynamics of the FVC inside and let image \"predict\" or \"provide\" one or several grouping criteria.
As long as the final model will be kept simple (linear model, for example), one still has a lot of space to reduce the amount of data which is enough for training a model. This will allow to create more homogenous groups and train the models in these groups.
", "rawMarkdown": "In general, my impression is that this task is quite specific.\nFirst of all, fighting for picture technical characteristics (pixels, slices etc) has no correlation with actual image content. And really cool that the experiment showed that no correlation has been found. If some correlation would have been found, I would bet on some bias in the data sample, which one definitely should not rely on as there would be no guarantee that the test data contains the same bias.\n\nBut what actually resulted the features like Lung Volume and Lung Area?\nTo be honest, I think the task is quite complicated and has many internal interdependencies (and for that too few data ;) ).\nAs I have mentioned before, there are clients with different dynamics of the disease --> not a good idea to mix them up in creating a model.\nMoreover, I see even deeper patterns in the data. There is even more variety in the dynamics of the disease caused by patient individual characteristics. Like two clients might be recovering, but one much faster than the other, because this patient has better \"input\" - higher lung volume and non-smoker and a female. Thus, I would first try to split the data to the meaningful groups with \"similar\" dynamics of the FVC inside and let image \"predict\" or \"provide\" one or several grouping criteria.\n\nAs long as the final model will be kept simple (linear model, for example), one still has a lot of space to reduce the amount of data which is enough for training a model. This will allow to create more homogenous groups and train the models in these groups.", "votes": null }, { "id": "1025934", "postDate": "09/24/2020 22:25:54", "content": "Hello Hannah, thanks for extending the discussion.
\nI didn't think of spliting the data by the FVC pattern because I thought I would be leaking the target information. But I guess it's like a feature engineering if we add this information just for the training set and not for the validation set. Did I get that right?
\nMy doubt was what to do for the test set, but rereading the ending insights of your notebook it's clear what you mean.
\nI'll inform if I had any promising results.
", "rawMarkdown": "Hello Hannah, thanks for extending the discussion.\n\nI didn't think of spliting the data by the FVC pattern because I thought I would be leaking the target information. But I guess it's like a feature engineering if we add this information just for the training set and not for the validation set. Did I get that right?\n\nMy doubt was what to do for the test set, but rereading the ending insights of your notebook it's clear what you mean.\n\nI'll inform if I had any promising results.", "votes": null }, { "id": "1026011", "postDate": "09/25/2020 01:34:46", "content": "I agree in principle that different clusters of patients could be better analysed by different models, but my concern is that there simply isn't enough data to make generalisation possible in that case. We really could have done with more samples in this dataset…
", "rawMarkdown": "I agree in principle that different clusters of patients could be better analysed by different models, but my concern is that there simply isn't enough data to make generalisation possible in that case. We really could have done with more samples in this dataset...", "votes": null }, { "id": "1026063", "postDate": "09/25/2020 03:58:43", "content": "@cascadenite The image features didn't work out really well (if you're thinking it would), as our team did integrate it with the model, but it resulted in poor CV, which I guess could be the indicator that the scans won't work out really good on the private LB.
\nEven though the competition's emphasis would be make use of the scans, but technically at this point, I'm not that confident that scans would be useful.
\nI might be wrong, but currently it feels that scans do not add up much value in improving LB/CV.
", "rawMarkdown": "cascadenite The image features didn't work out really well (if you're thinking it would), as our team did integrate it with the model, but it resulted in poor CV, which I guess could be the indicator that the scans won't work out really good on the private LB.\n\nEven though the competition's emphasis would be make use of the scans, but technically at this point, I'm not that confident that scans would be useful.\n\nI might be wrong, but currently it feels that scans do not add up much value in improving LB/CV.", "votes": null }, { "id": "1026285", "postDate": "09/25/2020 07:41:24", "content": "Yes, I agree. One might have also taken it as a feature engineering and adding the info to the targeting set. However, we have kind of not that much data to my taste to train a model which will be able to catch all the patterns itself, I fear. That is why I was more thinking of an approach where I overtake some tasks of the model :)
", "rawMarkdown": "Yes, I agree. One might have also taken it as a feature engineering and adding the info to the targeting set. However, we have kind of not that much data to my taste to train a model which will be able to catch all the patterns itself, I fear. That is why I was more thinking of an approach where I overtake some tasks of the model :)", "votes": null }, { "id": "1026579", "postDate": "09/25/2020 12:28:43", "content": "Apologies for the delayed response. I'm not convinced with your definition of lung volume, please do correct me if you meant something else. I'm going to write down mathematically what i have understood from your explaination about lung volume.
\nLet \\( I_1, … I_n \\) denote the CT scan images for a patient \\(P\\) and \\(A_1,…,A_n\\) denote the corresponding area for segmented lung in each of the images. Let \\(l\\) denote the slice thickness for each image, then the approx. length of the lung can be obtained as \\(L = l*n\\).
\nYour Definition of lung volume:
\n\\(Volume = (A_1+…+A_n)/n * L\\)
Ideal definition of lung volume:
\n\\(Volume = {A_1}*l +…+{A_n} * l = (A_1+…+A_n)*l \\)
Both the definitions are different. Does this make sense?
", "rawMarkdown": "Apologies for the delayed response. I'm not convinced with your definition of lung volume, please do correct me if you meant something else. I'm going to write down mathematically what i have understood from your explaination about lung volume.\n\nLet \\\\( I_1, ... I_n \\\\) denote the CT scan images for a patient \\\\(P\\\\) and \\\\(A_1,...,A_n\\\\) denote the corresponding area for segmented lung in each of the images. Let \\\\(l\\\\) denote the slice thickness for each image, then the approx. length of the lung can be obtained as \\\\(L = l*n\\\\).\n\n- Your Definition of lung volume:\n\\\\(Volume = (A_1+...+A_n)/n * L\\\\)\n\n- Ideal definition of lung volume:\n\\\\(Volume = {A_1}*l +...+{A_n} * l = (A_1+...+A_n)*l \\\\)\n\nBoth the definitions are different. Does this make sense?", "votes": null }, { "id": "1026596", "postDate": "09/25/2020 12:42:52", "content": "The two definitions you wrote down are identical?
\n$$ (A_1+\\ldots + A_n)/n * L = (A_1+\\ldots + A_n)/n * \\ell * n = (A_1 + \\ldots + A_n) * \\ell $$
Thank you all for your contributions, that's exactly the technical discussion I was looking for!
\nIn summary the winning model will probably be a not-too-complex Tab-Data model, which is only using some additional features, but from Images probably only a tissue-amount/lung-volume value.
", "rawMarkdown": "Thank you all for your contributions, that's exactly the technical discussion I was looking for!\n\nIn summary the winning model will probably be a not-too-complex Tab-Data model, which is only using some additional features, but from Images probably only a tissue-amount/lung-volume value.", "votes": null }, { "id": "1026665", "postDate": "09/25/2020 13:31:40", "content": "@tobiit, thanks a ton! I completely missed that, my bad. Now I completely agree with:
\n\n\nvolume of lung / volume of tissue = average area of lung / average area of tissue
\n
I think the quality tissue segmentation is the key here.
", "rawMarkdown": "tobiit, thanks a ton! I completely missed that, my bad. Now I completely agree with:\n\n> volume of lung / volume of tissue = average area of lung / average area of tissue\n\nI think the quality tissue segmentation is the key here.", "votes": null }, { "id": "1030375", "postDate": "09/28/2020 16:07:52", "content": "What @ronaldokun said about the logic makes sense to me. But to my understanding, part of the problem is precisely because clinicians are not able to offer a reasonably certain prognosis even with imaging.
", "rawMarkdown": "What @ronaldokun said about the logic makes sense to me. But to my understanding, part of the problem is precisely because clinicians are not able to offer a reasonably certain prognosis even with imaging.", "votes": null }, { "id": "1030456", "postDate": "09/28/2020 17:19:47", "content": "Yes, I think the whole point of this competition is to fill this gap. It seems a lot harder for this problem judging by the results so far.
", "rawMarkdown": "Yes, I think the whole point of this competition is to fill this gap. It seems a lot harder for this problem judging by the results so far.", "votes": null }, { "id": "1031748", "postDate": "09/29/2020 16:58:11", "content": "@abhishekgbhat CT report provided to us , has got many slices. Are they different CT reports or they part of Single CT scan report only ?
", "rawMarkdown": "abhishekgbhat CT report provided to us , has got many slices. Are they different CT reports or they part of Single CT scan report only ?", "votes": null }, { "id": "1031836", "postDate": "09/29/2020 17:55:42", "content": "@ronaldokun to your point of Percent feature.
\nWhen we use it for training and hence prediction we get better lb inspite of fact that it means for all patient visits we are saying patients FVC is same of expected FVC for given segment of patient,please correct me if m wrong in contrary when we try to correct this by not including FVC then we get,worse lb
.
\nIn short is your current score is making use of Percent in Train ?
Hello @jaideepvalani
\nYes, you are absolutely right. Our current best score makes use of percent but by the point made above, IMHO it seems conceptually wrong.
\nBut yes, we are using the percent the same way as yours in our submission because it gave way better scores in CV and LB but we are probably overfitting a lot to the Public LB.
", "rawMarkdown": "Hello @jaideepvalani \n\nYes, you are absolutely right. Our current best score makes use of percent but by the point made above, IMHO it seems conceptually wrong.\n\nBut yes, we are using the percent the same way as yours in our submission because it gave way better scores in CV and LB but we are probably overfitting a lot to the Public LB.", "votes": null }, { "id": "1032160", "postDate": "09/30/2020 02:27:39", "content": "@jaideepvalani CT scan for a particular patient is a collection of slices and all these belong to the same report. So to answer your question, for each patient we have just one CT report with many slices.
", "rawMarkdown": "jaideepvalani CT scan for a particular patient is a collection of slices and all these belong to the same report. So to answer your question, for each patient we have just one CT report with many slices.", "votes": null }, { "id": "1036519", "postDate": "10/03/2020 21:57:17", "content": "Hi everyone,
\nwe use image-derived features, and it helped to get +0.015 in LB above the tabular results.
\nI manually segmented 100+ CT scans for the lung itself, and it delivers perfect results in lung segmentation.
\nAlso made ~ 40k markups for segmentation of honeycombing, reticulation, GGO/GGR but the heterogenity of the dataset… well… kind-of makes the work pointless.
\nThe unets are generalising pretty poorly on tissue features like reticulation or GGO, but honeycombing is not bad.
\n3D Lung
I hope that after the competition your work on the manual segmentation doesn't go to waste. I'm sure there'll be other applications that would really benefit - deserves more than + 0.015
", "rawMarkdown": "I hope that after the competition your work on the manual segmentation doesn't go to waste. I'm sure there'll be other applications that would really benefit - deserves more than + 0.015", "votes": null }, { "id": "1037089", "postDate": "10/04/2020 15:44:02", "content": "@sandorkonya even if do lung seg for all the patients ct report,average volume of all the lungs should stand same,every one has got same lungs structure except that the tissues in it could be varrying but we lose that while we do segmentation,so is segmentation still useful ?
", "rawMarkdown": "sandorkonya even if do lung seg for all the patients ct report,average volume of all the lungs should stand same,every one has got same lungs structure except that the tissues in it could be varrying but we lose that while we do segmentation,so is segmentation still useful ?", "votes": null }, { "id": "1037278", "postDate": "10/04/2020 19:13:55", "content": "@jameschapman19 it won't go waste… i can give back something to the community =)
", "rawMarkdown": "jameschapman19 it won't go waste... i can give back something to the community =)", "votes": null }, { "id": "1037281", "postDate": "10/04/2020 19:18:28", "content": "@jaideepvalani
\nthe average lung volume is not the same! It depends from the constitution (height, BMI, sex) of the patient… and it also depends on how far the fibrosis is developed. The fibrosis itself makes a volumen depletion of the lung volume (makes it to shrink).
\nWhy would we lose the \"tissue of the lung\" while we segment?
@sandorkonya hope you get paid off by your domain expertise..
\nHow can we determine height from ct … any one trying to do
I have tried including the following features you mentioned:
\nBut I did observe some good correlation between:
\nYou can consider including some of the above features in your experiment.
\nAlso how do you define - image_area_cm2, slice_volume_cm3?
Hi @abhishekgbhat, I just saw your post yesterday night, sorry for the delay.
\nThe feautes are derived from Laura's Notebook pulmonary-dicom-preprocessing. E.g. img_area_cm2 is derived by the resolution.
How do you derive \"Average of (Tissue area)/(Lung Area)\"?
\nIs there a notebook available? Sounds like a lot of work using Hounsfield values & segmentation.
Hey @ChristianDenich, thanks for the clarification.
\n\n\nHow do you derive \"Average of (Tissue area)/(Lung Area)\"?
\n
No, I haven't seen any notebook with this feature. From the limited research I have done, the fiberosis tissue plays an important role in the severity of the disease. So I found out a way to segment the tissues within the lung. From this segmented image I obtained the number of tissue pixels and multiplied it with Pixel Spacing R and C. This gives the tissue area in each image. Assuming you have the lung area for each image, you can get this ratio for each image. Finally, if you have this ratio for all images of a particular patient you can simply take the average. The tough part is to segment the tissues effectively.
", "votes": null, "replies": [] }, { "id": 1023338, "author_name": "tobiit", "author_url": "", "post_date": "09/23/2020 06:25:40", "content": "So from a maths point of view, this should be the same as considering (Tissue volume)/(Lung volume) I guess.
\nI considered this feature and thought it gave me a slight improvement, but after changing some hyperparameters etc. I tried again without the feature and got a higher score than with the feature.
\nMaybe it works better with a better segmentation technique though. I just used the ranges of the HU scale for segmentation.
", "votes": null, "replies": [] }, { "id": 1024017, "author_name": "abhishekgbhat", "author_url": "", "post_date": "09/23/2020 15:35:13", "content": "This is not exactly the volume because im not considering the third dimension(i.e slice thickness) here. This feature tells us the following: For a particular patient, on an average what percent of the lung area does the tissue take up
", "votes": null, "replies": [] }, { "id": 1024030, "author_name": "tobiit", "author_url": "", "post_date": "09/23/2020 15:46:04", "content": "Yes, but to get the volume of lung, you would have to multiply the average lung area by the length of the lung (3rd dimension). Analoguously, you get the volume of tissue, when you multiply the average tissue area by the length of the lung. When you divide both things, the length cancels out, such that
\nvolume of lung / volume of tissue = average area of lung / average area of tissue.
\nDo you agree?
Apologies for the delayed response. I'm not convinced with your definition of lung volume, please do correct me if you meant something else. I'm going to write down mathematically what i have understood from your explaination about lung volume.
\nLet \\( I_1, … I_n \\) denote the CT scan images for a patient \\(P\\) and \\(A_1,…,A_n\\) denote the corresponding area for segmented lung in each of the images. Let \\(l\\) denote the slice thickness for each image, then the approx. length of the lung can be obtained as \\(L = l*n\\).
\nYour Definition of lung volume:
\n\\(Volume = (A_1+…+A_n)/n * L\\)
Ideal definition of lung volume:
\n\\(Volume = {A_1}*l +…+{A_n} * l = (A_1+…+A_n)*l \\)
Both the definitions are different. Does this make sense?
", "votes": null, "replies": [] }, { "id": 1026596, "author_name": "tobiit", "author_url": "", "post_date": "09/25/2020 12:42:52", "content": "The two definitions you wrote down are identical?
\n$$ (A_1+\\ldots + A_n)/n * L = (A_1+\\ldots + A_n)/n * \\ell * n = (A_1 + \\ldots + A_n) * \\ell $$
@tobiit, thanks a ton! I completely missed that, my bad. Now I completely agree with:
\n\n\nvolume of lung / volume of tissue = average area of lung / average area of tissue
\n
I think the quality tissue segmentation is the key here.
", "votes": null, "replies": [] }, { "id": 1031748, "author_name": "jaideepvalani", "author_url": "", "post_date": "09/29/2020 16:58:11", "content": "@abhishekgbhat CT report provided to us , has got many slices. Are they different CT reports or they part of Single CT scan report only ?
", "votes": null, "replies": [] }, { "id": 1032160, "author_name": "abhishekgbhat", "author_url": "", "post_date": "09/30/2020 02:27:39", "content": "@jaideepvalani CT scan for a particular patient is a collection of slices and all these belong to the same report. So to answer your question, for each patient we have just one CT report with many slices.
", "votes": null, "replies": [] } ] }, { "id": 1018277, "author_name": "aadhavvignesh", "author_url": "", "post_date": "09/19/2020 15:15:39", "content": "What I got to know from Image Data:
\nI had already mentioned that these features didn't work:
\nin cm^3in cm^2Segmented lung images didn't work as expected. Poor CV/LB still persists.
\nThis has led our team to put less focus on image data, and more on the available tabular data. I unfortunately can't reveal anything beyond this, because every team would love to keep their sauce 'secret' :P
\nAbout LB and rankings:
\nI personally feel that the LB is NOT a good indicator of your team's results. I guess that I can safely say that our team has a stable model (my instinct says so :P), and our rankings aren't good. A shake-up is imminent, so I do feel teams should not be looking at the LB scores, but indeed should have a good CV set up.
", "votes": null, "replies": [ { "id": 1018351, "author_name": "subzeroop", "author_url": "", "post_date": "09/19/2020 16:18:50", "content": "hey adhav does your team needs a member ? , i am currently seraching for a team , my current rank is 26
", "votes": null, "replies": [] }, { "id": 1022140, "author_name": "aadhavvignesh", "author_url": "", "post_date": "09/22/2020 10:37:55", "content": "I'm really sorry @subzeroop, our team is full. I hope you can find a team soon :)
", "votes": null, "replies": [] }, { "id": 1025632, "author_name": "cascadenite", "author_url": "", "post_date": "09/24/2020 17:21:10", "content": "Hi, I've been noticing too that the images are frustratingly not helping the public LB position but I'm going to keep focussing on them anyway. My hope is that the CT scans must have some value and that a big shakeup will occur where those models that make good use of the scans perform better on the private LB. I'm not looking at the technial aspects of the ct-scan (like pixel size, slice thickness) but I'm confident that the shapes, areas and textures of the slices themselves can be useful.
\nI must admit though that this hope is more an ensemble of sunk-cost fallacy + dumb-optimism + \"what is point of this competition if not\"…. :)
", "votes": null, "replies": [] }, { "id": 1025706, "author_name": "ronaldokun", "author_url": "", "post_date": "09/24/2020 18:08:22", "content": "I've insisted on the previous 2 months with the same hope…
", "votes": null, "replies": [] }, { "id": 1026063, "author_name": "aadhavvignesh", "author_url": "", "post_date": "09/25/2020 03:58:43", "content": "@cascadenite The image features didn't work out really well (if you're thinking it would), as our team did integrate it with the model, but it resulted in poor CV, which I guess could be the indicator that the scans won't work out really good on the private LB.
\nEven though the competition's emphasis would be make use of the scans, but technically at this point, I'm not that confident that scans would be useful.
\nI might be wrong, but currently it feels that scans do not add up much value in improving LB/CV.
", "votes": null, "replies": [] } ] }, { "id": 1018303, "author_name": "yimacs", "author_url": "", "post_date": "09/19/2020 15:32:43", "content": "it doesn't help for CV/LB.
", "votes": null, "replies": [] }, { "id": 1018991, "author_name": "ajay19", "author_url": "", "post_date": "09/20/2020 06:02:46", "content": "Tried image resolution , augmentation while training , DenseNet and Efficientnet ensembling but all in vain
", "votes": null, "replies": [] }, { "id": 1019284, "author_name": "gunesevitan", "author_url": "", "post_date": "09/20/2020 10:28:07", "content": "Thanks for sharing your findings. Even if they improve cv score, I wouldn't use them anyway. Since FVC is not dependent to them, it would be huge gamble to use them in your final submissions.
", "votes": null, "replies": [] }, { "id": 1019757, "author_name": "bigironsphere", "author_url": "", "post_date": "09/20/2020 16:45:34", "content": "These features contain diagnostic information for the CT scanner setup, any correlation they have with lung function will be completely incidental anyway.
", "votes": null, "replies": [] }, { "id": 1020565, "author_name": "prk007", "author_url": "", "post_date": "09/21/2020 09:07:22", "content": "The holdout predictions for private LB is 85% where as public LB is scored on 15% of the predictions, although features related pixel spacing, slice thickness are useless, others like lung volume, lung are useful.
\nWait for a huge LB shakeup once private LB is announced.
", "votes": null, "replies": [] }, { "id": 1024276, "author_name": "ronaldokun", "author_url": "", "post_date": "09/23/2020 18:19:18", "content": "We tried:
\nCenterCrop, Resampled to Pixel Spacing and SliceThickness equal to 1, Mask the Lung and added the following
\nPixel Voxel Statistics: Mean, STD, Kurtosis & Skew Distribution
\nPercentage in Lung Window
\nChest Circunference
\nLung Height
\nLung Volume in cm3 or liters (dm3)
\nHU Bins Distribution from 2 to 14 bins ( Just adding the HU distribution in bins ).
Any addition or combination of those items just made it worse in the LB, sometimes better CV but always worse LB.
", "votes": null, "replies": [ { "id": 1024639, "author_name": "aadhavvignesh", "author_url": "", "post_date": "09/24/2020 03:33:35", "content": "I agree with this. We've tried all possible combinations and on integrating them with the model gave us worse LB and worse CV in almost every case.
\nDoes your team know what exactly the problem is, because at this point making use of scans seems pointless. (not meant to be conclusive, but including them is making CV/LB worse)
\nDo you also think that the winners' solution would help the hosts if they don't use image data for the final model, as I guess the primary objective of the hosts would be to make use of the scans for determining pulmonary fibrosis?
", "votes": null, "replies": [] }, { "id": 1024676, "author_name": "ronaldokun", "author_url": "", "post_date": "09/24/2020 04:15:49", "content": "We are not sure what is wrong, but we think that the release of just the baseline week of the images and also the test tabular data was a fundamental flaw in this competition. We are supposed to predict 146 weeks - time series data - and we don't have a time series data to train on.
\nMedical data is already extremely heterogenous, it seems we have different distributions of patients in the training set ( this notebook by @gannadolinska shows we can consider at least 3 different distributions of patients, those which decrease lung capacity, those steady and those which get better. )
\nThe tabular data has the percent feature, another controversial addition given that is highly correlated with the target, and we also don't have it in the test data.
\nIn an ideal case the image in the baseline would have enough information to guide the model to the different distributions of patients and the evolution of them but we failed to extract any
\nWe pretty much gave up using the images entirely, it was a very frustrating couple of months.
\nEither someone has a bag of tricks no one else has implemented using the images or the organizers are going to pay for a baseline tabular model, overfitted and completely useless medically.
Yes, we had initially planned to extract some useful features using image data, and I even saw some teams trying to manually honeycomb lungs and trying to make it work. Eventually, everyone who had used images became frustrated due to poor results.
\nI too fear that the organizers are going to end up with a baseline tabular model, rendering the objective of the competition to be useless.
\nHope we get to see a novel approach from the winners, but for now I guess most of the LB is saturated with overfitting models.
", "votes": null, "replies": [] }, { "id": 1025705, "author_name": "ronaldokun", "author_url": "", "post_date": "09/24/2020 18:07:31", "content": "I understand the logic though: If a Radiologist can see and characterize the prognosis of decline by just looking into a baseline CT, then a properly designed model would be able to do it too because the information is there. Either we failed to extract it properly or the images are problematic and is very difficult to do it or the info is not really there.
", "votes": null, "replies": [] }, { "id": 1030375, "author_name": "douglaskgaraujo", "author_url": "", "post_date": "09/28/2020 16:07:52", "content": "What @ronaldokun said about the logic makes sense to me. But to my understanding, part of the problem is precisely because clinicians are not able to offer a reasonably certain prognosis even with imaging.
", "votes": null, "replies": [] }, { "id": 1030456, "author_name": "ronaldokun", "author_url": "", "post_date": "09/28/2020 17:19:47", "content": "Yes, I think the whole point of this competition is to fill this gap. It seems a lot harder for this problem judging by the results so far.
", "votes": null, "replies": [] }, { "id": 1031836, "author_name": "jaideepvalani", "author_url": "", "post_date": "09/29/2020 17:55:42", "content": "@ronaldokun to your point of Percent feature.
\nWhen we use it for training and hence prediction we get better lb inspite of fact that it means for all patient visits we are saying patients FVC is same of expected FVC for given segment of patient,please correct me if m wrong in contrary when we try to correct this by not including FVC then we get,worse lb
.
\nIn short is your current score is making use of Percent in Train ?
Hello @jaideepvalani
\nYes, you are absolutely right. Our current best score makes use of percent but by the point made above, IMHO it seems conceptually wrong.
\nBut yes, we are using the percent the same way as yours in our submission because it gave way better scores in CV and LB but we are probably overfitting a lot to the Public LB.
", "votes": null, "replies": [] } ] }, { "id": 1025914, "author_name": "gannadolinska", "author_url": "", "post_date": "09/24/2020 21:46:13", "content": "In general, my impression is that this task is quite specific.
\nFirst of all, fighting for picture technical characteristics (pixels, slices etc) has no correlation with actual image content. And really cool that the experiment showed that no correlation has been found. If some correlation would have been found, I would bet on some bias in the data sample, which one definitely should not rely on as there would be no guarantee that the test data contains the same bias.
But what actually resulted the features like Lung Volume and Lung Area?
\nTo be honest, I think the task is quite complicated and has many internal interdependencies (and for that too few data ;) ).
\nAs I have mentioned before, there are clients with different dynamics of the disease --> not a good idea to mix them up in creating a model.
\nMoreover, I see even deeper patterns in the data. There is even more variety in the dynamics of the disease caused by patient individual characteristics. Like two clients might be recovering, but one much faster than the other, because this patient has better \"input\" - higher lung volume and non-smoker and a female. Thus, I would first try to split the data to the meaningful groups with \"similar\" dynamics of the FVC inside and let image \"predict\" or \"provide\" one or several grouping criteria.
As long as the final model will be kept simple (linear model, for example), one still has a lot of space to reduce the amount of data which is enough for training a model. This will allow to create more homogenous groups and train the models in these groups.
", "votes": null, "replies": [ { "id": 1025934, "author_name": "ronaldokun", "author_url": "", "post_date": "09/24/2020 22:25:54", "content": "Hello Hannah, thanks for extending the discussion.
\nI didn't think of spliting the data by the FVC pattern because I thought I would be leaking the target information. But I guess it's like a feature engineering if we add this information just for the training set and not for the validation set. Did I get that right?
\nMy doubt was what to do for the test set, but rereading the ending insights of your notebook it's clear what you mean.
\nI'll inform if I had any promising results.
", "votes": null, "replies": [] }, { "id": 1026011, "author_name": "bigironsphere", "author_url": "", "post_date": "09/25/2020 01:34:46", "content": "I agree in principle that different clusters of patients could be better analysed by different models, but my concern is that there simply isn't enough data to make generalisation possible in that case. We really could have done with more samples in this dataset…
", "votes": null, "replies": [] }, { "id": 1026285, "author_name": "gannadolinska", "author_url": "", "post_date": "09/25/2020 07:41:24", "content": "Yes, I agree. One might have also taken it as a feature engineering and adding the info to the targeting set. However, we have kind of not that much data to my taste to train a model which will be able to catch all the patterns itself, I fear. That is why I was more thinking of an approach where I overtake some tasks of the model :)
", "votes": null, "replies": [] } ] }, { "id": 1026651, "author_name": "ChristianDenich", "author_url": "", "post_date": "09/25/2020 13:21:31", "content": "Thank you all for your contributions, that's exactly the technical discussion I was looking for!
\nIn summary the winning model will probably be a not-too-complex Tab-Data model, which is only using some additional features, but from Images probably only a tissue-amount/lung-volume value.
", "votes": null, "replies": [] }, { "id": 1036519, "author_name": "sandorkonya", "author_url": "", "post_date": "10/03/2020 21:57:17", "content": "Hi everyone,
\nwe use image-derived features, and it helped to get +0.015 in LB above the tabular results.
\nI manually segmented 100+ CT scans for the lung itself, and it delivers perfect results in lung segmentation.
\nAlso made ~ 40k markups for segmentation of honeycombing, reticulation, GGO/GGR but the heterogenity of the dataset… well… kind-of makes the work pointless.
\nThe unets are generalising pretty poorly on tissue features like reticulation or GGO, but honeycombing is not bad.
\n3D Lung
I hope that after the competition your work on the manual segmentation doesn't go to waste. I'm sure there'll be other applications that would really benefit - deserves more than + 0.015
", "votes": null, "replies": [] }, { "id": 1037089, "author_name": "jaideepvalani", "author_url": "", "post_date": "10/04/2020 15:44:02", "content": "@sandorkonya even if do lung seg for all the patients ct report,average volume of all the lungs should stand same,every one has got same lungs structure except that the tissues in it could be varrying but we lose that while we do segmentation,so is segmentation still useful ?
", "votes": null, "replies": [] }, { "id": 1037278, "author_name": "sandorkonya", "author_url": "", "post_date": "10/04/2020 19:13:55", "content": "@jameschapman19 it won't go waste… i can give back something to the community =)
", "votes": null, "replies": [] }, { "id": 1037281, "author_name": "sandorkonya", "author_url": "", "post_date": "10/04/2020 19:18:28", "content": "@jaideepvalani
\nthe average lung volume is not the same! It depends from the constitution (height, BMI, sex) of the patient… and it also depends on how far the fibrosis is developed. The fibrosis itself makes a volumen depletion of the lung volume (makes it to shrink).
\nWhy would we lose the \"tissue of the lung\" while we segment?
@sandorkonya hope you get paid off by your domain expertise..
\nHow can we determine height from ct … any one trying to do
i agree these features didnt helped me to get a better CV/LB .\nbtw would you like to merge team with me as i dont have a team ,
", "votes": null, "replies": [ { "id": 1022100, "author_name": "ChristianDenich", "author_url": "", "post_date": "09/22/2020 10:08:33", "content": "Thank you for the feedback, that supports our findings.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1017848": "Dear fellow Kagglers,\n\nmy team & I have tried various experiments with **newly derived features from Image data**, like mentioned in Laura's [Notebook: pulmonary-dicom-preprocessing.](https://www.kaggle.com/allunia/pulmonary-dicom-preprocessing)\n\nSome of those features were:\n\n- img resolution\n- slice-thickness\n- pixelspacing_r\n- pixelspacing_c\n- pixelspacing_area\n- window_width\n- image_area_cm2\n- slice_volume_cm3\n\nOur current finding is, that those features sadly DO NOT improve CV/LB.\n\nDo you have share the same experience & get similar results, or are any of those features working for you?\nDid I miss important features?\n\nTo be more precise:\nThe above mentioned features do NOT have any causal relationship with the actual medical condition or prognosis of the patients. It's basically a test if there is a correlation: we couldn't find any. Even if there is a correlation, this would only help in the competition, but obviously not in the real-world examples.\n\nUsing features mentioned by @aadhavvignesh and @abhishekgbhat **might** have a causal relationship with the actual prognosis and the FVC:\n- Lung Volume in cm^3\n- Lung Area in cm^2\n- Average of (Tissue area)/(Lung Area) across all images for a particular patient and FVC\n\n\nThank you all for adding some insights on fighting Pulmonary Fibrosis!\n\n@tanulsingh077 feel free to add!", "1017994": "I have tried including the following features you mentioned:\n- Slice Thickness \n- Pixel Spacing R\n- Pixel Spacing C\nNone of the above features had any correlation with FVC or decline in FVC over time.\n\nBut I did observe some good correlation between:\n- Lung volume and FVC\n- Average of (Tissue area)/(Lung Area) across all images for a particular patient and FVC\n- Average tissue area and decline in FVC over time\n\nYou can consider including some of the above features in your experiment. \nAlso how do you define - image_area_cm2, slice_volume_cm3?", "1018139": "i agree these features didnt helped me to get a better CV/LB .\nbtw would you like to merge team with me as i dont have a team ,", "1018277": "**What I got to know from Image Data:**\n\nI had already mentioned that these features didn't work:\n\n- Lung Volume `in cm^3`\n- Lung Area `in cm^2`\n- Image Resolution/Dimensions\n\nSegmented lung images didn't work as expected. Poor CV/LB still persists. \n\nThis has led our team to put less focus on image data, and more on the available tabular data. I unfortunately can't reveal anything beyond this, because every team would love to keep their sauce 'secret' :P\n\n**About LB and rankings:**\n\nI personally feel that the LB is **NOT** a good indicator of your team's results. I guess that I can safely say that our team has a stable model (my instinct says so :P), and our rankings aren't good. A shake-up is imminent, so I do feel teams should not be looking at the LB scores, but indeed should have a good CV set up.", "1018303": "it doesn't help for CV/LB.", "1018351": "hey adhav does your team needs a member ? , i am currently seraching for a team , my current rank is 26", "1018991": "Tried image resolution , augmentation while training , DenseNet and Efficientnet ensembling but all in vain", "1019284": "Thanks for sharing your findings. Even if they improve cv score, I wouldn't use them anyway. Since FVC is not dependent to them, it would be huge gamble to use them in your final submissions.", "1019757": "These features contain diagnostic information for the CT scanner setup, any correlation they have with lung function will be completely incidental anyway.", "1020565": "The holdout predictions for private LB is 85% where as public LB is scored on 15% of the predictions, although features related pixel spacing, slice thickness are useless, others like lung volume, lung are useful. \n\nWait for a huge LB shakeup once private LB is announced.", "1022098": "Hi @abhishekgbhat, I just saw your post yesterday night, sorry for the delay.\nThe feautes are derived from Laura's Notebook [pulmonary-dicom-preprocessing](https://www.kaggle.com/allunia/pulmonary-dicom-preprocessing). E.g. img_area_cm2 is derived by the resolution.\n\nHow do you derive \"Average of (Tissue area)/(Lung Area)\"?\nIs there a notebook available? Sounds like a lot of work using Hounsfield values & segmentation.", "1022100": "Thank you for the feedback, that supports our findings.", "1022140": "I'm really sorry @subzeroop, our team is full. I hope you can find a team soon :)", "1022812": "Hey @ChristianDenich, thanks for the clarification. \n\n> How do you derive \"Average of (Tissue area)/(Lung Area)\"?\n\nNo, I haven't seen any notebook with this feature. From the limited research I have done, the fiberosis tissue plays an important role in the severity of the disease. So I found out a way to segment the tissues within the lung. From this segmented image I obtained the number of tissue pixels and multiplied it with Pixel Spacing R and C. This gives the tissue area in each image. Assuming you have the lung area for each image, you can get this ratio for each image. Finally, if you have this ratio for all images of a particular patient you can simply take the average. The tough part is to segment the tissues effectively.", "1023338": "So from a maths point of view, this should be the same as considering (Tissue volume)/(Lung volume) I guess. \n\nI considered this feature and thought it gave me a slight improvement, but after changing some hyperparameters etc. I tried again without the feature and got a higher score than with the feature. \n\nMaybe it works better with a better segmentation technique though. I just used the ranges of the HU scale for segmentation.", "1024017": "This is not exactly the volume because im not considering the third dimension(i.e slice thickness) here. This feature tells us the following: For a particular patient, on an average what percent of the lung area does the tissue take up", "1024030": "Yes, but to get the volume of lung, you would have to multiply the average lung area by the length of the lung (3rd dimension). Analoguously, you get the volume of tissue, when you multiply the average tissue area by the length of the lung. When you divide both things, the length cancels out, such that\nvolume of lung / volume of tissue = average area of lung / average area of tissue.\nDo you agree?", "1024276": "We tried:\n\nCenterCrop, Resampled to Pixel Spacing and SliceThickness equal to 1, Mask the Lung and added the following\n\nPixel Voxel Statistics: Mean, STD, Kurtosis & Skew Distribution\nPercentage in Lung Window\nChest Circunference\nLung Height\nLung Volume in cm3 or liters (dm3)\nHU Bins Distribution from 2 to 14 bins ( Just adding the HU distribution in bins ).\n\nAny addition or combination of those items just made it worse in the LB, sometimes better CV but always worse LB.", "1024639": "I agree with this. We've tried all possible combinations and on integrating them with the model gave us worse LB and worse CV in almost every case.\n\nDoes your team know what exactly the problem is, because at this point making use of scans seems pointless. (not meant to be conclusive, but including them is making CV/LB worse)\n\nDo you also think that the winners' solution would help the hosts if they don't use image data for the final model, as I guess the primary objective of the hosts would be to make use of the scans for determining pulmonary fibrosis?", "1024676": "We are not sure what is wrong, but we think that the release of just the baseline week of the images and also the test tabular data was a fundamental flaw in this competition. We are supposed to predict 146 weeks - time series data - and we don't have a time series data to train on.\n\nMedical data is already extremely heterogenous, it seems we have different distributions of patients in the training set ( this notebook by @gannadolinska shows we can consider at least 3 different distributions of patients, those which decrease lung capacity, those steady and those which get better. ) \n\nThe tabular data has the percent feature, another controversial addition given that is highly correlated with the target, and we also don't have it in the test data. \n\nIn an ideal case the image in the baseline would have enough information to guide the model to the different distributions of patients and the evolution of them but we failed to extract any\n\n We pretty much gave up using the images entirely, it was a very frustrating couple of months. \n\nEither someone has a bag of tricks no one else has implemented using the images or the organizers are going to pay for a baseline tabular model, overfitted and completely useless medically.", "1025134": "Yes, we had initially planned to extract some useful features using image data, and I even saw some teams trying to manually honeycomb lungs and trying to make it work. Eventually, everyone who had used images became frustrated due to poor results.\n\nI too fear that the organizers are going to end up with a baseline tabular model, rendering the objective of the competition to be useless.\n\nHope we get to see a novel approach from the winners, but for now I guess most of the LB is saturated with overfitting models.", "1025632": "Hi, I've been noticing too that the images are frustratingly not helping the public LB position but I'm going to keep focussing on them anyway. My hope is that the CT scans must have some value and that a big shakeup will occur where those models that make good use of the scans perform better on the private LB. I'm not looking at the technial aspects of the ct-scan (like pixel size, slice thickness) but I'm confident that the shapes, areas and textures of the slices themselves can be useful.\n\nI must admit though that this hope is more an ensemble of sunk-cost fallacy + dumb-optimism + \"what is point of this competition if not\".... :)", "1025705": "I understand the logic though: If a Radiologist can see and characterize the prognosis of decline by just looking into a baseline CT, then a properly designed model would be able to do it too because the information is there. Either we failed to extract it properly or the images are problematic and is very difficult to do it or the info is not really there.", "1025706": "I've insisted on the previous 2 months with the same hope...", "1025914": "In general, my impression is that this task is quite specific.\nFirst of all, fighting for picture technical characteristics (pixels, slices etc) has no correlation with actual image content. And really cool that the experiment showed that no correlation has been found. If some correlation would have been found, I would bet on some bias in the data sample, which one definitely should not rely on as there would be no guarantee that the test data contains the same bias.\n\nBut what actually resulted the features like Lung Volume and Lung Area?\nTo be honest, I think the task is quite complicated and has many internal interdependencies (and for that too few data ;) ).\nAs I have mentioned before, there are clients with different dynamics of the disease --> not a good idea to mix them up in creating a model.\nMoreover, I see even deeper patterns in the data. There is even more variety in the dynamics of the disease caused by patient individual characteristics. Like two clients might be recovering, but one much faster than the other, because this patient has better \"input\" - higher lung volume and non-smoker and a female. Thus, I would first try to split the data to the meaningful groups with \"similar\" dynamics of the FVC inside and let image \"predict\" or \"provide\" one or several grouping criteria.\n\nAs long as the final model will be kept simple (linear model, for example), one still has a lot of space to reduce the amount of data which is enough for training a model. This will allow to create more homogenous groups and train the models in these groups.", "1025934": "Hello Hannah, thanks for extending the discussion.\n\nI didn't think of spliting the data by the FVC pattern because I thought I would be leaking the target information. But I guess it's like a feature engineering if we add this information just for the training set and not for the validation set. Did I get that right?\n\nMy doubt was what to do for the test set, but rereading the ending insights of your notebook it's clear what you mean.\n\nI'll inform if I had any promising results.", "1026011": "I agree in principle that different clusters of patients could be better analysed by different models, but my concern is that there simply isn't enough data to make generalisation possible in that case. We really could have done with more samples in this dataset...", "1026063": "cascadenite The image features didn't work out really well (if you're thinking it would), as our team did integrate it with the model, but it resulted in poor CV, which I guess could be the indicator that the scans won't work out really good on the private LB.\n\nEven though the competition's emphasis would be make use of the scans, but technically at this point, I'm not that confident that scans would be useful.\n\nI might be wrong, but currently it feels that scans do not add up much value in improving LB/CV.", "1026285": "Yes, I agree. One might have also taken it as a feature engineering and adding the info to the targeting set. However, we have kind of not that much data to my taste to train a model which will be able to catch all the patterns itself, I fear. That is why I was more thinking of an approach where I overtake some tasks of the model :)", "1026579": "Apologies for the delayed response. I'm not convinced with your definition of lung volume, please do correct me if you meant something else. I'm going to write down mathematically what i have understood from your explaination about lung volume.\n\nLet \\\\( I_1, ... I_n \\\\) denote the CT scan images for a patient \\\\(P\\\\) and \\\\(A_1,...,A_n\\\\) denote the corresponding area for segmented lung in each of the images. Let \\\\(l\\\\) denote the slice thickness for each image, then the approx. length of the lung can be obtained as \\\\(L = l*n\\\\).\n\n- Your Definition of lung volume:\n\\\\(Volume = (A_1+...+A_n)/n * L\\\\)\n\n- Ideal definition of lung volume:\n\\\\(Volume = {A_1}*l +...+{A_n} * l = (A_1+...+A_n)*l \\\\)\n\nBoth the definitions are different. Does this make sense?", "1026596": "The two definitions you wrote down are identical?\n$$ (A_1+\\ldots + A_n)/n * L = (A_1+\\ldots + A_n)/n * \\ell * n = (A_1 + \\ldots + A_n) * \\ell $$", "1026651": "Thank you all for your contributions, that's exactly the technical discussion I was looking for!\n\nIn summary the winning model will probably be a not-too-complex Tab-Data model, which is only using some additional features, but from Images probably only a tissue-amount/lung-volume value.", "1026665": "tobiit, thanks a ton! I completely missed that, my bad. Now I completely agree with:\n\n> volume of lung / volume of tissue = average area of lung / average area of tissue\n\nI think the quality tissue segmentation is the key here.", "1030375": "What @ronaldokun said about the logic makes sense to me. But to my understanding, part of the problem is precisely because clinicians are not able to offer a reasonably certain prognosis even with imaging.", "1030456": "Yes, I think the whole point of this competition is to fill this gap. It seems a lot harder for this problem judging by the results so far.", "1031748": "abhishekgbhat CT report provided to us , has got many slices. Are they different CT reports or they part of Single CT scan report only ?", "1031836": "ronaldokun to your point of Percent feature. \nWhen we use it for training and hence prediction we get better lb inspite of fact that it means for all patient visits we are saying patients FVC is same of expected FVC for given segment of patient,please correct me if m wrong in contrary when we try to correct this by not including FVC then we get,worse lb \n \n.\nIn short is your current score is making use of Percent in Train ?", "1032016": "Hello @jaideepvalani \n\nYes, you are absolutely right. Our current best score makes use of percent but by the point made above, IMHO it seems conceptually wrong.\n\nBut yes, we are using the percent the same way as yours in our submission because it gave way better scores in CV and LB but we are probably overfitting a lot to the Public LB.", "1032160": "jaideepvalani CT scan for a particular patient is a collection of slices and all these belong to the same report. So to answer your question, for each patient we have just one CT report with many slices.", "1036519": "Hi everyone,\n\nwe use image-derived features, and it helped to get +0.015 in LB above the tabular results.\n\nI manually segmented 100+ CT scans for the lung itself, and it delivers perfect results in lung segmentation.\n\nAlso made ~ 40k markups for segmentation of honeycombing, reticulation, GGO/GGR but the heterogenity of the dataset... well... kind-of makes the work pointless. \nThe unets are generalising pretty poorly on tissue features like reticulation or GGO, but honeycombing is not bad.\n\n\n[3D Lung](http://drkonya.com/projects/kaggle/lung3d.gif)", "1037057": "I hope that after the competition your work on the manual segmentation doesn't go to waste. I'm sure there'll be other applications that would really benefit - deserves more than + 0.015", "1037089": "sandorkonya even if do lung seg for all the patients ct report,average volume of all the lungs should stand same,every one has got same lungs structure except that the tissues in it could be varrying but we lose that while we do segmentation,so is segmentation still useful ?", "1037278": "jameschapman19 it won't go waste... i can give back something to the community =)", "1037281": "jaideepvalani \nthe average lung volume is not the same! It depends from the constitution (height, BMI, sex) of the patient... and it also depends on how far the fibrosis is developed. The fibrosis itself makes a volumen depletion of the lung volume (makes it to shrink).\nWhy would we lose the \"tissue of the lung\" while we segment?", "1038768": "sandorkonya hope you get paid off by your domain expertise..\nHow can we determine height from ct ... any one trying to do" }, "source": "meta" }