There are a couple of discussion topics that suggest we may be overfitting to a small dataset here, and that there isn't really any relationship between MGMT and image features anyway - so why are we trying to make a model which predicts this relationship?
\nI think the organisers' goal in this case is premised on the fact that there isn't currently any observed or known relationship between the two. The goal is to find whether DNNs can find new correlations that we hadn't conceived of previously.
\nThis article is an excellent example: https://www.nature.com/articles/s41591-020-01192-7
\nThe model learnt to interpret areas of a knee XR that were correlated with pain scores, which hadn't been identified previously. This isn't because the model was undertrained or overfitting, but because these relationships weren't known.
The gotcha with this article is that these findings were in the context of underserved minority populations, whereas our understanding of Western medicine is based on Anglosaxon norms.
\nAs for this particular competition, it might not reveal anything new about a relationship between image features of glioblastoma and MGMT, but that will be a novel discovery in itself.
", "messages": [ { "id": "1542885", "postDate": "10/12/2021 22:32:30", "content": "There are a couple of discussion topics that suggest we may be overfitting to a small dataset here, and that there isn't really any relationship between MGMT and image features anyway - so why are we trying to make a model which predicts this relationship?
\nI think the organisers' goal in this case is premised on the fact that there isn't currently any observed or known relationship between the two. The goal is to find whether DNNs can find new correlations that we hadn't conceived of previously.
\nThis article is an excellent example: https://www.nature.com/articles/s41591-020-01192-7
\nThe model learnt to interpret areas of a knee XR that were correlated with pain scores, which hadn't been identified previously. This isn't because the model was undertrained or overfitting, but because these relationships weren't known.
The gotcha with this article is that these findings were in the context of underserved minority populations, whereas our understanding of Western medicine is based on Anglosaxon norms.
\nAs for this particular competition, it might not reveal anything new about a relationship between image features of glioblastoma and MGMT, but that will be a novel discovery in itself.
", "rawMarkdown": "There are a couple of discussion topics that suggest we may be overfitting to a small dataset here, and that there isn't really any relationship between MGMT and image features anyway - so why are we trying to make a model which predicts this relationship?\n\nI think the organisers' goal in this case is premised on the fact that there isn't currently any observed or known relationship between the two. The goal is to find whether DNNs can find new correlations that we hadn't conceived of previously.\n\nThis article is an excellent example: https://www.nature.com/articles/s41591-020-01192-7\nThe model learnt to interpret areas of a knee XR that were correlated with pain scores, which hadn't been identified previously. This isn't because the model was undertrained or overfitting, but because these relationships weren't known. \n\nThe gotcha with this article is that these findings were in the context of underserved minority populations, whereas our understanding of Western medicine is based on Anglosaxon norms. \n\nAs for this particular competition, it might not reveal anything new about a relationship between image features of glioblastoma and MGMT, but that will be a novel discovery in itself.", "votes": null }, { "id": "1542887", "postDate": "10/12/2021 22:50:14", "content": "Going into this competition, that was exactly my thought process.
\nThough I never underestimate the GMs of Kaggle, especially those from the rising sun area, and we may be presently surprised come final submission write ups. Fingers crossed!
", "rawMarkdown": "Going into this competition, that was exactly my thought process.\n\nThough I never underestimate the GMs of Kaggle, especially those from the rising sun area, and we may be presently surprised come final submission write ups. Fingers crossed!", "votes": null }, { "id": "1543605", "postDate": "10/13/2021 17:01:03", "content": "I definitely interpreted this competition as \"find a way to determine methylation tag status for glioblastomas from MRIs alone, to reduce the number of invasive/dangerous brain biopsies that must be done, no known technique for this exists\". A noble goal to be sure, but it looks like the task may in fact be impossible at the resolution of current MRI machines.
", "rawMarkdown": "I definitely interpreted this competition as \"find a way to determine methylation tag status for glioblastomas from MRIs alone, to reduce the number of invasive/dangerous brain biopsies that must be done, no known technique for this exists\". A noble goal to be sure, but it looks like the task may in fact be impossible at the resolution of current MRI machines.", "votes": null }, { "id": "1544076", "postDate": "10/14/2021 03:45:16", "content": "Yeah I see, that’s interesting. Can I ask why you think it’s looking impossible?
", "rawMarkdown": "Yeah I see, that’s interesting. Can I ask why you think it’s looking impossible?", "votes": null }, { "id": "1545387", "postDate": "10/15/2021 07:05:58", "content": "@reubenschmidt , @maxbaugh neuroradiologist here. I am pretty confident that there will be highly sensitive imaging features - but maybe not with standard sequences.
\nIt had been already shown that analysis of the MRI is capable of determining the IDH mutation status and 1p/19q codeletion status (and many more other imaging biomarker) for diffuse of gliomas [1]. These are really important biomarkers that determine the overall survival and the treatment possibilities.
\nEven the referred article below mentions empiric observations about the difference between methylated and non-methylated tumors (like oedema, higher ADC value, perfusion). The fact that even human observations exist means to me that the chances are really good that machine vision will find highly sensitive features even in these basis-sequences that determine the methylation status with high accuracy! Also, mri diagnostic is not standardised. Many-many sequences and also spectroscopic and perfusion analyses are not even considered here (since not standardised collected) that could help massively to have an imaging biomarker.
[1]: for example https://www.kjronline.org/pdf/10.3348/kjr.2020.1450
", "rawMarkdown": "reubenschmidt , @maxbaugh neuroradiologist here. I am pretty confident that there will be highly sensitive imaging features - but maybe not with standard sequences.\n\nIt had been already shown that analysis of the MRI is capable of determining the IDH mutation status and 1p/19q codeletion status (and many more other imaging biomarker) for diffuse of gliomas [1]. These are really important biomarkers that determine the overall survival and the treatment possibilities.\nEven the referred article below mentions empiric observations about the difference between methylated and non-methylated tumors (like oedema, higher ADC value, perfusion). The fact that even human observations exist means to me that the chances are really good that machine vision will find highly sensitive features even in these basis-sequences that determine the methylation status with high accuracy! Also, mri diagnostic is not standardised. Many-many sequences and also spectroscopic and perfusion analyses are not even considered here (since not standardised collected) that could help massively to have an imaging biomarker.\n\n[1]: for example https://www.kjronline.org/pdf/10.3348/kjr.2020.1450", "votes": null }, { "id": "1548832", "postDate": "10/18/2021 15:02:14", "content": "Got 100% accuracy on training dataset, looks like model overfitted. but if the model is giving 100% acc,it is maybe finding some correlation or features. Tried to do gradcam heatmap to check where the model is looking at,heatmap was overlapping with tumor. Couldn't find anything conclusive
", "rawMarkdown": "Got 100% accuracy on training dataset, looks like model overfitted. but if the model is giving 100% acc,it is maybe finding some correlation or features. Tried to do gradcam heatmap to check where the model is looking at,heatmap was overlapping with tumor. Couldn't find anything conclusive", "votes": null } ], "comments": [ { "id": 1542887, "author_name": "authman", "author_url": "", "post_date": "10/12/2021 22:50:14", "content": "Going into this competition, that was exactly my thought process.
\nThough I never underestimate the GMs of Kaggle, especially those from the rising sun area, and we may be presently surprised come final submission write ups. Fingers crossed!
", "votes": null, "replies": [] }, { "id": 1543605, "author_name": "maxbaugh", "author_url": "", "post_date": "10/13/2021 17:01:03", "content": "I definitely interpreted this competition as \"find a way to determine methylation tag status for glioblastomas from MRIs alone, to reduce the number of invasive/dangerous brain biopsies that must be done, no known technique for this exists\". A noble goal to be sure, but it looks like the task may in fact be impossible at the resolution of current MRI machines.
", "votes": null, "replies": [ { "id": 1544076, "author_name": "reubenschmidt", "author_url": "", "post_date": "10/14/2021 03:45:16", "content": "Yeah I see, that’s interesting. Can I ask why you think it’s looking impossible?
", "votes": null, "replies": [] }, { "id": 1545387, "author_name": "sandorkonya", "author_url": "", "post_date": "10/15/2021 07:05:58", "content": "@reubenschmidt , @maxbaugh neuroradiologist here. I am pretty confident that there will be highly sensitive imaging features - but maybe not with standard sequences.
\nIt had been already shown that analysis of the MRI is capable of determining the IDH mutation status and 1p/19q codeletion status (and many more other imaging biomarker) for diffuse of gliomas [1]. These are really important biomarkers that determine the overall survival and the treatment possibilities.
\nEven the referred article below mentions empiric observations about the difference between methylated and non-methylated tumors (like oedema, higher ADC value, perfusion). The fact that even human observations exist means to me that the chances are really good that machine vision will find highly sensitive features even in these basis-sequences that determine the methylation status with high accuracy! Also, mri diagnostic is not standardised. Many-many sequences and also spectroscopic and perfusion analyses are not even considered here (since not standardised collected) that could help massively to have an imaging biomarker.
[1]: for example https://www.kjronline.org/pdf/10.3348/kjr.2020.1450
", "votes": null, "replies": [] } ] }, { "id": 1548832, "author_name": "pranavjadhav", "author_url": "", "post_date": "10/18/2021 15:02:14", "content": "Got 100% accuracy on training dataset, looks like model overfitted. but if the model is giving 100% acc,it is maybe finding some correlation or features. Tried to do gradcam heatmap to check where the model is looking at,heatmap was overlapping with tumor. Couldn't find anything conclusive
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1542885": "There are a couple of discussion topics that suggest we may be overfitting to a small dataset here, and that there isn't really any relationship between MGMT and image features anyway - so why are we trying to make a model which predicts this relationship?\n\nI think the organisers' goal in this case is premised on the fact that there isn't currently any observed or known relationship between the two. The goal is to find whether DNNs can find new correlations that we hadn't conceived of previously.\n\nThis article is an excellent example: https://www.nature.com/articles/s41591-020-01192-7\nThe model learnt to interpret areas of a knee XR that were correlated with pain scores, which hadn't been identified previously. This isn't because the model was undertrained or overfitting, but because these relationships weren't known. \n\nThe gotcha with this article is that these findings were in the context of underserved minority populations, whereas our understanding of Western medicine is based on Anglosaxon norms. \n\nAs for this particular competition, it might not reveal anything new about a relationship between image features of glioblastoma and MGMT, but that will be a novel discovery in itself.", "1542887": "Going into this competition, that was exactly my thought process.\n\nThough I never underestimate the GMs of Kaggle, especially those from the rising sun area, and we may be presently surprised come final submission write ups. Fingers crossed!", "1543605": "I definitely interpreted this competition as \"find a way to determine methylation tag status for glioblastomas from MRIs alone, to reduce the number of invasive/dangerous brain biopsies that must be done, no known technique for this exists\". A noble goal to be sure, but it looks like the task may in fact be impossible at the resolution of current MRI machines.", "1544076": "Yeah I see, that’s interesting. Can I ask why you think it’s looking impossible?", "1545387": "reubenschmidt , @maxbaugh neuroradiologist here. I am pretty confident that there will be highly sensitive imaging features - but maybe not with standard sequences.\n\nIt had been already shown that analysis of the MRI is capable of determining the IDH mutation status and 1p/19q codeletion status (and many more other imaging biomarker) for diffuse of gliomas [1]. These are really important biomarkers that determine the overall survival and the treatment possibilities.\nEven the referred article below mentions empiric observations about the difference between methylated and non-methylated tumors (like oedema, higher ADC value, perfusion). The fact that even human observations exist means to me that the chances are really good that machine vision will find highly sensitive features even in these basis-sequences that determine the methylation status with high accuracy! Also, mri diagnostic is not standardised. Many-many sequences and also spectroscopic and perfusion analyses are not even considered here (since not standardised collected) that could help massively to have an imaging biomarker.\n\n[1]: for example https://www.kjronline.org/pdf/10.3348/kjr.2020.1450", "1548832": "Got 100% accuracy on training dataset, looks like model overfitted. but if the model is giving 100% acc,it is maybe finding some correlation or features. Tried to do gradcam heatmap to check where the model is looking at,heatmap was overlapping with tumor. Couldn't find anything conclusive" }, "source": "meta" }