I read some paper that claimed that there is a correlation between CNN features and MGMT promoter. I also talked to a professional radiologist who told me that MGMT promoter can't be seen in an MRI scan at least not by a trained radiologist.
\nCould it be that everyone here is just overfitting to the small dataset given? which means it would not generalize to real-world data ie. never be good enough for deployment purposes…
It would be really interesting to see the GradCam visualization of the winning models.
", "messages": [ { "id": "1538158", "postDate": "10/08/2021 06:03:37", "content": "I read some paper that claimed that there is a correlation between CNN features and MGMT promoter. I also talked to a professional radiologist who told me that MGMT promoter can't be seen in an MRI scan at least not by a trained radiologist.
\nCould it be that everyone here is just overfitting to the small dataset given? which means it would not generalize to real-world data ie. never be good enough for deployment purposes…
It would be really interesting to see the GradCam visualization of the winning models.
", "rawMarkdown": "I read some paper that claimed that there is a correlation between CNN features and MGMT promoter. I also talked to a professional radiologist who told me that MGMT promoter can't be seen in an MRI scan at least not by a trained radiologist.\nCould it be that everyone here is just overfitting to the small dataset given? which means it would not generalize to real-world data ie. never be good enough for deployment purposes...\n\nIt would be really interesting to see the GradCam visualization of the winning models.", "votes": null }, { "id": "1538277", "postDate": "10/08/2021 07:57:41", "content": "@pranshu15, I think your conjecture is not only valid but also likely. But we'll see if the results can be used in the real-world, and the slightest chance is worth a try. The fact that trained humans cannot detect MGMT promoter may just indicates that we don't know what they should be trained to see. If the challenge results give hints on unrecognized patterns to look for, it might improve the cognition of radiologist - even if the CNN gives insufficient results.
\nFYI, I'm not a radiologist or other kind of physician, feel free to declare my comment nonsense.
", "rawMarkdown": "pranshu15, I think your conjecture is not only valid but also likely. But we'll see if the results can be used in the real-world, and the slightest chance is worth a try. The fact that trained humans cannot detect MGMT promoter may just indicates that we don't know what they should be trained to see. If the challenge results give hints on unrecognized patterns to look for, it might improve the cognition of radiologist - even if the CNN gives insufficient results.\n\nFYI, I'm not a radiologist or other kind of physician, feel free to declare my comment nonsense.", "votes": null }, { "id": "1538291", "postDate": "10/08/2021 08:28:53", "content": "You make a good point.
\nAlthough I doubt that a deep learning model can find a pattern so subtle in such a small dataset, it might just help nudge the research in the right direction.
I'm concerned that the top solutions will find a correlation in this small dataset that isn't related to MGMT at all, such as tumor size, location or an unrelated abnormality. Without some method of validation, how will we know?
\nLegitimizing such a solution could do more harm than good in a real-world clinical setting.
", "rawMarkdown": "I'm concerned that the top solutions will find a correlation in this small dataset that isn't related to MGMT at all, such as tumor size, location or an unrelated abnormality. Without some method of validation, how will we know?\n\nLegitimizing such a solution could do more harm than good in a real-world clinical setting.", "votes": null }, { "id": "1538447", "postDate": "10/08/2021 12:47:05", "content": "True, GradCam could tell us where the model is looking but why it's looking at that specific place and what is it making of that information will remain a mystery.
\nHow are we supposed to judge it for something that even we (humans) can't do.
Are we just supposed to accept it if the result matches the report of the \"Invasive Procedure\" for MGMT promoter? That doesn't seem right, specially in medical domain.
", "rawMarkdown": "True, GradCam could tell us where the model is looking but why it's looking at that specific place and what is it making of that information will remain a mystery.\nHow are we supposed to judge it for something that even we (humans) can't do. \n\nAre we just supposed to accept it if the result matches the report of the \"Invasive Procedure\" for MGMT promoter? That doesn't seem right, specially in medical domain.", "votes": null }, { "id": "1542883", "postDate": "10/12/2021 22:26:47", "content": "\n\nMGMT promoter can't be seen in an MRI scan at least not by a trained radiologist.
\n
I think the point is that there isn't any observed or known relationship between the two. The journey of discovery for this kind of project is to find whether DNNs can find new correlations that we never conceived of previously.
\nThis article is an excellent example: https://www.nature.com/articles/s41591-020-01192-7
\nThe model learnt to interpret new findings unidentified previously as correlated to pain scores. This isn't because the model was undertrained or overfitting, but because these relationships weren't known.
I agree with that. What I doubt is if we have enough data to learn something like that.
\nTo put things in perspective, the RSNA Pulmonary Embolism (CT) competition had more than 9000 scans in total. And it's not even that hard to find embolism in angio scans, whereas here we don't even know what we are looking for.
\nI guess we'll get the answer soon though.
Nice post.
\nI think there are subtle difference between target 1 and 0, but my model cannot find them.
\nI trided some approaches like 2dcnn + rnn and 3d cnn, but cv result was not be improved.
I continue to try other approach for last a few days.
\nI am looking forward to PB, and want to know tricks.
Do check out this notebook by @qitvision : https://www.kaggle.com/qitvision/private-lb-simulation/notebook
", "rawMarkdown": "Do check out this notebook by @qitvision : https://www.kaggle.com/qitvision/private-lb-simulation/notebook", "votes": null } ], "comments": [ { "id": 1538277, "author_name": "alexanderbader", "author_url": "", "post_date": "10/08/2021 07:57:41", "content": "@pranshu15, I think your conjecture is not only valid but also likely. But we'll see if the results can be used in the real-world, and the slightest chance is worth a try. The fact that trained humans cannot detect MGMT promoter may just indicates that we don't know what they should be trained to see. If the challenge results give hints on unrecognized patterns to look for, it might improve the cognition of radiologist - even if the CNN gives insufficient results.
\nFYI, I'm not a radiologist or other kind of physician, feel free to declare my comment nonsense.
", "votes": null, "replies": [ { "id": 1538291, "author_name": "pranshu15", "author_url": "", "post_date": "10/08/2021 08:28:53", "content": "You make a good point.
\nAlthough I doubt that a deep learning model can find a pattern so subtle in such a small dataset, it might just help nudge the research in the right direction.
I'm concerned that the top solutions will find a correlation in this small dataset that isn't related to MGMT at all, such as tumor size, location or an unrelated abnormality. Without some method of validation, how will we know?
\nLegitimizing such a solution could do more harm than good in a real-world clinical setting.
", "votes": null, "replies": [ { "id": 1538447, "author_name": "pranshu15", "author_url": "", "post_date": "10/08/2021 12:47:05", "content": "True, GradCam could tell us where the model is looking but why it's looking at that specific place and what is it making of that information will remain a mystery.
\nHow are we supposed to judge it for something that even we (humans) can't do.
Are we just supposed to accept it if the result matches the report of the \"Invasive Procedure\" for MGMT promoter? That doesn't seem right, specially in medical domain.
", "votes": null, "replies": [] } ] }, { "id": 1542883, "author_name": "reubenschmidt", "author_url": "", "post_date": "10/12/2021 22:26:47", "content": "\n\nMGMT promoter can't be seen in an MRI scan at least not by a trained radiologist.
\n
I think the point is that there isn't any observed or known relationship between the two. The journey of discovery for this kind of project is to find whether DNNs can find new correlations that we never conceived of previously.
\nThis article is an excellent example: https://www.nature.com/articles/s41591-020-01192-7
\nThe model learnt to interpret new findings unidentified previously as correlated to pain scores. This isn't because the model was undertrained or overfitting, but because these relationships weren't known.
I agree with that. What I doubt is if we have enough data to learn something like that.
\nTo put things in perspective, the RSNA Pulmonary Embolism (CT) competition had more than 9000 scans in total. And it's not even that hard to find embolism in angio scans, whereas here we don't even know what we are looking for.
\nI guess we'll get the answer soon though.
Nice post.
\nI think there are subtle difference between target 1 and 0, but my model cannot find them.
\nI trided some approaches like 2dcnn + rnn and 3d cnn, but cv result was not be improved.
I continue to try other approach for last a few days.
\nI am looking forward to PB, and want to know tricks.
Do check out this notebook by @qitvision : https://www.kaggle.com/qitvision/private-lb-simulation/notebook
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1538158": "I read some paper that claimed that there is a correlation between CNN features and MGMT promoter. I also talked to a professional radiologist who told me that MGMT promoter can't be seen in an MRI scan at least not by a trained radiologist.\nCould it be that everyone here is just overfitting to the small dataset given? which means it would not generalize to real-world data ie. never be good enough for deployment purposes...\n\nIt would be really interesting to see the GradCam visualization of the winning models.", "1538277": "pranshu15, I think your conjecture is not only valid but also likely. But we'll see if the results can be used in the real-world, and the slightest chance is worth a try. The fact that trained humans cannot detect MGMT promoter may just indicates that we don't know what they should be trained to see. If the challenge results give hints on unrecognized patterns to look for, it might improve the cognition of radiologist - even if the CNN gives insufficient results.\n\nFYI, I'm not a radiologist or other kind of physician, feel free to declare my comment nonsense.", "1538291": "You make a good point. \nAlthough I doubt that a deep learning model can find a pattern so subtle in such a small dataset, it might just help nudge the research in the right direction.", "1538439": "I'm concerned that the top solutions will find a correlation in this small dataset that isn't related to MGMT at all, such as tumor size, location or an unrelated abnormality. Without some method of validation, how will we know?\n\nLegitimizing such a solution could do more harm than good in a real-world clinical setting.", "1538447": "True, GradCam could tell us where the model is looking but why it's looking at that specific place and what is it making of that information will remain a mystery.\nHow are we supposed to judge it for something that even we (humans) can't do. \n\nAre we just supposed to accept it if the result matches the report of the \"Invasive Procedure\" for MGMT promoter? That doesn't seem right, specially in medical domain.", "1542883": "> MGMT promoter can't be seen in an MRI scan at least not by a trained radiologist.\n\nI think the point is that there isn't any observed or known relationship between the two. The journey of discovery for this kind of project is to find whether DNNs can find new correlations that we never conceived of previously.\n\nThis article is an excellent example: https://www.nature.com/articles/s41591-020-01192-7\nThe model learnt to interpret new findings unidentified previously as correlated to pain scores. This isn't because the model was undertrained or overfitting, but because these relationships weren't known.", "1543269": "I agree with that. What I doubt is if we have enough data to learn something like that.\nTo put things in perspective, the RSNA Pulmonary Embolism (CT) competition had more than 9000 scans in total. And it's not even *that* hard to find embolism in angio scans, whereas here we don't even know what we are looking for.\nI guess we'll get the answer soon though.", "1543354": "Nice post.\n\nI think there are subtle difference between target 1 and 0, but my model cannot find them.\nI trided some approaches like 2dcnn + rnn and 3d cnn, but cv result was not be improved.\n\nI continue to try other approach for last a few days.\nI am looking forward to PB, and want to know tricks.", "1553392": "Do check out this notebook by @qitvision : https://www.kaggle.com/qitvision/private-lb-simulation/notebook" }, "source": "meta" }