The above is a link to a peer-reviwed paper recently published in the prestigious journal Nature Machine Intelligence. They examined 2,212 studies published in 2020, of which 415 were included after initial screening and, after quality screening, 62 studies were included in the systematic review.
\nUnfortunately they found that \"…none of the [machine learning] models identified are of potential clinical use due to methodological flaws and/or underlying biases.\"
\nTheir main findings were:
\nThey then proceed to suggest a number of recommendations. The paper is well worth reading and is Open Access:
\n\nIn an even more recent publication AI was found to be cheating by taking 'shortcuts':
\n\n\n", "messages": [ { "id": "1316648", "postDate": "05/20/2021 18:38:08", "content": "\n\"The team found that, rather than learning genuine medical pathology, these models rely instead on shortcut learning to draw spurious associations between medically irrelevant factors and disease status. Here, the models ignored clinically significant indicators and relied instead on characteristics such as text markers or patient positioning that were specific to each dataset to predict whether someone had COVID-19.\"
\n
The above is a link to a peer-reviwed paper recently published in the prestigious journal Nature Machine Intelligence. They examined 2,212 studies published in 2020, of which 415 were included after initial screening and, after quality screening, 62 studies were included in the systematic review.
\nUnfortunately they found that \"…none of the [machine learning] models identified are of potential clinical use due to methodological flaws and/or underlying biases.\"
\nTheir main findings were:
\nThey then proceed to suggest a number of recommendations. The paper is well worth reading and is Open Access:
\n\nIn an even more recent publication AI was found to be cheating by taking 'shortcuts':
\n\n\n", "rawMarkdown": "[**Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans**](https://www.nature.com/articles/s42256-021-00307-0)\n\nThe above is a link to a peer-reviwed paper recently published in the prestigious journal ***Nature Machine Intelligence***. They examined 2,212 studies published in 2020, of which 415 were included after initial screening and, after quality screening, 62 studies were included in the systematic review. \n\nUnfortunately they found that \"...*none of the [machine learning] models identified are of potential clinical use due to methodological flaws and/or underlying biases.*\"\n\nTheir main findings were:\n\n* **Duplication and quality issues**: Source issues, Frankenstein datasets, Implicit biases in the source data\n* **Methodology issues**: \"...*Diagnostic studies commonly compare their models’ performance to that of RT–PCR. However, as the ground-truth labels are often determined by RT–PCR, there is no way to measure whether a model outperforms RT–PCR from accuracy, sensitivity or specificity metrics alone. Ideally, models should aim to match clinicians using all available clinical and radiomic data*...\"\n\nThey then proceed to suggest a number of recommendations. The paper is well worth reading and is Open Access:\n\n* [Roberts et. al. *Nature Machine Intelligence*, volume **3**, pages 199–217 (2021)](https://www.nature.com/articles/s42256-021-00307-0)\n\nIn an even more recent publication AI was found to be cheating by taking 'shortcuts':\n\n> \"*The team found that, rather than learning genuine medical pathology, these models rely instead on shortcut learning to draw spurious associations between medically irrelevant factors and disease status. Here, the models ignored clinically significant indicators and relied instead on characteristics such as text markers or patient positioning that were specific to each dataset to predict whether someone had COVID-19.*\"\n\n* [\"*AI for radiographic COVID-19 detection selects shortcuts over signal*\", Nature Machine Intelligence (2021)](https://www.nature.com/articles/s42256-021-00338-7)", "votes": null }, { "id": "1316661", "postDate": "05/20/2021 18:59:16", "content": "\"The team found that, rather than learning genuine medical pathology, these models rely instead on shortcut learning to draw spurious associations between medically irrelevant factors and disease status. Here, the models ignored clinically significant indicators and relied instead on characteristics such as text markers or patient positioning that were specific to each dataset to predict whether someone had COVID-19.\"
\n
This is a great resource. Thanks for sharing it :)
", "rawMarkdown": "This is a great resource. Thanks for sharing it :)", "votes": null } ], "comments": [ { "id": 1316661, "author_name": "hassiahk", "author_url": "", "post_date": "05/20/2021 18:59:16", "content": "This is a great resource. Thanks for sharing it :)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1316648": "[**Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans**](https://www.nature.com/articles/s42256-021-00307-0)\n\nThe above is a link to a peer-reviwed paper recently published in the prestigious journal ***Nature Machine Intelligence***. They examined 2,212 studies published in 2020, of which 415 were included after initial screening and, after quality screening, 62 studies were included in the systematic review. \n\nUnfortunately they found that \"...*none of the [machine learning] models identified are of potential clinical use due to methodological flaws and/or underlying biases.*\"\n\nTheir main findings were:\n\n* **Duplication and quality issues**: Source issues, Frankenstein datasets, Implicit biases in the source data\n* **Methodology issues**: \"...*Diagnostic studies commonly compare their models’ performance to that of RT–PCR. However, as the ground-truth labels are often determined by RT–PCR, there is no way to measure whether a model outperforms RT–PCR from accuracy, sensitivity or specificity metrics alone. Ideally, models should aim to match clinicians using all available clinical and radiomic data*...\"\n\nThey then proceed to suggest a number of recommendations. The paper is well worth reading and is Open Access:\n\n* [Roberts et. al. *Nature Machine Intelligence*, volume **3**, pages 199–217 (2021)](https://www.nature.com/articles/s42256-021-00307-0)\n\nIn an even more recent publication AI was found to be cheating by taking 'shortcuts':\n\n> \"*The team found that, rather than learning genuine medical pathology, these models rely instead on shortcut learning to draw spurious associations between medically irrelevant factors and disease status. Here, the models ignored clinically significant indicators and relied instead on characteristics such as text markers or patient positioning that were specific to each dataset to predict whether someone had COVID-19.*\"\n\n* [\"*AI for radiographic COVID-19 detection selects shortcuts over signal*\", Nature Machine Intelligence (2021)](https://www.nature.com/articles/s42256-021-00338-7)", "1316661": "This is a great resource. Thanks for sharing it :)" }, "source": "meta" }