There a lot of fields available in DICOM, but almost all of them are useless as they either contain single value or some kind of UID which is unique for each record. Gender and Age fields are naturally to use for models. \nThere are two other available fields:
\n\nThey cannot be true predictors for pneumonia, but on my naive model (no image info, Gender, Age, View Position, and Pixel Spacing only) that predicts if a patient has pneumonia, both View Position and Pixel Spacing improved the score.
\n\nI did not check what Pixel Spacing is, but may guess that it is somehow related to resolution.\nView position is how picture was made. Approximately 54% of images (in this competition) made in PA position and 45% in AP (test has approximately the same ratio as train). See explanation about position here.
\n\nI may try to explain predictive power of these two fields in this way. PA position is preferable one as resulting images are more clear, AP is made for very ill patients, so probably percent of patients with pneumonia is higher among very ill patients. As for Pixel Spacing, if my guess is correct about meaning of the field, more advanced equipment is used for ill patient group.
\n\nHope this info can help community in improving models or at least in spending less time to investigate.
", "messages": [ { "id": "377836", "postDate": "08/29/2018 20:19:29", "content": "There a lot of fields available in DICOM, but almost all of them are useless as they either contain single value or some kind of UID which is unique for each record. Gender and Age fields are naturally to use for models. \nThere are two other available fields:
\n\nThey cannot be true predictors for pneumonia, but on my naive model (no image info, Gender, Age, View Position, and Pixel Spacing only) that predicts if a patient has pneumonia, both View Position and Pixel Spacing improved the score.
\n\nI did not check what Pixel Spacing is, but may guess that it is somehow related to resolution.\nView position is how picture was made. Approximately 54% of images (in this competition) made in PA position and 45% in AP (test has approximately the same ratio as train). See explanation about position here.
\n\nI may try to explain predictive power of these two fields in this way. PA position is preferable one as resulting images are more clear, AP is made for very ill patients, so probably percent of patients with pneumonia is higher among very ill patients. As for Pixel Spacing, if my guess is correct about meaning of the field, more advanced equipment is used for ill patient group.
\n\nHope this info can help community in improving models or at least in spending less time to investigate.
", "rawMarkdown": "There a lot of fields available in DICOM, but almost all of them are useless as they either contain single value or some kind of UID which is unique for each record. Gender and Age fields are naturally to use for models. \nThere are two other available fields: \n\n - View Position\n - Pixel Spacing\n\nThey cannot be true predictors for pneumonia, but on my naive model (no image info, Gender, Age, View Position, and Pixel Spacing only) that predicts if a patient has pneumonia, both View Position and Pixel Spacing improved the score.\n\nI did not check what Pixel Spacing is, but may guess that it is somehow related to resolution.\nView position is how picture was made. Approximately 54% of images (in this competition) made in PA position and 45% in AP (test has approximately the same ratio as train). See explanation about position [here][1]. \n\nI may try to explain predictive power of these two fields in this way. PA position is preferable one as resulting images are more clear, AP is made for very ill patients, so probably percent of patients with pneumonia is higher among very ill patients. As for Pixel Spacing, if my guess is correct about meaning of the field, more advanced equipment is used for ill patient group.\n\nHope this info can help community in improving models or at least in spending less time to investigate.\n\n\n\n\n [1]: https://www.med-ed.virginia.edu/courses/rad/cxr/technique3chest.html", "votes": null }, { "id": "377876", "postDate": "08/29/2018 22:02:14", "content": "Hi Alex,
\n\nInteresting that View Position and Pixel Spacing were useful data in improving your predictions.\nAP positioning is indeed much more common for sicker patients, so it makes sense that your model can make use of it to improve accuracy. As a radiologist, I don't think I consciously factor that into my assessment of chest radiographs, but perhaps I do subconsciously. There are other \"tells\" on radiographs: for example, if an endotracheal tube, pleural drainage tube, or central venous catheter is on the image, the patient is probably fairly ill.
\nThese are not visualization of pneumonia per se, and intuitively I would not think they are useful for diagnosing pneumonia, because by the time these medical interventions have been made, a diagnosis has often been made already, whether it's pneumonia, pulmonary edema, pleural effusion, etc etc. So while you may find your model paying attention to these things, it would not necessarily make it into a better real-world tool for diagnosing pneumonia in the \"virgin\" state.
\"As for Pixel Spacing, if my guess is correct about meaning of the field, more advanced equipment is used for ill patient group.\"\nActually this is not true. If the patient is too ill to stand for a PA view (x-ray beam from behind, detector against the chest), then an AP view is taken, usually with a \"portable\" unit which generally isn't as good as the permanently installed units. Pixel spacing is mainly used to better estimate true size of something on the image (though x-ray magnification precludes precise measurement. A portable AP view has the further drawback of (in general) no anti-scatter grid in front of the detector (too much risk of grid cut-off from inconsistent geometry, i.e. very hard to guarantee the plate is perpendicular to the diverging x-ray beam and the beam is centered and at the correct distance from the plate). And finally, when getting a PA view, a lateral view is almost always obtained, which marginally increases sensitivity for detection of disease and helps with localization of disease. In the portable setting, a lateral view is almost never obtained. I suspect, in the interest of not complicating things too much, only frontal views (PA and AP) where obtained for this corpus of cases.
\n\n-Dan
", "rawMarkdown": "Hi Alex,\n\nInteresting that View Position and Pixel Spacing were useful data in improving your predictions.\nAP positioning is indeed much more common for sicker patients, so it makes sense that your model can make use of it to improve accuracy. As a radiologist, I don't think I consciously factor that into my assessment of chest radiographs, but perhaps I do subconsciously. There are other \"tells\" on radiographs: for example, if an endotracheal tube, pleural drainage tube, or central venous catheter is on the image, the patient is probably fairly ill. \nThese are not visualization of pneumonia per se, and intuitively I would not think they are useful for diagnosing pneumonia, because by the time these medical interventions have been made, a diagnosis has often been made already, whether it's pneumonia, pulmonary edema, pleural effusion, etc etc. So while you may find your model paying attention to these things, it would not necessarily make it into a better real-world tool for diagnosing pneumonia in the \"virgin\" state.\n\n\"As for Pixel Spacing, if my guess is correct about meaning of the field, more advanced equipment is used for ill patient group.\"\nActually this is not true. If the patient is too ill to stand for a PA view (x-ray beam from behind, detector against the chest), then an AP view is taken, usually with a \"portable\" unit which generally isn't as good as the permanently installed units. Pixel spacing is mainly used to better estimate true size of something on the image (though x-ray magnification precludes precise measurement. A portable AP view has the further drawback of (in general) no anti-scatter grid in front of the detector (too much risk of grid cut-off from inconsistent geometry, i.e. very hard to guarantee the plate is perpendicular to the diverging x-ray beam and the beam is centered and at the correct distance from the plate). And finally, when getting a PA view, a lateral view is almost always obtained, which marginally increases sensitivity for detection of disease and helps with localization of disease. In the portable setting, a lateral view is almost never obtained. I suspect, in the interest of not complicating things too much, only frontal views (PA and AP) where obtained for this corpus of cases.\n\n-Dan", "votes": null }, { "id": "378290", "postDate": "08/30/2018 05:38:24", "content": "Dan, thank you very much for such a detailed comment. Agree that View Position and Pixel Spacing are not appropriate for real world prediction.
\n\nAs for why only forntal view present in the dataset, I suppose you already answered - \"when getting a PA view, a lateral view is almost always obtained, .... In the portable setting, a lateral view is almost never obtained\"
\n\nI have a question, when a radiologist examines a chest x-ray image, does he/she pay attention to view position? Does it change anything? For example, I see something on the image, but maybe this is because of AP position, I need to check other/more factors.
", "rawMarkdown": "Dan, thank you very much for such a detailed comment. Agree that View Position and Pixel Spacing are not appropriate for real world prediction.\n\nAs for why only forntal view present in the dataset, I suppose you already answered - \"when getting a PA view, a lateral view is almost always obtained, .... In the portable setting, a lateral view is almost never obtained\"\n\nI have a question, when a radiologist examines a chest x-ray image, does he/she pay attention to view position? Does it change anything? For example, I see something on the image, but maybe this is because of AP position, I need to check other/more factors.", "votes": null }, { "id": "378628", "postDate": "08/30/2018 12:30:01", "content": "Alex, most radiologists do make a note of AP vs PA, but mostly to be aware that there is generally lower quality and more variation of positioning in an AP view. The two most important things to keep in mind are 1) that the heart is magnified more on the AP view, so the standard of what constitutes cardiac enlargement differ from AP to PA; and 2) PA images are virtually always obtained in the upright (standing or sitting) position, while an AP image (often done at the bedside) may be supine (lying down), sitting up, or anywhere in between. Lung physiology is dependent on position (gravitational effects) and this is taken into account when evaluating for pathology (e.g. upper lung vessel distention may be normal withe the patient supine but abnormal when sitting up).\nMy guess is, other than a bias towards sicker patients for AP studies, none of these other considerations are going to be important discriminators for training a model. For one thing, the images are downsampled to 1024x1024, when the originals were more like 4096x2048, so loss of fine structure detail in lungs (which can be important in diagnosing viral pneumonia) will make PA and AP views more equivalent than at original resolution.\nI haven't done it, but it should be easy enough to determine if the AP views are indeed biased to pneumonia by calculating the ratio of pneumonia to non-pneumonia cases in the corpus for AP vs PA images.
", "rawMarkdown": "Alex, most radiologists do make a note of AP vs PA, but mostly to be aware that there is generally lower quality and more variation of positioning in an AP view. The two most important things to keep in mind are 1) that the heart is magnified more on the AP view, so the standard of what constitutes cardiac enlargement differ from AP to PA; and 2) PA images are virtually always obtained in the upright (standing or sitting) position, while an AP image (often done at the bedside) may be supine (lying down), sitting up, or anywhere in between. Lung physiology is dependent on position (gravitational effects) and this is taken into account when evaluating for pathology (e.g. upper lung vessel distention may be normal withe the patient supine but abnormal when sitting up).\nMy guess is, other than a bias towards sicker patients for AP studies, none of these other considerations are going to be important discriminators for training a model. For one thing, the images are downsampled to 1024x1024, when the originals were more like 4096x2048, so loss of fine structure detail in lungs (which can be important in diagnosing viral pneumonia) will make PA and AP views more equivalent than at original resolution.\nI haven't done it, but it should be easy enough to determine if the AP views are indeed biased to pneumonia by calculating the ratio of pneumonia to non-pneumonia cases in the corpus for AP vs PA images.", "votes": null }, { "id": "378655", "postDate": "08/30/2018 13:08:12", "content": "Thanks for reply, Dan. I am asking about position to decide if AP images should be treated in some special way, maybe create separate model, not to use position directly as a feature.\nRegarding bias, AP has 37.5% with pneumonia, while PA only 9%. \nSo I guess that some models may use tubes and catheters as sign of pneumonia.
", "rawMarkdown": "Thanks for reply, Dan. I am asking about position to decide if AP images should be treated in some special way, maybe create separate model, not to use position directly as a feature.\nRegarding bias, AP has 37.5% with pneumonia, while PA only 9%. \nSo I guess that some models may use tubes and catheters as sign of pneumonia.", "votes": null } ], "comments": [ { "id": 377876, "author_name": "dchernoff", "author_url": "", "post_date": "08/29/2018 22:02:14", "content": "Hi Alex,
\n\nInteresting that View Position and Pixel Spacing were useful data in improving your predictions.\nAP positioning is indeed much more common for sicker patients, so it makes sense that your model can make use of it to improve accuracy. As a radiologist, I don't think I consciously factor that into my assessment of chest radiographs, but perhaps I do subconsciously. There are other \"tells\" on radiographs: for example, if an endotracheal tube, pleural drainage tube, or central venous catheter is on the image, the patient is probably fairly ill.
\nThese are not visualization of pneumonia per se, and intuitively I would not think they are useful for diagnosing pneumonia, because by the time these medical interventions have been made, a diagnosis has often been made already, whether it's pneumonia, pulmonary edema, pleural effusion, etc etc. So while you may find your model paying attention to these things, it would not necessarily make it into a better real-world tool for diagnosing pneumonia in the \"virgin\" state.
\"As for Pixel Spacing, if my guess is correct about meaning of the field, more advanced equipment is used for ill patient group.\"\nActually this is not true. If the patient is too ill to stand for a PA view (x-ray beam from behind, detector against the chest), then an AP view is taken, usually with a \"portable\" unit which generally isn't as good as the permanently installed units. Pixel spacing is mainly used to better estimate true size of something on the image (though x-ray magnification precludes precise measurement. A portable AP view has the further drawback of (in general) no anti-scatter grid in front of the detector (too much risk of grid cut-off from inconsistent geometry, i.e. very hard to guarantee the plate is perpendicular to the diverging x-ray beam and the beam is centered and at the correct distance from the plate). And finally, when getting a PA view, a lateral view is almost always obtained, which marginally increases sensitivity for detection of disease and helps with localization of disease. In the portable setting, a lateral view is almost never obtained. I suspect, in the interest of not complicating things too much, only frontal views (PA and AP) where obtained for this corpus of cases.
\n\n-Dan
", "votes": null, "replies": [] }, { "id": 378290, "author_name": "alexfir", "author_url": "", "post_date": "08/30/2018 05:38:24", "content": "Dan, thank you very much for such a detailed comment. Agree that View Position and Pixel Spacing are not appropriate for real world prediction.
\n\nAs for why only forntal view present in the dataset, I suppose you already answered - \"when getting a PA view, a lateral view is almost always obtained, .... In the portable setting, a lateral view is almost never obtained\"
\n\nI have a question, when a radiologist examines a chest x-ray image, does he/she pay attention to view position? Does it change anything? For example, I see something on the image, but maybe this is because of AP position, I need to check other/more factors.
", "votes": null, "replies": [ { "id": 378628, "author_name": "dchernoff", "author_url": "", "post_date": "08/30/2018 12:30:01", "content": "Alex, most radiologists do make a note of AP vs PA, but mostly to be aware that there is generally lower quality and more variation of positioning in an AP view. The two most important things to keep in mind are 1) that the heart is magnified more on the AP view, so the standard of what constitutes cardiac enlargement differ from AP to PA; and 2) PA images are virtually always obtained in the upright (standing or sitting) position, while an AP image (often done at the bedside) may be supine (lying down), sitting up, or anywhere in between. Lung physiology is dependent on position (gravitational effects) and this is taken into account when evaluating for pathology (e.g. upper lung vessel distention may be normal withe the patient supine but abnormal when sitting up).\nMy guess is, other than a bias towards sicker patients for AP studies, none of these other considerations are going to be important discriminators for training a model. For one thing, the images are downsampled to 1024x1024, when the originals were more like 4096x2048, so loss of fine structure detail in lungs (which can be important in diagnosing viral pneumonia) will make PA and AP views more equivalent than at original resolution.\nI haven't done it, but it should be easy enough to determine if the AP views are indeed biased to pneumonia by calculating the ratio of pneumonia to non-pneumonia cases in the corpus for AP vs PA images.
", "votes": null, "replies": [] }, { "id": 378655, "author_name": "alexfir", "author_url": "", "post_date": "08/30/2018 13:08:12", "content": "Thanks for reply, Dan. I am asking about position to decide if AP images should be treated in some special way, maybe create separate model, not to use position directly as a feature.\nRegarding bias, AP has 37.5% with pneumonia, while PA only 9%. \nSo I guess that some models may use tubes and catheters as sign of pneumonia.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "377836": "There a lot of fields available in DICOM, but almost all of them are useless as they either contain single value or some kind of UID which is unique for each record. Gender and Age fields are naturally to use for models. \nThere are two other available fields: \n\n - View Position\n - Pixel Spacing\n\nThey cannot be true predictors for pneumonia, but on my naive model (no image info, Gender, Age, View Position, and Pixel Spacing only) that predicts if a patient has pneumonia, both View Position and Pixel Spacing improved the score.\n\nI did not check what Pixel Spacing is, but may guess that it is somehow related to resolution.\nView position is how picture was made. Approximately 54% of images (in this competition) made in PA position and 45% in AP (test has approximately the same ratio as train). See explanation about position [here][1]. \n\nI may try to explain predictive power of these two fields in this way. PA position is preferable one as resulting images are more clear, AP is made for very ill patients, so probably percent of patients with pneumonia is higher among very ill patients. As for Pixel Spacing, if my guess is correct about meaning of the field, more advanced equipment is used for ill patient group.\n\nHope this info can help community in improving models or at least in spending less time to investigate.\n\n\n\n\n [1]: https://www.med-ed.virginia.edu/courses/rad/cxr/technique3chest.html", "377876": "Hi Alex,\n\nInteresting that View Position and Pixel Spacing were useful data in improving your predictions.\nAP positioning is indeed much more common for sicker patients, so it makes sense that your model can make use of it to improve accuracy. As a radiologist, I don't think I consciously factor that into my assessment of chest radiographs, but perhaps I do subconsciously. There are other \"tells\" on radiographs: for example, if an endotracheal tube, pleural drainage tube, or central venous catheter is on the image, the patient is probably fairly ill. \nThese are not visualization of pneumonia per se, and intuitively I would not think they are useful for diagnosing pneumonia, because by the time these medical interventions have been made, a diagnosis has often been made already, whether it's pneumonia, pulmonary edema, pleural effusion, etc etc. So while you may find your model paying attention to these things, it would not necessarily make it into a better real-world tool for diagnosing pneumonia in the \"virgin\" state.\n\n\"As for Pixel Spacing, if my guess is correct about meaning of the field, more advanced equipment is used for ill patient group.\"\nActually this is not true. If the patient is too ill to stand for a PA view (x-ray beam from behind, detector against the chest), then an AP view is taken, usually with a \"portable\" unit which generally isn't as good as the permanently installed units. Pixel spacing is mainly used to better estimate true size of something on the image (though x-ray magnification precludes precise measurement. A portable AP view has the further drawback of (in general) no anti-scatter grid in front of the detector (too much risk of grid cut-off from inconsistent geometry, i.e. very hard to guarantee the plate is perpendicular to the diverging x-ray beam and the beam is centered and at the correct distance from the plate). And finally, when getting a PA view, a lateral view is almost always obtained, which marginally increases sensitivity for detection of disease and helps with localization of disease. In the portable setting, a lateral view is almost never obtained. I suspect, in the interest of not complicating things too much, only frontal views (PA and AP) where obtained for this corpus of cases.\n\n-Dan", "378290": "Dan, thank you very much for such a detailed comment. Agree that View Position and Pixel Spacing are not appropriate for real world prediction.\n\nAs for why only forntal view present in the dataset, I suppose you already answered - \"when getting a PA view, a lateral view is almost always obtained, .... In the portable setting, a lateral view is almost never obtained\"\n\nI have a question, when a radiologist examines a chest x-ray image, does he/she pay attention to view position? Does it change anything? For example, I see something on the image, but maybe this is because of AP position, I need to check other/more factors.", "378628": "Alex, most radiologists do make a note of AP vs PA, but mostly to be aware that there is generally lower quality and more variation of positioning in an AP view. The two most important things to keep in mind are 1) that the heart is magnified more on the AP view, so the standard of what constitutes cardiac enlargement differ from AP to PA; and 2) PA images are virtually always obtained in the upright (standing or sitting) position, while an AP image (often done at the bedside) may be supine (lying down), sitting up, or anywhere in between. Lung physiology is dependent on position (gravitational effects) and this is taken into account when evaluating for pathology (e.g. upper lung vessel distention may be normal withe the patient supine but abnormal when sitting up).\nMy guess is, other than a bias towards sicker patients for AP studies, none of these other considerations are going to be important discriminators for training a model. For one thing, the images are downsampled to 1024x1024, when the originals were more like 4096x2048, so loss of fine structure detail in lungs (which can be important in diagnosing viral pneumonia) will make PA and AP views more equivalent than at original resolution.\nI haven't done it, but it should be easy enough to determine if the AP views are indeed biased to pneumonia by calculating the ratio of pneumonia to non-pneumonia cases in the corpus for AP vs PA images.", "378655": "Thanks for reply, Dan. I am asking about position to decide if AP images should be treated in some special way, maybe create separate model, not to use position directly as a feature.\nRegarding bias, AP has 37.5% with pneumonia, while PA only 9%. \nSo I guess that some models may use tubes and catheters as sign of pneumonia." }, "source": "meta" }