Hi all,
\n\nIs there a resource [aside from the video] to ensure that we are segmenting the LV properly? I've been hand-annotating several patient files and my calculations get me 20-40% smaller volumes than are recorded.
\n\nI'm concerned that this competition will be less about finding the volumes of the LV and more about finding what other people believe the volumes of the LV to be...
", "messages": [ { "id": "103202", "postDate": "12/30/2015 06:02:11", "content": "Hi all,
\n\nIs there a resource [aside from the video] to ensure that we are segmenting the LV properly? I've been hand-annotating several patient files and my calculations get me 20-40% smaller volumes than are recorded.
\n\nI'm concerned that this competition will be less about finding the volumes of the LV and more about finding what other people believe the volumes of the LV to be...
", "rawMarkdown": "Hi all,\r\n\r\nIs there a resource [aside from the video] to ensure that we are segmenting the LV properly? I've been hand-annotating several patient files and my calculations get me 20-40% smaller volumes than are recorded.\r\n\r\nI'm concerned that this competition will be less about finding the volumes of the LV and more about finding what other people believe the volumes of the LV to be...", "votes": null }, { "id": "103232", "postDate": "12/30/2015 13:31:45", "content": "Let me see if I can add some comments here. Let me start from the back.
\n\nYou say that you are "concerned that this competition will be less about finding the volumes of the LV and more about finding what other people believe the volumes of the LV to be...".
\n\nI would say that this is exactly what this competition is about. There are a number of board certified cardiologist that have manually analyzed the images and thus given their interpretation of what the volumes of the LV are at given points in time. If two cardiologists analyze the same images, there are variations (on the order of 10% is not totally unusual) due to the fact that it is a manual process and judgement calls are made as to what to include in the volumes. The point here is that it is not random "other people" of the street whose opinion of the volumes you are trying to find, it is the opinion of experts in the field. You could say if there was a simple way to automate this and get the "right" answer without some professional applying their expertise and experience, there would really be no need for this sort of competition. The point is that it is hard to do right, it takes skill and practice, and it has been difficult to teach computers to do the segmentations right.
\n\nI will also add that we have data from multiple institutions with the analysis done by multiple cardiologists, so we are hoping that will help to balance the data in terms of inter-observer variability.
\n\nNow to your concrete observations that you are underestimating the volumes. I think this is pretty common. The contours should follow the ventricle edges pretty tightly and automated segmentation algorithms tend to underestimate the volumes because of blurring along the edges. It is also common to exclude slices in the base part of the heart (near the valves) that actually belong in the LV. Since these slices are large, they can lead to large errors.
\n\nI know that you were really looking for the quick guide to doing cardiac segmentation right, but I know of no such guide, sorry.
\n\nHope this helps,\nMichael
", "rawMarkdown": "Let me see if I can add some comments here. Let me start from the back.\r\n\r\nYou say that you are \"concerned that this competition will be less about finding the volumes of the LV and more about finding what other people believe the volumes of the LV to be...\".\r\n\r\nI would say that this is exactly what this competition is about. There are a number of board certified cardiologist that have manually analyzed the images and thus given their interpretation of what the volumes of the LV are at given points in time. If two cardiologists analyze the same images, there are variations (on the order of 10% is not totally unusual) due to the fact that it is a manual process and judgement calls are made as to what to include in the volumes. The point here is that it is not random \"other people\" of the street whose opinion of the volumes you are trying to find, it is the opinion of experts in the field. You could say if there was a simple way to automate this and get the \"right\" answer without some professional applying their expertise and experience, there would really be no need for this sort of competition. The point is that it is hard to do right, it takes skill and practice, and it has been difficult to teach computers to do the segmentations right.\r\n\r\nI will also add that we have data from multiple institutions with the analysis done by multiple cardiologists, so we are hoping that will help to balance the data in terms of inter-observer variability. \r\n\r\nNow to your concrete observations that you are underestimating the volumes. I think this is pretty common. The contours should follow the ventricle edges pretty tightly and automated segmentation algorithms tend to underestimate the volumes because of blurring along the edges. It is also common to exclude slices in the base part of the heart (near the valves) that actually belong in the LV. Since these slices are large, they can lead to large errors. \r\n\r\nI know that you were really looking for the quick guide to doing cardiac segmentation right, but I know of no such guide, sorry.\r\n\r\nHope this helps,\r\nMichael", "votes": null }, { "id": "103248", "postDate": "12/30/2015 18:19:23", "content": "@adleberg
\n\nI'm curious if your manually measured results are more accurate in calculating ejection fraction (EF) value than they are in calculating the absolute LV volumes. I noticed that it is hard to accurately segment the very top and bottom slices of LV. These are the areas where cardiologists apply their individually biased guesswork and intuition. I made Evaluation formula modification post with this observation.
", "rawMarkdown": "adleberg \r\n\r\nI'm curious if your manually measured results are more accurate in calculating ejection fraction (EF) value than they are in calculating the absolute LV volumes. I noticed that it is hard to accurately segment the very top and bottom slices of LV. These are the areas where cardiologists apply their individually biased guesswork and intuition. I made [Evaluation formula modification][1] post with this observation.\r\n\r\n\r\n [1]: https://www.kaggle.com/c/second-annual-data-science-bowl/forums/t/17959/evaluation-formula-modification/101702#post101702 \"this post\"", "votes": null } ], "comments": [ { "id": 103232, "author_name": "michaelhansen", "author_url": "", "post_date": "12/30/2015 13:31:45", "content": "Let me see if I can add some comments here. Let me start from the back.
\n\nYou say that you are "concerned that this competition will be less about finding the volumes of the LV and more about finding what other people believe the volumes of the LV to be...".
\n\nI would say that this is exactly what this competition is about. There are a number of board certified cardiologist that have manually analyzed the images and thus given their interpretation of what the volumes of the LV are at given points in time. If two cardiologists analyze the same images, there are variations (on the order of 10% is not totally unusual) due to the fact that it is a manual process and judgement calls are made as to what to include in the volumes. The point here is that it is not random "other people" of the street whose opinion of the volumes you are trying to find, it is the opinion of experts in the field. You could say if there was a simple way to automate this and get the "right" answer without some professional applying their expertise and experience, there would really be no need for this sort of competition. The point is that it is hard to do right, it takes skill and practice, and it has been difficult to teach computers to do the segmentations right.
\n\nI will also add that we have data from multiple institutions with the analysis done by multiple cardiologists, so we are hoping that will help to balance the data in terms of inter-observer variability.
\n\nNow to your concrete observations that you are underestimating the volumes. I think this is pretty common. The contours should follow the ventricle edges pretty tightly and automated segmentation algorithms tend to underestimate the volumes because of blurring along the edges. It is also common to exclude slices in the base part of the heart (near the valves) that actually belong in the LV. Since these slices are large, they can lead to large errors.
\n\nI know that you were really looking for the quick guide to doing cardiac segmentation right, but I know of no such guide, sorry.
\n\nHope this helps,\nMichael
", "votes": null, "replies": [] }, { "id": 103248, "author_name": "pauljurczak", "author_url": "", "post_date": "12/30/2015 18:19:23", "content": "@adleberg
\n\nI'm curious if your manually measured results are more accurate in calculating ejection fraction (EF) value than they are in calculating the absolute LV volumes. I noticed that it is hard to accurately segment the very top and bottom slices of LV. These are the areas where cardiologists apply their individually biased guesswork and intuition. I made Evaluation formula modification post with this observation.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "103202": "Hi all,\r\n\r\nIs there a resource [aside from the video] to ensure that we are segmenting the LV properly? I've been hand-annotating several patient files and my calculations get me 20-40% smaller volumes than are recorded.\r\n\r\nI'm concerned that this competition will be less about finding the volumes of the LV and more about finding what other people believe the volumes of the LV to be...", "103232": "Let me see if I can add some comments here. Let me start from the back.\r\n\r\nYou say that you are \"concerned that this competition will be less about finding the volumes of the LV and more about finding what other people believe the volumes of the LV to be...\".\r\n\r\nI would say that this is exactly what this competition is about. There are a number of board certified cardiologist that have manually analyzed the images and thus given their interpretation of what the volumes of the LV are at given points in time. If two cardiologists analyze the same images, there are variations (on the order of 10% is not totally unusual) due to the fact that it is a manual process and judgement calls are made as to what to include in the volumes. The point here is that it is not random \"other people\" of the street whose opinion of the volumes you are trying to find, it is the opinion of experts in the field. You could say if there was a simple way to automate this and get the \"right\" answer without some professional applying their expertise and experience, there would really be no need for this sort of competition. The point is that it is hard to do right, it takes skill and practice, and it has been difficult to teach computers to do the segmentations right.\r\n\r\nI will also add that we have data from multiple institutions with the analysis done by multiple cardiologists, so we are hoping that will help to balance the data in terms of inter-observer variability. \r\n\r\nNow to your concrete observations that you are underestimating the volumes. I think this is pretty common. The contours should follow the ventricle edges pretty tightly and automated segmentation algorithms tend to underestimate the volumes because of blurring along the edges. It is also common to exclude slices in the base part of the heart (near the valves) that actually belong in the LV. Since these slices are large, they can lead to large errors. \r\n\r\nI know that you were really looking for the quick guide to doing cardiac segmentation right, but I know of no such guide, sorry.\r\n\r\nHope this helps,\r\nMichael", "103248": "adleberg \r\n\r\nI'm curious if your manually measured results are more accurate in calculating ejection fraction (EF) value than they are in calculating the absolute LV volumes. I noticed that it is hard to accurately segment the very top and bottom slices of LV. These are the areas where cardiologists apply their individually biased guesswork and intuition. I made [Evaluation formula modification][1] post with this observation.\r\n\r\n\r\n [1]: https://www.kaggle.com/c/second-annual-data-science-bowl/forums/t/17959/evaluation-formula-modification/101702#post101702 \"this post\"" }, "source": "meta" }