On Thursday 1/7, I will host an offline Q&A with Michael Hansen and Andrew Arai, the researchers who contributed the data and challenge for this second annual Data Science Bowl (bios below).
\n\nIf you'd like to ask the PIs a question, please post in this thread. Feel free to ask questions about the data, the nature of the problem, and anything else that could be useful during the competition. Questions are due by Wednesday, 1/6, 2PM EST.
\n\n\n", "messages": [ { "id": "103324", "postDate": "12/31/2015 20:30:58", "content": "Michael is a biomedical engineer with a PhD from University of Aarhus, Denmark. He works at the National Heart, Lung, and Blood Institute (NHLBI), where he focuses on fast MRI techniques for real-time imaging and interventional procedures. His particular areas of interest are fast pulse sequences, non-Cartesian imaging, real-time reconstruction, GPU based reconstruction, and motion correction. Michael obtained his PhD from Aarhus University, Denmark, on the topic of fast dynamic imaging with special focus on cardiac imaging and image reconstruction. During his training, he spent time at the ETH in Zurich, Philips Research in Hamburg, King’s College London, University College of London, and Great Ormond Street Hospital for Children in London. Before coming to the NIH, he also worked as a laboratory head at Novartis Institutes for BioMedical Research in Cambridge, MA.
\n \nAndrew joined the National Heart, Lung, and Blood Institute (NHLBI) in 1993. He is a Senior Investigator in the Laboratory of Cardiac Energetics in the Institute’s Division of Intramural Research. Andrew studies conditions and diseases that alter the heart’s supply and utilization of energy. His expertise is in the use of magnetic resonance imaging (MRI) to evaluate patients with heart attacks and coronary heart disease. Andrew received his MD from the University of Illinois College of Medicine, Chicago in 1986. He received his BA from Cornell University in Ithaca, NY in 1982. Andrew has authored or co-authored more than 65 papers that have appeared in peer-reviewed journals. He is a member of the National Institutes of Health Nuclear Magnetic Resonance Safety Committee and is on the Editorial Board of the Journal of the American College of Cardiology. Andrew is also is a reviewer for several peer-reviewed journals.
\n
On Thursday 1/7, I will host an offline Q&A with Michael Hansen and Andrew Arai, the researchers who contributed the data and challenge for this second annual Data Science Bowl (bios below).
\n\nIf you'd like to ask the PIs a question, please post in this thread. Feel free to ask questions about the data, the nature of the problem, and anything else that could be useful during the competition. Questions are due by Wednesday, 1/6, 2PM EST.
\n\n\n", "rawMarkdown": "On Thursday 1/7, I will host an offline Q&A with Michael Hansen and Andrew Arai, the researchers who contributed the data and challenge for this second annual Data Science Bowl (bios below). \r\n\r\nIf you'd like to ask the PIs a question, please post in this thread. Feel free to ask questions about the data, the nature of the problem, and anything else that could be useful during the competition. Questions are due **by Wednesday, 1/6, 2PM EST.** \r\n\r\n> Michael is a biomedical engineer with a PhD from University of Aarhus, Denmark. He works at the National Heart, Lung, and Blood Institute (NHLBI), where he focuses on fast MRI techniques for real-time imaging and interventional procedures. His particular areas of interest are fast pulse sequences, non-Cartesian imaging, real-time reconstruction, GPU based reconstruction, and motion correction. Michael obtained his PhD from Aarhus University, Denmark, on the topic of fast dynamic imaging with special focus on cardiac imaging and image reconstruction. During his training, he spent time at the ETH in Zurich, Philips Research in Hamburg, King’s College London, University College of London, and Great Ormond Street Hospital for Children in London. Before coming to the NIH, he also worked as a laboratory head at Novartis Institutes for BioMedical Research in Cambridge, MA.\r\n> \r\n> Andrew joined the National Heart, Lung, and Blood Institute (NHLBI) in 1993. He is a Senior Investigator in the Laboratory of Cardiac Energetics in the Institute’s Division of Intramural Research. Andrew studies conditions and diseases that alter the heart’s supply and utilization of energy. His expertise is in the use of magnetic resonance imaging (MRI) to evaluate patients with heart attacks and coronary heart disease. Andrew received his MD from the University of Illinois College of Medicine, Chicago in 1986. He received his BA from Cornell University in Ithaca, NY in 1982. Andrew has authored or co-authored more than 65 papers that have appeared in peer-reviewed journals. He is a member of the National Institutes of Health Nuclear Magnetic Resonance Safety Committee and is on the Editorial Board of the Journal of the American College of Cardiology. Andrew is also is a reviewer for several peer-reviewed journals.", "votes": null }, { "id": "103356", "postDate": "01/01/2016 14:55:10", "content": "Michael is a biomedical engineer with a PhD from University of Aarhus, Denmark. He works at the National Heart, Lung, and Blood Institute (NHLBI), where he focuses on fast MRI techniques for real-time imaging and interventional procedures. His particular areas of interest are fast pulse sequences, non-Cartesian imaging, real-time reconstruction, GPU based reconstruction, and motion correction. Michael obtained his PhD from Aarhus University, Denmark, on the topic of fast dynamic imaging with special focus on cardiac imaging and image reconstruction. During his training, he spent time at the ETH in Zurich, Philips Research in Hamburg, King’s College London, University College of London, and Great Ormond Street Hospital for Children in London. Before coming to the NIH, he also worked as a laboratory head at Novartis Institutes for BioMedical Research in Cambridge, MA.
\n \nAndrew joined the National Heart, Lung, and Blood Institute (NHLBI) in 1993. He is a Senior Investigator in the Laboratory of Cardiac Energetics in the Institute’s Division of Intramural Research. Andrew studies conditions and diseases that alter the heart’s supply and utilization of energy. His expertise is in the use of magnetic resonance imaging (MRI) to evaluate patients with heart attacks and coronary heart disease. Andrew received his MD from the University of Illinois College of Medicine, Chicago in 1986. He received his BA from Cornell University in Ithaca, NY in 1982. Andrew has authored or co-authored more than 65 papers that have appeared in peer-reviewed journals. He is a member of the National Institutes of Health Nuclear Magnetic Resonance Safety Committee and is on the Editorial Board of the Journal of the American College of Cardiology. Andrew is also is a reviewer for several peer-reviewed journals.
\n
Hello ! Great to have this kind of Q&A :)
\n\nI have some questions about how the ground truth was calculated. It was said that it is based on manual / semi-automatic segmentation of the heart by experts:
\n\nFor each case how many experts segmentation did you use to compute the ground truth volume? Only one or do you try to get a consensus by using several experts segmentation?
How many different experts contributed to the challenge? Were the same experts used to segment the validation and test data?
How was chosen the minimum and maximum volume? Is it the min/max of the volume along the whole sequence or do you restrict the frame used? For example it is possible the volume after the contraction to be larger than the initial volume, yet when calculation an ejection fraction it would maybe be more relevant to take the maximum volume before the contraction?
Is the method we develop supposed to be 100% automatic for both the train and test set processing? Or can we do some manual adjustment with the train set when building the algorithm?
First, thanks for providing this data set. I'm really enjoying working with it. Here's a couple of questions:
\n\nWhat was the motivation for choosing to evaluate the models based on predictions of systolic and diastolic volumes rather than on ejection fraction (EF)? My, admittedly shallow, understanding is that EF is the clinically relevant number and I'm pretty certain that it would be faster and more accurate to predict EF directly than to predict the individual volumes. At least that seems likely with the methods that I am using.
What scores would correspond to a model that would be useful in a clinical setting? Right now scores seem to be saturating at around 0.02, but I suspect that there are still some significant improvements to be found (not that I know what those are!).
Hi,
\n\nThere are bunch of metadata attributes (around 90) whose meanings are not very clear. If we knew the meanings, we can use them properly in our models. Please let us know the meanings of the below items, which are metadata in DICOM images.
\n\n@Udaya Indeed, DICOM files come with a lot of metadata. Some of these fields were anonymized and will therefore not be useful (e.g. PatientBirthDate). Some should be self explanatory (e.g. PatientSex). Most are technical acquisition parameters or unrelated information that will not be useful for this task.
\n\nRather than asking Dr. Hansen to rewrite the manual on every DICOM field, may I suggest that you Google those that are confusing or that you expect to be useful? Searching for "FieldName dicom field" will get you what you need.
", "rawMarkdown": "Udaya Indeed, DICOM files come with a lot of metadata. Some of these fields were anonymized and will therefore not be useful (e.g. PatientBirthDate). Some should be self explanatory (e.g. PatientSex). Most are technical acquisition parameters or unrelated information that will not be useful for this task.\r\n\r\nRather than asking Dr. Hansen to rewrite the manual on every DICOM field, may I suggest that you Google those that are confusing or that you expect to be useful? Searching for \"FieldName dicom field\" will get you what you need.", "votes": null }, { "id": "103886", "postDate": "01/07/2016 15:24:10", "content": "How will the results be validated? Do you expect a segmentation output that will be manually validated at some point? In a real world setting this makes sense. On the other hand, if the problem is posed as a regression problem, irrespective of the result of segmentation, if any, a low error can be achieved (fit). I don't claim that posing this as a regression problem is wrong, but in the absence of a segmentation output that can be validated by an expert, I am not sure if the results will be clinically relevant. What is the PI's take on this?
", "rawMarkdown": "How will the results be validated? Do you expect a segmentation output that will be manually validated at some point? In a real world setting this makes sense. On the other hand, if the problem is posed as a regression problem, irrespective of the result of segmentation, if any, a low error can be achieved (fit). I don't claim that posing this as a regression problem is wrong, but in the absence of a segmentation output that can be validated by an expert, I am not sure if the results will be clinically relevant. What is the PI's take on this?", "votes": null }, { "id": "104034", "postDate": "01/08/2016 17:35:14", "content": "@IAsIam - Any method that results from this competition will go through additional research and prospective trials, and the workflow will be worked out. And, any method that wins is likely to have some form of embedded segmentation in it.
", "rawMarkdown": "IAsIam - Any method that results from this competition will go through additional research and prospective trials, and the workflow will be worked out. And, any method that wins is likely to have some form of embedded segmentation in it.", "votes": null }, { "id": "104088", "postDate": "01/09/2016 00:49:17", "content": "@Shannon, Just to be sure, is a segmentation output required as part of the solution or not?
", "rawMarkdown": "Shannon, Just to be sure, is a segmentation output required as part of the solution or not?", "votes": null } ], "comments": [ { "id": 103356, "author_name": "", "author_url": "", "post_date": "01/01/2016 14:55:10", "content": "Hello ! Great to have this kind of Q&A :)
\n\nI have some questions about how the ground truth was calculated. It was said that it is based on manual / semi-automatic segmentation of the heart by experts:
\n\nFor each case how many experts segmentation did you use to compute the ground truth volume? Only one or do you try to get a consensus by using several experts segmentation?
How many different experts contributed to the challenge? Were the same experts used to segment the validation and test data?
How was chosen the minimum and maximum volume? Is it the min/max of the volume along the whole sequence or do you restrict the frame used? For example it is possible the volume after the contraction to be larger than the initial volume, yet when calculation an ejection fraction it would maybe be more relevant to take the maximum volume before the contraction?
Is the method we develop supposed to be 100% automatic for both the train and test set processing? Or can we do some manual adjustment with the train set when building the algorithm?
First, thanks for providing this data set. I'm really enjoying working with it. Here's a couple of questions:
\n\nWhat was the motivation for choosing to evaluate the models based on predictions of systolic and diastolic volumes rather than on ejection fraction (EF)? My, admittedly shallow, understanding is that EF is the clinically relevant number and I'm pretty certain that it would be faster and more accurate to predict EF directly than to predict the individual volumes. At least that seems likely with the methods that I am using.
What scores would correspond to a model that would be useful in a clinical setting? Right now scores seem to be saturating at around 0.02, but I suspect that there are still some significant improvements to be found (not that I know what those are!).
Hi,
\n\nThere are bunch of metadata attributes (around 90) whose meanings are not very clear. If we knew the meanings, we can use them properly in our models. Please let us know the meanings of the below items, which are metadata in DICOM images.
\n\n@Udaya Indeed, DICOM files come with a lot of metadata. Some of these fields were anonymized and will therefore not be useful (e.g. PatientBirthDate). Some should be self explanatory (e.g. PatientSex). Most are technical acquisition parameters or unrelated information that will not be useful for this task.
\n\nRather than asking Dr. Hansen to rewrite the manual on every DICOM field, may I suggest that you Google those that are confusing or that you expect to be useful? Searching for "FieldName dicom field" will get you what you need.
", "votes": null, "replies": [] }, { "id": 103886, "author_name": "iasiam", "author_url": "", "post_date": "01/07/2016 15:24:10", "content": "How will the results be validated? Do you expect a segmentation output that will be manually validated at some point? In a real world setting this makes sense. On the other hand, if the problem is posed as a regression problem, irrespective of the result of segmentation, if any, a low error can be achieved (fit). I don't claim that posing this as a regression problem is wrong, but in the absence of a segmentation output that can be validated by an expert, I am not sure if the results will be clinically relevant. What is the PI's take on this?
", "votes": null, "replies": [] }, { "id": 104034, "author_name": "shannonlantzy", "author_url": "", "post_date": "01/08/2016 17:35:14", "content": "@IAsIam - Any method that results from this competition will go through additional research and prospective trials, and the workflow will be worked out. And, any method that wins is likely to have some form of embedded segmentation in it.
", "votes": null, "replies": [] }, { "id": 104088, "author_name": "iasiam", "author_url": "", "post_date": "01/09/2016 00:49:17", "content": "@Shannon, Just to be sure, is a segmentation output required as part of the solution or not?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "103324": "On Thursday 1/7, I will host an offline Q&A with Michael Hansen and Andrew Arai, the researchers who contributed the data and challenge for this second annual Data Science Bowl (bios below). \r\n\r\nIf you'd like to ask the PIs a question, please post in this thread. Feel free to ask questions about the data, the nature of the problem, and anything else that could be useful during the competition. Questions are due **by Wednesday, 1/6, 2PM EST.** \r\n\r\n> Michael is a biomedical engineer with a PhD from University of Aarhus, Denmark. He works at the National Heart, Lung, and Blood Institute (NHLBI), where he focuses on fast MRI techniques for real-time imaging and interventional procedures. His particular areas of interest are fast pulse sequences, non-Cartesian imaging, real-time reconstruction, GPU based reconstruction, and motion correction. Michael obtained his PhD from Aarhus University, Denmark, on the topic of fast dynamic imaging with special focus on cardiac imaging and image reconstruction. During his training, he spent time at the ETH in Zurich, Philips Research in Hamburg, King’s College London, University College of London, and Great Ormond Street Hospital for Children in London. Before coming to the NIH, he also worked as a laboratory head at Novartis Institutes for BioMedical Research in Cambridge, MA.\r\n> \r\n> Andrew joined the National Heart, Lung, and Blood Institute (NHLBI) in 1993. He is a Senior Investigator in the Laboratory of Cardiac Energetics in the Institute’s Division of Intramural Research. Andrew studies conditions and diseases that alter the heart’s supply and utilization of energy. His expertise is in the use of magnetic resonance imaging (MRI) to evaluate patients with heart attacks and coronary heart disease. Andrew received his MD from the University of Illinois College of Medicine, Chicago in 1986. He received his BA from Cornell University in Ithaca, NY in 1982. Andrew has authored or co-authored more than 65 papers that have appeared in peer-reviewed journals. He is a member of the National Institutes of Health Nuclear Magnetic Resonance Safety Committee and is on the Editorial Board of the Journal of the American College of Cardiology. Andrew is also is a reviewer for several peer-reviewed journals.", "103356": "Hello ! Great to have this kind of Q&A :)\r\n\r\nI have some questions about how the ground truth was calculated. It was said that it is based on manual / semi-automatic segmentation of the heart by experts:\r\n\r\n- For each case how many experts segmentation did you use to compute the ground truth volume? Only one or do you try to get a consensus by using several experts segmentation?\r\n\r\n- How many different experts contributed to the challenge? Were the same experts used to segment the validation and test data?\r\n\r\n- How was chosen the minimum and maximum volume? Is it the min/max of the volume along the whole sequence or do you restrict the frame used? For example it is possible the volume after the contraction to be larger than the initial volume, yet when calculation an ejection fraction it would maybe be more relevant to take the maximum volume before the contraction?\r\n\r\n- Is the method we develop supposed to be 100% automatic for both the train and test set processing? Or can we do some manual adjustment with the train set when building the algorithm?", "103508": "First, thanks for providing this data set. I'm really enjoying working with it. Here's a couple of questions:\r\n\r\n* What was the motivation for choosing to evaluate the models based on predictions of systolic and diastolic volumes rather than on ejection fraction (EF)? My, admittedly shallow, understanding is that EF is the clinically relevant number and I'm pretty certain that it would be faster and more accurate to predict EF directly than to predict the individual volumes. At least that seems likely with the methods that I am using.\r\n\r\n* What scores would correspond to a model that would be useful in a clinical setting? Right now scores seem to be saturating at around 0.02, but I suspect that there are still some significant improvements to be found (not that I know what those are!).", "103647": "Hi,\r\n\r\nThere are bunch of metadata attributes (around 90) whose meanings are not very clear. If we knew the meanings, we can use them properly in our models. Please let us know the meanings of the below items, which are metadata in DICOM images.\r\n\r\n - PhotometricInterpretation\r\n - Rows\r\n - WindowCenter\r\n - InPlanePhaseEncodingDirection\r\n - SeriesTime\r\n - InstanceNumber\r\n - NominalInterval\r\n - PatientSex\r\n - SOPInstanceUID\r\n - AcquisitionMatrix\r\n - ReferencedImageSequence\r\n - PositionReferenceIndicator\r\n - SpecificCharacterSet\r\n - TransmitCoilName\r\n - Columns\r\n - PixelSpacing\r\n - RepetitionTime\r\n - EchoTrainLength\r\n - PercentPhaseFieldOfView\r\n - SeriesNumber\r\n - ImagedNucleus\r\n - MRAcquisitionType\r\n - NumberOfFrames\r\n - HighBit\r\n - AcquisitionNumber\r\n - StudyTime\r\n - SmallestImagePixelValue\r\n - SoftwareVersions\r\n - BodyPartExamined\r\n - PatientBirthDate\r\n - PatientAge\r\n - PixelBandwidth\r\n - BitsStored\r\n - RefdPatientSequence\r\n - SAR\r\n - AngioFlag\r\n - ImagePositionPatient\r\n - ScanningSequence\r\n - Manufacturer\r\n - WindowWidth\r\n - ReferencedStudySequence\r\n - SliceLocation\r\n - FlipAngle\r\n - VariableFlipAngleFlag\r\n - LargestImagePixelValue\r\n - CommentsOnThePerformedProcedureStep\r\n - InstanceCreationTime\r\n - RefdImageSequence\r\n - ManufacturerModelName\r\n - NumberOfAverages\r\n - AcquisitionTime\r\n - SequenceName\r\n - EchoNumbers\r\n - PercentSampling\r\n - SpacingBetweenSlices\r\n - PatientName\r\n - ImageType\r\n - SOPClassUID\r\n - EchoTime\r\n - SeriesDescription\r\n - ReferencedPatientSequence\r\n - SequenceVariant\r\n - BitsAllocated\r\n - PerformedProcedureStepStartTime\r\n - ImagingFrequency\r\n - Modality\r\n - PatientID\r\n - SamplesPerPixel\r\n - ScanOptions\r\n - NumberOfPhaseEncodingSteps\r\n - PerformedProcedureStepID\r\n - MagneticFieldStrength\r\n - TriggerTime\r\n - RefdStudySequence\r\n - PatientPosition\r\n - ImageOrientationPatient\r\n - CardiacNumberOfImages\r\n - dBdt\r\n - PixelRepresentation\r\n - SliceThickness\r\n - PixelData\r\n - WindowCenterWidthExplanation\r\n - PatientAddress\r\n - PatientTelephoneNumbers\r\n - ContrastBolusTotalDose\r\n - ContrastBolusAgent\r\n - ContrastBolusIngredient\r\n - ContrastBolusVolume\r\n - ContrastBolusIngredientConcentration\r\n - ContrastFlowDuration", "103798": "Udaya Indeed, DICOM files come with a lot of metadata. Some of these fields were anonymized and will therefore not be useful (e.g. PatientBirthDate). Some should be self explanatory (e.g. PatientSex). Most are technical acquisition parameters or unrelated information that will not be useful for this task.\r\n\r\nRather than asking Dr. Hansen to rewrite the manual on every DICOM field, may I suggest that you Google those that are confusing or that you expect to be useful? Searching for \"FieldName dicom field\" will get you what you need.", "103886": "How will the results be validated? Do you expect a segmentation output that will be manually validated at some point? In a real world setting this makes sense. On the other hand, if the problem is posed as a regression problem, irrespective of the result of segmentation, if any, a low error can be achieved (fit). I don't claim that posing this as a regression problem is wrong, but in the absence of a segmentation output that can be validated by an expert, I am not sure if the results will be clinically relevant. What is the PI's take on this?", "104034": "IAsIam - Any method that results from this competition will go through additional research and prospective trials, and the workflow will be worked out. And, any method that wins is likely to have some form of embedded segmentation in it.", "104088": "Shannon, Just to be sure, is a segmentation output required as part of the solution or not?" }, "source": "meta" }