Hi,
\nIn order to properly compare CT scans, we need to resample the DICOM images, putting images from all patients in the same coordinates, and apply appropriate padding before feeding the 3D images to a Neural Network.
\nEvery CT scan has N slices, each slice with width and height. Resampling width and height is trivially done using PixelSpacing metadata.
\nIn order to resample along the Z axis, we need SliceThickness AND SpacingBetweenSlices. We have SliceThickness from all. The problem is SpacingBetweenSlices: this information is critical and is missing in 134 CT scans. (This information clearly explains why some CT scans have so many slices, while others have so few).
\nHere's the list of patients and the missing information.
\nhttps://www.dropbox.com/s/jmjj93dxnwcbvme/missing_data.csv?dl=0
The alternative would be to assume every patients' lungs have exactly the same size in mm along the Z axis, which would be a very wrong approximation.
\nCan someone help? @wcukierski @ahmedhshahin ?
\nThanks!
", "messages": [ { "id": "973968", "postDate": "08/17/2020 16:28:59", "content": "Hi,
\nIn order to properly compare CT scans, we need to resample the DICOM images, putting images from all patients in the same coordinates, and apply appropriate padding before feeding the 3D images to a Neural Network.
\nEvery CT scan has N slices, each slice with width and height. Resampling width and height is trivially done using PixelSpacing metadata.
\nIn order to resample along the Z axis, we need SliceThickness AND SpacingBetweenSlices. We have SliceThickness from all. The problem is SpacingBetweenSlices: this information is critical and is missing in 134 CT scans. (This information clearly explains why some CT scans have so many slices, while others have so few).
\nHere's the list of patients and the missing information.
\nhttps://www.dropbox.com/s/jmjj93dxnwcbvme/missing_data.csv?dl=0
The alternative would be to assume every patients' lungs have exactly the same size in mm along the Z axis, which would be a very wrong approximation.
\nCan someone help? @wcukierski @ahmedhshahin ?
\nThanks!
", "rawMarkdown": "Hi,\n\nIn order to properly compare CT scans, we need to resample the DICOM images, putting images from all patients in the same coordinates, and apply appropriate padding before feeding the 3D images to a Neural Network.\n\nEvery CT scan has N slices, each slice with width and height. Resampling width and height is trivially done using PixelSpacing metadata.\n\nIn order to resample along the Z axis, we need SliceThickness **AND SpacingBetweenSlices**. We have SliceThickness from all. **The problem is SpacingBetweenSlices: this information is critical and is missing in 134 CT scans**. (This information clearly explains why some CT scans have so many slices, while others have so few).\n\nHere's the list of patients and the missing information.\nhttps://www.dropbox.com/s/jmjj93dxnwcbvme/missing_data.csv?dl=0\n\nThe alternative would be to assume every patients' lungs have exactly the same size in mm along the Z axis, which would be a very wrong approximation.\n\nCan someone help? @wcukierski @ahmedhshahin ?\n\nThanks!", "votes": null }, { "id": "974023", "postDate": "08/17/2020 17:09:17", "content": "I spent today and yesterday looking at this. The critical information here is the last element of the ImagePositionPatient metadata. That tells you the z-axis (feet to head) location of the image with respect to the patient. I've chosen to ignore a similar piece of metadata, SliceLocation, because those coordinates are note explicitly in the frame of reference of the patient (I think it's in the CT machine's frame).
", "rawMarkdown": "I spent today and yesterday looking at this. The critical information here is the last element of the ImagePositionPatient metadata. That tells you the z-axis (feet to head) location of the image with respect to the patient. I've chosen to ignore a similar piece of metadata, SliceLocation, because those coordinates are note explicitly in the frame of reference of the patient (I think it's in the CT machine's frame).", "votes": null }, { "id": "974038", "postDate": "08/17/2020 17:23:00", "content": "Thanks @samlin20 , you are completely right… Just found a discussion on Stackoverflow, saying that Spacing Between Slices is not reliable: it's better to use Image Position Patient.
\nGet distance between slices in DICOM
\nThanks!
", "rawMarkdown": "Thanks @samlin20 , you are completely right... Just found a discussion on Stackoverflow, saying that Spacing Between Slices is not reliable: it's better to use Image Position Patient.\n\n[Get distance between slices in DICOM](https://stackoverflow.com/questions/37730772/get-distance-between-slices-in-dicom)\n\nThanks!", "votes": null }, { "id": "974066", "postDate": "08/17/2020 17:46:12", "content": "Hi Carlos,
\nI am wondering if @samlin20 's comment fixes the problem. Regarding the ImagePositionPatient, please check my comment here \"on your post too :)\"
\nhttps://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/165856#948302
\nThanks for your efforts and best of luck!
Thanks @ahmedhshahin, @samlin20 's answer is correct! Btw, just realized: we don't need slice thickness at all, the only thing that matters to reconstruct comparable 3D images from the CT scans is the mean space between slices (yes, mean, because there are some CT scans with eventually one or another slice missing, which is totally fine).
\nThanks!
Yes, by quickly reading your comments and without getting into much detail, I think I can agree with you.
", "rawMarkdown": "Yes, by quickly reading your comments and without getting into much detail, I think I can agree with you.", "votes": null }, { "id": "974147", "postDate": "08/17/2020 18:43:32", "content": "Well, slice thickness may still be useful because it defines what thickness the image describes. From my analyses, most CT scans have a slice thickness that is (perhaps exactly) 1/5 of the typical distance between slices.
", "rawMarkdown": "Well, slice thickness may still be useful because it defines what thickness the image describes. From my analyses, most CT scans have a slice thickness that is (perhaps exactly) 1/5 of the typical distance between slices.", "votes": null }, { "id": "974223", "postDate": "08/17/2020 20:01:09", "content": "Wait! @ahmedhshahin , CT scans ID00132637202222178761324 and ID00128637202219474716089 don't have neither ImagePositionPatient nor SpacingBetweenSlices.
\n2 questions:
\nThanks!
", "rawMarkdown": "Wait! @ahmedhshahin , CT scans ID00132637202222178761324 and ID00128637202219474716089 don't have **neither ImagePositionPatient nor SpacingBetweenSlices**.\n\n2 questions:\n- What should we do in these cases? As there are only 2, I can assume their lungs' size in Z axis is the average of the group... but I'd like your pov\n- In the private set, should we expect to find other CT scans without neither of these attributes? (That's the important question)\n\nThanks!", "votes": null }, { "id": "974317", "postDate": "08/17/2020 22:08:33", "content": "ID00132637202222178761324 - 407 images. Evenly spaced. Presumably 0.625 mm or so.
\nID00128637202219474716089 - has 48 images. If you look at the images, you'll see that there are two images close together, then a skip before the next two images. So they are not equally spaced. I would guess each image is about 1.25 mm. and each pair of images are spaced 12.5 cm apart. So, 1.25 mm, 1.25 mm then a gap of 10 mm, then it repeats. All conjecture. Clinically, doing a thin slice every 10 mm is not unusual. I'm not used to seeing 2 slices every 10 mm.
", "rawMarkdown": "ID00132637202222178761324 - 407 images. Evenly spaced. Presumably 0.625 mm or so.\n\nID00128637202219474716089 - has 48 images. If you look at the images, you'll see that there are two images close together, then a skip before the next two images. So they are not equally spaced. I would guess each image is about 1.25 mm. and each pair of images are spaced 12.5 cm apart. So, 1.25 mm, 1.25 mm then a gap of 10 mm, then it repeats. All conjecture. Clinically, doing a thin slice every 10 mm is not unusual. I'm not used to seeing 2 slices every 10 mm.", "votes": null }, { "id": "974320", "postDate": "08/17/2020 22:10:24", "content": "perfect, I eyeballed and guessed 0.7mm in the first case and 5mm in the 2nd case, very close to your guesses :)
\ngood enough!
\nCheers!
", "rawMarkdown": "perfect, I eyeballed and guessed 0.7mm in the first case and 5mm in the 2nd case, very close to your guesses :)\n\ngood enough!\n\nCheers!", "votes": null }, { "id": "974357", "postDate": "08/17/2020 23:39:14", "content": "CT scans typically have fixed minimum sizes based on both the technology and routine protocols. Slice thickness is usually a multiple of that minimum size.
\nSome scanners can scan at 0.5 mm increments (so expect 0.5 mm, 1.0 mm, 1.5 mm, etc). Others at 0.625 mm (so expect 0.625 mm, 1.25 mm (very popular), 2.5 mm, 3.75 mm, 5.0 mm, etc).
", "rawMarkdown": "CT scans typically have fixed minimum sizes based on both the technology and routine protocols. Slice thickness is usually a multiple of that minimum size.\n\nSome scanners can scan at 0.5 mm increments (so expect 0.5 mm, 1.0 mm, 1.5 mm, etc). Others at 0.625 mm (so expect 0.625 mm, 1.25 mm (very popular), 2.5 mm, 3.75 mm, 5.0 mm, etc).", "votes": null }, { "id": "975522", "postDate": "08/18/2020 10:34:13", "content": "Hi, I've had a comment about these two or three scans here: https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/165856#948302
\nregarding your second question, ImagePositionPatient and SliceThickness are available in all slices and patients in the test set, hence there are no problems with the models.
@carlossouza is you current score using image features ?
\nI was wondering in what way images can be useful they are static ones not taken over weeks ?
I spent today and yesterday looking at this. The critical information here is the last element of the ImagePositionPatient metadata. That tells you the z-axis (feet to head) location of the image with respect to the patient. I've chosen to ignore a similar piece of metadata, SliceLocation, because those coordinates are note explicitly in the frame of reference of the patient (I think it's in the CT machine's frame).
", "votes": null, "replies": [] }, { "id": 974038, "author_name": "carlossouza", "author_url": "", "post_date": "08/17/2020 17:23:00", "content": "Thanks @samlin20 , you are completely right… Just found a discussion on Stackoverflow, saying that Spacing Between Slices is not reliable: it's better to use Image Position Patient.
\nGet distance between slices in DICOM
\nThanks!
", "votes": null, "replies": [] }, { "id": 974066, "author_name": "ahmedhshahin", "author_url": "", "post_date": "08/17/2020 17:46:12", "content": "Hi Carlos,
\nI am wondering if @samlin20 's comment fixes the problem. Regarding the ImagePositionPatient, please check my comment here \"on your post too :)\"
\nhttps://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/165856#948302
\nThanks for your efforts and best of luck!
Thanks @ahmedhshahin, @samlin20 's answer is correct! Btw, just realized: we don't need slice thickness at all, the only thing that matters to reconstruct comparable 3D images from the CT scans is the mean space between slices (yes, mean, because there are some CT scans with eventually one or another slice missing, which is totally fine).
\nThanks!
Well, slice thickness may still be useful because it defines what thickness the image describes. From my analyses, most CT scans have a slice thickness that is (perhaps exactly) 1/5 of the typical distance between slices.
", "votes": null, "replies": [] }, { "id": 977033, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/19/2020 08:48:48", "content": "@carlossouza is you current score using image features ?
\nI was wondering in what way images can be useful they are static ones not taken over weeks ?
Yes, by quickly reading your comments and without getting into much detail, I think I can agree with you.
", "votes": null, "replies": [ { "id": 974223, "author_name": "carlossouza", "author_url": "", "post_date": "08/17/2020 20:01:09", "content": "Wait! @ahmedhshahin , CT scans ID00132637202222178761324 and ID00128637202219474716089 don't have neither ImagePositionPatient nor SpacingBetweenSlices.
\n2 questions:
\nThanks!
", "votes": null, "replies": [ { "id": 974317, "author_name": "richardepstein", "author_url": "", "post_date": "08/17/2020 22:08:33", "content": "ID00132637202222178761324 - 407 images. Evenly spaced. Presumably 0.625 mm or so.
\nID00128637202219474716089 - has 48 images. If you look at the images, you'll see that there are two images close together, then a skip before the next two images. So they are not equally spaced. I would guess each image is about 1.25 mm. and each pair of images are spaced 12.5 cm apart. So, 1.25 mm, 1.25 mm then a gap of 10 mm, then it repeats. All conjecture. Clinically, doing a thin slice every 10 mm is not unusual. I'm not used to seeing 2 slices every 10 mm.
", "votes": null, "replies": [ { "id": 974320, "author_name": "carlossouza", "author_url": "", "post_date": "08/17/2020 22:10:24", "content": "perfect, I eyeballed and guessed 0.7mm in the first case and 5mm in the 2nd case, very close to your guesses :)
\ngood enough!
\nCheers!
", "votes": null, "replies": [ { "id": 974357, "author_name": "richardepstein", "author_url": "", "post_date": "08/17/2020 23:39:14", "content": "CT scans typically have fixed minimum sizes based on both the technology and routine protocols. Slice thickness is usually a multiple of that minimum size.
\nSome scanners can scan at 0.5 mm increments (so expect 0.5 mm, 1.0 mm, 1.5 mm, etc). Others at 0.625 mm (so expect 0.625 mm, 1.25 mm (very popular), 2.5 mm, 3.75 mm, 5.0 mm, etc).
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 975522, "author_name": "ahmedhshahin", "author_url": "", "post_date": "08/18/2020 10:34:13", "content": "Hi, I've had a comment about these two or three scans here: https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/165856#948302
\nregarding your second question, ImagePositionPatient and SliceThickness are available in all slices and patients in the test set, hence there are no problems with the models.