![\"TD-CNN\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5552115%2F534d58cf5303d09d363a0cc01a42f1f6%2FTDCNN.png?generation=1728574147484838&alt=media\")
Thanks to the organizers for this competition, and my partner Simon Veitner @simonveitner for the help!
\nOur solution was built from a two stage process:
\nThe code is available at https://github.com/phalendj/rsna2024.
\nLooking through the training data, we had the different studies, which were then divided into different series covering the different image types Sagittal T2/STIR, Sagittal T1, and Axial T2. In some studies there were multiple series of a single image type. Within the different image types, there were what I called 'Stacks', which were instances that formed a regular grid of voxels, so all had the same orientation and spacing.
\nWithin a single stack, we can come up with the affine matrix that maps the voxel location to a point in the world space via an Affine matrix that can be constructed from the image orientation and location data in the dicom metadata. We then would use this affine matrix for each stack to translate points from one pixel space into another. To construct the full affine matrix, we followed (https://nipy.org/nibabel/dicom/dicom_orientation.html)[https://nipy.org/nibabel/dicom/dicom_orientation.html]. Let $$(Z_x \\Delta k, Z_y \\Delta k, Z_z \\Delta k) = S_1 - S_0$$ be the difference in the zero voxel points of two instances, then the affine matrix is defined $P = A n$
\n$$ \\left[\\begin{array}{c}
\nP_x \\\\
\nP_y \\\\
\nP_z \\\\
\n1 \\end{array}
\n\\right] =
\n\\left[
\n\\begin{array}{cccc}
\nX_x \\Delta i & Y_x \\Delta j & Z_x \\Delta k & S_x \\\\
\nX_y \\Delta i & Y_y \\Delta j & Z_y \\Delta k & S_y \\\\
\nX_z \\Delta i & Y_z \\Delta j & Z_z \\Delta k & S_z \\\\
\n0 & 0 & 0 & 1 \\\\
\n\\end{array}
\n\\right]
\n\\left[\\begin{array}{c}
\ni \\\\
\nj \\\\
\nk \\\\
\n1 \\end{array}
\n\\right]
\n$$
And conversions from one pixel space to another are done using the matrix $$ A^{\\prime -1} A$$.
\nMany of the vertebrae we found to be mislabelled, usually off by one. We went through a few iterations of finding where our model for keypoints did not match the training data and corrected any training data.
\nFor this we focussed on the Sagittal T2/STIR images in order to find the locations of the vertebrae. First we found the projected instance numbers using a TD-CNN on all of the Sagittal T2/STIR instances of the largest stack,
\n
We then output the probability that each image was the correct image.
\nUsing the same Sagittal T2/STIR images, we used a 2.5D CNN with 15 channel input and timm_efficientnet_b2 in the segmentation-models-pytorch package. We then used Heatmap loss as outlined in https://www.kaggle.com/code/sacuscreed/sagittal-t2-sagittal-level-segmentation by Angel Jacinto Sanchez Ruiz (@sacuscreed).
\nFinally, after taking the training data and translating each label coordinate to the world space, we found the average offset in mm for the foraminal locations and the subarticular locations. We used this to generate a file 'test_label_coordinates.csv' for the next step.
\nGiven the size of the areas we would pull for our diagnosis model, we found that this was good enough to move to the next level.
\nAfter finding locations, we used a 224x224 image for input into a TDCNN, same as outlined above. We would find the center location for a level, then cut an image of size 40-50mm around that location and scale to the input size. We found that the different studies had a wide variety of pixel spacings, so this gave a standard size of the target area. We would then train this with Weighted CrossEntropyLoss. The model we submitted with was densenet121.
For spinal, the size of a batch for training would be (B, 5(Levels), 11(Images), 224, 224), for foraminal narrowing (B, 5(Levels), 2 (Sides), 11(Images), 224, 224), and for subarticular stenosis (B, 5(Levels), 7(Images), 224, 224). For Subarticular Stenosis, we found applying a flip to the right side images gave a better model since everything was in a standard format.
\nAfter all of the training, we did add a master model that had all the same inputs, but joined all outputs for a single level into a large input for a classifier head. We found there was a lot of correllation in the different diagnoses for a single spinal level, but not as much among the different levels.
\nThanks to the organizers for this competition, and my partner Simon Veitner @simonveitner for the help!
\nOur solution was built from a two stage process:
\nThe code is available at https://github.com/phalendj/rsna2024.
\nLooking through the training data, we had the different studies, which were then divided into different series covering the different image types Sagittal T2/STIR, Sagittal T1, and Axial T2. In some studies there were multiple series of a single image type. Within the different image types, there were what I called 'Stacks', which were instances that formed a regular grid of voxels, so all had the same orientation and spacing.
\nWithin a single stack, we can come up with the affine matrix that maps the voxel location to a point in the world space via an Affine matrix that can be constructed from the image orientation and location data in the dicom metadata. We then would use this affine matrix for each stack to translate points from one pixel space into another. To construct the full affine matrix, we followed (https://nipy.org/nibabel/dicom/dicom_orientation.html)[https://nipy.org/nibabel/dicom/dicom_orientation.html]. Let $$(Z_x \\Delta k, Z_y \\Delta k, Z_z \\Delta k) = S_1 - S_0$$ be the difference in the zero voxel points of two instances, then the affine matrix is defined $P = A n$
\n$$ \\left[\\begin{array}{c}
\nP_x \\\\
\nP_y \\\\
\nP_z \\\\
\n1 \\end{array}
\n\\right] =
\n\\left[
\n\\begin{array}{cccc}
\nX_x \\Delta i & Y_x \\Delta j & Z_x \\Delta k & S_x \\\\
\nX_y \\Delta i & Y_y \\Delta j & Z_y \\Delta k & S_y \\\\
\nX_z \\Delta i & Y_z \\Delta j & Z_z \\Delta k & S_z \\\\
\n0 & 0 & 0 & 1 \\\\
\n\\end{array}
\n\\right]
\n\\left[\\begin{array}{c}
\ni \\\\
\nj \\\\
\nk \\\\
\n1 \\end{array}
\n\\right]
\n$$
And conversions from one pixel space to another are done using the matrix $$ A^{\\prime -1} A$$.
\nMany of the vertebrae we found to be mislabelled, usually off by one. We went through a few iterations of finding where our model for keypoints did not match the training data and corrected any training data.
\nFor this we focussed on the Sagittal T2/STIR images in order to find the locations of the vertebrae. First we found the projected instance numbers using a TD-CNN on all of the Sagittal T2/STIR instances of the largest stack,
\n
We then output the probability that each image was the correct image.
\nUsing the same Sagittal T2/STIR images, we used a 2.5D CNN with 15 channel input and timm_efficientnet_b2 in the segmentation-models-pytorch package. We then used Heatmap loss as outlined in https://www.kaggle.com/code/sacuscreed/sagittal-t2-sagittal-level-segmentation by Angel Jacinto Sanchez Ruiz (@sacuscreed).
\nFinally, after taking the training data and translating each label coordinate to the world space, we found the average offset in mm for the foraminal locations and the subarticular locations. We used this to generate a file 'test_label_coordinates.csv' for the next step.
\nGiven the size of the areas we would pull for our diagnosis model, we found that this was good enough to move to the next level.
\nAfter finding locations, we used a 224x224 image for input into a TDCNN, same as outlined above. We would find the center location for a level, then cut an image of size 40-50mm around that location and scale to the input size. We found that the different studies had a wide variety of pixel spacings, so this gave a standard size of the target area. We would then train this with Weighted CrossEntropyLoss. The model we submitted with was densenet121.
For spinal, the size of a batch for training would be (B, 5(Levels), 11(Images), 224, 224), for foraminal narrowing (B, 5(Levels), 2 (Sides), 11(Images), 224, 224), and for subarticular stenosis (B, 5(Levels), 7(Images), 224, 224). For Subarticular Stenosis, we found applying a flip to the right side images gave a better model since everything was in a standard format.
\nAfter all of the training, we did add a master model that had all the same inputs, but joined all outputs for a single level into a large input for a classifier head. We found there was a lot of correllation in the different diagnoses for a single spinal level, but not as much among the different levels.
\n