{ "id": 507906, "title": "Error when reading some DICOM series with SimpleITK (DICOMs with different dimensions)", "url": "/competitions/rsna-2024-lumbar-spine-degenerative-classification/discussion/507906", "author_name": "", "post_date": "2024-05-27T18:19:05.125953700Z", "votes": 2, "comment_count": 3, "views": 0, "content": "

Hello. I'm trying to convert the DICOM series to NIfTI (nii.gz), and to read the data I'm using SimpleITK:

\n
import SimpleITK  sitk\nfrom pathlib import Path\n\ndef dicom:\n    dicom_dir = .parent\n    output_path = \n    output_dir = output_path.parent\n    output_dir.mkdir(parents=True, exist_ok=True)\n\n    :\n        reader = sitk.\n        dicom_names = reader.)\n        reader.\n        image = reader.\n        sitk.)\n    except Exception  e:\n        print(f)\n        error_paths.append(str(dicom_dir))\n
\n

However, I get the following error in 35 series of the whole dataset (below is an example):

\n
 processing rsna--lumbar-spine-degenerative-classification/train_images//: Exception thrown in SimpleITK ImageSeriesReader_Execute: /tmp/SimpleITK-build/ITK-prefix/include/ITK-./itkImageFileReader.hxx::\n returns IO region that does not fully contain the requested region. Requested region: ImageRegion (x7ffee92f7780)\n  : \n  :\n  :\n region: ImageRegion (x7ffee92f7740)\n  : \n  :\n  :\n
\n

I believe this is due to the frames having different shapes (particularly in the case above, 640x640 and 608x608).

\n

The following series have this error:

\n
[\n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/4/', \n'rsna--lumbar-spine-degenerative-classification/train_images/2/2', \n'rsna--lumbar-spine-degenerative-classification/train_images/8/', \n'rsna--lumbar-spine-degenerative-classification/train_images/0/', \n'rsna--lumbar-spine-degenerative-classification/train_images/5/2', \n'rsna--lumbar-spine-degenerative-classification/train_images/6/', \n'rsna--lumbar-spine-degenerative-classification/train_images/1/', \n'rsna--lumbar-spine-degenerative-classification/train_images/0', \n'rsna--lumbar-spine-degenerative-classification/train_images/9', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/4', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/7', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/2', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/0'\n]\n
", "messages": [ { "id": "2839844", "postDate": "05/27/2024 18:19:05", "content": "

Hello. I'm trying to convert the DICOM series to NIfTI (nii.gz), and to read the data I'm using SimpleITK:

\n
import SimpleITK  sitk\nfrom pathlib import Path\n\ndef dicom:\n    dicom_dir = .parent\n    output_path = \n    output_dir = output_path.parent\n    output_dir.mkdir(parents=True, exist_ok=True)\n\n    :\n        reader = sitk.\n        dicom_names = reader.)\n        reader.\n        image = reader.\n        sitk.)\n    except Exception  e:\n        print(f)\n        error_paths.append(str(dicom_dir))\n
\n

However, I get the following error in 35 series of the whole dataset (below is an example):

\n
 processing rsna--lumbar-spine-degenerative-classification/train_images//: Exception thrown in SimpleITK ImageSeriesReader_Execute: /tmp/SimpleITK-build/ITK-prefix/include/ITK-./itkImageFileReader.hxx::\n returns IO region that does not fully contain the requested region. Requested region: ImageRegion (x7ffee92f7780)\n  : \n  :\n  :\n region: ImageRegion (x7ffee92f7740)\n  : \n  :\n  :\n
\n

I believe this is due to the frames having different shapes (particularly in the case above, 640x640 and 608x608).

\n

The following series have this error:

\n
[\n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/4/', \n'rsna--lumbar-spine-degenerative-classification/train_images/2/2', \n'rsna--lumbar-spine-degenerative-classification/train_images/8/', \n'rsna--lumbar-spine-degenerative-classification/train_images/0/', \n'rsna--lumbar-spine-degenerative-classification/train_images/5/2', \n'rsna--lumbar-spine-degenerative-classification/train_images/6/', \n'rsna--lumbar-spine-degenerative-classification/train_images/1/', \n'rsna--lumbar-spine-degenerative-classification/train_images/0', \n'rsna--lumbar-spine-degenerative-classification/train_images/9', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/4', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/7', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/2', \n'rsna--lumbar-spine-degenerative-classification/train_images/', \n'rsna--lumbar-spine-degenerative-classification/train_images/0'\n]\n
", "rawMarkdown": "Hello. I'm trying to convert the DICOM series to NIfTI (nii.gz), and to read the data I'm using SimpleITK:\n\n```python3\nimport SimpleITK as sitk\nfrom pathlib import Path\n\ndef dicom_to_nifti_sitk(row, error_paths: list[str]):\n dicom_dir = Path(row[\"dcm_path\"]).parent\n output_path = Path(row[\"nifti_path\"])\n output_dir = output_path.parent\n output_dir.mkdir(parents=True, exist_ok=True)\n\n try:\n reader = sitk.ImageSeriesReader()\n dicom_names = reader.GetGDCMSeriesFileNames(str(dicom_dir))\n reader.SetFileNames(dicom_names)\n image = reader.Execute()\n sitk.WriteImage(image, str(output_path))\n except Exception as e:\n print(f\"Error processing {dicom_dir}: {e}\")\n error_paths.append(str(dicom_dir))\n\n```\n\nHowever, I get the following error in 35 series of the whole dataset (below is an example):\n```\nError processing rsna-2024-lumbar-spine-degenerative-classification/train_images/4193490688/549563660: Exception thrown in SimpleITK ImageSeriesReader_Execute: /tmp/SimpleITK-build/ITK-prefix/include/ITK-5.3/itkImageFileReader.hxx:335:\nImageIO returns IO region that does not fully contain the requested region. Requested region: ImageRegion (0x7ffee92f7780)\n Dimension: 3\n Index: [0, 0, 0]\n Size: [640, 640, 1]\nStreamableRegion region: ImageRegion (0x7ffee92f7740)\n Dimension: 3\n Index: [0, 0, 0]\n Size: [608, 608, 1]\n```\n\nI believe this is due to the frames having different shapes (particularly in the case above, 640x640 and 608x608).\n\nThe following series have this error:\n```\n[\n'rsna-2024-lumbar-spine-degenerative-classification/train_images/74782131/3401861580', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/114899184/1364910156', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/335455502/790180412', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/582364168/3242313646', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/594735110/4075111689', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/766494595/519827232', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/809072026/2594153114', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/979209761/3101981332', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1258848546/178314290', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1271819130/396937199', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1537608176/2603019387', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1538136131/533252904', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1603568458/3451074679', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1973833645/1328374636', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1992037544/2051789894', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1995123254/1133656256', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2083466060/2440822236', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2334206006/4191635045', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2470505035/1297079442', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2568819355/3366910731', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2755347468/3708423406', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2780132468/4151611107', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2795583238/3365980706', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3339741647/4195187002', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3542237003/458336097', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3651144029/1415702790', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3707028884/3238050001', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3884015124/1519537115', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3885334932/4045988081', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3930841971/2216414062', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3956571539/4190692765', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/4024872715/2479444365', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/4165566893/807064932', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/4165566893/4033318924', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/4193490688/549563660'\n]\n```", "votes": null }, { "id": "2839877", "postDate": "05/27/2024 18:42:12", "content": "

There seems to be 2 (or more) series in the same folder.
\nI've used the following code to load all dicom-imgs of one of the series you mentioned:

\n
 pathlib  Path\n pydicom\nROOT_DICOMS = \n\nstudy_id = \nseries_id = \n\nseries_path = \nall_dcms = (Path(series_path).glob())\n\n dcm  all_dcms:\n    dicom_img = pydicom.dcmread(dcm)\n    ()\n
\n

and I've got some dicom images of size 640x640, and other 608x608. SimpleITK cannot stack dicoms of different image-size (and it is right).

\n

I think we can assume that multiple series got dumped in the same folder.

", "rawMarkdown": "There seems to be 2 (or more) series in the same folder.\nI've used the following code to load all dicom-imgs of one of the series you mentioned:\n\n```\nfrom pathlib import Path\nimport pydicom\nROOT_DICOMS = \"/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/\"\n\nstudy_id = \"335455502\"\nseries_id = \"790180412\"\n\nseries_path = f\"{ROOT_DICOMS}/train_images/{study_id}/{series_id}\"\nall_dcms = list(Path(series_path).glob(\"./*.dcm\"))\n\nfor dcm in all_dcms:\n dicom_img = pydicom.dcmread(dcm)\n print(f\"Image shape: {dicom_img.pixel_array.shape}\")\n```\n\nand I've got some dicom images of size `640x640`, and other `608x608`. SimpleITK cannot stack dicoms of different image-size (and it is right).\n\nI think we can assume that multiple series got dumped in the same folder.", "votes": null }, { "id": "2845649", "postDate": "05/30/2024 16:08:59", "content": "

In addition to what Benardo has suggested, I have come up with the following solution to get groupings of slices:

\n
 collections  defaultdict  dd\n pathlib  Path\n typing  , , , , \n\n numpy  np\n numpy.typing  NDArray\n pydicom  FileDataset, dcmread\n\n ():\n     np.allclose(value, , , abs_tol)\n\n ():\n     np.allclose(value, , , abs_tol)\n\n ():\n    row_cosine, column_cosine = image_orientation[:], image_orientation[:]\n\n     (\n        _almost_zero(np.dot(row_cosine, column_cosine), )\n         _almost_one(np.linalg.norm(row_cosine), )\n         _almost_one(np.linalg.norm(column_cosine), )\n    )\n\n ():\n    invariant_properties = (\n        (, ),\n        (, ),\n        (, ),\n        (, ),\n        (, ),\n        (, ),\n        (,  x: (np.array(x).tolist())),\n        (, ),\n        (, ),\n    )\n    convert_if_not_none =  x, f: (x  x    f(x))\n     (\n        [\n            convert_if_not_none((dataset, property_name, ), convert_type)\n             property_name, convert_type  invariant_properties\n        ]\n    )\n\n ():\n    image_orientation = np.array(dataset.ImageOrientationPatient)\n\n    \n      is_image_orientation_valid(image_orientation):\n         \n\n     group_id, value  image_orientation_groups.items():\n         np.allclose(image_orientation, value, atol=):\n             group_id\n\n    new_group_id = (image_orientation_groups)\n    image_orientation_groups[new_group_id] = image_orientation\n     new_group_id\n\n ():\n    image_orientation_groups = {: np.array(datasets[].ImageOrientationPatient)}\n\n    groups: dd[, []] = dd()\n\n     idx, dataset  (datasets):\n        group_id = get_or_create_image_orientation_group(\n            dataset, image_orientation_groups\n        )\n         group_id   :\n            group_key = formatted_invariants(dataset) + (group_id,)\n            groups[group_key].append(idx)\n\n     (groups.values(), key= x: (x), reverse=)\n\n ():\n     (( dcm: ([(attr  dcm)  attr  attrnames]), slices))\n\n\n ():\n    \n    req_attrs = [, ]\n    slices = remove_slices_without_attributes(all_slices, req_attrs)\n     (slices) > , \n    groups = find_invariant_groupings(slices)\n\n     (groups) > , \n\n     [[slices[idx]  idx  groups[group_id]]  group_id  ((groups))]\n
\n

Each of the list of slices returned by group_slices function is a group that can be combined together.
\nYou can check out combine_slices function from a library called dicom_numpy to convert this list of PyDicom objects to a 3D NumPy volume.

\n

Some Corrections:
\nAdditional grouping with ImageOrientationPatient attribute is also needed. Based on the criteria used by combine_slices function, I used a tolerance of 1e-5. Also giving an example of how to use this:

\n
train_images_path = Path()\nstudy_id, series_id = , \nseries_path = train_images_path / study_id / series_id\n series_path.exists()\nall_dcms = [pydicom.dcmread(path)  path  series_path.glob()]\n\ngroups = group_slices(sort_by_slice_position(all_dcms))\n((groups), ((, groups)))  \n\n\n dicom_numpy  combine_slices\n\n group  groups:\n    vol, affine = combine_slices(group)\n    (vol.shape)\n    \n\n\n\n\n\n
", "rawMarkdown": "In addition to what Benardo has suggested, I have come up with the following solution to get groupings of slices:\n```python\nfrom collections import defaultdict as dd\nfrom pathlib import Path\nfrom typing import Any, Dict, List, Optional, Tuple\n\nimport numpy as np\nfrom numpy.typing import NDArray\nfrom pydicom import FileDataset, dcmread\n\ndef _almost_zero(value, abs_tol):\n return np.allclose(value, 0.0, 1e-09, abs_tol)\n\ndef _almost_one(value, abs_tol):\n return np.allclose(value, 1.0, 1e-09, abs_tol)\n\ndef is_image_orientation_valid(image_orientation):\n row_cosine, column_cosine = image_orientation[:3], image_orientation[3:]\n\n return (\n _almost_zero(np.dot(row_cosine, column_cosine), 1e-4)\n and _almost_one(np.linalg.norm(row_cosine), 1e-4)\n and _almost_one(np.linalg.norm(column_cosine), 1e-4)\n )\n\ndef formatted_invariants(dataset: FileDataset):\n invariant_properties = (\n (\"Modality\", str),\n (\"SOPClassUID\", str),\n (\"SeriesInstanceUID\", str),\n (\"Rows\", int),\n (\"Columns\", int),\n (\"SamplesPerPixel\", int),\n (\"PixelSpacing\", lambda x: tuple(np.array(x).tolist())),\n (\"PixelRepresentation\", int),\n (\"BitsAllocated\", int),\n )\n convert_if_not_none = lambda x, f: (x if x is None else f(x))\n return tuple(\n [\n convert_if_not_none(getattr(dataset, property_name, None), convert_type)\n for property_name, convert_type in invariant_properties\n ]\n )\n\ndef get_or_create_image_orientation_group(\n dataset: FileDataset, image_orientation_groups: Dict[int, NDArray[Any]]\n):\n image_orientation = np.array(dataset.ImageOrientationPatient)\n\n # Rejected dataset not to be included in grouping\n if not is_image_orientation_valid(image_orientation):\n return None\n\n for group_id, value in image_orientation_groups.items():\n if np.allclose(image_orientation, value, atol=1e-5):\n return group_id\n\n new_group_id = len(image_orientation_groups)\n image_orientation_groups[new_group_id] = image_orientation\n return new_group_id\n\ndef find_invariant_groupings(datasets: List[FileDataset]):\n image_orientation_groups = {0: np.array(datasets[0].ImageOrientationPatient)}\n\n groups: dd[Any, list[int]] = dd(list)\n\n for idx, dataset in enumerate(datasets):\n group_id = get_or_create_image_orientation_group(\n dataset, image_orientation_groups\n )\n if group_id is not None:\n group_key = formatted_invariants(dataset) + (group_id,)\n groups[group_key].append(idx)\n\n return sorted(groups.values(), key=lambda x: len(x), reverse=True)\n\ndef remove_slices_without_attributes(slices: List[FileDataset], attrnames: List[str]):\n return list(filter(lambda dcm: all([(attr in dcm) for attr in attrnames]), slices))\n\n\ndef group_slices(all_slices: List[FileDataset]):\n \"\"\"Filter slices and a single groupping\"\"\"\n req_attrs = [\"ImageOrientationPatient\", \"ImagePositionPatient\"]\n slices = remove_slices_without_attributes(all_slices, req_attrs)\n assert len(slices) > 0, f\"No slices with all these attrs: {', '.join(req_attrs)}\"\n groups = find_invariant_groupings(slices)\n\n assert len(groups) > 0, f\"Did not find any groups in slices.\"\n\n return [[slices[idx] for idx in groups[group_id]] for group_id in range(len(groups))]\n```\n\nEach of the list of slices returned by `group_slices` function is a group that can be combined together.\nYou can check out `combine_slices` function from a library called `dicom_numpy` to convert this list of PyDicom objects to a 3D NumPy volume.\n\n**Some Corrections:**\nAdditional grouping with ImageOrientationPatient attribute is also needed. Based on the criteria used by `combine_slices` function, I used a tolerance of 1e-5. Also giving an example of how to use this:\n\n```python\ntrain_images_path = Path(\"../input/rsna-2024-lumbar-spine-degenerative-classification/train_images_path\")\nstudy_id, series_id = \"46494080\", \"1543341132\"\nseries_path = train_images_path / study_id / series_id\nassert series_path.exists()\nall_dcms = [pydicom.dcmread(path) for path in series_path.glob(\"*.dcm\")]\n\ngroups = group_slices(sort_by_slice_position(all_dcms))\nprint(len(groups), list(map(len, groups))) # Output: 3, [5, 5, 5]\n\n# If you have installed `dicom-numpy` by `%pip install dicom-numpy`\nfrom dicom_numpy import combine_slices\n\nfor group in groups:\n vol, affine = combine_slices(group)\n print(vol.shape)\n # Your code to process these volumes\n\n# Output:\n# (512, 512, 5)\n# (512, 512, 5)\n# (512, 512, 5)\n```", "votes": null }, { "id": "2851317", "postDate": "06/02/2024 17:14:07", "content": "

Even with this ImageOrientationPatient attribute addendum, there are still corner cases. E.g.: study_id, series_id = \"100206310\", \"1012284084\". This pertains to one irregular inter-slice distance within a group which suggests that these are 2 separate groups (just happening to have the same orientation).

", "rawMarkdown": "Even with this `ImageOrientationPatient` attribute addendum, there are still corner cases. E.g.: `study_id, series_id = \"100206310\", \"1012284084\"`. This pertains to one irregular inter-slice distance within a group which suggests that these are 2 separate groups (just happening to have the same orientation).", "votes": null } ], "comments": [ { "id": 2839877, "author_name": "bernardohenz", "author_url": "", "post_date": "05/27/2024 18:42:12", "content": "

There seems to be 2 (or more) series in the same folder.
\nI've used the following code to load all dicom-imgs of one of the series you mentioned:

\n
 pathlib  Path\n pydicom\nROOT_DICOMS = \n\nstudy_id = \nseries_id = \n\nseries_path = \nall_dcms = (Path(series_path).glob())\n\n dcm  all_dcms:\n    dicom_img = pydicom.dcmread(dcm)\n    ()\n
\n

and I've got some dicom images of size 640x640, and other 608x608. SimpleITK cannot stack dicoms of different image-size (and it is right).

\n

I think we can assume that multiple series got dumped in the same folder.

", "votes": null, "replies": [] }, { "id": 2845649, "author_name": "coderrkj", "author_url": "", "post_date": "05/30/2024 16:08:59", "content": "

In addition to what Benardo has suggested, I have come up with the following solution to get groupings of slices:

\n
 collections  defaultdict  dd\n pathlib  Path\n typing  , , , , \n\n numpy  np\n numpy.typing  NDArray\n pydicom  FileDataset, dcmread\n\n ():\n     np.allclose(value, , , abs_tol)\n\n ():\n     np.allclose(value, , , abs_tol)\n\n ():\n    row_cosine, column_cosine = image_orientation[:], image_orientation[:]\n\n     (\n        _almost_zero(np.dot(row_cosine, column_cosine), )\n         _almost_one(np.linalg.norm(row_cosine), )\n         _almost_one(np.linalg.norm(column_cosine), )\n    )\n\n ():\n    invariant_properties = (\n        (, ),\n        (, ),\n        (, ),\n        (, ),\n        (, ),\n        (, ),\n        (,  x: (np.array(x).tolist())),\n        (, ),\n        (, ),\n    )\n    convert_if_not_none =  x, f: (x  x    f(x))\n     (\n        [\n            convert_if_not_none((dataset, property_name, ), convert_type)\n             property_name, convert_type  invariant_properties\n        ]\n    )\n\n ():\n    image_orientation = np.array(dataset.ImageOrientationPatient)\n\n    \n      is_image_orientation_valid(image_orientation):\n         \n\n     group_id, value  image_orientation_groups.items():\n         np.allclose(image_orientation, value, atol=):\n             group_id\n\n    new_group_id = (image_orientation_groups)\n    image_orientation_groups[new_group_id] = image_orientation\n     new_group_id\n\n ():\n    image_orientation_groups = {: np.array(datasets[].ImageOrientationPatient)}\n\n    groups: dd[, []] = dd()\n\n     idx, dataset  (datasets):\n        group_id = get_or_create_image_orientation_group(\n            dataset, image_orientation_groups\n        )\n         group_id   :\n            group_key = formatted_invariants(dataset) + (group_id,)\n            groups[group_key].append(idx)\n\n     (groups.values(), key= x: (x), reverse=)\n\n ():\n     (( dcm: ([(attr  dcm)  attr  attrnames]), slices))\n\n\n ():\n    \n    req_attrs = [, ]\n    slices = remove_slices_without_attributes(all_slices, req_attrs)\n     (slices) > , \n    groups = find_invariant_groupings(slices)\n\n     (groups) > , \n\n     [[slices[idx]  idx  groups[group_id]]  group_id  ((groups))]\n
\n

Each of the list of slices returned by group_slices function is a group that can be combined together.
\nYou can check out combine_slices function from a library called dicom_numpy to convert this list of PyDicom objects to a 3D NumPy volume.

\n

Some Corrections:
\nAdditional grouping with ImageOrientationPatient attribute is also needed. Based on the criteria used by combine_slices function, I used a tolerance of 1e-5. Also giving an example of how to use this:

\n
train_images_path = Path()\nstudy_id, series_id = , \nseries_path = train_images_path / study_id / series_id\n series_path.exists()\nall_dcms = [pydicom.dcmread(path)  path  series_path.glob()]\n\ngroups = group_slices(sort_by_slice_position(all_dcms))\n((groups), ((, groups)))  \n\n\n dicom_numpy  combine_slices\n\n group  groups:\n    vol, affine = combine_slices(group)\n    (vol.shape)\n    \n\n\n\n\n\n
", "votes": null, "replies": [ { "id": 2851317, "author_name": "coderrkj", "author_url": "", "post_date": "06/02/2024 17:14:07", "content": "

Even with this ImageOrientationPatient attribute addendum, there are still corner cases. E.g.: study_id, series_id = \"100206310\", \"1012284084\". This pertains to one irregular inter-slice distance within a group which suggests that these are 2 separate groups (just happening to have the same orientation).

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2839844": "Hello. I'm trying to convert the DICOM series to NIfTI (nii.gz), and to read the data I'm using SimpleITK:\n\n```python3\nimport SimpleITK as sitk\nfrom pathlib import Path\n\ndef dicom_to_nifti_sitk(row, error_paths: list[str]):\n dicom_dir = Path(row[\"dcm_path\"]).parent\n output_path = Path(row[\"nifti_path\"])\n output_dir = output_path.parent\n output_dir.mkdir(parents=True, exist_ok=True)\n\n try:\n reader = sitk.ImageSeriesReader()\n dicom_names = reader.GetGDCMSeriesFileNames(str(dicom_dir))\n reader.SetFileNames(dicom_names)\n image = reader.Execute()\n sitk.WriteImage(image, str(output_path))\n except Exception as e:\n print(f\"Error processing {dicom_dir}: {e}\")\n error_paths.append(str(dicom_dir))\n\n```\n\nHowever, I get the following error in 35 series of the whole dataset (below is an example):\n```\nError processing rsna-2024-lumbar-spine-degenerative-classification/train_images/4193490688/549563660: Exception thrown in SimpleITK ImageSeriesReader_Execute: /tmp/SimpleITK-build/ITK-prefix/include/ITK-5.3/itkImageFileReader.hxx:335:\nImageIO returns IO region that does not fully contain the requested region. Requested region: ImageRegion (0x7ffee92f7780)\n Dimension: 3\n Index: [0, 0, 0]\n Size: [640, 640, 1]\nStreamableRegion region: ImageRegion (0x7ffee92f7740)\n Dimension: 3\n Index: [0, 0, 0]\n Size: [608, 608, 1]\n```\n\nI believe this is due to the frames having different shapes (particularly in the case above, 640x640 and 608x608).\n\nThe following series have this error:\n```\n[\n'rsna-2024-lumbar-spine-degenerative-classification/train_images/74782131/3401861580', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/114899184/1364910156', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/335455502/790180412', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/582364168/3242313646', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/594735110/4075111689', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/766494595/519827232', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/809072026/2594153114', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/979209761/3101981332', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1258848546/178314290', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1271819130/396937199', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1537608176/2603019387', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1538136131/533252904', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1603568458/3451074679', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1973833645/1328374636', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1992037544/2051789894', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/1995123254/1133656256', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2083466060/2440822236', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2334206006/4191635045', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2470505035/1297079442', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2568819355/3366910731', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2755347468/3708423406', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2780132468/4151611107', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/2795583238/3365980706', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3339741647/4195187002', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3542237003/458336097', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3651144029/1415702790', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3707028884/3238050001', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3884015124/1519537115', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3885334932/4045988081', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3930841971/2216414062', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/3956571539/4190692765', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/4024872715/2479444365', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/4165566893/807064932', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/4165566893/4033318924', \n'rsna-2024-lumbar-spine-degenerative-classification/train_images/4193490688/549563660'\n]\n```", "2839877": "There seems to be 2 (or more) series in the same folder.\nI've used the following code to load all dicom-imgs of one of the series you mentioned:\n\n```\nfrom pathlib import Path\nimport pydicom\nROOT_DICOMS = \"/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/\"\n\nstudy_id = \"335455502\"\nseries_id = \"790180412\"\n\nseries_path = f\"{ROOT_DICOMS}/train_images/{study_id}/{series_id}\"\nall_dcms = list(Path(series_path).glob(\"./*.dcm\"))\n\nfor dcm in all_dcms:\n dicom_img = pydicom.dcmread(dcm)\n print(f\"Image shape: {dicom_img.pixel_array.shape}\")\n```\n\nand I've got some dicom images of size `640x640`, and other `608x608`. SimpleITK cannot stack dicoms of different image-size (and it is right).\n\nI think we can assume that multiple series got dumped in the same folder.", "2845649": "In addition to what Benardo has suggested, I have come up with the following solution to get groupings of slices:\n```python\nfrom collections import defaultdict as dd\nfrom pathlib import Path\nfrom typing import Any, Dict, List, Optional, Tuple\n\nimport numpy as np\nfrom numpy.typing import NDArray\nfrom pydicom import FileDataset, dcmread\n\ndef _almost_zero(value, abs_tol):\n return np.allclose(value, 0.0, 1e-09, abs_tol)\n\ndef _almost_one(value, abs_tol):\n return np.allclose(value, 1.0, 1e-09, abs_tol)\n\ndef is_image_orientation_valid(image_orientation):\n row_cosine, column_cosine = image_orientation[:3], image_orientation[3:]\n\n return (\n _almost_zero(np.dot(row_cosine, column_cosine), 1e-4)\n and _almost_one(np.linalg.norm(row_cosine), 1e-4)\n and _almost_one(np.linalg.norm(column_cosine), 1e-4)\n )\n\ndef formatted_invariants(dataset: FileDataset):\n invariant_properties = (\n (\"Modality\", str),\n (\"SOPClassUID\", str),\n (\"SeriesInstanceUID\", str),\n (\"Rows\", int),\n (\"Columns\", int),\n (\"SamplesPerPixel\", int),\n (\"PixelSpacing\", lambda x: tuple(np.array(x).tolist())),\n (\"PixelRepresentation\", int),\n (\"BitsAllocated\", int),\n )\n convert_if_not_none = lambda x, f: (x if x is None else f(x))\n return tuple(\n [\n convert_if_not_none(getattr(dataset, property_name, None), convert_type)\n for property_name, convert_type in invariant_properties\n ]\n )\n\ndef get_or_create_image_orientation_group(\n dataset: FileDataset, image_orientation_groups: Dict[int, NDArray[Any]]\n):\n image_orientation = np.array(dataset.ImageOrientationPatient)\n\n # Rejected dataset not to be included in grouping\n if not is_image_orientation_valid(image_orientation):\n return None\n\n for group_id, value in image_orientation_groups.items():\n if np.allclose(image_orientation, value, atol=1e-5):\n return group_id\n\n new_group_id = len(image_orientation_groups)\n image_orientation_groups[new_group_id] = image_orientation\n return new_group_id\n\ndef find_invariant_groupings(datasets: List[FileDataset]):\n image_orientation_groups = {0: np.array(datasets[0].ImageOrientationPatient)}\n\n groups: dd[Any, list[int]] = dd(list)\n\n for idx, dataset in enumerate(datasets):\n group_id = get_or_create_image_orientation_group(\n dataset, image_orientation_groups\n )\n if group_id is not None:\n group_key = formatted_invariants(dataset) + (group_id,)\n groups[group_key].append(idx)\n\n return sorted(groups.values(), key=lambda x: len(x), reverse=True)\n\ndef remove_slices_without_attributes(slices: List[FileDataset], attrnames: List[str]):\n return list(filter(lambda dcm: all([(attr in dcm) for attr in attrnames]), slices))\n\n\ndef group_slices(all_slices: List[FileDataset]):\n \"\"\"Filter slices and a single groupping\"\"\"\n req_attrs = [\"ImageOrientationPatient\", \"ImagePositionPatient\"]\n slices = remove_slices_without_attributes(all_slices, req_attrs)\n assert len(slices) > 0, f\"No slices with all these attrs: {', '.join(req_attrs)}\"\n groups = find_invariant_groupings(slices)\n\n assert len(groups) > 0, f\"Did not find any groups in slices.\"\n\n return [[slices[idx] for idx in groups[group_id]] for group_id in range(len(groups))]\n```\n\nEach of the list of slices returned by `group_slices` function is a group that can be combined together.\nYou can check out `combine_slices` function from a library called `dicom_numpy` to convert this list of PyDicom objects to a 3D NumPy volume.\n\n**Some Corrections:**\nAdditional grouping with ImageOrientationPatient attribute is also needed. Based on the criteria used by `combine_slices` function, I used a tolerance of 1e-5. Also giving an example of how to use this:\n\n```python\ntrain_images_path = Path(\"../input/rsna-2024-lumbar-spine-degenerative-classification/train_images_path\")\nstudy_id, series_id = \"46494080\", \"1543341132\"\nseries_path = train_images_path / study_id / series_id\nassert series_path.exists()\nall_dcms = [pydicom.dcmread(path) for path in series_path.glob(\"*.dcm\")]\n\ngroups = group_slices(sort_by_slice_position(all_dcms))\nprint(len(groups), list(map(len, groups))) # Output: 3, [5, 5, 5]\n\n# If you have installed `dicom-numpy` by `%pip install dicom-numpy`\nfrom dicom_numpy import combine_slices\n\nfor group in groups:\n vol, affine = combine_slices(group)\n print(vol.shape)\n # Your code to process these volumes\n\n# Output:\n# (512, 512, 5)\n# (512, 512, 5)\n# (512, 512, 5)\n```", "2851317": "Even with this `ImageOrientationPatient` attribute addendum, there are still corner cases. E.g.: `study_id, series_id = \"100206310\", \"1012284084\"`. This pertains to one irregular inter-slice distance within a group which suggests that these are 2 separate groups (just happening to have the same orientation)." }, "source": "meta" }