{
  "id": 176879,
  "title": "It appears there are missing slices",
  "url": "/competitions/osic-pulmonary-fibrosis-progression/discussion/176879",
  "author_name": "Ji Wong Park",
  "post_date": "2020-08-24T01:38:56.388000",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<pre><code>/kaggle/working$ py convert.py \ndir: ID00279637202272164826258\ndir: ID00190637202244450116191\ndir: ID00184637202242062969203\ndir: ID00398637202303897337979\ndir: ID00051637202185848464638\ndir: ID00376637202297677828573\ndir: ID00421637202311550012437\nThe slice spacing is non-uniform. Slice spacings:\n[9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1.\n 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1.\n 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9.]\nTraceback (most recent call last):\n  File \"convert.py\", line 33, in &lt;module&gt;\n    npy_file = extract_voxel_data(list_dicom)\n  File \"convert.py\", line 14, in extract_voxel_data\n    voxel_ndarray, ijk_to_xyz = dicom_numpy.combine_slices(datasets)\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 72, in combine_slices\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 160, in _validate_slices_form_uniform_grid\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 235, in _check_for_missing_slices\ndicom_numpy.exceptions.DicomImportException: It appears there are missing slices\n</code></pre>\n<p>and here is convert.py</p>\n<h1>convert.py</h1>\n<pre><code>#!/usr/bin/env python\n\nimport os\nfrom pathlib import Path\nimport pydicom\nimport dicom_numpy\n\ntrain_path='/kaggle/input/osic-pulmonary-fibrosis-progression/train'\ntrain_new_path='/kaggle/working/new_train'\n\ndef extract_voxel_data(list_of_dicom_files):\n    datasets = [pydicom.dcmread(f) for f in list_of_dicom_files]\n    try:\n        voxel_ndarray, ijk_to_xyz = dicom_numpy.combine_slices(datasets)\n    except dicom_numpy.DicomImportException as e:\n        # invalid DICOM data\n        raise\n    return voxel_ndarray\n\n\nif __name__ == '__main__':\n    Path(train_new_path).mkdir(parents=True, exist_ok=True)\n    list_all = os.listdir(train_path)\n    print('---------------------------------------------------')\n    for dir_ in list_all:\n        print('dir: {}'.format(dir_))\n        dname = os.path.join(train_path, dir_)\n        new_dname = os.path.join(train_new_path, dir_)\n        list_files = os.listdir(dname)\n        list_dicom = []\n        for file_ in list_files:\n            list_dicom.append(os.path.join(dname, file_))\n        npy_file = extract_voxel_data(list_dicom)\n        npy_file.tofile(new_dname+'.npy')\n        print('---------------------------------------------------')\n</code></pre>",
  "messages": [
    {
      "id": 985870,
      "postDate": "2020-08-26T04:27:01.260Z",
      "content": "<p>There are missing slices in many of the files. I think it’s up to us to try to piece together the data based on metadata etc. Perhaps we can train our convnets with cutout to teach them to ignore the missing slices.</p>",
      "rawMarkdown": "There are missing slices in many of the files. I think it’s up to us to try to piece together the data based on metadata etc. Perhaps we can train our convnets with cutout to teach them to ignore the missing slices.",
      "votes": 1,
      "replies": [
        {
          "id": 986637,
          "postDate": "2020-08-26T16:45:49.910Z",
          "content": "<p>yes .. we tried for 1 file ..<br>\nPlease sort them by slice location ..we tried for 1 image and it is working .. Below is the sample code ..<br>\n<a href=\"https://pydicom.github.io/pydicom/dev/auto_examples/image_processing/reslice.html\" target=\"_blank\">https://pydicom.github.io/pydicom/dev/auto_examples/image_processing/reslice.html</a></p>",
          "rawMarkdown": "yes .. we tried for 1 file ..\nPlease sort them by slice location ..we tried for 1 image and it is working .. Below is the sample code ..\nhttps://pydicom.github.io/pydicom/dev/auto_examples/image_processing/reslice.html",
          "votes": 1
        },
        {
          "id": 986638,
          "postDate": "2020-08-26T16:46:35.043Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 983031,
      "postDate": "2020-08-24T01:38:56.390Z",
      "content": "<pre><code>/kaggle/working$ py convert.py \ndir: ID00279637202272164826258\ndir: ID00190637202244450116191\ndir: ID00184637202242062969203\ndir: ID00398637202303897337979\ndir: ID00051637202185848464638\ndir: ID00376637202297677828573\ndir: ID00421637202311550012437\nThe slice spacing is non-uniform. Slice spacings:\n[9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1.\n 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1.\n 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9.]\nTraceback (most recent call last):\n  File \"convert.py\", line 33, in &lt;module&gt;\n    npy_file = extract_voxel_data(list_dicom)\n  File \"convert.py\", line 14, in extract_voxel_data\n    voxel_ndarray, ijk_to_xyz = dicom_numpy.combine_slices(datasets)\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 72, in combine_slices\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 160, in _validate_slices_form_uniform_grid\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 235, in _check_for_missing_slices\ndicom_numpy.exceptions.DicomImportException: It appears there are missing slices\n</code></pre>\n<p>and here is convert.py</p>\n<h1>convert.py</h1>\n<pre><code>#!/usr/bin/env python\n\nimport os\nfrom pathlib import Path\nimport pydicom\nimport dicom_numpy\n\ntrain_path='/kaggle/input/osic-pulmonary-fibrosis-progression/train'\ntrain_new_path='/kaggle/working/new_train'\n\ndef extract_voxel_data(list_of_dicom_files):\n    datasets = [pydicom.dcmread(f) for f in list_of_dicom_files]\n    try:\n        voxel_ndarray, ijk_to_xyz = dicom_numpy.combine_slices(datasets)\n    except dicom_numpy.DicomImportException as e:\n        # invalid DICOM data\n        raise\n    return voxel_ndarray\n\n\nif __name__ == '__main__':\n    Path(train_new_path).mkdir(parents=True, exist_ok=True)\n    list_all = os.listdir(train_path)\n    print('---------------------------------------------------')\n    for dir_ in list_all:\n        print('dir: {}'.format(dir_))\n        dname = os.path.join(train_path, dir_)\n        new_dname = os.path.join(train_new_path, dir_)\n        list_files = os.listdir(dname)\n        list_dicom = []\n        for file_ in list_files:\n            list_dicom.append(os.path.join(dname, file_))\n        npy_file = extract_voxel_data(list_dicom)\n        npy_file.tofile(new_dname+'.npy')\n        print('---------------------------------------------------')\n</code></pre>",
      "rawMarkdown": "```\n/kaggle/working$ py convert.py \ndir: ID00279637202272164826258\ndir: ID00190637202244450116191\ndir: ID00184637202242062969203\ndir: ID00398637202303897337979\ndir: ID00051637202185848464638\ndir: ID00376637202297677828573\ndir: ID00421637202311550012437\nThe slice spacing is non-uniform. Slice spacings:\n[9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1.\n 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1.\n 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9.]\nTraceback (most recent call last):\n  File \"convert.py\", line 33, in <module>\n    npy_file = extract_voxel_data(list_dicom)\n  File \"convert.py\", line 14, in extract_voxel_data\n    voxel_ndarray, ijk_to_xyz = dicom_numpy.combine_slices(datasets)\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 72, in combine_slices\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 160, in _validate_slices_form_uniform_grid\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 235, in _check_for_missing_slices\ndicom_numpy.exceptions.DicomImportException: It appears there are missing slices\n```\n\nand here is convert.py\n\n# convert.py\n```\n#!/usr/bin/env python\n\nimport os\nfrom pathlib import Path\nimport pydicom\nimport dicom_numpy\n\ntrain_path='/kaggle/input/osic-pulmonary-fibrosis-progression/train'\ntrain_new_path='/kaggle/working/new_train'\n\ndef extract_voxel_data(list_of_dicom_files):\n    datasets = [pydicom.dcmread(f) for f in list_of_dicom_files]\n    try:\n        voxel_ndarray, ijk_to_xyz = dicom_numpy.combine_slices(datasets)\n    except dicom_numpy.DicomImportException as e:\n        # invalid DICOM data\n        raise\n    return voxel_ndarray\n\n\nif __name__ == '__main__':\n    Path(train_new_path).mkdir(parents=True, exist_ok=True)\n    list_all = os.listdir(train_path)\n    print('---------------------------------------------------')\n    for dir_ in list_all:\n        print('dir: {}'.format(dir_))\n        dname = os.path.join(train_path, dir_)\n        new_dname = os.path.join(train_new_path, dir_)\n        list_files = os.listdir(dname)\n        list_dicom = []\n        for file_ in list_files:\n            list_dicom.append(os.path.join(dname, file_))\n        npy_file = extract_voxel_data(list_dicom)\n        npy_file.tofile(new_dname+'.npy')\n        print('---------------------------------------------------')\n```\n",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 985870,
      "author_name": "Sam Lin",
      "author_url": "",
      "post_date": "2020-08-26T04:27:01.260000",
      "content": "<p>There are missing slices in many of the files. I think it’s up to us to try to piece together the data based on metadata etc. Perhaps we can train our convnets with cutout to teach them to ignore the missing slices.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 986637,
          "author_name": "Pulkit Mehta",
          "author_url": "",
          "post_date": "2020-08-26T16:45:49.910000",
          "content": "<p>yes .. we tried for 1 file ..<br>\nPlease sort them by slice location ..we tried for 1 image and it is working .. Below is the sample code ..<br>\n<a href=\"https://pydicom.github.io/pydicom/dev/auto_examples/image_processing/reslice.html\" target=\"_blank\">https://pydicom.github.io/pydicom/dev/auto_examples/image_processing/reslice.html</a></p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 986638,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-08-26T16:46:35.043000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "985870": "There are missing slices in many of the files. I think it’s up to us to try to piece together the data based on metadata etc. Perhaps we can train our convnets with cutout to teach them to ignore the missing slices.",
    "983031": "```\n/kaggle/working$ py convert.py \ndir: ID00279637202272164826258\ndir: ID00190637202244450116191\ndir: ID00184637202242062969203\ndir: ID00398637202303897337979\ndir: ID00051637202185848464638\ndir: ID00376637202297677828573\ndir: ID00421637202311550012437\nThe slice spacing is non-uniform. Slice spacings:\n[9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1.\n 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1.\n 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9. 1. 9.]\nTraceback (most recent call last):\n  File \"convert.py\", line 33, in <module>\n    npy_file = extract_voxel_data(list_dicom)\n  File \"convert.py\", line 14, in extract_voxel_data\n    voxel_ndarray, ijk_to_xyz = dicom_numpy.combine_slices(datasets)\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 72, in combine_slices\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 160, in _validate_slices_form_uniform_grid\n  File \"/usr/local/lib/python3.8/dist-packages/dicom_numpy-0.4.0-py3.8.egg/dicom_numpy/combine_slices.py\", line 235, in _check_for_missing_slices\ndicom_numpy.exceptions.DicomImportException: It appears there are missing slices\n```\n\nand here is convert.py\n\n# convert.py\n```\n#!/usr/bin/env python\n\nimport os\nfrom pathlib import Path\nimport pydicom\nimport dicom_numpy\n\ntrain_path='/kaggle/input/osic-pulmonary-fibrosis-progression/train'\ntrain_new_path='/kaggle/working/new_train'\n\ndef extract_voxel_data(list_of_dicom_files):\n    datasets = [pydicom.dcmread(f) for f in list_of_dicom_files]\n    try:\n        voxel_ndarray, ijk_to_xyz = dicom_numpy.combine_slices(datasets)\n    except dicom_numpy.DicomImportException as e:\n        # invalid DICOM data\n        raise\n    return voxel_ndarray\n\n\nif __name__ == '__main__':\n    Path(train_new_path).mkdir(parents=True, exist_ok=True)\n    list_all = os.listdir(train_path)\n    print('---------------------------------------------------')\n    for dir_ in list_all:\n        print('dir: {}'.format(dir_))\n        dname = os.path.join(train_path, dir_)\n        new_dname = os.path.join(train_new_path, dir_)\n        list_files = os.listdir(dname)\n        list_dicom = []\n        for file_ in list_files:\n            list_dicom.append(os.path.join(dname, file_))\n        npy_file = extract_voxel_data(list_dicom)\n        npy_file.tofile(new_dname+'.npy')\n        print('---------------------------------------------------')\n```\n"
  }
}