{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.14","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"colab":{"provenance":[]},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":71549,"databundleVersionId":8561470,"sourceType":"competition"},{"sourceId":9485544,"sourceType":"datasetVersion","datasetId":5770401}],"dockerImageVersionId":30775,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"---\n# Understanding the Anatomical Aspects of Lumbar Spine Degeneration Detection and Classification\n# Also overview of the dataset\n---","metadata":{"id":"P0oyTIGvVomK"}},{"cell_type":"markdown","source":"**Table of Contents**\n---\n\n\n\n1.   Introduction\n2.   The Spine: Basic Overview\n3.   Lumbar Spine Anatomy\n     *  Vertebrae\n     *  Intervertebral Discs\n     *  Facet Joints\n4.  Vertebral Levels and Their Significance\n5.  Intervertebral Disc Anatomy\n6.  Lumbar Disc Degeneration and Zones of Degeneration\n7.  Degenerative Spine Conditions\n    *   Canal Stenosis\n    *   Foraminal Narrowing\n    *   Subarticular Stenosis\n    \n\n8.  MRI Imaging of the Lumbar Spine\n    *   T1-Weighted Imaging\n    *   T2-Weighted Imaging\n    *   STIR Imaging\n9.  Severity Classification Criteria\n10. MRI in the Diagnosis of Spinal Stenosis\n11. Dataset Overview and Key Files\n12. Conclusion\n\n---\n\n","metadata":{"id":"flJpFfxOVwav"}},{"cell_type":"markdown","source":"**Introduction**\n---\n\nIn this notebook, we explore the anatomical aspects essential for developing a deep learning model aimed at detecting and classifying the severity of lumbar spine degeneration using MRI scans. Understanding the anatomy is crucial for accurate data annotation, model training, and interpretation of results. The dataset used in this study includes DICOM (.dcm) files and accompanying metadata in CSV format, encompassing data from 1975 patients. This comprehensive dataset provides a solid foundation for training and evaluating machine learning models, offering a valuable resource in the development of automated diagnostic tools for lumbar spine degeneration.\n\n----\n\n","metadata":{"id":"tyqO-fLWZKrx"}},{"cell_type":"markdown","source":"Let's briefly discuss the anatomy relevant to this dataset to understand what we're asking you to detect.","metadata":{}},{"cell_type":"markdown","source":"**The Spine: Basic Overview**  \n---\n\nThe human spine is a column of bones (vertebrae) and intervertebral discs that provide structural support and protect the spinal cord. It is divided into five sections:\n\n   *  **Cervical Spine** (Neck): C1-C7 vertebrae\n   *  **Thoracic Spine** (Upper Back): T1-T12 vertebrae\n   *  **Lumbar Spine** (Lower Back): L1-L5 vertebrae\n   *  **Sacral Spine** (Pelvis): S1-S5 fused vertebrae\n   *  **Coccyx** (Tailbone): 3-4 fused vertebrae\n","metadata":{"id":"ReJXioNDmgDG"}},{"cell_type":"code","source":"from IPython.display import Image\n\nprint(\"\\t\\tHuman Spine Diagram\\n\")\n# Resize the image to desired width and height\nImage(\"/kaggle/input/diagrams/Images/spine.png\",width=400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.344942Z","iopub.execute_input":"2024-10-11T06:03:38.345471Z","iopub.status.idle":"2024-10-11T06:03:38.410338Z","shell.execute_reply.started":"2024-10-11T06:03:38.345406Z","shell.execute_reply":"2024-10-11T06:03:38.408399Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n**Basic Functions of the Spine:**\n\n  * **Protection:** Encloses and protects the spinal cord and nerve roots.\n  * **Mobility:** Allows for bending, twisting, and other movements.\n  * **Support:** Bears the weight of the head and trunk, transferring forces to the pelvis and lower limbs.\n\n----","metadata":{"id":"Vx-daLI83VP0"}},{"cell_type":"markdown","source":"**Lumbar Spine Anatomy**\n---\n\nThe lumbar spine refers to the lower back region, consisting of five vertebrae (L1 to L5) and several related anatomical structures that are prone to degeneration and can lead to pain or nerve impingement. Understanding these structures will enhance the ability to interpret MRI images and classify conditions more effectively.\n","metadata":{"id":"kAIlLwE-oWnX"}},{"cell_type":"markdown","source":"**1. Vertebrae**\n\nThe lumbar vertebrae are **larger and stronger** than the vertebrae in the upper spine because they **bear more weight**. Each vertebra consists of:\n\n   * **Body:** The thick, anterior part that bears the weight.\n   * **Vertebral Arch:** A bony ring forming the spinal canal, where the spinal cord passes through.\n   * **Spinous and Transverse Processes:** Projections that provide points of attachment for muscles and ligaments.\n\n**2. Intervertebral Discs**\n\nBetween each pair of lumbar vertebrae is an intervertebral disc. These discs act as s**hock absorbers**, **cushioning the bones** during movement and **preventing the vertebrae from rubbing** against each other.\n\n   Structure of an Intervertebral Disc:\n\n   * **Nucleus Pulposus:** The soft, gel-like center that absorbs shock.\n   * **Annulus Fibrosus:** The tough, outer layer made of collagen fibers, which keeps the disc in place.\n\n**3. Facet Joints**\n\nEach vertebra connects to the one above and below it through facet joints, which allow **controlled motion** and provide **stability** to the spine.\n\n   * Facet joints are located at the back of the spine, between the vertebrae.\n   * Degeneration in these joints can result in pain and limited motion.\n\n\n","metadata":{"id":"MdISOo-M37q-"}},{"cell_type":"code","source":"print(\"\\t\\tLumbar Spine Region\\n\")\nImage(\"/kaggle/input/diagrams/Images/lumbar.png\", width=400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.412921Z","iopub.execute_input":"2024-10-11T06:03:38.413378Z","iopub.status.idle":"2024-10-11T06:03:38.436622Z","shell.execute_reply.started":"2024-10-11T06:03:38.413331Z","shell.execute_reply":"2024-10-11T06:03:38.434693Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Vertebral Levels and Their Significance**\n---\n\nUnderstanding the vertebral levels is crucial for precise diagnosis and treatment planning. Each level from L1/L2 to L5/S1 has distinct anatomical and functional characteristics.\n\n\n  *  L1-L2:\tUpper lumbar region, less mobility compared to lower levels.\n  *  L2-L3:\tMid-lumbar region, common site for **disc herniations**.\n  *  L3-L4:\tLower lumbar region, significant for **weight-bearing**.\n  *  L4-L5:\tHighly mobile, frequent site for **degenerative** changes.\n  *  L5-S1:\tJunction between lumbar spine and sacrum, critical for stability.\n\n\n","metadata":{"id":"cuvob5D0ugRI"}},{"cell_type":"code","source":"print(\"\\tLumbar Spine Labelling\\n\")\nImage(\"/kaggle/input/diagrams/Images/lumbar label.png\", width=300)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.438796Z","iopub.execute_input":"2024-10-11T06:03:38.439280Z","iopub.status.idle":"2024-10-11T06:03:38.462574Z","shell.execute_reply.started":"2024-10-11T06:03:38.439231Z","shell.execute_reply":"2024-10-11T06:03:38.460899Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Intervertebral Disc Anatomy**\n---\n\nIntervertebral disc is a cushion-like structure between the vertebrae of the spine.\n\n* **Nucleus pulposus:** The soft, jelly-like center of the intervertebral disc.\n* **Annulus fibrosus:** The tough, outer ring of the intervertebral disc.\n* **Spinal cord:** A long, bundled structure of nerves that runs through the spinal canal.\n* **Neural foramen:** A passageway for nerves to exit the spinal column.\n* **Nerve root:** The beginning of a spinal nerve, which branches off from the spinal cord.\n\n\n","metadata":{"id":"_IDlHfJ1yLCs"}},{"cell_type":"code","source":"print(\"\\tIntervertebral Disc Diagram\\n\")\nImage(\"/kaggle/input/diagrams/Images/disc .png\", width=400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:04:07.485164Z","iopub.execute_input":"2024-10-11T06:04:07.485662Z","iopub.status.idle":"2024-10-11T06:04:07.503570Z","shell.execute_reply.started":"2024-10-11T06:04:07.485616Z","shell.execute_reply":"2024-10-11T06:04:07.501933Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n**Lumbar Disc Degeneration**\n---\n\nA condition where the soft, gel-like nucleus of a lumbar disc bulges or ruptures through the outer, tougher annulus.\n\n**Zones of Degeneration**\n---\n\n* **Central zone:** The innermost part of the disc.\n* **Subarticular zone:** Between the central zone and the facet joint.\n* **Foraminal zone:** Toward the nerve root foramen.\n* **Extraforaminal zone:** Beyond the nerve root foramen.","metadata":{"id":"6-9vmpMo2VBH"}},{"cell_type":"code","source":"print(\"\\t\\tZones of Degeneration\\n\")\nImage(\"/kaggle/input/diagrams/Images/zone.png\", width=450)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.500888Z","iopub.execute_input":"2024-10-11T06:03:38.501316Z","iopub.status.idle":"2024-10-11T06:03:38.521271Z","shell.execute_reply.started":"2024-10-11T06:03:38.501261Z","shell.execute_reply":"2024-10-11T06:03:38.519573Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Degenerative Spine Conditions**\n---\n\nAs the spine ages or due to mechanical stress, certain degenerative conditions may develop in the lumbar region, affecting the spinal nerves. Your model will classify these conditions into normal/mild, moderate, or severe grades based on the compression and narrowing.\n\n1. **Canal Stenosis**\n\nSpinal canal stenosis is a condition where the spinal canal becomes narrowed, compressing the spinal cord or the cauda equina (a bundle of nerve roots). This can result in significant neurological symptoms, including pain, numbness, and muscle weakness.\n\n  * Unlike foraminal or subarticular stenosis, canal stenosis does not affect one side of the spine but can occur at specific lumbar levels (L1/L2 to L5/S1).\n\n\n\n\n","metadata":{"id":"fz5AIHLSp0Mg"}},{"cell_type":"markdown","source":"2. **Foraminal Narrowing**\n\nForaminal narrowing, or foraminal stenosis, occurs when the foramen becomes smaller due to disc degeneration, bone spurs, or ligament thickening. This can compress the spinal nerves exiting the spinal canal, leading to symptoms such as pain, numbness, or weakness.\n\n   * Left/Right Neural Foraminal Narrowing: The narrowing can happen on either the left or right side of the spine at any lumbar level (L1/L2 to L5/S1).","metadata":{}},{"cell_type":"markdown","source":"3. **Subarticular Stenosis**\n\nSubarticular stenosis involves the narrowing of the space where the nerve roots exit the spine, which may lead to impingement of the nerves before they reach the foramina. This condition is also known as lateral recess stenosis.\n\n  * Left/Right Subarticular Stenosis: Can occur on either side of the spine, usually due to disc herniation or bone spurs.\n  \n  ","metadata":{}},{"cell_type":"code","source":"print(\"\\t\\t\\t\\t\\tTypes of Stenosis\\n\")\nImage(\"/kaggle/input/diagrams/Images/stenosis.png\", width=800)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.523255Z","iopub.execute_input":"2024-10-11T06:03:38.524151Z","iopub.status.idle":"2024-10-11T06:03:38.564984Z","shell.execute_reply.started":"2024-10-11T06:03:38.524097Z","shell.execute_reply":"2024-10-11T06:03:38.563635Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**MRI Imaging of the Lumbar Spine**\n---\n\n**Magnetic Resonance Imaging (MRI)** is a valuable tool for evaluating the lumbar spine. By using strong magnetic fields and radio waves, it provides detailed images of the bones, discs, spinal cord, and surrounding tissues.\n\nKey **MRI sequences** used in lumbar spine imaging include:\n\n1. **T1-Weighted Imaging:**\n  \n   * Appearance: Bone appears bright, while soft tissues and cerebrospinal fluid (CSF) appear dark.\n  \n   * Purpose: Excellent for evaluating bone anatomy, cortical bone, and certain types of tumors.\n\n2. **T2-Weighted Imaging:**\n  \n   * Appearance: Bone appears dark, while soft tissues and CSF appear bright.\n  \n   * Purpose: Ideal for detecting inflammation, edema, fluid collections, and certain types of tumors.\n\n3. **STIR (Short T1 Inversion Recovery) Imaging:**\n  \n   * Appearance: Similar to T2-weighted, but with increased suppression of fat   \n   * Purpose: Used to better visualize inflammatory changes, edema, and lesions within the bone marrow.\n\n*Note: In the dataset, Sagital T2 and STIR images are likely the same. This is because STIR is a specific type of T2-weighted sequence that is optimized for detecting fluid-containing tissues.*","metadata":{"id":"prlFIn45qy-c"}},{"cell_type":"code","source":"print(\"\\t\\t\\t\\tMRI in Sagital T1 and T2\\n\")\nImage(\"/kaggle/input/diagrams/Images/t1-t2.png\", width=700)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.566716Z","iopub.execute_input":"2024-10-11T06:03:38.567147Z","iopub.status.idle":"2024-10-11T06:03:38.598129Z","shell.execute_reply.started":"2024-10-11T06:03:38.567101Z","shell.execute_reply":"2024-10-11T06:03:38.596598Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**MRI Views:**\n\n1. **Sagittal Plane:**\nDivides the body into left and right halves.\n\n2. **Coronal Plane:**\nDivides the body into anterior (front) and posterior (back) halves.\n\n3. **Axial Plane:**\nDivides the body into superior (top) and inferior (bottom) halves.\n\n*Note: The dataset contains sagittal and axial MRI scans, but no coronal images.*","metadata":{"id":"s-kd_TaW6o6B"}},{"cell_type":"code","source":"print(\"\\t\\tMRI Veiw Planes\\n\")\nImage(\"/kaggle/input/diagrams/Images/view.png\", width=400)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.600059Z","iopub.execute_input":"2024-10-11T06:03:38.600679Z","iopub.status.idle":"2024-10-11T06:03:38.624244Z","shell.execute_reply.started":"2024-10-11T06:03:38.600604Z","shell.execute_reply":"2024-10-11T06:03:38.622419Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Severity Classification Criteria**\n---\n\nEach condition is classified into three severity levels based on the degree of compression:\n\n   * **Normal/Mild** (0 - 0.33): Minimal or no compression, unlikely to cause significant symptoms.\n   * **Moderate** (0.34 - 0.66): Noticeable compression with potential symptoms.\n   * **Severe** (0.67 - 1.00): Significant compression, likely causing severe symptoms.\n\n","metadata":{"id":"0o6ebG6UrZgR"}},{"cell_type":"code","source":"print(\"\\t\\t\\t\\t\\t\\tSeverity\\n\")\nImage(\"/kaggle/input/diagrams/Images/severity.png\", width=800)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.626004Z","iopub.execute_input":"2024-10-11T06:03:38.626474Z","iopub.status.idle":"2024-10-11T06:03:38.669777Z","shell.execute_reply.started":"2024-10-11T06:03:38.626400Z","shell.execute_reply":"2024-10-11T06:03:38.668195Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**MRI in the Diagnosis of Spinal Stenosis**\n----\n\n**The Role of Radiologists**\n\n* **Radiologists** specialize in interpreting medical images, including MRI scans.\n* They identify and diagnose spinal stenosis and other conditions based on MRI findings.\n* Radiologists prepare **detailed reports** outlining their findings and recommendations.\n* **Neurologists** or **orthopedic surgeons** may also review MRI images and the radiologist's report.\n* They confirm the diagnosis and determine the appropriate course of treatment.","metadata":{"id":"xweX0Ws8Ails"}},{"cell_type":"code","source":"print(\"\\t\\tDisc label\\n\")\nImage(\"/kaggle/input/diagrams/Images/disc label.png\", width=350)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.675064Z","iopub.execute_input":"2024-10-11T06:03:38.676194Z","iopub.status.idle":"2024-10-11T06:03:38.706731Z","shell.execute_reply.started":"2024-10-11T06:03:38.676091Z","shell.execute_reply":"2024-10-11T06:03:38.705217Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**MRI Techniques for Spinal Stenosis**\n\n* **Visualizing the Spinal Canal:** MRI provides detailed images of the spinal canal, allowing for measurement of its size and identification of narrowing.\n\n* **Examining Nerve Roots:** MRI can visualize nerve roots exiting the spinal cord and assess their relationship to surrounding bony structures. Foraminal narrowing occurs when nerve roots are compressed by encroaching bone or soft tissues.\n\n* **Assessing Facet Joints:** MRI can evaluate the size and condition of facet joints, located between vertebrae. Subarticular stenosis occurs when these joints become enlarged or develop bony overgrowths that narrow the spinal canal.\n\n* **Detecting Soft Tissue Changes:** MRI can identify soft tissue abnormalities, such as herniated discs or spinal tumors, which can contribute to spinal stenosis.","metadata":{"id":"WXaQoZpVDNsE"}},{"cell_type":"code","source":"print(\"\\t\\tHerniated(Degenerative) Disc\\n\")\nImage(\"/kaggle/input/diagrams/Images/mri.png\", width=500)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.708979Z","iopub.execute_input":"2024-10-11T06:03:38.709908Z","iopub.status.idle":"2024-10-11T06:03:38.748664Z","shell.execute_reply.started":"2024-10-11T06:03:38.709845Z","shell.execute_reply":"2024-10-11T06:03:38.746638Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Dataset Overview:**\n---\n\n* Total Files: **147,320 files**.\n* Total Size: **35.34 GB**.\n* File Types: Primarily **DICOM** (.dcm) files and **CSV** files.\n\nThe test dataset contains MRI data from only **one patient**, while the training dataset includes MRI data from **1975 patients**. Both datasets contain sagittal and axial MRI scans in T1 and T2 weighting.","metadata":{"id":"nlduRZnSanh_"}},{"cell_type":"markdown","source":"**Directory Structure:**\n\n* **kaggle/input/rsna-2024-lumbar-spine-degenerative-classification:** This is the root directory of the dataset.\n * **test_images:**\n   * **44036939 (study_id):** A subdirectory containing test images.\n     * **2828203845 (series_id):** A subdirectory within the previous one, likely representing a specific test case.\n       * **1.dcm, 2.dcm, 3.dcm, ... :** DICOM image files.\n     * **... :** Indicates additional subdirectories with similar structure.\n * **test_series_descriptions.csv:** A CSV file containing descriptions for the test series.\n * **train_images:**\n   * **4003253 (study_id):** A subdirectory containing training images.\n     * **702807833 (series_id):** A subdirectory within the previous one, likely representing a specific training case.\n       * **1.dcm, 2.dcm, 3.dcm, ... :** DICOM image files.\n     * **... :** Indicates additional subdirectories with similar structure.\n   * **... :** Indicates additional subdirectories with similar structure.\n * **train_label_coordinates.csv:** A CSV file containing label coordinates for the training MRI images.\n * **train_series_descriptions.csv:** A CSV file containing descriptions for the training series.\n * **train.csv:** A CSV file likely containing information about the training data.","metadata":{"id":"xKylASZnhrXP"}},{"cell_type":"code","source":"print(\"\\t\\t\\tDataset Structure\\n\")\nImage(\"/kaggle/input/diagrams/Images/dataset.png\", width=500)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:38.753014Z","iopub.execute_input":"2024-10-11T06:03:38.755322Z","iopub.status.idle":"2024-10-11T06:03:38.779042Z","shell.execute_reply.started":"2024-10-11T06:03:38.755242Z","shell.execute_reply":"2024-10-11T06:03:38.777832Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# standard libraries \nimport numpy as np\nimport pandas as pd\n","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:03:38.781168Z","iopub.execute_input":"2024-10-11T06:03:38.782088Z","iopub.status.idle":"2024-10-11T06:03:40.370665Z","shell.execute_reply.started":"2024-10-11T06:03:38.782021Z","shell.execute_reply":"2024-10-11T06:03:40.369033Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# read data\npath = '../input/rsna-2024-lumbar-spine-degenerative-classification/'\n\ndf_train  = pd.read_csv(path + 'train.csv')\ndf_train_label = pd.read_csv(path + 'train_label_coordinates.csv')\ndf_train_desc  = pd.read_csv(path + 'train_series_descriptions.csv')\ndf_test_desc   = pd.read_csv(path + 'test_series_descriptions.csv')\ndf_sub         = pd.read_csv(path + 'sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:03:40.372520Z","iopub.execute_input":"2024-10-11T06:03:40.373259Z","iopub.status.idle":"2024-10-11T06:03:40.596006Z","shell.execute_reply.started":"2024-10-11T06:03:40.373193Z","shell.execute_reply":"2024-10-11T06:03:40.594598Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Key Files and Their Contents:**\n---\n\n1. **train.csv:**\n\n  * Purpose: Contains labels for the training set.\n  * 26 Columns:\n     * **study_id**: Unique identifier for each study, which may include multiple image series.\n     * **[condition]_[level]**: Labels for specific conditions, with severity levels: Normal/Mild, Moderate, or Severe.\n[spinal_canal_stenosis/left_neural_foraminal_narrowing/right_neural_foraminal_narrowing/left_subarticular_stenosis/right_subarticular_stenosis]_[l1_l2/l2_l3/l3_l4/l4_l5/l5_s1]\n\n *Note that some entries may have incomplete labels.*","metadata":{"id":"lmudo1cQe2zw"}},{"cell_type":"code","source":"df_train.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:03:40.597551Z","iopub.execute_input":"2024-10-11T06:03:40.597971Z","iopub.status.idle":"2024-10-11T06:03:40.642915Z","shell.execute_reply.started":"2024-10-11T06:03:40.597924Z","shell.execute_reply":"2024-10-11T06:03:40.641600Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"2. **train_label_coordinates.csv:**\n\n  * Purpose: Provides coordinates for labeled regions in training images.\n  * 7 Columns:\n    * **study_id**: Identifies the study.\n    * **series_id**: Identifies the specific series within the study.\n    * **instance_number**: Indicates the slice number within the 3D stack of MRI.\n    * **condition**: Specifies conditions like spinal canal stenosis, left/right neural foraminal narrowing, and left/right subarticular stenosis, with some conditions evaluated on spine.\n    * **level**: Specifies the vertebral level involved (e.g., l3_l4).\n    * **[x/y]**: Coordinates marking the center of the labeled region of MRI.","metadata":{"id":"UvQIFpUle_bb"}},{"cell_type":"code","source":"df_train_label.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:03:40.645024Z","iopub.execute_input":"2024-10-11T06:03:40.645477Z","iopub.status.idle":"2024-10-11T06:03:40.668043Z","shell.execute_reply.started":"2024-10-11T06:03:40.645406Z","shell.execute_reply":"2024-10-11T06:03:40.666342Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"3. **[train/test]_series_descriptions.csv:**\n\n  * Purpose: Provides metadata on each series.\n  * 3 Columns:\n    * **study_id**: Study identifier.\n    * **series_id**: Series identifier.\n    * **series_description**: Describes the scan orientation, which is essential for understanding the anatomical view (e.g., Axial T2, Sagittal T1, Sagittal T2/STIR).","metadata":{"id":"QS4EWIv2e_LX"}},{"cell_type":"code","source":"df_train_desc.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:03:40.670681Z","iopub.execute_input":"2024-10-11T06:03:40.671781Z","iopub.status.idle":"2024-10-11T06:03:40.688221Z","shell.execute_reply.started":"2024-10-11T06:03:40.671707Z","shell.execute_reply":"2024-10-11T06:03:40.686776Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_test_desc.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:03:40.690088Z","iopub.execute_input":"2024-10-11T06:03:40.690674Z","iopub.status.idle":"2024-10-11T06:03:40.704973Z","shell.execute_reply.started":"2024-10-11T06:03:40.690607Z","shell.execute_reply":"2024-10-11T06:03:40.703540Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"4. **sample_submission.csv:**\n\n  * Purpose: Provides a template for how predictions should be formatted.\n  * 4 Columns:\n    * **row_id**: A unique identifier combining study ID, condition, and level (e.g., 4003253_spinal_canal_stenosis_l3_l4).\n    * **[normal_mild/moderate/severe]** - The three prediction columns.","metadata":{"id":"JokRL01ifI8V"}},{"cell_type":"code","source":"df_sub.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:03:40.706920Z","iopub.execute_input":"2024-10-11T06:03:40.707574Z","iopub.status.idle":"2024-10-11T06:03:40.724248Z","shell.execute_reply.started":"2024-10-11T06:03:40.707505Z","shell.execute_reply":"2024-10-11T06:03:40.722659Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"5. **[train/test]_images/[study_id]/[series_id]/[instance_number].dcm:**\n\n  \n  * **train_images**: Contains the training images in DICOM format.\n  * **test_images**: Contains the testing images in DICOM format.\n  * Purpose: DICOM files containing the MRI images for each study, organized by study and series.\n","metadata":{"id":"MSPs39D3e--f"}},{"cell_type":"code","source":"print(\"\\t\\t\\t[Train/Test]_images Folder\\n\")\nImage(\"/kaggle/input/diagrams/Images/train_test img.png\", width=800)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:40.726017Z","iopub.execute_input":"2024-10-11T06:03:40.726476Z","iopub.status.idle":"2024-10-11T06:03:40.756917Z","shell.execute_reply.started":"2024-10-11T06:03:40.726401Z","shell.execute_reply":"2024-10-11T06:03:40.755344Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**DICOM**\n\n* **DICOM (Digital Imaging and Communications in Medicine):** A global standard for handling, storing, transmitting, and displaying medical imaging data.\n\n* **File Format and Protocol**: DICOM files combine image data with metadata (patient information, scan details, etc.), ensuring comprehensive information is included with each image.\n\n* **Widely Used:** Predominant in radiology and other medical imaging fields, including MRI, CT, ultrasound, and X-rays.","metadata":{"id":"TfIMm9WHf-Hh"}},{"cell_type":"code","source":"print(\"\\t\\t\\tRepresentation of Metadata and MRI image using DICOM viewer application\\n\")\nImage(\"/kaggle/input/diagrams/Images/dicom.png\", width=900)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:40.758582Z","iopub.execute_input":"2024-10-11T06:03:40.759030Z","iopub.status.idle":"2024-10-11T06:03:40.790073Z","shell.execute_reply.started":"2024-10-11T06:03:40.758983Z","shell.execute_reply":"2024-10-11T06:03:40.788763Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import pydicom\ndicom_file_path = '../input/rsna-2024-lumbar-spine-degenerative-classification/train_images/4003253/702807833/7.dcm'\ndicom_data = pydicom.dcmread(dicom_file_path)\nprint(\"This is the Metadata\")\nprint(dicom_data)","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2024-10-11T06:03:40.792272Z","iopub.execute_input":"2024-10-11T06:03:40.792790Z","iopub.status.idle":"2024-10-11T06:03:42.638934Z","shell.execute_reply.started":"2024-10-11T06:03:40.792733Z","shell.execute_reply":"2024-10-11T06:03:42.637463Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nprint(\"This is the corresponding MRI\")\npixel_array = dicom_data.pixel_array\n# Image ko display karna\nplt.imshow(pixel_array, cmap='gray')\nplt.show()\n","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2024-10-11T06:03:42.640673Z","iopub.execute_input":"2024-10-11T06:03:42.641195Z","iopub.status.idle":"2024-10-11T06:03:43.092956Z","shell.execute_reply.started":"2024-10-11T06:03:42.641133Z","shell.execute_reply":"2024-10-11T06:03:43.091597Z"},"trusted":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\n# List out all of the Studies we have on patients.\npart_1 = os.listdir('/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_images')\npart_1 = list(filter(lambda x: x.find('.DS') == -1, part_1))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:43.094842Z","iopub.execute_input":"2024-10-11T06:03:43.095404Z","iopub.status.idle":"2024-10-11T06:03:43.243091Z","shell.execute_reply.started":"2024-10-11T06:03:43.095339Z","shell.execute_reply":"2024-10-11T06:03:43.241410Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_meta_f = pd.read_csv('/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_series_descriptions.csv')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:43.245168Z","iopub.execute_input":"2024-10-11T06:03:43.245769Z","iopub.status.idle":"2024-10-11T06:03:43.265370Z","shell.execute_reply.started":"2024-10-11T06:03:43.245702Z","shell.execute_reply":"2024-10-11T06:03:43.263513Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"p1 = [(x, f\"/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_images/{x}\") for x in part_1]\nmeta_obj = { p[0]: { 'folder_path': p[1], \n                    'SeriesInstanceUIDs': [] \n                   } \n            for p in p1 }","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:43.268143Z","iopub.execute_input":"2024-10-11T06:03:43.269079Z","iopub.status.idle":"2024-10-11T06:03:43.282875Z","shell.execute_reply.started":"2024-10-11T06:03:43.269013Z","shell.execute_reply":"2024-10-11T06:03:43.281541Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"for m in meta_obj:\n    meta_obj[m]['SeriesInstanceUIDs'] = list(\n        filter(lambda x: x.find('.DS') == -1, \n               os.listdir(meta_obj[m]['folder_path'])\n              )\n    )","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:43.290409Z","iopub.execute_input":"2024-10-11T06:03:43.290978Z","iopub.status.idle":"2024-10-11T06:03:48.895052Z","shell.execute_reply.started":"2024-10-11T06:03:43.290927Z","shell.execute_reply":"2024-10-11T06:03:48.893807Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from tqdm import tqdm\n# grabs the correspoding series descriptions\nfor k in tqdm(meta_obj):\n    for s in meta_obj[k]['SeriesInstanceUIDs']:\n        if 'SeriesDescriptions' not in meta_obj[k]:\n            meta_obj[k]['SeriesDescriptions'] = []\n        try:\n            meta_obj[k]['SeriesDescriptions'].append(\n                df_meta_f[(df_meta_f['study_id'] == int(k)) & \n                (df_meta_f['series_id'] == int(s))]['series_description'].iloc[0])\n        except:\n            print(\"Failed on\", s, k)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:48.896909Z","iopub.execute_input":"2024-10-11T06:03:48.897352Z","iopub.status.idle":"2024-10-11T06:03:52.844374Z","shell.execute_reply.started":"2024-10-11T06:03:48.897307Z","shell.execute_reply":"2024-10-11T06:03:52.843148Z"},"trusted":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"meta_obj[list(meta_obj.keys())[1]]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:52.846081Z","iopub.execute_input":"2024-10-11T06:03:52.846566Z","iopub.status.idle":"2024-10-11T06:03:52.855264Z","shell.execute_reply.started":"2024-10-11T06:03:52.846517Z","shell.execute_reply":"2024-10-11T06:03:52.853760Z"},"trusted":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import glob\n\npatient = df_train.iloc[1]\nptobj = meta_obj[str(patient['study_id'])]\n\nim_list_dcm = {}\nfor idx, i in enumerate(ptobj['SeriesInstanceUIDs']):\n    im_list_dcm[i] = {'images': [], 'description': ptobj['SeriesDescriptions'][idx]}\n    images = glob.glob(f\"{ptobj['folder_path']}/{ptobj['SeriesInstanceUIDs'][idx]}/*.dcm\")\n    for j in sorted(images, key=lambda x: int(x.split('/')[-1].replace('.dcm', ''))):\n        im_list_dcm[i]['images'].append({\n            'SOPInstanceUID': j.split('/')[-1].replace('.dcm', ''), \n            'dicom': pydicom.dcmread(j) })","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:52.856710Z","iopub.execute_input":"2024-10-11T06:03:52.857228Z","iopub.status.idle":"2024-10-11T06:03:53.523513Z","shell.execute_reply.started":"2024-10-11T06:03:52.857158Z","shell.execute_reply":"2024-10-11T06:03:53.522134Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Function to display images\ndef display_images(images, title, max_images_per_row=4):\n    # Calculate the number of rows needed\n    num_images = len(images)\n    num_rows = (num_images + max_images_per_row - 1) // max_images_per_row  # Ceiling division\n\n    # Create a subplot grid\n    fig, axes = plt.subplots(num_rows, max_images_per_row, figsize=(5, 1.5 * num_rows))\n    \n    # Flatten axes array for easier looping if there are multiple rows\n    if num_rows > 1:\n        axes = axes.flatten()\n    else:\n        axes = [axes]  # Make it iterable for consistency\n\n    # Plot each image\n    for idx, image in enumerate(images):\n        ax = axes[idx]\n        ax.imshow(image, cmap='gray')  # Assuming grayscale for simplicity, change cmap as needed\n        ax.axis('off')  # Hide axes\n\n    # Turn off unused subplots\n    for idx in range(num_images, len(axes)):\n        axes[idx].axis('off')\n    fig.suptitle(title, fontsize=16)\n\n    plt.tight_layout()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:53.525209Z","iopub.execute_input":"2024-10-11T06:03:53.525645Z","iopub.status.idle":"2024-10-11T06:03:53.535664Z","shell.execute_reply.started":"2024-10-11T06:03:53.525598Z","shell.execute_reply":"2024-10-11T06:03:53.534076Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Representation of MRI slices in the data\n\nIn this section, we will load and visualize the patient data, including the corresponding scans and diagnoses.","metadata":{"_kg_hide-output":true}},{"cell_type":"code","source":"\nfor i in im_list_dcm:\n    display_images([x['dicom'].pixel_array for x in im_list_dcm[i]['images']], \n                   im_list_dcm[i]['description'])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:03:53.537237Z","iopub.execute_input":"2024-10-11T06:03:53.537781Z","iopub.status.idle":"2024-10-11T06:04:00.615642Z","shell.execute_reply.started":"2024-10-11T06:03:53.537721Z","shell.execute_reply":"2024-10-11T06:04:00.614066Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import pydicom as dicom\nfrom matplotlib.colors import Normalize\nimport matplotlib.patches as patches\n\ntrain_label_coordinates=df_train_label\ntrain_label_coordinates['series_description'] = df_train_desc.series_description","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:04:00.617179Z","iopub.execute_input":"2024-10-11T06:04:00.617683Z","iopub.status.idle":"2024-10-11T06:04:00.630829Z","shell.execute_reply.started":"2024-10-11T06:04:00.617630Z","shell.execute_reply":"2024-10-11T06:04:00.629003Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def mrt(id, ser, inst):\n    lag=20\n    path2 = path+'train_images/' + str(id) +'/' + str(ser)+'/' + str(inst) + '.dcm'\n\n    ds = dicom.dcmread(path2)\n    fig, ax = plt.subplots(figsize=(16, 8))\n    from matplotlib.colors import LogNorm \n\n    # Use Normalize for linear scaling\n    norm = Normalize(vmin=ds.pixel_array.min(), vmax=ds.pixel_array.max())\n    ax.imshow(ds.pixel_array, cmap='gray', norm=norm)  # Display the image with 'gray' colormap\n\n    # Create a legend\n    legend_elements = []\n\n    # Plot the coordinates for the current condition\n    ab = train_label_coordinates[(train_label_coordinates.study_id==id) & \n                                          (train_label_coordinates.instance_number==inst)&\n                                         (train_label_coordinates.series_id==ser)]\n\n    a = 25 * max(ds.pixel_array.shape)/640\n    for _, row in ab.iterrows():\n        x, y = row['x'], row['y']\n\n        rect2 = patches.Rectangle((x - a, y - a), 2*a, 2*a, linewidth=2, edgecolor='white', facecolor='none')\n        rect1 = patches.Rectangle((x - a, y - a), 2*a, 2*a, linewidth=2, facecolor='white', alpha = 0.25)\n\n        ax.add_patch(rect2)\n        ax.add_patch(rect1)\n\n        # Add the condition to the legend\n        legend_elements.append(patches.Patch(facecolor='none', edgecolor='r', ))\n\n    # Add title\n    title = f\"{ab.series_description.unique()}, Study: {id}, Series: {ser}, Instance: {inst}\"\n    ax.set_title(title, fontsize=20)\n\n    # Display additional columns:\n    for _, row in ab.iterrows():\n        text = f\"level {row['level']}, {row['condition']}\"\n        ax.text(row['x'] + lag, row['y']+np.random.randint(-15, 15), text, fontsize=10, color='white', verticalalignment='center_baseline')\n    \n    plt.show() ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:04:00.632723Z","iopub.execute_input":"2024-10-11T06:04:00.633265Z","iopub.status.idle":"2024-10-11T06:04:00.654321Z","shell.execute_reply.started":"2024-10-11T06:04:00.633189Z","shell.execute_reply":"2024-10-11T06:04:00.652854Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Ploting the labels on MRI image\n\nNow that we have patient data loaded, we’ll proceed to visualize the patient DICOM images along with the locations of any annotated pathologies.","metadata":{}},{"cell_type":"code","source":"#Case #1\nid = 4003253\nser= 702807833\ninst=8\n\nmrt(id, ser, inst)","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:04:00.656387Z","iopub.execute_input":"2024-10-11T06:04:00.656888Z","iopub.status.idle":"2024-10-11T06:04:01.263315Z","shell.execute_reply.started":"2024-10-11T06:04:00.656839Z","shell.execute_reply":"2024-10-11T06:04:01.262001Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#Case #2\nid = 4003253\nser= 2448190387\ninst=11\n\nmrt(id, ser, inst)","metadata":{"execution":{"iopub.status.busy":"2024-10-11T06:04:01.265207Z","iopub.execute_input":"2024-10-11T06:04:01.265698Z","iopub.status.idle":"2024-10-11T06:04:01.814633Z","shell.execute_reply.started":"2024-10-11T06:04:01.265644Z","shell.execute_reply":"2024-10-11T06:04:01.813235Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Animated video of the combined MRI instances","metadata":{}},{"cell_type":"code","source":"import glob\nfrom matplotlib import animation, rc\n\nrc('animation', html='jshtml')\n\ndef load_dicom(filename):\n    ds = pydicom.dcmread(filename)\n    return ds.pixel_array\n\ndef load_dicom_line(path):\n    t_paths = sorted(\n        glob.glob(os.path.join(path, \"*\")), \n        key=lambda x: int(os.path.splitext(os.path.basename(x))[0].split(\"-\")[-1]),\n    )\n    images = []\n    for filename in t_paths:\n        data = load_dicom(filename)\n        if data.max() == 0:\n            continue\n        images.append(data)\n    return images\n\ndef create_animation(ims):\n    fig = plt.figure(figsize=(6, 6))\n    plt.axis('off')\n    im = plt.imshow(ims[0], cmap=\"gray\")\n\n    def animate_func(i):\n        im.set_array(ims[i])\n        return [im]\n\n    return animation.FuncAnimation(fig, animate_func, frames=len(ims), interval=1000//24)\n\npath_to_folder = \"/kaggle/input/rsna-2024-lumbar-spine-degenerative-classification/train_images/100206310/1792451510\"\nimages = load_dicom_line(path_to_folder)\nanim = create_animation(images)\nanim","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-10-11T06:04:01.816479Z","iopub.execute_input":"2024-10-11T06:04:01.817013Z","iopub.status.idle":"2024-10-11T06:04:04.711050Z","shell.execute_reply.started":"2024-10-11T06:04:01.816951Z","shell.execute_reply":"2024-10-11T06:04:04.708809Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Conclusion**\n---\n\nIn this notebook, we covered the key anatomical structures of the lumbar spine,  along with their significance in lumbar spine degeneration. We also explored various degenerative conditions such as canal stenosis, foraminal narrowing, and subarticular stenosis, which are critical for detecting and classifying lumbar spine conditions using MRI scans. Additionally, we provided an overview of the dataset, which includes CSV files and DICOM (.dcm) files containing metadata. This dataset comprises 1975 patients, offering a valuable resource for training and evaluating models for lumbar spine condition classification. Have fun with the challenge!\n\n---\n\n","metadata":{"id":"udjTlHQZPk_J"}}]}