{"cells":[{"metadata":{},"cell_type":"markdown","source":"# RANZCR CLiP - Catheter and Line Position Challenge - Exploratory Data Analysis\n\nQuick Exploratory Data Analysis for [RANZCR CLiP - Catheter and Line Position Challenge](https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification) challenge    \n\nIn this competition, you’ll detect the presence and position of catheters and lines on chest x-rays. Use machine learning to train and test your model on 40,000 images to categorize a tube that is poorly placed."},{"metadata":{},"cell_type":"markdown","source":"![](https://storage.googleapis.com/kaggle-competitions/kaggle/23870/logos/header.png?t=2020-12-01-04-28-05)"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"top\"></a>\n\n<div class=\"list-group\" id=\"list-tab\" role=\"tablist\">\n<h3 class=\"list-group-item list-group-item-action active\" data-toggle=\"list\" style='color:white; background:#6E848D; border:0' role=\"tab\" aria-controls=\"home\"><center>Quick Navigation</center></h3>\n\n* [Overview](#1)\n    \n* [Annotations](#2)\n    \n* [ETT - Abnormal](#4)\n* [ETT - Borderline](#5)\n* [ETT - Normal](#6)\n* [NGT - Abnormal](#7)\n* [NGT - Borderline](#8)\n* [NGT - Incompletely Imaged](#9)\n* [NGT - Normal](#10)\n* [CVC - Abnormal](#11)\n* [CVC - Borderline](#12)\n* [CVC - Normal](#13)\n* [Swan Ganz Catheter Present](#14)\n    \n    \n* [Submission](#100)"},{"metadata":{},"cell_type":"markdown","source":"<a id=\"1\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>Overview<center><h2>"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport ast\nimport random\n\nimport numpy as np\nimport pandas as pd\nimport cv2\nimport matplotlib.pyplot as plt\nimport seaborn as sns","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"BASE_DIR = \"../input/ranzcr-clip-catheter-line-classification/\"\nos.listdir(BASE_DIR)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(os.path.join(BASE_DIR, \"train.csv\"), index_col=0)\ndf_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.iloc[:, :-1].sum()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def visualize_batch(image_ids):\n    plt.figure(figsize=(16, 12))\n    \n    for ind, image_id in enumerate(image_ids):\n        plt.subplot(3, 4, ind + 1)\n        image = cv2.imread(os.path.join(BASE_DIR, \"train\", f\"{image_id}.jpg\"))\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n        plt.imshow(image)\n        plt.axis(\"off\")\n    \n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def print_statistics(df, col):\n    print(\"Distribution:\")\n    print(df[col].value_counts())\n    print()\n    print(f\"Percent of 1: {df[col].mean():.5f}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"2\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>Annotations<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"**train_annotations.csv** these are segmentation annotations for training samples that have them. They are included solely as additional information for competitors."},{"metadata":{"trusted":true},"cell_type":"code","source":"df_annot = pd.read_csv(os.path.join(BASE_DIR, \"train_annotations.csv\"))\ndf_annot.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def plot_image_with_annotations(row_ind):\n    row = df_annot.iloc[row_ind]\n    image_path = os.path.join(BASE_DIR, \"train\", row[\"StudyInstanceUID\"] + \".jpg\")\n    label = row[\"label\"]\n    data = np.array(ast.literal_eval(row[\"data\"]))\n    \n    plt.figure(figsize=(10, 5))\n    image = cv2.imread(image_path)\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    plt.subplot(1, 2, 1)\n    plt.imshow(image)\n    plt.subplot(1, 2, 2)\n    plt.imshow(image)\n    plt.scatter(data[:, 0], data[:, 1])\n    \n    plt.suptitle(label, fontsize=15)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_image_with_annotations(8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for i in range(5):\n    plot_image_with_annotations(random.randint(0, 15000))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"4\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>ETT - Abnormal<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"endotracheal tube placement abnormal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"ETT - Abnormal\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"5\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>ETT - Borderline<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"endotracheal tube placement borderline abnormal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"ETT - Borderline\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"6\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>ETT - Normal<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"endotracheal tube placement normal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"ETT - Normal\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"7\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>NGT - Abnormal<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"nasogastric tube placement abnormal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"NGT - Abnormal\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"8\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>NGT - Borderline<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"nasogastric tube placement borderline abnormal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"NGT - Borderline\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"9\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>NGT - Incompletely Imaged<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"nasogastric tube placement inconclusive due to imaging"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"NGT - Incompletely Imaged\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"10\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>NGT - Normal<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"nasogastric tube placement borderline normal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"NGT - Normal\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"11\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>CVC - Abnormal<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"central venous catheter placement abnormal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"CVC - Abnormal\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"12\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>CVC - Borderline<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"central venous catheter placement borderline abnormal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"CVC - Borderline\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"13\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>CVC - Normal<center><h2>"},{"metadata":{},"cell_type":"markdown","source":"central venous catheter placement normal"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"CVC - Normal\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"14\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>Swan Ganz Catheter Present<center><h2>"},{"metadata":{"trusted":true},"cell_type":"code","source":"col_name = \"Swan Ganz Catheter Present\"\nprint_statistics(df_train, col_name)\ntmp_df = df_train[df_train[col_name] == 1]\nvisualize_batch(random.sample(tmp_df.index.tolist(), 12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"100\"></a>\n<h2 style='background:#6E848D; border:0; color:white'><center>Submission<center><h2>"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_submission = pd.read_csv(os.path.join(BASE_DIR, \"sample_submission.csv\"), index_col=0)\ndf_submission","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_submission.to_csv(\"submission.csv\")","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}