{"cells":[{"metadata":{},"cell_type":"markdown","source":"### Introduction\n\n\nThe global cancer burden is estimated to have risen to 18.1 million new cases and 9.6 million deaths in 2018. One in 5 men and one in 6 women worldwide develop cancer during their lifetime, and one in 8 men and one in 11 women die from the disease.\nThe clinical radiologists and radiation oncologists who treat cancers have catheters and lines inserted during the course of emergency  in the patients during treatment. If not positioned correctly they can lead to serious complications\n\nThe dataset consist of insertion of 3 catheters\n\n**Endotracheal tube**-An endotracheal tube is a flexible plastic tube that is placed through the mouth into the trachea (windpipe) to help a patient breathe. \n\n**Nasogastric tube**-A nasogastric  tube is a flexible tube of rubber or plastic that is passed through the nose, down through the esophagus, and into the stomach.\n\n**Central Venous Catheter**-A central venous catheter is a thin, flexible tube that is inserted into a vein, usually below the right collarbone, and guided (threaded) into a large vein above the right side of the heart called the superior vena cava.\n"},{"metadata":{},"cell_type":"markdown","source":"What we need to do? \nWe will detect the presence and position of catheters and lines on chest x-rays i.e we will predict predict a probability of the tube  placed in a Normal position ,Abnormal Position, Boarderline.\n\n Metric: Area Under the ROC curve"},{"metadata":{},"cell_type":"markdown","source":"![auc.png](attachment:auc.png)","attachments":{"auc.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAWkAAAFKCAIAAABQITmBAAAMZklEQVR42uzdP2sbaR7A8dGSd2ECyyKlOFzdNUaxr9hOPjBpcoRttpPZJlKjLmU6N3KaxerSLOHSHIazui3WRKS6SqSIzHIQ8jrmsGb0fyxnFEkZzXw+bOGVHhtnsvny/OaZrB+EYRgApPSdSwBoB6AdgHYA2gGgHYB2ANoBaAegHQDaAWgHoB2AdqRWGvHbA9v3t+vvvuRP3wNXCgpeirlX/vv38K9/aAdwXyxWCId2QKFLkRiO89If2gFicb8oFkdHR9fX11/+WdoBxYpF4rYibTi0A4qyrVhihXBoBxRiW7H2cGgHFC4Wc1YLh3ZADmeQ7dAOyPO2Yr1zinZAsWKx9nBoB+S8FBsKh3ZAzmOxoXBoB+Q5FpsLh3bAmmORkVJsOhzaAfmMxabDoR2Qw1hsIRzagVjkLRbbCYd2IBa5isXWwqEdiEV+YrHNcGgHepFDWwiHdiAWwqEd6EUhYzFnO+HQDsQC7UAvimprc4p2sMO9EIsshEM70Avh0A4MI8KxRX4O/upuzh+XFj0+v0la3D39snULK+9ct2vJiP5ZHovpf/wHluVwBEFQCsMwaxdl/DO4M/i9zbWj0uwlvVNtD941ylMxOO4kf4n6VXhRm8lG0srZL5erLYZGZDYch4eH9h2bVm0PwsigXR2+0muedSeBiXMwWRaGV/Xo3c7xaTdYWHmblHDuS+7a/mJJOGwudn3H4X7H+pUbL+rNYQD6H2+CWjkIumfxzqR+Nb11qF0M2v3hpqXz8rxVa8yunN6LlBvvwob9BRkLh3ZszP6jYSi6/x7tOVq1ucz845/VZq8XBL0PgyAoj1cG9Se1nIwkepHjcGjHmnVP46kjDsDNx/5MSmbi8Wg/CHqjLUowWpn5dEiGcGjHuvSalVJz9v7HRS1Xv0K9EA7t2LgdPRNZIRl6UeRwOGdZVy4mJyK9ZmXyRMZwLgniuWTO7DyzbOU3SMaSgxJHJMKhHWtWbryOzlOnTmgrf4kPbf/1n7kkTI5Vohsc45WTT85OMpyqCod2bLgeL6LnNjov463H+JVeszJ5kmPmoY/4AGa8MugczzxK2j0tlU673zYZfmu/lcyGQzvWrfakPrd7qF1MngMbP2deSXroo3YxebSsMnkm/a5HUiUj39uN7H+T2rHmeLTiP//jrUdQu0h6OnT46OjccUy58W7yyOn03ZT1HdtIhjllXfx9lqJYcu/TxRGORff+fRZntJKBcKxCO4qVDNUQDu3ARkM4tIONVUMyhEM7sNEQDu1ANdjlcGiH8QTh0A7VQDi0A9UQDu1ANchbOLRDNRAO7VANMi834dAO1YAV+Tv4WQ+Hvxefgzkll78u+45MV8NlyUE48jSnaIdqIBxmFuFAOOw7VMNlEQ7tIEU4VEM4tAPbDeG4LsKvVDtsNxCOVbhXKhwIh32HaiAc9h3CgXBoh3AIh3CYWVgpHKohHNqB7QaxwobDzCIcoB3CwbbmFBdBO4SD1OEo8pwyzf2OjYRDNYTDvgPhQDi0QzgQDu0QDoRDO4QD4dAO4UA4tAPhEA60QzgQDu0A4dAOmw6EQzuEA+HQDoRDONAO4WA54dCOzRIO0I5VNh3kbE5xEbTDtELqcJhTtMO0gnBoRyY3HcIhHGjHKtMKwoF2pA6HTYdwoB2pCYdwoB2mFeFAO0wrCId2mFYQDu0wrSAc2mHTgXCgHTYdwoF22HRwD+HQjm1vOoQDtMO0Urg5xUXQDtMKqcNhTtEOmw6EQztsOhAO7bDpQDjQDpsO4UA7EA60Y/sDi02HcKAdCAfaYdOBcGgHCId22HQgHNoBwoF2pNp0IBxoxyoMLLtCOLQD0I4dH1hsOrI/p7gI2gGpw2FO0Y4sbjoQDrRjFQYW4UA7EA60YysDi02HcKAdCAfagXCgHQYWhEM7QDjQDoQD7TCwCAfagXCgHbCUcGiHgQW0AzY8p7gI2gGpw2FOKUo7TktLPD6/CW7OHy++cdqdfIXEBcNPNbAIBwXdd+w/KgeDD73FNzrHs/1Y1GtWttwPhINttuMiHLmqR6/Ur8YvXdQmC8cvj9Z1Xs6modoexCsG7Wrcj7Ouqy8cuN8RqbVGafgwSF5RbryI+9L/eLPFgQXhIMvtSDv0bI+bHcJBhttxc/5zM7oDUn9SS17SPT3uLF+BcLALHqzji3SOS53pf6+2W7Nh6DUrpebMgsGFdAgHhW/HtPpVeE8Wqu3Bu8Y25hWns8JBxmeWqeOX5HDE5yzRIUuvWXFCKxxoRwrlxuvoHMYJrXCgHSnrER/Rzj8BYmARDrRjqdqTKB62HsKBdqSLx+j5sY1vPdga4SigUhiGmfueSqXog6/53sws8DUODw+ztu/YBuHYzpziIphZIHU4zCnaAcKBdiAcaMdq3OwQDrQD4UA7EA60A+FAO3LFzQ7hQDsQDrQD4UA7EA7IVTv8RAXhQDu+ihulX0k4MLMA2sGG5xQXAe0gdTjMKRS0HZ4KEw60A+FAOxAOtAPhAO0QDuGg8O1wo1Q40A6EA+1AONAOhANy1Q5/fVY40I6v4kapcKAdCAfagXCgHewW4UA7AO1YH0+UJs4pLgLaQepwmFPQDoQD7UA40A6EA+3YYW6UCgfagXCgHQgH2oFwgHYIBxS+HUX+33YIB9qxBkU7ZBEOtINVCAfaAWgHG55TXAS0g9ThMKegHetRnKfRhQPtQDjQDoQD7UA4IK0HLoFwZMrnt788e9Uffrj//M2vT/eSl70/O2pdTr9w5+K5lcu+KPYdwrG76ej93h993P+99zk5Bkdz4bhd/OrZ0dHZ+3tX9l89++XtZ/+ZFbQdOT5kKfqoEqfj5OQk+nP+2/vFbUkcg/3nb65Hzk6idy9bk3pMVgYnZ+OVb57v+1Nv32HHkdt0HLWOohpcXs/tJH6L55mTs+nJ46A1asLl63hTMbXyunUwXrn39NdrM4t2CEde03EQHCTG4/31aM/x08Hsp+5Vf4zi0f/fp+mVwy+GdghHrsVbhf3vH95uJRLi8fnTn9EHPzxc2DnsPfwh+uDPT5+nVkqHdghH/tMRbRX2f6wOw3Bwx9iCdiAcS9IhHlm3e8935OyQRTjmbmX0Xz07ejX71uXrtz8dPN2L55J+PJcczI4tc/PMkpXYd5DDdCQaP+jx8Pv92Rcmnz8+VolucIxXLh7zoh27PKe4CMkDy9QzG9MPboxasff055PR7mTqObDphz7iA5jxyuCyNfMs2PuzhUfIKMjM4gZH/nx++3ruXsfIwdFJcHkZxePp7dhy0Do7uRyG4rJ1NL9VmXno46D15vmfw8fbF8agE5fcvkM4ciF+Dn0xHZMbppMp5aCV9HTo8NHR1ux57N7TXyePnI7cbm1azm3XoBSGYea+p1Ip+iDxe9vde6XCwQ45PDzM1b5DOMDMYlQB7UA40A6EA7RDOEA7hAO0QzhAO7ZkVw5ohQPtQDhAO4QDtCObhAPtANCODc8pLgLakV3ZPGRxgwPtQDhAO4QDtEM4QDuEA7RDOEA7EA7IYTuycEArHGDfIRygHcIB2iEcoB3CAdpRVMIBO9yO3f1ZcKAdxZpTXATQjtThMKeAdggHaIdwgHYIB2iHcIB2ZM+mD2iFA+w7hAO0QzhAO4QDtEM4gEK2QzhAO4QDtCPJeg9ohQPsO1YhHKAdgHZskv8ZB2jHKuEwp4B2CAdoh3CAdmxa2gNa4QDtsOMA7RAO0A7hAO0QDqCQ7RAO0I559x6yCAdohx0HaIdwgHZkk3CAdgDaseE5xe8ZaEfqcJhTQDvuN31AKxygHXYcoB3CAYVUCsMwc99TqeQ3Br6te8vgjBbI6cwCZNCDXdwsAfYdgHYA2gGgHYB2ANoBaAegHQDaAWgHa9I9LSU47Q7fvDl/fOd7dy8oPT6/cWG1g3y7+dhf8u7gQ2/xxc7xbD8W9ZoV/dAOiqHaHoRTLmoz79av4tev6nE/Xs6mYfLpg3Y17sdZ12XVDojUWqM0fBgkryg3XsR96X+089AOSG//UdlF0A4IhrdGf25Gd0DqT2rJS7qnx53lK9AOcqTXrNxxlBLEt0dvVeJyVNut2h2fHpWj2h5cSEcxPHAJ+DL1q/CeLFTbg3cN84p9B4Uwe84yH4fxOUvCe9OfHh2y9JoVJ7TaASmUG6+jcxgntNoBKesRH9HOPwGCdsBStSdRPGw9tAPSxWP0/JitRyGU/EADwL4D0A5AOwDtANAOQDsA7QC0A9AOAO0AtAPYov8HAAD//0/USFVHooMhAAAAAElFTkSuQmCC"}}},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport cv2\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Import the Libraries"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-output":true,"_kg_hide-input":true},"cell_type":"code","source":"import os\nfrom os import listdir\nimport pandas as pd\nimport numpy as np\nimport glob\nimport tqdm\nfrom typing import Dict\nimport matplotlib.pyplot as plt\n\n%matplotlib inline\n\n\n\nfrom colorama import Fore, Back, Style\n#plotly\n!pip install chart_studio\nimport plotly.express as px\nimport chart_studio.plotly as py\nimport plotly.graph_objs as go\nfrom plotly.offline import iplot\nimport cufflinks\ncufflinks.go_offline()\ncufflinks.set_config_file(world_readable=True, theme='pearl')\n\nfrom colorama import Fore, Back, Style\n\nimport seaborn as sns\nsns.set(style=\"whitegrid\")\nimport pydicom\nimport warnings\nwarnings.filterwarnings('ignore')\n\nplt.style.use('fivethirtyeight')\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train=pd.read_csv('../input/ranzcr-clip-catheter-line-classification/train.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Columns\nStudyInstanceUID - unique ID for each image\n\nETT - Abnormal - endotracheal tube placement abnormal\n\nETT - Borderline - endotracheal tube placement borderline abnormal\n\nETT - Normal - endotracheal tube placement normal\n\nNGT - Abnormal - nasogastric tube placement abnormal\n\nNGT - Borderline - nasogastric tube placement borderline abnormal\n\nNGT - Incompletely Imaged - nasogastric tube placement inconclusive due to imaging\n\nNGT - Normal - nasogastric tube placement borderline normal\n\nCVC - Abnormal - central venous catheter placement abnormal\n\nCVC - Borderline - central venous catheter placement borderline abnormal\n\nCVC - Normal - central venous catheter placement normal\n\nSwan Ganz Catheter Present\n\nPatientID - unique ID for each patient in the dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(Fore.YELLOW + 'Training data shape: ',Style.RESET_ALL,train.shape)\ntrain.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels = train[train.columns[1:-1]].columns.tolist()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.iloc[:, :-1].sum()[1:]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt = px.bar(train.iloc[:, :-1].sum()[1:],x=labels,y=np.sum(train[labels]), template = 'seaborn',title=\"Positions of Tube\")\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(Fore.YELLOW +\"Total Patients in Train set: \",Style.RESET_ALL,train['PatientID'].count())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(Fore.YELLOW + \"The total patient ids are\",Style.RESET_ALL,f\"{train['PatientID'].count()},\", Fore.BLUE + \"from those the unique ids are\", Style.RESET_ALL, f\"{train['PatientID'].value_counts().shape[0]}.\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"ETT = ['ETT - Abnormal', 'ETT - Borderline', 'ETT - Normal']\nNGT = ['NGT - Abnormal', 'NGT - Borderline', 'NGT - Incompletely Imaged', 'NGT - Normal']\nCVC = ['CVC - Abnormal', 'CVC - Borderline', 'CVC - Normal']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train[ETT].sum(axis=1).value_counts().iplot(kind='bar',\n                                              yTitle='Counts', \n                                              linecolor='black', \n                                              opacity=0.7,\n                                              color='RED',\n                                              theme='pearl',\n                                              bargap=0.5,\n                                              gridcolor='white',\n                                              title='Distribution of ETT')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train[NGT].sum(axis=1).value_counts().iplot(kind='bar',\n                                              yTitle='Counts', \n                                              linecolor='black', \n                                              opacity=0.7,\n                                              color='RED',\n                                              theme='pearl',\n                                              bargap=0.5,\n                                              gridcolor='white',\n                                              title='Distribution of NGT')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train[CVC].sum(axis=1).value_counts().iplot(kind='bar',\n                                              yTitle='Counts', \n                                              linecolor='black', \n                                              opacity=0.7,\n                                              color='blue',\n                                              theme='pearl',\n                                              bargap=0.5,\n                                              gridcolor='white',\n                                              title='Distribution of CVC')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Lets Look at some images"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_annot=pd.read_csv(\"../input/ranzcr-clip-catheter-line-classification/train_annotations.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_annot.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"path='../input/ranzcr-clip-catheter-line-classification/'\ndef plot(data, imageclass):\n    \n    fig, ax = plt.subplots(1, 2, figsize = (25,12))\n    temp = data[data[imageclass]==1]\n    \n    for i in range(2):\n        idx = temp.index[i]\n        image_id = temp.loc[idx, 'StudyInstanceUID']\n        image_file = cv2.imread(''.join([path, 'train/', image_id, '.jpg']))\n        image_file = cv2.cvtColor(image_file, cv2.COLOR_BGR2RGB)\n        ax[i].imshow(image_file)\n        ax[i].set_title(imageclass)\n        ax[i].set_xticklabels([])\n        ax[i].set_yticklabels([])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"\nplot(train, 'ETT - Abnormal')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plot(train, 'NGT - Abnormal')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plot(train, 'CVC - Borderline')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plot(train, 'Swan Ganz Catheter Present')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### To be continued"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}