{"cells":[{"metadata":{},"cell_type":"markdown","source":"# About the competition and dataset\n \nThe competition is related to healthcare domain which expects you to detect the position of catheters and lines on chest x-rays.\n\n### What is Catheters?\nCatheter is a soft hollow tube passed into the body for specific reasons. For example, a urinary catheter is passed into the body to drain out the urine from the body through these tubes if a person cannot empty the urine in the normal way. If these catheters are not introduced into the body at the right positions, it can cause serious complications to the person's body.\n\nBelow is an image of a catheter:\n![Chest catheter](https://5.imimg.com/data5/EI/OF/UN/SELLER-134485/chest-drainage-catheter-250x250.jpg)\n\nThe dataset given in the competition is about catheter tubes placed in the upper parts of the human body whose position can be understood from a chest x-ray.\n\nAccording to the competition description, the gold standard for the confirmation of tube positions are chest radiographs. But for a human expert, he/she will have to manually check the chest x-rays to confirm whether these tubes are in the optimal position which costs him/her a lot of time. So, the goal of this competition is to train machine learning models to predict whether a tube is placed in the optimal position or not, given a chest x-ray. Earlier detection of malpositioned catheters is even more important during this pandemic as COVID-19 patients are increasing.\n\n### Getting familiar with the dataset\n\nThe dataset contains a lot of biological terms which may be confusing if you haven't looked back at your biology lessons for a long time. But we will discuss them together. \n\nThe dataset contains three types of tubes for which we will have to make predictions.\n\n1. **Endotracheal tube(ETT)** - This is nothing but a tube passed through the trachea(or windpipe) which is a pipe like structure which helps in the passage of air(breathing). Endotracheal tube is a flexible plastic tube placed through the mouth into the trachea to help a patient breathe.\n\n\n\n\n2. **Nasogastric tube(NGT)** - This is a flexible tube made of either rubber or plastic which is passed through the nose to the esophagus(a tube connecting the throat with the stomach) into the stomach.This is used for feeding food and medicine to the stomach through the nose.\n\n\n3. **Central venous catheter(CVC)** - This is a thin, flexible tube inserted into the body through a vein usually below the collar bone and is guided into a large vein above the right side of the heart called superior vena cava. CVC is introduced to give fluids, blood or other drugs.\n\n\n\n\n### Going deeper into the dataset\n\nThe ultimate goal of the competition is that given an image of the chest x-ray, our model should be able to make one of the following predictions:\n\n1. **ETT - Abnormal(endotracheal tube placement abnormal)** --> This means that the endotracheal tube is positioned incorrectly and requires immediate repositioning.\n\n2. **ETT - Borderline(endotracheal tube placement borderline)** --> This means that the endotracheal tube may require some repositioning but in most cases it will work fine in the current position.\n\n3. **ETT - Normal(endotracheal tube placement normal)** --> This means that the endotracheal tube is placed correctly and no repositioning is required.\n\n4. **NGT - Abnormal(nasogastric tube placement abnormal)** --> Nasogastric tube is placed incorrectly, immediate repositioning is required.\n\n5. **NGT - Borderline(nasogastric tube placement borderline)** --> Nasogastric tube may require some repositioning but still it will work fine in the current position.\n\n6. **NGT - Incompletely Imaged(nasogastric tube placement inconclusive due to imaging)** --> Not able to conclude whether nasogastric tube placement requires repositioning or not due to problems in imaging.\n\n7. **NGT - Normal(nasogastric tube placement normal)** --> Nasogastric tube placement is correct and no repositioning is required.\n\n8. **CVC - Abnormal(central venous catheter placement abnormal)** --> Central venous catheter placement is abnormal and requires immediate repositioning.\n\n9. **CVC - Borderline(central venous catheter placement borderline)** --> Catheter may require some repositioning but still works fine in the current position.\n\n10. **CVC - Normal(central venous catheter placement normal)** - Catheter placement is correct and no repositioning is required.\n\n11. **Swan Ganz Catheter Present** - Swan Ganz Catheter is a thin tube passed into the right side of the heart and the arteries leading to the lungs. This is used for monitoring heart's function, blood flow and pressures in and around the heart. \n\n\n\n### Understanding the dataset structure\n\n1. **train** --> This folder contains all the chest x-ray images for training the model.\n\n2. **train.csv** --> This csv file contains the name of the chest x-ray image, patient id and all the labels as described in the above section. This is a sample from the train.csv file:"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import pandas as pd\ntrain = pd.read_csv('../input/ranzcr-clip-catheter-line-classification/train.csv')\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"3. train_annotations.csv --> This csv file contains the position of the catheter tube('data' column) in the x-ray image."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from ast import literal_eval\nannot = pd.read_csv('../input/ranzcr-clip-catheter-line-classification/train_annotations.csv')\nannot['data'] = annot['data'].apply(literal_eval)\nannot.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Basic EDA"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom ast import literal_eval","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/ranzcr-clip-catheter-line-classification/train.csv')\nannot = pd.read_csv('../input/ranzcr-clip-catheter-line-classification/train_annotations.csv')\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"annot.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.shape, annot.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**As we've said earlier, train.csv has image id, patient id along with their labels and train_annotations.csv has image id and their corresponding annotation points. From the shapes of the csv files we can understand that the dataset has x-ray images with and without catheter tubes.**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train.info(), annot.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"annot['data'][0][0]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**From the above output, we can see that the 'data' column is not in a way that we had expected. So, let's change it.**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"annot['data'] = annot['data'].apply(literal_eval)\nannot['data'][0][0], annot['data'].head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(f\"Out of {train.shape[0]} there are {train['PatientID'].nunique()} unique patients\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"patient_count = train['PatientID'].value_counts()\n\nplt.figure(figsize=(10, 6))\nsns.countplot(x=patient_count);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that that there are multiple x-rays for some of the patients"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"label_count = train.drop(['StudyInstanceUID', 'PatientID'], axis=1).sum(axis=0)\n\nsns.barplot(x=label_count.index, y=label_count)\nplt.xticks(rotation=90);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**For each of the categoies(ETT, NGT and CVC) the normal images are higher in number and abnormal images has lower number of images**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"normal, borderline, abnormal = 0, 0, 0\nfor i in label_count.index:\n if 'abnormal' in i.lower():\n abnormal += label_count[i]\n elif 'borderline' in i.lower():\n borderline += label_count[i]\n else:\n normal += label_count[i]\n \nsns.barplot(x=['normal_imgs', 'borderline_imgs', 'abnormal_imgs'], \n y=[normal, borderline, abnormal]);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**We can conclude that, overall, normal images are higher in number than abnormal images**"},{"metadata":{},"cell_type":"markdown","source":"### Visualizing the chest x-rays"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def plot_image(img_id):\n img_path = '../input/ranzcr-clip-catheter-line-classification/train/'+img_id+'.jpg'\n img = plt.imread(img_path)\n annotation = annot[annot['StudyInstanceUID']==img_id]['data'].values[0]\n label = str(annot[annot['StudyInstanceUID']==img_id]['label'].values[0])\n \n x = [x[0] for x in annotation]\n y = [x[1] for x in annotation]\n \n plt.figure(figsize=(10, 6))\n plt.subplot(1, 2, 1)\n plt.title('Actual Image')\n plt.imshow(img, cmap='gray')\n plt.axis('off')\n \n plt.subplot(1, 2, 2)\n plt.title(label)\n plt.imshow(img, cmap='gray')\n plt.axis('off')\n plt.plot(x, y, 'r-', linewidth=4)\n plt.show()\n\nstart_idx = 100\nend_idx = start_idx+10\nfor i in range(start_idx, end_idx, 2):\n plot_image(annot['StudyInstanceUID'][i])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**If you enjoyed the notebook, it will be great if you upvote my work**"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}