{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Competition\n# SIIM-FISABIO-RSNA COVID-19 Detection\n<font color = 'blue'>Identify and localize COVID-19 abnormalities on chest radiographs</font>","metadata":{}},{"cell_type":"markdown","source":"- Currently, COVID-19 can be diagnosed via polymerase chain reaction to detect genetic material from the virus or chest radiograph. However, it can take a few hours and sometimes days before the molecular test results are back.\n- By contrast, chest radiographs can be obtained in minutes","metadata":{}},{"cell_type":"markdown","source":"If successful, you'll help radiologists diagnose the millions of COVID-19 patients more confidently and quickly. This will also enable doctors to see the extent of the disease and help them make decisions regarding treatment. Depending upon severity, affected patients may need hospitalization, admission into an intensive care unit, or supportive therapies like mechanical ventilation. As a result of better diagnosis, more patients will quickly receive the best care for their condition, which could mitigate the most severe effects of the virus.","metadata":{}},{"cell_type":"markdown","source":"# Competition Rules\n\n- CPU Notebook <= 9 hours run-time\n- GPU Notebook <= 9 hours run-time\n- Internet access disabled\n- Freely & publicly available external data is allowed, including pre-trained models\n- Submission file must be named submission.csv","metadata":{}},{"cell_type":"markdown","source":"# Evaluation Metric\n- <font color = 'blue'>Standard PASCAL VOC 2010 **mean Average Precision (mAP)** at IoU > 0.5</font>","metadata":{}},{"cell_type":"code","source":"!conda install gdcm -c conda-forge -y","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%matplotlib inline\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\n\nfrom tqdm import tqdm\nfrom glob import glob\nimport gc\n\nimport cv2\nimport matplotlib.pyplot as plt\nimport matplotlib\nimport seaborn as sns\nfrom IPython.display import display\n\nplt.rcParams[\"figure.figsize\"] = (12, 8)\nplt.rcParams['axes.titlesize'] = 16\nsns.set_palette('Set3_r')\n\nimport pydicom as dicom\nimport ast\n\nimport warnings\nwarnings.filterwarnings('ignore')\n\nprint(os.listdir('/kaggle/input/siim-covid19-detection/'))\n\nfrom time import time, strftime, gmtime\nstart = time()\nimport datetime\nprint(str(datetime.datetime.now()))","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"base_dir = '/kaggle/input/siim-covid19-detection/'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_label = pd.read_csv(base_dir + 'train_study_level.csv')\nprint(train_label.shape)\ntrain_label.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_img = pd.read_csv(base_dir + 'train_image_level.csv')\nprint(train_img.shape)\ntrain_img.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.read_csv(base_dir + 'sample_submission.csv')\nprint(sub.shape)\nsub.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_dicom = glob(base_dir + 'train/*/*/*.dcm')\ntest_dicom = glob(base_dir + 'test/*/*/*.dcm')\nprint(f\"Number of train dicom files: {len(train_dicom)}\")\nprint(f\"Number of test dicom files: {len(test_dicom)}\")","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_img['dcm_path'] = train_img['StudyInstanceUID'].apply(lambda x: glob(base_dir + 'train/' + x + '/*/*.dcm')[0])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_label['StudyInstanceUID'] = train_label['id'].apply(lambda x: x.split('_')[0])\ntrain_img = train_img.merge(train_label.iloc[:, 1:], on = 'StudyInstanceUID')\nprint(train_img.shape)\ntrain_img.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Check any multi-label\ntrain_img.iloc[:, 5:].sum(axis = 1).value_counts()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# EDA","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (16, 12))\nplt.suptitle('Countplot of Classes')\nfor i, c in enumerate(train_label.columns[1:5]):\n    plt.subplot(2, 2, i + 1)\n    ax = sns.countplot(train_label[c])\n    for p in ax.patches:\n        ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- There can be multiple bounding boxes for a given case image\n- We create a feature with number of BB for a given image","metadata":{}},{"cell_type":"code","source":"train_img['bb_num'] = train_img['label'].apply(lambda x: len(x.split()) / 6)\nplt.title('Number of Bounding Boxes per Image')\nax = sns.countplot(data = train_img, x = 'bb_num')\nfor p in ax.patches:\n        ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_label(x):\n    if len(x) == 1 and x[0] == 'none':\n        return 'None'\n    elif len(x) == 1 and x[0] == 'opacity':\n        return 'Opacity'\n    elif len(x) == 2:\n        return 'Double Opacity'\n    elif len(x) > 2:\n        return 'More than 2 Opacity'","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.title('Countplot of Label Cateogries')\ntrain_img['label_category'] = train_img['label'].apply(lambda x: x.split()[::6]).apply(get_label)\nax = sns.countplot(data = train_img, x = 'label_category')\nfor p in ax.patches:\n        ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"__Display Sample Dicom__","metadata":{}},{"cell_type":"code","source":"sample = dicom.dcmread(train_img['dcm_path'][0])\nsample","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#train_img['gender'] = train_img['dcm_path'].apply(lambda x: dicom.dcmread(x).PatientSex)\n#ax = sns.countplot(data = train_img, x = 'gender')\n#for p in ax.patches:\n#       ax.annotate(str(p.get_height()), (p.get_x() * 1.005, p.get_height() * 1.005))","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Display Sample X-Ray Images","metadata":{}},{"cell_type":"code","source":"def display_dicom_images(idx):\n    f, ax = plt.subplots(3, 3, figsize = (15, 10))\n    f.subplots_adjust(hspace = .1, wspace = .1)\n    ax = ax.flatten()\n\n    for i, ind in enumerate(idx):\n        dicom_path = train_img['dcm_path'].loc[ind]\n        label = train_img['label_category'].loc[ind]\n        #classes = train_img.iloc[:, 5:8].columns[train_img.iloc[ind, 5:8].values.argmax()].split()[0]\n        dicom_file = dicom.dcmread(dicom_path)\n        img = dicom_file.pixel_array\n        #print(img.shape)\n        ax[i].imshow(img, cmap = 'gray')\n        ax[i].set_xticklabels([])\n        ax[i].set_yticklabels([])\n        ax[i].set_title(f\"{label}\", fontsize = 10)\n        if label != 'None':\n            boxes = ast.literal_eval(train_img['boxes'].loc[ind])\n            #print(label)\n            for bb in boxes:\n                bbox = [bb['x'], bb['y'], bb['width'], bb['height']]\n                p = matplotlib.patches.Rectangle((bbox[0], bbox[1]),\n                                                 bbox[2],\n                                                 bbox[3],\n                                                 color = 'red', fc = 'none')\n                ax[i].add_patch(p)\n    plt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"idx = np.random.choice(train_img.index, 9)\ndisplay_dicom_images(idx)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"idx = np.random.choice(train_img.index, 9)\ndisplay_dicom_images(idx)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# WIP","metadata":{}},{"cell_type":"code","source":"finish = time()\nprint(strftime(\"%H:%M:%S\", gmtime(finish - start)))","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}