{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"[<font size=\"5\">Description of COVID-19 Detection Problem</font>](#1)\n\nFive times more deadly than the flu, COVID-19 causes significant morbidity and mortality. Like other pneumonias, pulmonary infection with COVID-19 results in inflammation and fluid in the lungs. COVID-19 looks very similar to other viral and bacterial pneumonias on chest radiographs, which makes it difficult to diagnose. This computer vision model for detection and localization of COVID-19 would help doctors provide a quick and confident diagnosis. As a result, patients could get the right treatment before the most severe effects of the virus take hold.\n\nCurrently, COVID-19 can be diagnosed via polymerase chain reaction to detect genetic material from the virus or chest radiograph. However, it can take a few hours and sometimes days before the molecular test results are back. By contrast, chest radiographs can be obtained in minutes. While guidelines exist to help radiologists differentiate COVID-19 from other types of infection, their assessments vary. In addition, non-radiologists could be supported with better localization of the disease, such as with a visual bounding box.\n\nIn this competition, the task is to identify and localize COVID-19 abnormalities on chest radiographs. In particular, categorization of the radiographs as negative for pneumonia or typical, indeterminate, or atypical for COVID-19.","metadata":{}},{"cell_type":"code","source":"#first we import some important libraries\nimport numpy as np #for using arrays and matrices\nimport pandas as pd #for converting files to dataframe format, opening for example \nimport matplotlib #for plotting graphs and tables\nimport matplotlib.pyplot as plt\nimport pydicom as dicom # Dicom (Digital Imaging) \nimport os #allows us to use the operating system interation\nfrom tqdm import tqdm # allows you to output a smart progress bar by wrapping around any iterable","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:24.276853Z","iopub.execute_input":"2021-06-14T12:05:24.277295Z","iopub.status.idle":"2021-06-14T12:05:24.283156Z","shell.execute_reply.started":"2021-06-14T12:05:24.277260Z","shell.execute_reply":"2021-06-14T12:05:24.282099Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<font size=\"5\">Category of Radiographs </font>\n\n\n1. NEGATIVE FOR PNEUMONIA - No lung opacities\n\n2. TYPICAL APPEARANCE - Multifocal bilateral, peripheral opacities with rounded morphology, lower lung–predominant distribution\n\n3. INDETERMINATE APPEARANCE - Absence of typical findings AND unilateral, central or upper lung predominant distribution\n\n4. ATYPICAL APPEARANCE - Pneumothorax, pleural effusion, pulmonary edema, lobar consolidation, solitary lung nodule or mass, diffuse tiny nodules, cavity","metadata":{}},{"cell_type":"markdown","source":"<font size =\"5\"> Data Files </font>\n\n\n1. train_study_level.csv - the train study-level metadata, with one row for each study, including correct labels.\n    \n2. train_image_level.csv - the train image-level metadata, with one row for each image, including both correct labels and any bounding boxes in a dictionary format. Some images in both test and train have multiple bounding boxes.\n    \n3. sample_submission.csv - a sample submission file containing all image- and study-level IDs.\n\n4. train folder - comprises 6334 chest scans in DICOM format, stored in paths with the form study/series/image\n    \n5. test folder - The hidden test dataset is of roughly the same scale as the training dataset. Studies in the test set may contain more than one label.","metadata":{}},{"cell_type":"markdown","source":"[<font size=\"5\">Interpretating the Problem </font>](#2)\n\nThe problem is about classification and detection and we are told to find in a test set, a *bounding box*, which is a rectangular box for identifying or even, detecting where a certain object is, and also it's said to find a class.\nWith the labels 'Negative for Pneumonia' 'Typical Appearance' 'Indeterminate Appearance' 'Atypical Appearance', we have to make a prediction string, for example:\n> atypical 1 0 0 1 1\n","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"path = '/kaggle/input/siim-covid19-detection/'\ntrain_image_level = pd.read_csv(path + \"train_image_level.csv\")\n","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:24.284367Z","iopub.execute_input":"2021-06-14T12:05:24.284773Z","iopub.status.idle":"2021-06-14T12:05:24.328834Z","shell.execute_reply.started":"2021-06-14T12:05:24.284743Z","shell.execute_reply":"2021-06-14T12:05:24.328018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"identity = train_image_level[\"id\"] #we will write an specific column, in this case id\nprint(identity)\n\n#if we wanted to show the first two columns we would use train_image_level[\"id\", \"boxes\"]\n\n","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:24.330234Z","iopub.execute_input":"2021-06-14T12:05:24.330670Z","iopub.status.idle":"2021-06-14T12:05:24.336457Z","shell.execute_reply.started":"2021-06-14T12:05:24.330639Z","shell.execute_reply":"2021-06-14T12:05:24.335844Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_image_level.describe()\ntrain_image_level.head(8) #we show the first 8 lines\n","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:24.337653Z","iopub.execute_input":"2021-06-14T12:05:24.338058Z","iopub.status.idle":"2021-06-14T12:05:24.383530Z","shell.execute_reply.started":"2021-06-14T12:05:24.338029Z","shell.execute_reply":"2021-06-14T12:05:24.382831Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_image_level.loc[0,:] #for showing the first row in all columsn\n#train_image_level.loc[row, column]\n","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:24.384567Z","iopub.execute_input":"2021-06-14T12:05:24.385008Z","iopub.status.idle":"2021-06-14T12:05:24.390915Z","shell.execute_reply.started":"2021-06-14T12:05:24.384976Z","shell.execute_reply":"2021-06-14T12:05:24.390209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[<font size=\"5\">Plotting in the Graphic Form </font>](#3)","metadata":{}},{"cell_type":"code","source":"#extraction of an image \n\ndef extract_image(i):\n    path_train = path + 'train/' + training_set.loc[i, 'StudyInstanceUID']\n    last_folder_in_path = os.listdir(path_train)[0]\n    path_train = path_train + '/{}/'.format(last_folder_in_path)\n    img_id = training_set.loc[i, 'id_y'].replace('_image','.dcm')\n    print(img_id)\n    data_file = dicom.dcmread(path_train + img_id)\n    img = data_file.pixel_array\n    return img","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:24.392012Z","iopub.execute_input":"2021-06-14T12:05:24.392458Z","iopub.status.idle":"2021-06-14T12:05:24.403684Z","shell.execute_reply.started":"2021-06-14T12:05:24.392415Z","shell.execute_reply":"2021-06-14T12:05:24.402687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(3,4, figsize=(20,20)) #create 4 figures in horizontal and 3 in vertical with size (x,y) = (20,20)\nfig.subplots_adjust(hspace=0.1, wspace=0.1) #(hspace, wspace) = (spacebetween two graphs in vertical, spacebetween two graphs in horizontal)\n\n#above are some explanation for the plotting in the problem\n#observe that figures are blank\n","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:24.405134Z","iopub.execute_input":"2021-06-14T12:05:24.405569Z","iopub.status.idle":"2021-06-14T12:05:25.593455Z","shell.execute_reply.started":"2021-06-14T12:05:24.405524Z","shell.execute_reply":"2021-06-14T12:05:25.592449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_study_level = pd.read_csv(path + \"train_study_level.csv\")\ntrain_study_level_key = train_study_level.id.str[:-6]\ntraining_set = pd.merge(left = train_study_level, right = train_image_level, how = 'right', left_on = train_study_level_key, right_on = 'StudyInstanceUID')\ntraining_set.drop(['id_x'], axis = 1)\n","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:10:25.232424Z","iopub.execute_input":"2021-06-14T12:10:25.232916Z","iopub.status.idle":"2021-06-14T12:10:25.273150Z","shell.execute_reply.started":"2021-06-14T12:10:25.232885Z","shell.execute_reply":"2021-06-14T12:10:25.272260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_set.head()","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:25.631106Z","iopub.execute_input":"2021-06-14T12:05:25.631420Z","iopub.status.idle":"2021-06-14T12:05:25.645767Z","shell.execute_reply.started":"2021-06-14T12:05:25.631388Z","shell.execute_reply":"2021-06-14T12:05:25.645054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_set.loc[0,'label'] #we show the row 0 with column label\n","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:25.646827Z","iopub.execute_input":"2021-06-14T12:05:25.647098Z","iopub.status.idle":"2021-06-14T12:05:25.665553Z","shell.execute_reply.started":"2021-06-14T12:05:25.647071Z","shell.execute_reply":"2021-06-14T12:05:25.664554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#for a generic axes ax we have the following important functions\n\n#ax.set_xticklabels([]) #ticks just denoting data points on axes in this case [] blank\n#ax.set_yticklabels([]) #ticks just denoting data points on axes in this case [] blanck","metadata":{"execution":{"iopub.status.busy":"2021-06-14T12:05:25.666762Z","iopub.execute_input":"2021-06-14T12:05:25.667095Z","iopub.status.idle":"2021-06-14T12:05:25.673531Z","shell.execute_reply.started":"2021-06-14T12:05:25.667064Z","shell.execute_reply":"2021-06-14T12:05:25.672588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}}]}