{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Problem description\n\n****\n\nFive times more deadly than the flu, COVID-19 causes significant morbidity and mortality. Like other pneumonias, pulmonary infection with COVID-19 results in inflammation and fluid in the lungs. COVID-19 looks very similar to other viral and bacterial pneumonias on chest radiographs, which makes it difficult to diagnose. This computer vision model for detection and localization of COVID-19 would help doctors provide a quick and confident diagnosis. As a result, patients could get the right treatment before the most severe effects of the virus take hold.\n\n\nCurrently, COVID-19 can be diagnosed via polymerase chain reaction to detect genetic material from the virus or chest radiograph. However, it can take a few hours and sometimes days before the molecular test results are back. By contrast, chest radiographs can be obtained in minutes. While guidelines exist to help radiologists differentiate COVID-19 from other types of infection, their assessments vary. In addition, non-radiologists could be supported with better localization of the disease, such as with a visual bounding box.\n\n\nIn this competition, the task is to identify and localize COVID-19 abnormalities on chest radiographs. In particular, categorization of the radiographs as negative for pneumonia or typical, indeterminate, or atypical for COVID-19.","metadata":{}},{"cell_type":"markdown","source":"**Categorization of the radiographs:**\n\n* NEGATIVE FOR PNEUMONIA - No lung opacities\n\n* TYPICAL APPEARANCE - Multifocal bilateral, peripheral opacities with rounded morphology, lower lung–predominant distribution\n\n* INDETERMINATE APPEARANCE - Absence of typical findings AND unilateral, central or upper lung predominant distribution\n\n* ATYPICAL APPEARANCE - Pneumothorax, pleural effusion, pulmonary edema, lobar consolidation, solitary lung nodule or mass, diffuse tiny nodules, cavity","metadata":{}},{"cell_type":"markdown","source":"**Input data:**\n\n* train_study_level.csv - the train study-level metadata, with one row for each study, including correct labels.\n* train_image_level.csv - the train image-level metadata, with one row for each image, including both correct labels and any bounding boxes in a dictionary format. Some images in both test and train have multiple bounding boxes.\n* sample_submission.csv - a sample submission file containing all image- and study-level IDs.\n* train folder - comprises 6334 chest scans in DICOM format, stored in paths with the form study/series/image\n* test folder - The hidden test dataset is of roughly the same scale as the training dataset. Studies in the test set may contain more than one label.","metadata":{}},{"cell_type":"markdown","source":"# Content table\n\n****\n\n1. Importing the libraries\n2. Importing the datasets\n3. Data exploration\n4. Read Dicom files\n5. Feature engineering\n6. Making the model\n7. Compiling the model\n8. References","metadata":{}},{"cell_type":"markdown","source":"# Importing the libraries\n****","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sn\nimport pydicom as dicom # Dicom (Digital Imaging in Medicine) - medical image datasets, storage and transfer\nimport os\nfrom tqdm import tqdm # allows you to output a smart progress bar by wrapping around any iterable\nimport glob # retrieve files/pathnames matching a specified pattern\nimport pprint # pretty-print” arbitrary Python data structures\nimport ast # \nfrom pydicom.pixel_data_handlers.util import apply_voi_lut #\nimport wandb #\nimport keras\nfrom tensorflow.keras import models\nfrom tensorflow.keras import layers\nfrom tensorflow.keras import optimizers\nfrom textwrap import wrap\n\npd.set_option('display.max_columns', 500)","metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-07-14T12:43:29.564487Z","iopub.execute_input":"2021-07-14T12:43:29.564822Z","iopub.status.idle":"2021-07-14T12:43:35.960112Z","shell.execute_reply.started":"2021-07-14T12:43:29.564791Z","shell.execute_reply":"2021-07-14T12:43:35.959178Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Importing the datasets\n****","metadata":{}},{"cell_type":"code","source":"path = '/kaggle/input/siim-covid19-detection/'\ntrain_dir = '/kaggle/input/siim-covid19-detection/train'\ntest_dir = '/kaggle/input/siim-covid19-detection/test'\n\ntrain_image_level = pd.read_csv(path + \"train_image_level.csv\")\ntrain_study_level = pd.read_csv(path + \"train_study_level.csv\")","metadata":{"execution":{"iopub.status.busy":"2021-07-14T12:43:35.961538Z","iopub.execute_input":"2021-07-14T12:43:35.961899Z","iopub.status.idle":"2021-07-14T12:43:36.017477Z","shell.execute_reply.started":"2021-07-14T12:43:35.961843Z","shell.execute_reply":"2021-07-14T12:43:36.016570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data exploration\n****","metadata":{}},{"cell_type":"markdown","source":"Let's have a look inside the train_image_level:","metadata":{}},{"cell_type":"code","source":"train_image_level.head()","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:50:06.369373Z","iopub.execute_input":"2021-06-29T20:50:06.369906Z","iopub.status.idle":"2021-06-29T20:50:06.40526Z","shell.execute_reply.started":"2021-06-29T20:50:06.369861Z","shell.execute_reply":"2021-06-29T20:50:06.40396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_image_level.describe()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are 6334 unique values in the train_image_level dataframe.","metadata":{}},{"cell_type":"markdown","source":"Now let's have a look inside train_study_level dataset:","metadata":{}},{"cell_type":"code","source":"train_study_level.head()","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:50:11.06051Z","iopub.execute_input":"2021-06-29T20:50:11.060891Z","iopub.status.idle":"2021-06-29T20:50:11.07491Z","shell.execute_reply.started":"2021-06-29T20:50:11.060858Z","shell.execute_reply":"2021-06-29T20:50:11.073602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_study_level.describe()","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:54:07.817515Z","iopub.execute_input":"2021-06-29T20:54:07.817938Z","iopub.status.idle":"2021-06-29T20:54:07.855521Z","shell.execute_reply.started":"2021-06-29T20:54:07.817907Z","shell.execute_reply":"2021-06-29T20:54:07.854215Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Distribution of Appearances**","metadata":{}},{"cell_type":"markdown","source":"Note:\n\nWhen you use enumerate(), the function gives you back two loop variables:\n\n1. The count of the current iteration\n2. The value of the item at the current iteration","metadata":{}},{"cell_type":"code","source":"columns = ['Negative for Pneumonia', 'Typical Appearance', 'Indeterminate Appearance', 'Atypical Appearance']\nsum = []\n\n# label rotation for clear view\nfig, ax = plt.subplots()\nax.set_xticklabels(labels = columns, rotation = 45)\n\nfor column in columns:\n    plt.bar(column, train_study_level[column].sum())","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:54:10.656962Z","iopub.execute_input":"2021-06-29T20:54:10.657327Z","iopub.status.idle":"2021-06-29T20:54:10.839987Z","shell.execute_reply.started":"2021-06-29T20:54:10.657297Z","shell.execute_reply":"2021-06-29T20:54:10.838732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are 6054 rows in the train_study_level dataframe. The number of unique values in study dataframe differs from the unique values in the images dataframe. Let's check how many studies have more than 1 image linked.","metadata":{}},{"cell_type":"code","source":"train_study_level_key = train_study_level.id.str[:-6]\ntraining_set = pd.merge(left = train_study_level, right = train_image_level, how = 'right', left_on = train_study_level_key, right_on = 'StudyInstanceUID')\ntraining_set.drop(['id_x'], axis = 1)","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:54:14.730492Z","iopub.execute_input":"2021-06-29T20:54:14.7309Z","iopub.status.idle":"2021-06-29T20:54:14.77152Z","shell.execute_reply.started":"2021-06-29T20:54:14.730871Z","shell.execute_reply":"2021-06-29T20:54:14.770514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's have a look at these studies with multiple images:","metadata":{}},{"cell_type":"code","source":"training_set[training_set.groupby('StudyInstanceUID')['id_y'].transform('size') > 1].sort_values('StudyInstanceUID')","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:54:17.629597Z","iopub.execute_input":"2021-06-29T20:54:17.630039Z","iopub.status.idle":"2021-06-29T20:54:17.683687Z","shell.execute_reply.started":"2021-06-29T20:54:17.630009Z","shell.execute_reply":"2021-06-29T20:54:17.682505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Read Dicom files\n****","metadata":{}},{"cell_type":"markdown","source":"Function used to locate image from the path:","metadata":{}},{"cell_type":"code","source":"def extract_image(i):\n    path_train = path + 'train/' + training_set.loc[i, 'StudyInstanceUID']\n    last_folder_in_path = os.listdir(path_train)[0]\n    path_train = path_train + '/{}/'.format(last_folder_in_path)\n    img_id = training_set.loc[i, 'id_y'].replace('_image','.dcm')\n    \n    print(img_id)\n    \n    data_file = dicom.dcmread(path_train + img_id)\n    img = data_file.pixel_array\n    return img","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:54:30.47915Z","iopub.execute_input":"2021-06-29T20:54:30.479483Z","iopub.status.idle":"2021-06-29T20:54:30.488053Z","shell.execute_reply.started":"2021-06-29T20:54:30.479455Z","shell.execute_reply":"2021-06-29T20:54:30.486864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Images and rectangles visualization**","metadata":{}},{"cell_type":"code","source":"fig, axes = plt.subplots(3,3, figsize=(20,16))\nfig.subplots_adjust(hspace=.1, wspace=.1)\naxes = axes.ravel()\n\nfor row in range(9):\n    img = extract_image(row)\n    if (training_set.loc[row,'boxes'] == training_set.loc[row,'boxes']):\n        boxes = ast.literal_eval(training_set.loc[row,'boxes'])\n        for box in boxes:\n            p = matplotlib.patches.Rectangle((box['x'], box['y']),\n                                              box['width'], box['height'],\n                                              ec = 'r', fc = 'none', lw = 2.\n                                            )\n            axes[row].add_patch(p)\n    axes[row].imshow(img, cmap = 'gray')\n    axes[row].set_title(training_set.loc[row, 'label'].split(' ')[0])\n    axes[row].set_xticklabels([])\n    axes[row].set_yticklabels([])","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:54:34.058498Z","iopub.execute_input":"2021-06-29T20:54:34.058914Z","iopub.status.idle":"2021-06-29T20:54:44.219498Z","shell.execute_reply.started":"2021-06-29T20:54:34.058879Z","shell.execute_reply":"2021-06-29T20:54:44.218277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Feature engineering\n****","metadata":{}},{"cell_type":"markdown","source":"**Opacity_Count** - Count the number of opacities in the image","metadata":{}},{"cell_type":"code","source":"Opacity_Count = training_set['label'].str.count('opacity')\ntraining_set['Opacity_Count'] = Opacity_Count.values","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:54:44.22138Z","iopub.execute_input":"2021-06-29T20:54:44.221735Z","iopub.status.idle":"2021-06-29T20:54:44.24126Z","shell.execute_reply.started":"2021-06-29T20:54:44.221697Z","shell.execute_reply":"2021-06-29T20:54:44.239681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Rectange_Area** - Sum of areas of rectangles - assumption : the bigger the rectangle - the bigger the opacity","metadata":{}},{"cell_type":"code","source":"image_rectangles_areas = []\n\nfor row in range(6334):#len(training_set.index)):\n    image_rectangles_area_sum = 0\n    rectangle_area = 0\n    if (training_set.loc[row,'boxes'] == training_set.loc[row,'boxes']):\n        boxes = ast.literal_eval(training_set.loc[row,'boxes'])\n        for box in boxes:\n            rectangle_area = box['width'] * box['height']\n            image_rectangles_area_sum = image_rectangles_area_sum + rectangle_area\n        image_rectangles_areas.append(image_rectangles_area_sum)\n    else: # nan values\n        image_rectangles_area_sum = image_rectangles_area_sum + rectangle_area\n        image_rectangles_areas.append(image_rectangles_area_sum)","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:54:58.703104Z","iopub.execute_input":"2021-06-29T20:54:58.703453Z","iopub.status.idle":"2021-06-29T20:54:59.131253Z","shell.execute_reply.started":"2021-06-29T20:54:58.703422Z","shell.execute_reply":"2021-06-29T20:54:59.130109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_set['Rectangle_Area'] = image_rectangles_areas","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:55:01.424227Z","iopub.execute_input":"2021-06-29T20:55:01.424667Z","iopub.status.idle":"2021-06-29T20:55:01.432211Z","shell.execute_reply.started":"2021-06-29T20:55:01.424635Z","shell.execute_reply":"2021-06-29T20:55:01.431031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Creating buckets - rectangle areas**","metadata":{}},{"cell_type":"markdown","source":"Distribution of the rectangle areas","metadata":{}},{"cell_type":"code","source":"training_set['Rectangle_Area'] = round(training_set['Rectangle_Area'],2)","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:55:04.415219Z","iopub.execute_input":"2021-06-29T20:55:04.415612Z","iopub.status.idle":"2021-06-29T20:55:04.422443Z","shell.execute_reply.started":"2021-06-29T20:55:04.415582Z","shell.execute_reply":"2021-06-29T20:55:04.420828Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#pd.qcut(training_set['Rectangle_Area'], q = 4)\n\n#training_set.boxplot(by = \"Negative for Pneumonia\",column = ['Rectangle_Area'],grid = True, layout=(1, 1))\n\ncut_labels_4 = ['0', '<1e6', '<2e6', '<4e6', '<8e6']\ncut_bins = [-1, 0, 1000000, 2000000, 4000000, 8000000]\ntraining_set['Rectangle_Area_Bin'] = pd.cut(training_set['Rectangle_Area'], bins = cut_bins, labels = cut_labels_4)","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:55:06.178421Z","iopub.execute_input":"2021-06-29T20:55:06.178853Z","iopub.status.idle":"2021-06-29T20:55:06.189706Z","shell.execute_reply.started":"2021-06-29T20:55:06.178752Z","shell.execute_reply":"2021-06-29T20:55:06.188047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"columns = ['Negative for Pneumonia', 'Typical Appearance', 'Indeterminate Appearance', 'Atypical Appearance']\n\nplt.figure(figsize = (16, 14))\nsn.set(font_scale = 1.2)\nsn.set_style('ticks')\n\nfor i, column in enumerate(columns):\n    plt.subplot(3, 3, i + 1)\n    sn.countplot(data = training_set, x = 'Rectangle_Area_Bin', hue = column, palette = ['#d02f52',\"#55a0ee\"])\n    \nsn.despine()","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:55:06.682066Z","iopub.execute_input":"2021-06-29T20:55:06.682416Z","iopub.status.idle":"2021-06-29T20:55:07.429958Z","shell.execute_reply.started":"2021-06-29T20:55:06.682386Z","shell.execute_reply":"2021-06-29T20:55:07.42858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"opacity = sorted(list(training_set['Rectangle_Area_Bin'].value_counts().index))\n\nfor i in opacity:\n    Count_Series = training_set[training_set['Rectangle_Area_Bin'] == i].iloc[:,[1, 2, 3, 4]].sum()\n    fig = plt.figure(figsize=(12,3))\n    sn.barplot(x = Count_Series.index, y = Count_Series.values/sum(training_set['Rectangle_Area_Bin'] == i))\n    plt.title('Rectangle_Area_Bin : {} '.format(i))\n    plt.plot()","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:55:12.488386Z","iopub.execute_input":"2021-06-29T20:55:12.488846Z","iopub.status.idle":"2021-06-29T20:55:12.529216Z","shell.execute_reply.started":"2021-06-29T20:55:12.488812Z","shell.execute_reply":"2021-06-29T20:55:12.52709Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Rectangle area and opacity count","metadata":{}},{"cell_type":"code","source":"\nopacity = sorted(list(training_set['Opacity_Count'].value_counts().index))\n\nfor i in opacity:\n    Count_Series = training_set[training_set['Opacity_Count'] == i].iloc[:,[1, 2, 3, 4]].sum()\n    fig = plt.figure(figsize=(12,3))\n    sn.barplot(x = Count_Series.index, y = Count_Series.values/sum(training_set['Opacity_Count'] == i))\n    plt.title('OpacityCount : {} '.format(i))\n    plt.plot()","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:55:13.897466Z","iopub.execute_input":"2021-06-29T20:55:13.897873Z","iopub.status.idle":"2021-06-29T20:55:13.936373Z","shell.execute_reply.started":"2021-06-29T20:55:13.897803Z","shell.execute_reply":"2021-06-29T20:55:13.934191Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**TBD**: Position of the rectangle by quadrants (4 bins - 4 quadrants)","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Image metadata**","metadata":{}},{"cell_type":"code","source":"training_paths = []\ntrain_directory = \"../input/siim-covid19-detection/train/\"\n\nfor sid in tqdm(training_set['StudyInstanceUID']):\n    training_paths.append(glob.glob(os.path.join(train_directory, sid +\"/*/*\"))[0])\n\ntraining_set['path'] = training_paths","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:55:17.213279Z","iopub.execute_input":"2021-06-29T20:55:17.213694Z","iopub.status.idle":"2021-06-29T20:55:38.144045Z","shell.execute_reply.started":"2021-06-29T20:55:17.213664Z","shell.execute_reply":"2021-06-29T20:55:38.142756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The pixel values are in the range of 0 to 255. It is easier for us to normalize the data between 0 to 1 and we can do that just by dividing our train and test set by 255.","metadata":{}},{"cell_type":"code","source":"voi_lut=True\nfix_monochrome=True\n\ndef dicom_dataset_to_dict(filename,func):\n    \"\"\"Credit: https://github.com/pydicom/pydicom/issues/319\n               https://www.kaggle.com/raddar/convert-dicom-to-np-array-the-correct-way\n    \"\"\"\n    \n    dicom_header = dicom.dcmread(filename) \n    \n    #====== DICOM FILE DATA ======\n    dicom_dict = {}\n    repr(dicom_header)\n    for dicom_value in dicom_header.values():\n        if dicom_value.tag == (0x7fe0, 0x0010):\n            #discard pixel data\n            continue\n        if type(dicom_value.value) == dicom.dataset.Dataset:\n            dicom_dict[dicom_value.name] = dicom_dataset_to_dict(dicom_value.value)\n        else:\n            v = _convert_value(dicom_value.value)\n            dicom_dict[dicom_value.name] = v\n      \n    del dicom_dict['Pixel Representation']\n    \n    if func != 'metadata_df':\n        #====== DICOM IMAGE DATA ======\n        # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to \"human-friendly\" view\n        if voi_lut:\n            data = apply_voi_lut(dicom_header.pixel_array, dicom_header)\n        else:\n            data = dicom_header.pixel_array\n        # depending on this value, X-ray may look inverted - fix that:\n        if fix_monochrome and dicom_header.PhotometricInterpretation == \"MONOCHROME1\":\n            data = np.amax(data) - data\n        data = data - np.min(data)\n        data = data / np.max(data)\n        modified_image_data = (data * 255).astype(np.uint8)\n    \n        return dicom_dict, modified_image_data\n    \n    else:\n        return dicom_dict\n\ndef _sanitise_unicode(s):\n    return s.replace(u\"\\u0000\", \"\").strip()\n\ndef _convert_value(v):\n    t = type(v)\n    if t in (list, int, float):\n        cv = v\n    elif t == str:\n        cv = _sanitise_unicode(v)\n    elif t == bytes:\n        s = v.decode('ascii', 'replace')\n        cv = _sanitise_unicode(s)\n    elif t == dicom.valuerep.DSfloat:\n        cv = float(v)\n    elif t == dicom.valuerep.IS:\n        cv = int(v)\n    else:\n        cv = repr(v)\n    return cv\n\n","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:56:31.686935Z","iopub.execute_input":"2021-06-29T20:56:31.687364Z","iopub.status.idle":"2021-06-29T20:56:31.701596Z","shell.execute_reply.started":"2021-06-29T20:56:31.687333Z","shell.execute_reply":"2021-06-29T20:56:31.700416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Getting the dictionary data to the dataframe and dropping the columns not needed","metadata":{}},{"cell_type":"code","source":"metadata = []\n\nfor filename in training_set.path:\n    try:\n        data_di = dicom_dataset_to_dict(filename,'metadata_df')\n        metadata.append(data_di)\n    except:\n        continue\n\ndicom_data_df = pd.DataFrame(metadata)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-29T20:56:38.050498Z","iopub.execute_input":"2021-06-29T20:56:38.050977Z","iopub.status.idle":"2021-06-29T21:11:55.498276Z","shell.execute_reply.started":"2021-06-29T20:56:38.050943Z","shell.execute_reply":"2021-06-29T21:11:55.497193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dicom_data_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dicom_data_df.drop(['Specific Character Set', 'SOP Class UID','SOP Instance UID','Study Date','Study Time','Accession Number','Patient ID','Accession Number','Rows','Columns'], axis=1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Get the metadata information as new columns in an existing dataframe","metadata":{}},{"cell_type":"code","source":"training_set_merged = pd.merge(left = training_set, right = dicom_data_df, how = 'left', left_on = 'StudyInstanceUID', right_on = 'Study Instance UID')\ntraining_set_merged.head()\ntraining_set = training_set_merged","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To be able to use y_col attribute in the ImageDataGenerator, we will create new column 'Class' containing the type of pneumonia.","metadata":{}},{"cell_type":"code","source":"for row in range(6334):\n    if training_set['Negative for Pneumonia'] == 1:\n        pneumonia_class = 'Negative for Pneumonia'\n    elif training_set['Typical Appearance'] == 1:\n        pneumonia_class = 'Typical Appearance'\n    elif training_set['Indeterminate Appearance'] == 1:\n        pneumonia_class = 'Indeterminate Appearance'\n    else:\n        pneumonia_class = 'Atypical Appearance'\n\n    training_set['Class'] = pneumonia_class","metadata":{"execution":{"iopub.status.busy":"2021-06-29T21:11:55.500004Z","iopub.execute_input":"2021-06-29T21:11:55.500507Z","iopub.status.idle":"2021-06-29T21:11:55.542078Z","shell.execute_reply.started":"2021-06-29T21:11:55.500467Z","shell.execute_reply":"2021-06-29T21:11:55.539672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_set.head()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**TBD**: Outliers and irregularities in the data","metadata":{}},{"cell_type":"markdown","source":"# Building the model\n****","metadata":{}},{"cell_type":"markdown","source":"EfficientNet is used:\n\nThe EfficientNet scaling method uniformly scales network width, depth, and resolution with a set of fixed scaling coefficients. The base EfficientNet-B0 network is based on the inverted bottleneck residual blocks of MobileNetV2, in addition to squeeze-and-excitation blocks.","metadata":{}},{"cell_type":"markdown","source":"**Understanding how EfficientNets works a little better:**","metadata":{}},{"cell_type":"markdown","source":"The core idea about Efficient Nets is the use of compound scaling - using a weighted scale of three inter-connected hyper parameters of the model - Resolution of the input, Depth of the Network and Width of the Network.\n\n\n![](https://warehouse-camo.ingress.cmh1.psfhosted.org/fe998467d67d4e76b3f0c81fd7d52db053735d7c/68747470733a2f2f6c617465782e636f6465636f67732e636f6d2f706e672e6c617465783f5c696e6c696e652673706163653b5c6470697b3330307d2673706163653b5c62675f77686974652673706163653b5c626567696e7b616c69676e2a7d2673706163653b64657074683a262673706163653b642673706163653b3d2673706163653b5c616c7068612673706163653b5e2673706163653b5c7068692673706163653b5c5c2673706163653b77696474683a262673706163653b772673706163653b3d2673706163653b5c626574612673706163653b5e2673706163653b5c7068692673706163653b5c5c2673706163653b7265736f6c7574696f6e3a262673706163653b722673706163653b3d2673706163653b5c67616d6d612673706163653b5e2673706163653b5c7068692673706163653b5c656e647b616c69676e2a7d//)\n\n\nWhen phi, the compound coefficient, is initially set to 1, we get the base configuration - in this case EfficientNetB0. We then use this configuration in a grid search to find the coefficients alpha, beta and gamma which optimize the following objective under the constraint:\n\n\n![](https://warehouse-camo.ingress.cmh1.psfhosted.org/bc03bbc347eef78c683053ad5e24f5e348c5562b/68747470733a2f2f6c617465782e636f6465636f67732e636f6d2f706e672e6c617465783f5c696e6c696e652673706163653b5c6470697b3330307d2673706163653b5c626567696e7b616c69676e2a7d2673706163653b5c616c7068612673706163653b5c63646f742673706163653b5c626574612673706163653b5e2673706163653b322673706163653b5c63646f742673706163653b5c67616d6d612673706163653b5e2673706163653b322673706163653b265c617070726f782673706163653b322673706163653b5c5c2673706163653b5c616c7068612673706163653b5c67652673706163653b312c2673706163653b5c626574612673706163653b5c67652673706163653b26312c2673706163653b5c67616d6d612673706163653b5c67652673706163653b312673706163653b5c656e647b616c69676e2a7d)\n\n\nOnce these coefficients for alpha, beta and gamma are found, then simply scale phi, the compound coeffieints by different amounts to get a family of models with more capacity and possibly better performance.","metadata":{}},{"cell_type":"markdown","source":"**EfficientNet pros:**\n\nBy using shortcuts directly between the bottlenecks which connects a much fewer number of channels compared to expansion layers, combined with depthwise separable convolution which effectively **reduces computation** by almost a factor of k^2, compared to traditional layers. Where k stands for the kernel size, specifying the height and width of the 2D convolution window.\n\nThe second benefit of EfficientNet, it scales more efficiently by carefully balancing network depth, width, and resolution, which lead to **better performance**.","metadata":{}},{"cell_type":"markdown","source":"****\nNext we set up the infrastructure to run a training job on our dataset. We choose the number of epochs to train for. The more epochs, the better your model is likely to fit your data but training will run for longer.\n\nNext, we set up the network to build the correct number of layers for the number of classes we have in our dataset.","metadata":{}},{"cell_type":"markdown","source":"**Layers:**\n\n\n**The pooling layer** operates upon each feature map separately to create a new set of the same number of pooled feature maps.\nPooling involves selecting a pooling operation, much like a filter to be applied to feature maps.\nAverage Pooling: Calculate the average value for each patch on the feature map.\n\n**Dense** implements the operation: output = activation(dot(input, kernel) + bias) where activation is the element-wise activation function passed as the activation argument, kernel is a weights matrix created by the layer, and bias is a bias vector created by the layer (only applicable if use_bias is True).\nDense is the only actual network layer in the model.\nA Dense layer feeds all outputs from the previous layer to all its neurons, each neuron providing one output to the next layer. A Dense(10) has ten neurons.\n\n**Model** groups layers into an object with training and inference features.\n\n**Adam** is a replacement optimization algorithm for stochastic gradient descent for training deep learning models. Adam combines the best properties of the AdaGrad and RMSProp algorithms to provide an optimization algorithm that can handle sparse gradients on noisy problems.\n\nLoss is a prediction error of Neural Net. And the method to calculate the loss is called Loss Function. In simple words, the Loss is used to calculate the gradients. And gradients are used to update the weights of the Neural Net.\n\n**CategoricalCrossentropy** - crossentropy loss function when there are two or more label classes","metadata":{}},{"cell_type":"markdown","source":"****","metadata":{}},{"cell_type":"code","source":"EFNS = [efn.EfficientNetB0, efn.EfficientNetB1, efn.EfficientNetB2, efn.EfficientNetB3, \n        efn.EfficientNetB4, efn.EfficientNetB5, efn.EfficientNetB6, efn.EfficientNetB7]\n\ndef build_model(dim = IMG_SIZES[0], ef = 0):\n    inputs = tf.keras.layers.Input(shape = (*dim, 3))\n    base = EFNS[ef](input_shape = (*dim,3), weights = 'imagenet', include_top = False)\n    x = base(inputs)\n    \n    # pooling layer\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    # Dense\n    x = tf.keras.layers.Dense(64, activation = 'relu')(x)\n    x = tf.keras.layers.Dense(4, activation = 'softmax')(x)\n    \n    model = tf.keras.Model(inputs = inputs, outputs = x)\n    \n    opt = tf.keras.optimizers.Adam(learning_rate = 0.001)\n    \n    loss = tf.keras.losses.CategoricalCrossentropy(label_smoothing = 0.01)\n    \n    auc = tf.keras.metrics.AUC(curve = 'ROC', multi_label = True)\n    \n    acc = tf.keras.metrics.CategoricalAccuracy()\n    \n    f1  = tfa.metrics.F1Score(num_classes = 4,average = 'macro',threshold = None)\n    \n    model.compile(optimizer = opt,loss = loss,metrics = [auc, acc, f1])\n    \n    return model","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training the model\n****","metadata":{}},{"cell_type":"markdown","source":"1. Specify where the training and test folders are\n2. Use Keras's **ImageDataGenerator** to **augment** the training data. If you haven't used this library before, or are new to data augmentation, take a look at this link: http://keras.io/preprocessing/image/\n3. Use a pre-trained model called **EfficientNet**\n4. Make predictions on the test images in the test zip file and format the submission.csv file to hold our own submissions!","metadata":{}},{"cell_type":"markdown","source":"Image data augmentation is a technique that can be used to artificially expand the size of a training dataset by creating modified versions of images in the dataset.\n\n**Augmenting the images: ImageDataGenerator:**\n\n* generate **two generators** - one for training, and another for validation. These are stored in train_generator and val_generator. For both, we apply a series of distortions.\n* instead of storing all these new images in a directory, we use the method flow_from_dataframe to dynamically load these images as we train the model\n* **flow_from_dataframe** - takes the dataframe and the path to a directory + generates batches. The generated batches contain augmented/normalized data.\n* all the distortions we made for **train_gen** are not applied to test_gen. This is because we don't want to augment the data in the test directory.","metadata":{}},{"cell_type":"markdown","source":"* **training** dataset - used to fit the model\n* **validation** dataset - used to provide an unbiased evaluation of a model fit on the training dataset while tuning model hyperparameters","metadata":{}},{"cell_type":"code","source":"from keras.preprocessing.image import ImageDataGenerator\n\ndata_generator = ImageDataGenerator(\n    rescale = 1/255,\n    validation_split = 0.10,\n    rotation_range = 40,\n    width_shift_range = 0.2,\n    height_shift_range = 0.2,\n    shear_range = 0.2,\n    zoom_range = 0.2,\n    horizontal_flip = True,\n    fill_mode = 'nearest'\n)\n\ntrain_dg = data_generator.flow_from_dataframe(\ndataframe = training_set,\ndirectory = ,\nx_col = ,\ny_col = ,\ntarget_size = (),\nsubset = ,\nbatch_size = 1024,\nshuffle = True,\nclass_mode = 'categorical')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"folds = 5\nepochs = [12] * folds\n\n# train and validation subsets\n\nnum_of_train_files = len(train_image_level)\n\nprint(num_of_train_files)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T12:43:42.592075Z","iopub.execute_input":"2021-07-14T12:43:42.592404Z","iopub.status.idle":"2021-07-14T12:43:42.597884Z","shell.execute_reply.started":"2021-07-14T12:43:42.592373Z","shell.execute_reply":"2021-07-14T12:43:42.596764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**KFold cross validation**\n****\nProvides train/test indices to split data in train/test sets. Split dataset into k consecutive folds (without shuffling by default).\n\nEach fold is then used once as a validation while the k - 1 remaining folds form the training set.\n\nWhen shuffle is True, random_state affects the ordering of the indices, which controls the randomness of each fold. Otherwise, this parameter has no effect. \n\nsplit(X[, y, groups]) - Generate indices to split data into training and test set.","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import KFold\n\nskf = KFold(n_splits = folds, shuffle = True, random_state = 0)\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.arange(num_of_train_files)\nskf.split(np.arange(num_of_train_files))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# References\n****\n\n* https://github.com/pydicom/pydicom/issues/319\n* https://www.kaggle.com/songseungwon/siim-covid-19-detection-10-step-tutorial-1\n* https://www.kaggle.com/ruchi798/siim-covid-19-detection-eda-data-augmentation#DICOM-data\n* https://www.kaggle.com/awsaf49/siim-covid-19-study-level-train-tpu/comments\n* https://towardsdatascience.com/train-validation-and-test-sets-72cb40cba9e7\n* https://www.kaggle.com/arjunrao2000/beginners-guide-efficientnet-with-keras","metadata":{}}]}