{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## **RSNA Pneumonia Detection**\n\nPneumonia is an infection in one or both lungs. Bacteria, viruses, and fungi cause it. The infection causes inflammation in the air sacs in your lungs, which are called alveoli.\n\nCXRs are the most commonly performed diagnostic imaging study. A number of factors such as \npositioning of the patient and depth of inspiration can alter the appearance of the CXR, complicating \ninterpretation further. In addition, clinicians are faced with reading high volumes of images every shift.\n\nAutomating Pneumonia screening in chest radiographs, providing affected area details through bounding box.","metadata":{"id":"jbsUHUkcNgMg"}},{"cell_type":"markdown","source":"## **Objective:**\nThe objective of this project is to build an algorithm to locate the position of inflammation in a medical image. The algorithm needs to  locate lung opacities on chest radiographs automatically\n\nThe objective of the project is,\n* Learn to how to do build an Object Detection Model\n* Use transfer learning to fine-tune a model.\n* Learn to set the optimizers, loss functions, epochs, learning rate, batch size, checkpointing, early stopping etc.\n* Read different research papers of given domain to obtain the knowledge of advanced models for the given problem.\n\n\n#### Acknowledgment for the datasets: https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/overview/acknowledgements","metadata":{"id":"XYm7TlOX1tgN"}},{"cell_type":"markdown","source":"### **1.0 Importing and installing the necessary Libraries**","metadata":{"id":"_6icbKjAIQiv"}},{"cell_type":"code","source":"!pip install tqdm\n!pip install pydicom\n!pip install -U albumentations","metadata":{"id":"rCqKX8LJXwX2","outputId":"9f175f42-f3c8-4f37-9f20-10883600d08b","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%matplotlib inline","metadata":{"id":"bXhuVFq9Jmb-","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tensorflow as tf","metadata":{"id":"zvwFW9QrJJVk","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Initialize the random number generator\nimport random\nrandom.seed(0)\n\n# Ignore the warnings\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"id":"P-zTGWRF2wnQ","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from glob import glob\nimport os\nimport time\nimport math\nimport fnmatch\n\nfrom zipfile import ZipFile\nfrom tqdm import tqdm_notebook\n\nimport numpy as np\nimport pandas as pd\n\nimport cv2\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as patches \nfrom matplotlib.patches import Rectangle\nimport pydicom as dicom\nimport seaborn as sns\n\nfrom sklearn.utils import shuffle\nfrom skimage.measure import label, regionprops\n\nimport albumentations as A\n\nfrom tensorflow import keras\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.python.keras.utils.data_utils import Sequence\nfrom tensorflow.keras.layers import Conv2D, Input, Flatten, Dense, Dropout, Concatenate, BatchNormalization, Conv2DTranspose\nfrom tensorflow.keras.models import Model, Sequential, load_model\nimport tensorflow.keras.backend as K\nfrom tensorflow.keras.applications import VGG16\nfrom tensorflow.keras.applications.vgg16 import preprocess_input\n\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau\nfrom tensorflow.keras.losses import binary_crossentropy\n","metadata":{"id":"R3Pf-mzZIQi5","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **1.1. EDA and Visualization**\n\n*   Importing Data\n*   Analysing the dimensions of data\n*   Visualizing the data\n","metadata":{"id":"4dqgYZr826s7"}},{"cell_type":"code","source":"rootDir='/kaggle/input/'\nworkingDir='/kaggle/working/'\nzipFilename=rootDir+'rsna-pneumonia-detection-challenge.zip'\ndatasetPath=rootDir+'rsna-pneumonia-detection-challenge/'","metadata":{"id":"FlR0Rw5ZOwhh","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The input folder contains 4 important information\n* stage_2_train_labels.csv - CSV file containing the patient id, bounding boxes and target label\n* stage_2_detailed_class_info.csv - CSV file containing the detail informaiton of patientid and the corresponding label\n* stage_2_train_images - directory contains train images in DICOM format\n* stage_2_test_images - directory contains test images in DICOM format","metadata":{"id":"NSHzxBkoIQi7"}},{"cell_type":"code","source":"trainImagesDir=datasetPath+'stage_2_train_images/'\ntestImagesDir=datasetPath+'stage_2_test_images/'\nsampleSubmission=datasetPath+'stage_2_sample_submission.csv'\nclassInfo=datasetPath+'stage_2_detailed_class_info.csv'\nrsnaLink=datasetPath+'GCP Credits Request Link - RSNA.txt'\ntrainLabels=datasetPath+'stage_2_train_labels.csv'","metadata":{"id":"btEyb0dOIQi8","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_train_path = os.listdir(trainImagesDir)\nimage_test_path = os.listdir(testImagesDir)\nprint(\"Number of images in train set:\", len(image_train_path),\"\\nNumber of images in test set:\", len(image_test_path))","metadata":{"id":"nsQ5padnIQi9","outputId":"fc83b7e2-263e-4e9c-8b2c-dae5b4bc17fd","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Loading the data\n# There are two input files given - Detailed class info and train labels\nclass_info_df = pd.read_csv(classInfo)\ntrain_labels_df = pd.read_csv(trainLabels) ","metadata":{"id":"GObLdVDE3WTQ","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Detailed class info -  rows: {}, columns: {}\".format(class_info_df.shape[0], class_info_df.shape[1]))\nprint(\"Train labels -  rows: {}, columns: {}\".format(train_labels_df.shape[0], train_labels_df.shape[1]))","metadata":{"id":"BxrzZNb03KzL","outputId":"b808b49f-b49d-402f-e9cb-f49899f36837","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are only 26683 images in the image directory, but the csv file contains 30227 rows. There are more rows than the images.","metadata":{"id":"q9_COwIk5IZn"}},{"cell_type":"code","source":"class_info_df.head(10)","metadata":{"id":"dPXjudVp4KDB","outputId":"7341d7df-862b-41e4-f8e5-d4f39069dc45","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_labels_df.head(10)","metadata":{"id":"343oKZ1pIQi_","outputId":"338cf6e4-998b-4c4a-e277-16d83e355f55","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In Detailed class info dataset , the detailed information about the type of class associated with a certain patientId is given. It has 3 entries \"Lung Opacity\", \"Normal\" and \"No Lung Opacity/Not Normal\"\n\nThe CSV file contains PatientId, bounding box details with (x,y) coordinates and width and height that encapsulates the box. It also contains the Target variable. For target variable 0, the bounding box values has NaN values.\n","metadata":{"id":"fVfoZd0oIQi_"}},{"cell_type":"markdown","source":"If we look closely, there are duplicate entries for patientId in the csv files. We can observe row #4 and #5, row #8 and #9 have same patientId values, aka, the patient is identified with pneumonia at multiple areas in lungs\n\nCheck the unique patient ID in the train dataset","metadata":{"id":"oMaOVkdAIQjA"}},{"cell_type":"code","source":"print(\"Unique patientId in  train_class_df: \", train_labels_df['patientId'].nunique())","metadata":{"id":"fjoEVuzGARr7","outputId":"349e0ca4-80c5-4e66-b98d-53b89e98ebbc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### **Checking missing data in two datasets**","metadata":{"id":"TFwZ4Klz5sFp"}},{"cell_type":"code","source":"train_labels_df.info()","metadata":{"id":"rrmoRrKaIQjB","outputId":"0acffec1-d0c0-4183-8fa1-525e2a6263bf","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For the info of the data, we observe that of the total 30227 rows, 9555 rows has non null. So, all bounding boxes are either defined or not defined.","metadata":{"id":"1MZ0uLKRIQjB"}},{"cell_type":"code","source":"print(train_labels_df[train_labels_df.Target==0].shape[0])\nprint(train_labels_df[train_labels_df.Target==1].shape[0])","metadata":{"id":"OSltZzEaIQjC","outputId":"51b5f9c7-0010-4923-8de8-a530e19e8456","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see from above that the total number of patientIds that are identified with Pneumonia are 9555 and it matches to the non null values. It can be inferred from this that all pneumonia data set has bounding boxes defined and for normal patients, no bounding boxes exist.","metadata":{"id":"sTMOHOJ-IQjC"}},{"cell_type":"code","source":"def missing_data(data):\n    total = data.isnull().sum().sort_values(ascending = False)\n    percent = (data.isnull().sum()/data.isnull().count()*100).sort_values(ascending = False)\n    return np.transpose(pd.concat([total, percent], axis=1, keys=['Total', 'Percent']))\nmissing_data(train_labels_df)","metadata":{"id":"TaRU7BYs5ySh","outputId":"9a4ca2de-76fe-4a69-f080-de74a5224b8c","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"missing_data(class_info_df)","metadata":{"id":"fOrPX3Ww59yH","outputId":"62b882c2-ece0-492f-aab5-9809effa6eb7","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"68.38% of values are missing for x,y, height and width in train labels for target 0 (not Lung opacity) in train labels dataset","metadata":{"id":"mK3AQyFV6SSs"}},{"cell_type":"markdown","source":"#### **Checking class distribution in Detailed class info dataset**","metadata":{"id":"5rjQod1u6qju"}},{"cell_type":"code","source":"plt.rc('axes', labelsize=15)\nplt.rc('axes', titlesize=20)\nsns.set_palette('Set2')","metadata":{"id":"nN2eJOn67ewk","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, ax = plt.subplots(1,1, figsize=(6,4))\ntotal = float(class_info_df.shape[0])\nsns.countplot(class_info_df['class'], order = class_info_df['class'].value_counts().index)\nfor p in ax.patches:\n    height = p.get_height()\n    ax.text(p.get_x()+p.get_width()/2.,\n            height + 3,\n            '{:1.2f}%'.format(100*height/total),\n            ha=\"center\") \nplt.show()","metadata":{"id":"I2ISqt1N6s_l","outputId":"812df04b-6788-474e-eabc-e2fd8982ca14","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# More details on classes - No Lung Opacity / Not Normal, Lung Opacity, Normal\n\ndef get_feature_distribution(data, feature):\n    # Get the count for each label\n    label_counts = data[feature].value_counts()\n\n    # Get total number of samples\n    total_samples = data.shape[0]\n\n    # Count the number of items in each class\n    print(\"{:<30s}:   count(percentage)\".format(feature))\n    for i in range(len(label_counts)):\n        label = label_counts.index[i]\n        count = label_counts.values[i]\n        percent = round((count / total_samples) * 100, 2)\n        print(\"{:<30s}:   {}({}%)\".format(label, count, percent))\n\nget_feature_distribution(class_info_df, 'class')\n","metadata":{"id":"t8eTSkX96v-A","outputId":"5685acae-0cbb-4e74-8805-2dbb1ae09f68","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"No Lung Opacity / Not Normal and Normal have together the same percent (68.39%) as the percent of missing values for target window in class details information.\n\nIn the train set, the percent of data with pneumonia is therefore 31.61%.","metadata":{"id":"mu4jkvAB8qkk"}},{"cell_type":"code","source":"train_labels_df.Target.unique()","metadata":{"id":"3gwAxD95IQjD","outputId":"ad6a56f7-ee71-4dc6-e6a8-603aaccdec1d","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The target has two classifications 0 and 1 namely Normal and Pneumonia","metadata":{"id":"K38YsiXDIQjD"}},{"cell_type":"markdown","source":"#### **Merging train labels and Detailed class info datasets to get more insights**","metadata":{"id":"RpR0m1SF90oo"}},{"cell_type":"code","source":"train_labels_df.shape[0], class_info_df.shape[0]","metadata":{"id":"8q6hTb-fQvlW","outputId":"ff7d50ba-affe-48f7-d80a-df3af599a1ba","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# merging the two datasets (train and class detail info) using Patient ID as the merge criteria\ntrain_class_df = train_labels_df.merge(class_info_df, left_on='patientId', right_on='patientId', how='inner')","metadata":{"id":"3rihhjiG921Z","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_class_df.sample(5)","metadata":{"id":"5NoaF9Uc95Hw","outputId":"986cc31c-e35e-46cf-dcdf-68b1101b6333","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#plotting the number of examinations for each class detected, grouped by Target value\nfig, ax = plt.subplots(nrows=1,figsize=(12,6))\ntmp = train_class_df.groupby('Target')['class'].value_counts()\ndf = pd.DataFrame(data={'Freq': tmp.values}, index=tmp.index).reset_index()\nsns.barplot(ax=ax, x='Target', y='Freq', hue='class', data=df)\nplt.title(\"Chest examination - Frequency of Targets\")\nplt.show()","metadata":{"id":"R0KPaQuh97lH","outputId":"00d944cf-7948-46ab-8ec9-c9c0774930d1","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Plot frequency distribution graph for bounding box detection for Lung Opacity ","metadata":{"id":"rfxKxQen-4WV"}},{"cell_type":"code","source":"target1 = train_class_df[train_class_df['Target']==1]\nsns.set_style('whitegrid')\nplt.figure()\nfig, ax = plt.subplots(2,2,figsize=(12,12))\nsns.distplot(target1['x'],kde=True,bins=50, color=\"orange\", ax=ax[0,0])\nsns.distplot(target1['y'],kde=True,bins=50, color=\"green\", ax=ax[0,1])\nsns.distplot(target1['width'],kde=True,bins=50, color=\"brown\", ax=ax[1,0])\nsns.distplot(target1['height'],kde=True,bins=50, color=\"blue\", ax=ax[1,1])\nlocs, labels = plt.xticks()\nplt.tick_params(axis='both', which='major', labelsize=12)\nplt.show()","metadata":{"id":"yV3uMvHc-3WF","outputId":"7cad2e9c-2f56-4f21-cb39-24a53cbb9005","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Exploring Dicom image files - Reading training & test files","metadata":{"id":"bYmLiMPH_iv4"}},{"cell_type":"markdown","source":"#### **Extracting a single image and processing DICOM information**","metadata":{"id":"ZFij71jtBQ3T"}},{"cell_type":"code","source":"samplePatientID = list(train_class_df[:3].T.to_dict().values())[0]['patientId']\nsamplePatientID = samplePatientID+'.dcm'\ndicom_file_path = os.path.join(trainImagesDir, samplePatientID)\ndicom_file_dataset = dicom.read_file(dicom_file_path)\ndicom_file_dataset","metadata":{"id":"gxKZzjTKBYOa","outputId":"2e2cc34c-2543-4303-fa43-d7e88f71d22d","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It is observed that some useful information are available in the DICOM metadata with predictive values, for example:\n\nPatient sex, Patient age, Modality, Body part examined, View position, Rows & Columns, Pixel Spacing","metadata":{"id":"UJVtWrsUBxxF"}},{"cell_type":"markdown","source":"#### **Plotting dicom images with Target = 1**","metadata":{"id":"-gjXhG5uCFDz"}},{"cell_type":"code","source":"def show_dicom_images(data, bbox=False):\n    img_data = list(data.T.to_dict().values())\n    f, ax = plt.subplots(3,3, figsize=(16,25))\n    for i,data_row in enumerate(img_data):\n        patientImage = data_row['patientId']+'.dcm'\n        imagePath = os.path.join(trainImagesDir, patientImage)\n        data_row_img_data = dicom.read_file(imagePath)\n        modality = data_row_img_data.Modality\n        age = data_row_img_data.PatientAge\n        sex = data_row_img_data.PatientSex\n        data_row_img = dicom.dcmread(imagePath)\n        ax[i//3, i%3].imshow(data_row_img.pixel_array, cmap=plt.cm.bone) \n        ax[i//3, i%3].axis('off')\n        ax[i//3, i%3].set_title('ID: {}\\nModality: {} Age: {} Sex: {} Target: {}\\nClass: {}\\nBounding box: {}:{}:{}:{}'.format(\n                data_row['patientId'],\n                modality, age, sex, data_row['Target'], data_row['class'], \n                data_row['x'],data_row['y'],data_row['width'],data_row['height']))\n        if bbox:\n          rows = train_class_df[train_class_df['patientId']==data_row['patientId']]\n          box_data = list(rows.T.to_dict().values())\n          for j, row in enumerate(box_data):\n              ax[i//3, i%3].add_patch(Rectangle(xy=(row['x'], row['y']),\n                          width=row['width'],height=row['height'], \n                          color=\"yellow\",alpha = 0.1))   \n    plt.tight_layout()\n    plt.show()","metadata":{"id":"KU7qAkG2CHtA","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"show_dicom_images(train_class_df[train_class_df['Target']==1].sample(9))","metadata":{"id":"1f8CesHLCUjs","outputId":"69ca7337-3082-46f5-c3b2-b99c51e587c3","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next step is to represent the images with the overlay boxes superposed. For this, the whole dataset with Target = 1 has to be parsed and all coordinates of the windows showing a Lung Opacity on the same image have to be gathered.","metadata":{"id":"gUenByJcEE_P"}},{"cell_type":"code","source":"show_dicom_images(train_class_df[train_class_df['Target']==1].sample(9), bbox=True)","metadata":{"id":"RSnWHJqNEQD4","outputId":"85fc98d8-8f7c-4ff5-ecf0-0926117fc7e1","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For some of the images with Target=1, we could see multiple areas (boxes/rectangles) with Lung Opacity. \n","metadata":{"id":"mbsRDWgiEwuG"}},{"cell_type":"code","source":"print('A maximum of {} areas are detected in Lungs for pneumonia patient'.format(max(train_labels_df.patientId.value_counts())))","metadata":{"id":"WrDhxKIqIQjH","outputId":"fab2d705-5c2d-4200-b535-629a4f5259d4","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### **Plotting DICOM images with Target = 0**","metadata":{"id":"v-jFIfFNGEWb"}},{"cell_type":"code","source":"show_dicom_images(train_class_df[train_class_df['Target']==0].sample(9))","metadata":{"id":"7XkttMmPGFDX","outputId":"e93f7952-d6e6-4e37-f6b2-21b0ed1d86e6","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### **Adding metadata information from Dicom data to train and test datasets**","metadata":{"id":"iMAf0tr6Hlvu"}},{"cell_type":"code","source":"# parsing the DICOM meta information and add it to the train dataset\nvars = ['Modality', 'PatientAge', 'PatientSex', 'BodyPartExamined', 'ViewPosition', 'ConversionType', 'Rows', 'Columns', 'PixelSpacing']\n\ndef process_dicom_data(data_df, data_path):\n    for var in vars:\n        data_df[var] = None\n    image_names = os.listdir(data_path)\n    for i, img_name in tqdm_notebook(enumerate(image_names)):\n        imagePath = os.path.join(data_path,img_name)\n        data_row_img_data = dicom.read_file(imagePath)\n        idx = (data_df['patientId']==data_row_img_data.PatientID)\n        data_df.loc[idx,'Modality'] = data_row_img_data.Modality\n        data_df.loc[idx,'PatientAge'] = pd.to_numeric(data_row_img_data.PatientAge)\n        data_df.loc[idx,'PatientSex'] = data_row_img_data.PatientSex\n        data_df.loc[idx,'BodyPartExamined'] = data_row_img_data.BodyPartExamined\n        data_df.loc[idx,'ViewPosition'] = data_row_img_data.ViewPosition\n        data_df.loc[idx,'ConversionType'] = data_row_img_data.ConversionType\n        data_df.loc[idx,'Rows'] = data_row_img_data.Rows\n        data_df.loc[idx,'Columns'] = data_row_img_data.Columns  \n        data_df.loc[idx,'PixelSpacing'] = str.format(\"{:4.3f}\",data_row_img_data.PixelSpacing[0])","metadata":{"id":"YxB0Eq8fHn3O","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"process_dicom_data(train_class_df, trainImagesDir)","metadata":{"id":"huVrzNlXHsd2","outputId":"46207f32-4452-4a26-ddce-e1f9660ec2a5","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Creating Test dataset with similar information as that of train set\n# stage_2_sample_submission.csv - Contains patientIds for the test set. sample submission contains one box per image, but there is no limit to the number of bounding boxes that can be assigned to a given image.\ntest_class_df = pd.read_csv(sampleSubmission)\ntest_class_df.head()","metadata":{"id":"Me62MbRPIk7f","outputId":"766d0db1-f386-4e91-a7fa-7b9ff091d572","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_class_df = test_class_df.drop('PredictionString',1)\nprocess_dicom_data(test_class_df, testImagesDir)","metadata":{"id":"eHQuXCOHInmF","outputId":"e2c86a94-b3d6-4b78-da54-c7f658e1e795","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Checking how many modalities are used\nprint(\"Modalities: train:\",train_class_df['Modality'].unique(), \"test:\", test_class_df['Modality'].unique())","metadata":{"id":"rO1QjB9wIflu","outputId":"b365cf5e-ce35-4a1b-acf7-9823640d8545","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The meaning of this modality is CR - Computer Radiography","metadata":{"id":"OMRA3IMQI2-r"}},{"cell_type":"code","source":"# checking if other body parts than 'CHEST' appears in the data\nprint(\"Body Part Examined: train:\",train_class_df['BodyPartExamined'].unique(), \"test:\", test_class_df['BodyPartExamined'].unique())","metadata":{"id":"6SLBEkDqI5Ga","outputId":"2d5c3db2-6744-4a92-e9f1-1b79ee4852c1","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# View Position is a radiographic view associated with the Patient Position. Let's check the View Positions distribution for the both datasets\nprint(\"View Position: train:\",train_class_df['ViewPosition'].unique(), \"test:\", test_class_df['ViewPosition'].unique())","metadata":{"id":"lTRvPJZPI7RS","outputId":"a8276fb9-3998-4d62-f543-4d4f11211e23","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Train dataset-checking the distribution of PA and AP\nget_feature_distribution(train_class_df,'ViewPosition')","metadata":{"id":"mhVQnNTKI-Nz","outputId":"8b259103-a8f4-40f9-aa8d-d37ca199ed8d","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Both AP and PA body positions are present in the data. The meaning of these view positions are :\n\nAP - Anterior/Posterior; PA - Posterior/Anterior.","metadata":{"id":"-wp1JW1qJBUt"}},{"cell_type":"markdown","source":"Test dataset : Checking the distribution of AP and PA positions for the test set","metadata":{"id":"hWk9yHNrJGdu"}},{"cell_type":"code","source":"get_feature_distribution(test_class_df,'ViewPosition')","metadata":{"id":"ndKNWBslJIc-","outputId":"860c5627-5f31-4024-e37d-f327db32bdcc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Conversion Type: Let's check the Conversion Type data\nprint(\"Conversion Type: train:\",train_class_df['ConversionType'].unique(), \"test:\", test_class_df['ConversionType'].unique())","metadata":{"id":"uRYStWB5JKdh","outputId":"d42b3311-62b4-4cfd-889b-2e54d6502529","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Both train and test have only WSD Conversion Type Data. The meaning of this Conversion Type is WSD: Workstation","metadata":{"id":"L2mqDChoJOli"}},{"cell_type":"code","source":"# Rows and columns\nprint(\"Rows: train:\",train_class_df['Rows'].unique(), \"test:\", test_class_df['Rows'].unique())\nprint(\"Columns: train:\",train_class_df['Columns'].unique(), \"test:\", test_class_df['Columns'].unique())","metadata":{"id":"0e-Mc32cJNb_","outputId":"4c671af0-4544-4302-a579-020353a05b5b","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Only {Rows:Columns} {1024:1024} are present in both train and test.\n\nEven though the size of the images are same, we can observe that each image has different aspect ratio, that is, not all images occupy the same space. Also, the brightness is different.","metadata":{"id":"cDjLV7zYJTIr"}},{"cell_type":"markdown","source":"**Distribution of patient age for the test data set**","metadata":{"id":"jbtc11XqJVLm"}},{"cell_type":"code","source":"#Test dataset\nfig, (ax) = plt.subplots(nrows=1,figsize=(16,6))\nsns.countplot(test_class_df['PatientAge'], ax=ax)\nplt.title(\"Test set: Patient Age\")\nplt.xticks(rotation=90)\nplt.show()","metadata":{"id":"_qlIhoIXJXUI","outputId":"ebb2aef8-3f9f-45a2-f4f9-d0af432b8ddb","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Distribution of Patient Sex for the test data. We can see few age for the dataset is mentioned as 412, which is an incorrect data","metadata":{"id":"nvCfL0fGJdYO"}},{"cell_type":"code","source":"# Test Data\nsns.countplot(test_class_df['PatientSex'])\nplt.title(\"Test set: Patient Sex\")\nplt.show()","metadata":{"id":"Ag5WTNl4JnxO","outputId":"5271f9a5-b4ee-4c25-f6c0-3a0a02648e59","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Conclusion**\n\nAfter exploring both the tabular and DICOM data, we were able to:\n\n1. discover duplications in the tabular data\n2. explore the DICOM images\n3. extract meta information from the DICOM data\n4. add features to the tabular data from the meta information in DICOM data\n5. further analyze the distribution of the data with the newly added features from DICOM metadata\n\n\nAll these findings are useful for building a model.","metadata":{"id":"Bot1XMKfJrfd"}},{"cell_type":"markdown","source":"#### **Preprocess the dataset for model input**","metadata":{"id":"TiRtLsKQJvue"}},{"cell_type":"code","source":"def update_dataset(path, df1):\n    pid=[]\n    label=[]\n    bbox=[]\n\n    for name, group in df1.groupby(['patientId','Target']):\n        pid.append(path+group['patientId'].tolist()[0]+'.dcm')\n        label.append(group['Target'].tolist()[0])\n        if group['Target'].tolist()[0] == 1:\n            ibbox=[]\n            for row in group.iterrows():\n                ibbox.append([row[1]['x'], row[1]['y'], row[1]['width'], row[1]['height']])\n            bbox.append(ibbox)\n        else:\n            bbox.append([])\n    df = pd.DataFrame({'patientId':pid, 'bboxes': bbox, 'label':label})\n    return df","metadata":{"id":"zynLElMJIQjE","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can observe that the non-null values are 9555 which matches to the patients that have pneumonia problem","metadata":{"id":"h3VpckVjIQjE"}},{"cell_type":"code","source":"df=update_dataset(trainImagesDir, train_labels_df)\nprint(df.shape)\ndf.head()","metadata":{"id":"sZudJPdpIQjF","outputId":"7be4ebb5-2435-424a-e756-b53563a779d2","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Total number of patients that are normal are {}'.format(df[df.label==0].shape[0]))\nprint('Total number of patients that have pneumonia are {}'.format(df[df.label==1].shape[0]))","metadata":{"id":"NMU248HSIQjF","outputId":"5cc9e8d5-4039-46d7-c2b5-e31e515fd487","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"imgWidth=224\nimgHeight=224\nimgChannels=3\nimgSize=(imgHeight, imgWidth)\nbatchSize=64\nlabelDict={0:'normal', 1:'lung opacity'}","metadata":{"id":"ZTDb7JAHIQjG","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# try:\n#     tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n#     tf.config.experimental_connect_to_cluster(tpu)\n#     tf.tpu.experimental.initialize_tpu_system(tpu)\n#     strategy = tf.distribute.experimental.TPUStrategy(tpu)\n# except ValueError:\n#     strategy = tf.distribute.get_strategy() # for CPU and single GPU\n#     print('Number of replicas:', strategy.num_replicas_in_sync)","metadata":{"id":"D43s0FH5IQjG","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def loadImage(row, axis):\n    image_path = row.patientId\n    img = dicom.dcmread(image_path).pixel_array\n    axis.imshow(img, cmap='gray')\n    lbl=labelDict.get(row.label)\n    bboxes=row.bboxes\n    for bbox in bboxes:\n        x=bbox[0]\n        y=bbox[1]\n        w=bbox[2]\n        h=bbox[3]\n        rect = patches.Rectangle((x,y), w, h, linewidth=2, edgecolor='red', fill=False)\n        axis.add_patch(rect)\n    axis.set_title(lbl)\n    ","metadata":{"id":"v2iU4XxHZS84","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def loadImages(df):\n    cols=5\n    rows=4\n    idx=0\n    f,axarr=plt.subplots(rows,cols,figsize=(18,10))\n    for r in range(rows):\n        for c in range(cols):\n            axis=axarr[r,c]\n            loadImage(df.iloc[idx], axis)\n            idx+=1\n    plt.tight_layout()","metadata":{"id":"btE4Aa3RIQjH","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loadImages(df)","metadata":{"id":"jrMyHgvLIQjI","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let us print the image with maximum bounding boxes","metadata":{"id":"yKcH5vDJYqx2"}},{"cell_type":"code","source":"max_bbox_idx=np.argmax([len(x) for x in df.bboxes])\nf,axarry=plt.subplots(1,1,figsize=(5,5))\nloadImage(df.iloc[max_bbox_idx], axarry)","metadata":{"id":"WoxeTU9qYzbE","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"c=math.ceil(df.shape[0]*0.7)\ntrain_df,val_df=df[:c],df[c:]\nprint(train_df.shape, val_df.shape)","metadata":{"id":"yy0m_y1QIQjK","outputId":"237d64a5-2236-4332-c2d8-48440d563ed4","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# image transformer object\ntransform = A.Compose([\n        A.RandomRotate90(),\n        A.Flip(),\n        A.Transpose(),\n        A.OneOf([\n            A.IAAAdditiveGaussianNoise(),\n            A.GaussNoise(),\n        ], p=0.2),\n        A.OneOf([\n            A.MotionBlur(p=.2),\n            A.MedianBlur(blur_limit=3, p=0.1),\n            A.Blur(blur_limit=3, p=0.1),\n        ], p=0.2),\n        A.ShiftScaleRotate(shift_limit=0.0625, scale_limit=0.2, rotate_limit=45, p=0.2),\n        A.OneOf([\n            A.OpticalDistortion(p=0.3),\n            A.GridDistortion(p=.1),\n            A.IAAPiecewiseAffine(p=0.3),\n        ], p=0.2),\n        A.OneOf([\n            A.CLAHE(clip_limit=2),\n            A.IAASharpen(),\n            A.IAAEmboss(),\n            A.RandomBrightnessContrast(),            \n        ], p=0.3),\n        A.HueSaturationValue(p=0.3),\n    ])\n","metadata":{"id":"GPMLXN7KIc-J","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Custom datagenerator with mask approach\nclass CustomDataGen(Sequence):\n    \n    def __init__(self, df, x_col, y_col, batch_size, input_size=(224, 224, 3), preprocess_function=None, shuffle=True, predict=False, include_labels=False):\n        self.df = df.copy(deep=True)\n        self.x_col = x_col\n        self.y_col = y_col\n        self.batch_size = batch_size\n        self.input_size = input_size\n        self.shuffle = shuffle\n        self.predict = predict\n        self.n = df.shape[0]\n        self.n_class = df[y_col['output1']].nunique()\n        self.preprocess_function=preprocess_function\n        self.include_labels=include_labels\n    \n    def on_epoch_end(self):\n        if self.shuffle:\n            shuffle(self.df.patientId)\n    \n    def __loadinput(self, img_path, lbl, bboxes):\n        image = dicom.dcmread(img_path).pixel_array\n        img_size = (image.shape[0],image.shape[1])\n        image = cv2.resize(image,(self.input_size[0], self.input_size[1]))\n        if len(image.shape) !=3 or image.shape[2]!=3:\n            image = np.stack([image] * 3, axis=-1)\n        # Add data augmention, for train and validation\n        if not self.include_labels:\n            image = transform(image=image)['image']\n        if self.preprocess_function != None:\n            image = self.preprocess_function(image)\n        if self.predict:\n            return image\n        else:\n            mask = np.zeros((self.input_size[0], self.input_size[1]))\n            for i, bbox in enumerate(bboxes):\n                x, y, w, h = bbox\n                x1=math.floor(x*self.input_size[1]/img_size[1])\n                y1=math.floor(y*self.input_size[0]/img_size[0])\n                x2=math.ceil((x+w)*self.input_size[1]/img_size[1])\n                y2=math.ceil((y+h)*self.input_size[0]/img_size[0])\n                mask[y1:y2, x1:x2] = 1\n            if self.include_labels:\n                return image, lbl, mask\n            else:\n                return image, mask\n\n    \n    def __loaddata(self, batches):\n        # Generates data containing batch_size samples\n        path_batch = batches[self.x_col['input']]\n        classes_batch = batches[self.y_col['output1']]\n        bboxes_batch = batches[self.y_col['output2']]\n\n        img_batch = [self.__loadinput(filename, lbl, bboxes) for filename, lbl, bboxes in zip(path_batch, classes_batch, bboxes_batch)]\n        return img_batch\n    \n    def __getitem__(self, index):\n        batches = self.df[index * self.batch_size:(index + 1) * self.batch_size]\n        batchdata = self.__loaddata(batches)\n        if self.predict:\n            return batchdata\n        else:\n            if self.include_labels:\n                image, lbl, mask = zip(*batchdata)\n                return np.array(image), np.array(lbl), np.array(mask)\n            else:\n                image, mask = zip(*batchdata)\n                return np.array(image), np.array(mask)\n\n    def __len__(self):\n        return self.n // self.batch_size\n","metadata":{"id":"hVpBoHu6IQjK","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Need to identify the distribution (Pneumonia and normal) of the data used\nConsider a portion of the dataset for train (2000) and validation (25%) of train size","metadata":{"id":"-FTEwZNePa3j"}},{"cell_type":"code","source":"trainsize=2000\nvalsize=int(0.25*trainsize)\ntrainpartial=train_df[:trainsize]\nvalpartial=val_df[:valsize]\nprint('Distribution of labels in the train data are ', \n      (trainpartial.label.value_counts().values/trainpartial.label.value_counts().values.sum())*100)\nprint('Distribution of labels in the validation data are ', \n      (valpartial.label.value_counts().values/valpartial.label.value_counts().values.sum())*100)","metadata":{"id":"xawDNny-Pa3k","outputId":"f13d68be-46cd-4385-9c37-55941146cd51","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The distribution of the data in both train and validation is good. Also, the distribution of same label data across the train adn test are also good to proceed","metadata":{"id":"oOtbSI2IPa3k"}},{"cell_type":"code","source":"traingen = CustomDataGen( trainpartial,\n                          x_col={'input':'patientId'},\n                          y_col={'output1': 'label', 'output2': 'bboxes'},\n                          batch_size=batchSize, input_size=(imgSize),\n                          preprocess_function=preprocess_input\n                        )\n\nvalgen = CustomDataGen(   valpartial,\n                          x_col={'input':'patientId'},\n                          y_col={'output1': 'label', 'output2': 'bboxes'},\n                          batch_size=batchSize, input_size=(imgSize),\n                          preprocess_function=preprocess_input\n                        )\n","metadata":{"id":"WnGmTF5-IQjL","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def create_vgg16_unetmodel():\n    print('Creating VGG16 model')\n    model = VGG16(input_shape=(imgHeight, imgWidth, 3), include_top=False, weights=\"imagenet\")\n    for layer in model.layers:\n        layer.trainable = False\n\n    #encoder - get output layers for concatenate with the upsampling layer\n    block1 = model.get_layer(\"block1_conv2\").output\n    block2 = model.get_layer(\"block2_conv2\").output\n    block3 = model.get_layer(\"block3_conv3\").output\n    block4 = model.get_layer(\"block4_conv3\").output\n    block5 = model.get_layer(\"block5_conv3\").output\n    # block6 = model.get_layer(\"block5_pool\").output\n\n    #Decoder block\n    x = Conv2DTranspose(512, (2, 2), strides=(2, 2), padding='same') (block5)\n    x = Concatenate()([x, block4])\n    x = Conv2D(512, (3, 3), activation='relu', padding='same') (x)\n    x = Conv2D(512, (3, 3), activation='relu', padding='same') (x)\n    x = Conv2D(512, (3, 3), activation='relu', padding='same') (x)\n    x= BatchNormalization()(x)\n    x=Dropout(0.5)(x)\n\n    x = Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same') (x)\n    x = Concatenate()([x, block3])\n    x = Conv2D(256, (3, 3), activation='relu', padding='same') (x)\n    x = Conv2D(256, (3, 3), activation='relu', padding='same') (x)\n    x = Conv2D(256, (3, 3), activation='relu', padding='same') (x)\n    x= BatchNormalization()(x)\n    x=Dropout(0.5)(x)\n\n    x = Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same') (x)\n    x = Concatenate()([x, block2])\n    x = Conv2D(128, (3, 3), activation='relu', padding='same') (x)\n    x = Conv2D(128, (3, 3), activation='relu', padding='same') (x)\n    x= BatchNormalization()(x)\n    x=Dropout(0.5)(x)\n\n    x = Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same') (x)\n    x = Concatenate()([x, block1])\n    x = Conv2D(64, (3, 3), activation='relu', padding='same') (x)\n    x = Conv2D(64, (3, 3), activation='relu', padding='same') (x)\n    x= BatchNormalization()(x)\n\n    x = Conv2D(1, kernel_size=1, activation=\"sigmoid\")(x)\n\n    vgg16unetmodel = Model(inputs=model.input, outputs=x)\n    \n    return vgg16unetmodel\n","metadata":{"id":"o2uPSuvRIQjL","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model=create_vgg16_unetmodel()\nmodel.summary()","metadata":{"id":"-SiSvIaLIQjM","outputId":"668157e9-3b3e-4f8b-b55a-cf5a71e371cc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def iou_loss(y_true, y_pred):\n    y_true = tf.reshape(y_true, [-1])\n    y_pred = tf.reshape(y_pred, [-1])\n    intersection = tf.reduce_sum(float(y_true) * float(y_pred))\n    score = (intersection + 1.) / (tf.reduce_sum(float(y_true)) + tf.reduce_sum(float(y_pred)) - intersection + 1.)\n    return 1 - score\n\n# combine bce loss and iou loss\ndef iou_bce_loss(y_true, y_pred):\n    return 0.5 * keras.losses.binary_crossentropy(y_true, y_pred) + 0.5 * iou_loss(y_true, y_pred)\n\n# mean iou as a metric\ndef mean_iou(y_true, y_pred):\n    y_pred = tf.round(y_pred)\n    intersect = tf.reduce_sum(float(y_true) * float(y_pred), axis=[1])\n    union = tf.reduce_sum(float(y_true),axis=[1]) + tf.reduce_sum(float(y_pred),axis=[1])\n    smooth = tf.ones(tf.shape(intersect))\n    return tf.reduce_mean((intersect + smooth) / (union - intersect + smooth))\n","metadata":{"id":"r-u6VnpVIp_p","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_path=workingDir+\"model_unet_vgg16.h5\"\ncheckpoint = ModelCheckpoint(model_path, monitor=\"val_loss\", verbose=1, save_best_only=True)\nstop = EarlyStopping(monitor=\"val_loss\", patience=5)\nreduce_lr = ReduceLROnPlateau(monitor=\"val_loss\", factor=0.1, patience=3, min_lr=1e-6, verbose=1)\nadam=Adam(learning_rate=0.01, beta_1=0.9,beta_2=0.99)\nmodel.compile(loss=iou_bce_loss, optimizer=adam, metrics=[mean_iou])","metadata":{"id":"l4VnEg6xdwJH","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"start=time.time()\ntry:\n    history = model.fit(  traingen,\n                validation_data = valgen,\n                epochs=10,\n                batch_size=batchSize,\n                callbacks=[stop, reduce_lr, checkpoint]\n            )\nexcept Exception as e:\n    print(e)\nprint('Time taken:',time.time()-start)","metadata":{"id":"2R1Ls3vBIQjN","outputId":"851a5931-e251-4f0f-9a3b-d477727b1279","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"727s 24s/step - loss: 0.4701 - mean_iou: 0.8283 - val_loss: 0.4835 - val_mean_iou: 0.8162","metadata":{"id":"MVLjjtH9Pa3m"}},{"cell_type":"code","source":"plt.figure(figsize=(25,6))\nplt.subplot(221)\nplt.plot(history.epoch, history.history[\"loss\"], label=\"Train loss\")\nplt.plot(history.epoch, history.history[\"val_loss\"], label=\"Valid loss\")\nplt.legend()\nplt.subplot(222)\nplt.plot(history.epoch, history.history[\"mean_iou\"], label=\"Train miou\")\nplt.plot(history.epoch, history.history[\"val_mean_iou\"], label=\"Valid miou\")\nplt.legend()\nplt.show()","metadata":{"id":"SZrTGweMIQjN","outputId":"6e889743-c0bc-4af1-e572-41ca10fb367d","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Model Evaluation with transfer learning**","metadata":{"id":"d2ecJxEWIzaa"}},{"cell_type":"code","source":"def load_saved_model(modelname):\n    if os.path.exists(modelname):\n        return load_model(model_path, compile=False)\n    else:\n        print('model does not exist at {}'.format(modelname))","metadata":{"id":"Vc4RhmAGIQjO","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model=load_saved_model(model_path)","metadata":{"id":"_wW-iCv0IQjO","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predgen = CustomDataGen(  val_df[:100],\n                          x_col={'input':'patientId'},\n                          y_col={'output1': 'label', 'output2': 'bboxes'},\n                          batch_size=batchSize, input_size=(imgSize),\n                          preprocess_function=preprocess_input,\n                          include_labels=True\n                        )","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred=[]\ntrue=[]\nimgs=[]\nlabels=[]\nfor images, lbls, truemasks in tqdm_notebook(predgen):\n    imgs.extend(images)\n    labels.extend(lbls)\n    true.extend(truemasks)\n    pred_mask=model.predict(x=[images])\n    pred.extend(pred_mask)\nimgs=np.asarray(imgs)\nlabels=np.asarray(labels)\ntrue=np.asarray(true)\npred=np.asarray(pred).squeeze()","metadata":{"id":"8yiNZCv_I5ns","outputId":"56c5c3c4-4ce2-4415-fec2-c64aeba03956","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(imgs.shape, labels.shape, true.shape, pred.shape)","metadata":{"id":"_rJVlSUYI7PQ","outputId":"a52708ef-dd02-47ca-9ba0-7539d40b811f","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def extract_bboxes(mask):\n    lbl_0 = label(mask)\n    bboxes = regionprops(lbl_0)\n    img_boxes=[]\n    for bbox in bboxes:\n        y1, x1, y2, x2 = bbox.bbox\n        w, h = x2-x1, y2-y1\n        if w > mask.shape[1]*0.1 and h > mask.shape[0]*0.1: # keep bounding boxes that are atleast 1% of the image size elimiating small areas detected\n            img_boxes.append([x1,y1,w,h])\n    \n    return img_boxes","metadata":{"id":"ZXPalbMGI8oP","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def loadImage(img, lbl, act, pred, axis):\n    axis.imshow(img[:,:,0], cmap=plt.cm.bone)\n    pred = pred[:,:] > 0.5\n    img_boxes = extract_bboxes(pred)\n    for bbox in img_boxes:\n        # print('Found bounding box at {}'.format(bbox))\n        x,y,w,h=bbox\n        rect1 = patches.Rectangle((x,y), w, h, linewidth=2, edgecolor='green', fill=False)\n        axis.add_patch(rect1)\n\n    img_boxes=extract_bboxes(act)\n    for bbox in img_boxes:\n        # print('Found bounding box at {}'.format(bbox))\n        x,y,w,h=bbox\n        rect2 = patches.Rectangle((x,y), w, h, linewidth=2, edgecolor='red', fill=False)\n        axis.add_patch(rect2)\n    axis.set_title('Actual vs Predicted\\n'+'label:'+labelDict.get(lbl))\n    ","metadata":{"id":"7gA3anK4I-Ku","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_actual_preds(imgs, lbls, act, pred):\n    cols=3\n    rows=4\n    idx=0\n    f,axarr=plt.subplots(rows,cols,figsize=(18,10))\n    for r in range(rows):\n        for c in range(cols):\n            axis=axarr[r,c]\n            loadImage(imgs[idx], lbls[idx], act[idx], pred[idx], axis)\n            idx+=1\n    plt.tight_layout()","metadata":{"id":"dBjckgisI_3s","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_actual_preds(imgs, labels, true, pred)","metadata":{"id":"r0igho1bJBjL","outputId":"1d86c929-6e30-4d72-e017-04659120f5f6","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"id":"xXIpycsQPa3s"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"id":"jO1_R7uiQgb9"},"execution_count":null,"outputs":[]}]}