{"cells":[{"metadata":{"id":"5C4n2fpmHVlO"},"cell_type":"markdown","source":"The goal is to build a pneumonia detection system, to locate the position of inflammation in an image.\n\n**Overview:**\n*  The input data for pneumonia detection is available in kaggle, images are DICOM images and label and position of pneumonia is available in .csv file for training.\n*  Focus will be to build a convolutional network to train the given image files and predict the sample image to detect pneumonia.\n\n**APPROACH:**\n1.  The DICOM image file contain a combination of header metadata as well as underlying raw image arrays for pixel data. \nTo manipulate the DICOM files python **pydicom** module is used.\n2.  Used Connected components to separate multiple areas of predicted pneumonia\nby drawing a bounding box.\n3.  Create a custom generator to load the data in memory and split the files in training(90%) and validation (10%) set .\n4.  Iniially Built a network consists of a number of residual blocks with convolutions and downsampling blocks with max pooling,there is a single upsampling layer which converts the output to the same shape as the input.\n\n\n\n**OBSERVATION**\n1. Forced to use Custom data generator as the file loading was consuming more memory.\n2. Loss function used adding both Iou and BinaryCrossEntropy,though IOU could have been the most aptable but for accuracy score to be high we will use BCE too.\n3. The accuracy of the model was around ~97%, but the training time was more 6-7 hrs with 10 epochs.Here we are showing 3 epochs ,the slowness may be dute to high momentum =0.99 .\n\n\n\n\n\n\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install --upgrade pip","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -q efficientnet","execution_count":null,"outputs":[]},{"metadata":{"id":"Q7fjBEw2DGwj","executionInfo":{"status":"ok","timestamp":1596218433022,"user_tz":-330,"elapsed":6953,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"afae3bd0-5f0f-4be3-d3a0-c96a53400104","trusted":true},"cell_type":"code","source":"\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n%matplotlib inline\nimport matplotlib.pyplot as plt\nimport csv\nimport os,math,re\nimport tensorflow as tf\nfrom tensorflow import keras\nimport pydicom as dcm\nimport pylab\nimport random\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten\nfrom keras.layers import Conv2D, MaxPooling2D\nfrom skimage import io\nfrom skimage import measure\nfrom skimage.transform import resize\nfrom sklearn.metrics import f1_score, precision_score, recall_score, confusion_matrix\n","execution_count":null,"outputs":[]},{"metadata":{"id":"HTuaRv68AyRL","executionInfo":{"status":"ok","timestamp":1596217471127,"user_tz":-330,"elapsed":791,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"7f6cba8b-cb5f-4d7a-b72e-38efee85dfd7","trusted":true},"cell_type":"code","source":"print(tf.__version__)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# detect and init the TPU\ntpu = tf.distribute.cluster_resolver.TPUClusterResolver()\ntf.config.experimental_connect_to_cluster(tpu)\ntf.tpu.experimental.initialize_tpu_system(tpu)\n\n# instantiate a distribution strategy\ntpu_strategy = tf.distribute.experimental.TPUStrategy(tpu)","execution_count":null,"outputs":[]},{"metadata":{"id":"wz9ATX1fDjmR","executionInfo":{"status":"ok","timestamp":1596218521914,"user_tz":-330,"elapsed":900,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"0c09dae6-1917-4e2a-c1ac-439f74658c90","trusted":true},"cell_type":"code","source":"#loading and reading data set in pandas dataframe\nos.chdir('/kaggle/input/rsna-pneumonia-detection-challenge/')\n\n#os.chdir('/content/competitions/rsna-pneumonia-detection-challenge')\ndata_detailed_class_info = pd.read_csv('stage_2_detailed_class_info.csv')\ndata_train_labels = pd.read_csv('stage_2_train_labels.csv')\ndata_sample_submission = pd.read_csv('stage_2_sample_submission.csv')\nprint(\"=============================================================================================================\")\nprint(\"Data stage_2_detailed_class_info.csv : \")\nprint(\"=============================================================================================================\")\nprint(\"# stage_2_detailed_class_info.csv -  shape of the data       : \" , data_detailed_class_info.shape)\nprint(\"# stage_2_detailed_class_info.csv -  Empty data count        : \" , data_detailed_class_info.isnull().sum())\nprint(\"\")\nprint(\"\")\nprint(data_detailed_class_info.head(3))\nprint(\"\")\nprint('# Count of Unique patientId values in stage_2_detailed_class_info.csv file : ',data_detailed_class_info['patientId'].nunique() )    \nprint(\"\")\nprint(\"=============================================================================================================\")\nprint(\"Data stage_2_train_labels.csv : \")\nprint(\"=============================================================================================================\")\nprint(\"# stage_2_train_labels.csv -  shape of the data       : \" , data_train_labels.shape)\nprint(\"# stage_2_train_labels.csv -  Empty data count        : \" , data_train_labels.isnull().sum())\nprint(\"\")\nprint(\"\")\nprint(data_train_labels.head(10))\nprint(\"\")\nprint('# Count of Unique patientId values in stage_2_train_labels.csv file : ',data_train_labels['patientId'].nunique() )\nprint(\"\")\nprint(\"=============================================================================================================\")\nprint(\"Data stage_2_sample_submission.csv : \")\nprint(\"=============================================================================================================\")\nprint(\"# stage_2_sample_submission.csv -  shape of the data       : \" , data_sample_submission.shape)\nprint(\"# stage_2_sample_submission.csv -  Empty data count        : \" , data_sample_submission.isnull().sum())\nprint(\"\")\nprint(\"\")\nprint(data_sample_submission.head(3))\nprint(\"\")\nprint('# Count of Unique patientId values in stage_2_sample_submission.csv file : ',data_sample_submission['patientId'].nunique() )","execution_count":null,"outputs":[]},{"metadata":{"id":"SIQn7FbaEEAD","executionInfo":{"status":"ok","timestamp":1596218525895,"user_tz":-330,"elapsed":886,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"0525ffae-c778-40f4-87f0-a91a64009cba","trusted":true},"cell_type":"code","source":"df_detailed_class_info=data_detailed_class_info\ndf_data_train_labels=data_train_labels\nprint(df_detailed_class_info.groupby('class').size().reset_index(name='count'))\ndfTemp=df_detailed_class_info.groupby('class').size()\nax=dfTemp.plot(kind='bar',color=list('yrg'),figsize=(10, 10), fontsize=8)\nax.set_xlabel(\"class\", fontsize=20)\nax.set_ylabel(\"count\", fontsize=20)\nax.set_title('Pneumonia Class Count')\n\nfor p in ax.patches:\n  ax.annotate(np.round(p.get_height(),decimals=2),(p.get_x()+p.get_width()/2., p.get_height()),ha='center',va='center',xytext=(0, 10),textcoords='offset points')\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"id":"TAKAUhlaEJj_","executionInfo":{"status":"ok","timestamp":1596218564656,"user_tz":-330,"elapsed":1300,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"9eae08c1-e452-40ed-8bf7-cb8dc592a785","trusted":true},"cell_type":"code","source":"#Merging the data frame of 'stage_2_detailed_class_info.csv' and 'stage_2_train_labels.csv'\ndata_combined=df_data_train_labels.merge(df_detailed_class_info,left_on='patientId', right_on='patientId', how='inner')\nprint(\"Merged data :\",data_combined.shape)\ndfNew=data_combined.groupby(['Target','patientId']).size().reset_index(name='count')\nax=dfNew.groupby('count').size().plot(kind='bar',color=list('yrgb'),figsize=(10, 10),fontsize=8)\nax.set_xlabel(\"count of Pneumonia spotted per patient\", fontsize=20)\nax.set_ylabel(\"count of patients\", fontsize=20)\nax.set_title('Pneumonia locations per Image')\nfor p in ax.patches:\n  ax.annotate(np.round(p.get_height(),decimals=2),(p.get_x()+p.get_width()/2., p.get_height()),ha='center',va='center',xytext=(0, 10),textcoords='offset points')\nplt.show()\nprint(dfNew.groupby('count').size())\nprint(\"Minimum width of Pneumonia : \",data_combined['width'].min())\nprint(\"Minimum height of Pneumonia : \",data_combined['height'].min())\ndfOps=data_combined.groupby('Target').size().reset_index(name='count')\ndfOps['Target']=dfOps['Target'].replace(0,'Absence').replace(1,'Presence')\nprint(\"Summary of abscence and presence of pneumonia: \\n \" ,dfOps.head())\nprint(\"\")\nprint(\"Example of Patient with presence of pneumonia with multiple areas : \\n\", df_data_train_labels.iloc[4:6])\nprint(\"Example of Patient where pneumonia absent : \\n\", df_data_train_labels.iloc[0:1])\nplt.figure()\nplt.title('Pneumonia width lengths')\nplt.hist(data_combined[data_combined['Target'] > 0 ]['width'], bins=np.linspace(0,1000,50))\nplt.show()\nplt.figure()\nplt.title('Pneumonia height lengths')\nplt.hist(data_combined[data_combined['Target'] > 0 ]['height'], bins=np.linspace(0,1000,50))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"XWQSINmoEhYH"},"cell_type":"markdown","source":"\n\nWe can see for stage_2_detailed_class_info.csv file there are 3 categories of people as per the lung conditions:\n1.   Lung Opacity\n2.   No Lung Opacity/Not Normal\n3.   Normal\n\nAbove bar graph says the dataset have more entries of \"No Lung Opacity/Not Normal\".\n\nThere are missing entries in stage_2_train_labels.csv for location (feature \"x\",\"y\",\"width\",height\") which means absence of Pneumonia.\n\nUnique values of PatientId are 26684 present in file stage_2_detailed_class_info.csv ,so patient can contain more than one location entry.\n\nPneumonia presence and absence count is also mentioned above,also a graph showing patient having pneumonia per Image(Fig Title: Pneumonia locations per Image).\nThe above histograms show density of width and height of Pneumonia ,~200\n\n\n\n","execution_count":null},{"metadata":{"id":"O0M9Lh43GCNj","executionInfo":{"status":"ok","timestamp":1596218568793,"user_tz":-330,"elapsed":1477,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"8e7bf09c-a01a-4dee-8d4a-c3da8bb8572e","trusted":true},"cell_type":"code","source":"#onlyfiles = next(os.walk('/content/competitions/rsna-pneumonia-detection-challenge/stage_2_train_images/'))[2] #dir is your directory path as string\nonlyfiles = next(os.walk('/kaggle/input/rsna-pneumonia-detection-challenge/stage_2_train_images/'))[2]\n\nprint(\"Number of dcm files present in directory: \",len(onlyfiles))\n#os.chdir('/content/competitions/rsna-pneumonia-detection-challenge/stage_2_train_images/')\nos.chdir('/kaggle/input/rsna-pneumonia-detection-challenge/stage_2_train_images/')\n#trying to view the dcm file for a particular patientId\npatientId = data_combined['patientId'][5]\ndcm_file = '%s.dcm' % patientId\ndcm_data = dcm.read_file(dcm_file)\nprint(\"Format of .dcm files for a particular random patientId:\")\nprint(\"===================================================================\")\nprint(dcm_data)\nim = dcm_data.pixel_array\nprint(type(im))\nprint(im.dtype)\nprint(im.shape)\npylab.imshow(im, cmap=pylab.cm.gist_gray)","execution_count":null,"outputs":[]},{"metadata":{"id":"2g-sKwNDPeAG"},"cell_type":"markdown","source":"The dicom files gives patient details and image details of patient,such as \"age\",\"sex\",\"Body Part examined\" etc of Patient. Image detais like \"Compression Method\",\"Pixel Data\",\"Size(RowsxColums)\",\"Photmetric Interpretation\" etc . as it monochrome it should be a black and white pics or varying tone of one colour.\n\nFrom the above image we can see the image is correctly displayed for a specific patientId. Now we need to identify the Pneumonia spot ,so we need to write some common functions for drawing the spot provided in the input file stage_2_train_labels.csv .","execution_count":null},{"metadata":{"id":"UdEC4yc2PfA8","executionInfo":{"status":"ok","timestamp":1596218498256,"user_tz":-330,"elapsed":842,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"trusted":true},"cell_type":"code","source":"#common functions to draw pneumonia location.\ndef getPneumoniaLocation(dataFrame):\n    #create a list of location column\n    l_get_loc= lambda rec: [rec['y'],rec['x'],rec['height'],rec['width']]\n    #create python dictionary to stored from the dataframe patients label and location\n    dictPatient={}\n    for index, row in dataFrame.iterrows():\n        l_patientid = row['patientId']\n        #add patients in dictionary \n        if l_patientid not in dictPatient:\n            dictPatient[l_patientid] = {\n                #'dicom': '/content/competitions/rsna-pneumonia-detection-challenge/stage_2_train_images/%s.dcm' % l_patientid,\n                'dicom': '/kaggle/input/rsna-pneumonia-detection-challenge/stage_2_train_images/%s.dcm' % l_patientid,\n                'label': row['Target'],\n                'location': []}\n         \n        #if patient is having pneumonia then add the details of location in dictionary\n        if dictPatient[l_patientid]['label'] == 1:\n            dictPatient[l_patientid]['location'].append(l_get_loc(row))\n    \n    return dictPatient\n\n\ndef displayPneumoniaImage(dataPatient):\n    #open the file present in the input data key value 'dicom'\n    workingFile = dcm.read_file(dataPatient['dicom'])\n    im = workingFile.pixel_array\n    #convert from single channel to 3 channel\n    im = np.stack([im] * 3, axis=2)\n    for rectangle in dataPatient['location']:\n        rgb = np.floor(np.random.rand(3) * 256).astype('int')\n        #converts coordinates to integer\n        rectangle = [int(b) for b in rectangle]\n        y1, x1, height, width = rectangle\n        y2 = y1 + height\n        x2 = x1 + width\n        im[y1:y1 + 10, x1:x2] = rgb\n        im[y2:y2 + 10, x1:x2] = rgb\n        im[y1:y2, x1:x1 + 10] = rgb\n        im[y1:y2, x2:x2 + 10] = rgb\n\n    pylab.figure(figsize=(10,10))\n    pylab.imshow(im, cmap=pylab.cm.gist_gray)\n    pylab.axis('off')\n\n#common function to shuffle file names and split  file names to training and validation set\ndef shuffleFileNames(folder,setName):\n    filenames = os.listdir(folder)\n    random.shuffle(filenames)\n#Split data into train and validation samples, assumption is to split 90% train and 10% validation\n    n_valid_samples = 2700\n    train_filenames = filenames[n_valid_samples:]\n    valid_filenames = filenames[:n_valid_samples]\n    if setName == \"Training\":\n        return train_filenames\n    if setName == \"Validation\":\n        return valid_filenames\n    \n    return train_filenames \n","execution_count":null,"outputs":[]},{"metadata":{"id":"XuN1k6tJh3hJ","executionInfo":{"status":"ok","timestamp":1596217949364,"user_tz":-330,"elapsed":2699,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"a7be966c-9d91-4cd0-d97e-2c05163eb60c","trusted":true},"cell_type":"code","source":"print(\"Random pateintId's picked to show the bounding box images : \\n\" ,data_train_labels.loc[data_train_labels['patientId'].isin(['0100515c-5204-4f31-98e0-f35e4b00004a','00704310-78a8-4b38-8475-49f4573b2dbb','00436515-870c-4b36-a041-de91049b9ab4','01adfd2f-7bc7-4cef-ab68-a0992752b620'])])","execution_count":null,"outputs":[]},{"metadata":{"id":"LRf9RAzWPjdk","executionInfo":{"status":"error","timestamp":1596218536098,"user_tz":-330,"elapsed":1000,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"7c38c533-2b2b-4258-f347-0ffb08e0443f","trusted":true},"cell_type":"code","source":"pneumoniaLocation=getPneumoniaLocation(data_combined)\ndisplayPneumoniaImage(pneumoniaLocation['00436515-870c-4b36-a041-de91049b9ab4'])","execution_count":null,"outputs":[]},{"metadata":{"id":"-wQAYPE0ee2Z","executionInfo":{"status":"ok","timestamp":1596217956721,"user_tz":-330,"elapsed":10033,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"d89f4651-b82a-4c84-948e-a68bc8b61c1b","trusted":true},"cell_type":"code","source":"displayPneumoniaImage(pneumoniaLocation['00704310-78a8-4b38-8475-49f4573b2dbb'])","execution_count":null,"outputs":[]},{"metadata":{"id":"S3gJ7ZKgP4Ds"},"cell_type":"markdown","source":"The above two patient have pneumonia in two locations ,above figure displays the same.","execution_count":null},{"metadata":{"id":"GcI2b8vEgvmA","executionInfo":{"status":"ok","timestamp":1596217956722,"user_tz":-330,"elapsed":10020,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"71e1bf82-ef9b-4716-9dc4-a2ea6e4f4cfc","trusted":true},"cell_type":"code","source":"displayPneumoniaImage(pneumoniaLocation['0100515c-5204-4f31-98e0-f35e4b00004a'])","execution_count":null,"outputs":[]},{"metadata":{"id":"hDK4PhcFkQPQ","executionInfo":{"status":"ok","timestamp":1596217958048,"user_tz":-330,"elapsed":11334,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"5da85e1f-6727-4c5e-9e1d-210984f15247","trusted":true},"cell_type":"code","source":"displayPneumoniaImage(pneumoniaLocation['01adfd2f-7bc7-4cef-ab68-a0992752b620'])","execution_count":null,"outputs":[]},{"metadata":{"id":"2saBaS8Akgxj"},"cell_type":"markdown","source":"The above two patient have pneumonia in single locations ,above figure displays the same.","execution_count":null},{"metadata":{"id":"EJ_2uYoGP5Eh","executionInfo":{"status":"ok","timestamp":1596217958050,"user_tz":-330,"elapsed":11326,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"8acca222-8730-4bec-cd22-d0c3a63b9665","trusted":true},"cell_type":"code","source":"patientid= data_combined['patientId'][0]\ndisplayPneumoniaImage(pneumoniaLocation[patientid])","execution_count":null,"outputs":[]},{"metadata":{"id":"G2psiZEbP9yc"},"cell_type":"markdown","source":"The above Image is not having pneumonia,thus there is no detector/marking of pneumonia","execution_count":null},{"metadata":{"id":"LaFevxw4P_65"},"cell_type":"markdown","source":"**MODEL BUILDING**","execution_count":null},{"metadata":{"id":"ANtQSj9aQKII","executionInfo":{"status":"ok","timestamp":1596218581124,"user_tz":-330,"elapsed":890,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"trusted":true},"cell_type":"code","source":"def getDictLocationFileName(dataFrame):\n    pneumoniaLocation={}\n    for index,row in dataFrame.iterrows():\n        if row['Target'] == 1:\n            filename = row[0]\n            location = row[1:5]\n            location = [int(float(i)) for i in location]\n            if filename in pneumoniaLocation:\n                pneumoniaLocation[filename].append(location)\n            else:\n                pneumoniaLocation[filename] = [location]\n    \n    return pneumoniaLocation\n            \ndef iou_loss(y_true, y_pred):\n    y_true = tf.reshape(y_true, [-1])\n    y_pred = tf.reshape(y_pred, [-1])\n    intersection = tf.reduce_sum(y_true * y_pred)\n    score = (intersection + 1.) / (tf.reduce_sum(y_true) + tf.reduce_sum(y_pred) - intersection + 1.)\n    return 1 - score\n\ndef iou_bce_loss(y_true, y_pred):\n    return 0.5 * keras.losses.binary_crossentropy(y_true, y_pred) + 0.5 * iou_loss(y_true, y_pred)\n\ndef cosine_annealing(x):\n    lr = 0.001\n    epochs = 25\n    return lr*(np.cos(np.pi*x/epochs)+1.)/2\n\ndef mean_iou(y_true, y_pred):\n    y_pred = tf.round(y_pred)\n    intersect = tf.reduce_sum(y_true * y_pred, axis=[1, 2, 3])\n    union = tf.reduce_sum(y_true, axis=[1, 2, 3]) + tf.reduce_sum(y_pred, axis=[1, 2, 3])\n    smooth = tf.ones(tf.shape(intersect))\n    return tf.reduce_mean((intersect + smooth) / (union - intersect + smooth))","execution_count":null,"outputs":[]},{"metadata":{"id":"DklYpCNuQQhq","executionInfo":{"status":"ok","timestamp":1596218583388,"user_tz":-330,"elapsed":845,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"trusted":true},"cell_type":"code","source":"#To handle large data set avoid memory consumption during loading of large files \n#by using Keras we will create data generator class ,with multiprocessing batches will be loading and also images will be trained.\n\nclass dataGenerator(keras.utils.Sequence):\n    def __init__(self, folder, filenames, pneumonia_locations=None, batch_size=32, image_size=256, shuffle=True, augment=False, predict=False):\n        self.folder = folder\n        self.filenames = filenames\n        self.pneumonia_locations = pneumonia_locations\n        self.batch_size = batch_size\n        self.image_size = image_size\n        self.shuffle = shuffle\n        self.augment = augment\n        self.predict = predict\n        self.on_epoch_end()\n        \n    def __load__(self, filename):\n        # load dicom file as numpy array\n        img = dcm.dcmread(os.path.join(self.folder, filename)).pixel_array\n        # create empty mask\n        msk = np.zeros(img.shape)\n        # get filename without extension\n        filename = filename.split('.')[0]\n        # if image contains pneumonia\n        if filename in self.pneumonia_locations:\n            # loop through pneumonia\n            for location in self.pneumonia_locations[filename]:\n                # add 1's at the location of the pneumonia\n                x, y, w, h = location\n                msk[y:y+h, x:x+w] = 1\n        # resize both image and mask\n        img = resize(img, (self.image_size, self.image_size), mode='reflect')\n        msk = resize(msk, (self.image_size, self.image_size), mode='reflect') > 0.5\n        # if augment then horizontal flip half the time\n        if self.augment and random.random() > 0.5:\n            img = np.fliplr(img)\n            msk = np.fliplr(msk)\n        # add trailing channel dimension\n        img = np.expand_dims(img, -1)\n        msk = np.expand_dims(msk, -1)\n        return img, msk\n    \n    def __loadpredict__(self, filename):\n        # load dicom file as numpy array\n        img = dcm.dcmread(os.path.join(self.folder, filename)).pixel_array\n        # resize image\n        img = resize(img, (self.image_size, self.image_size), mode='reflect')\n        # add trailing channel dimension\n        img = np.expand_dims(img, -1)\n        return img      \n    \n    #function which returns batch of images and filenames while training and while predicted returns masks and images\n    def __getitem__(self, index):\n        filenames= self.filenames[index*self.batch_size:(index+1)*self.batch_size]\n        if self.predict:\n            # load files\n            imgs = [self.__loadpredict__(filename) for filename in filenames]\n            # create numpy batch\n            imgs = np.array(imgs)\n            return imgs, filenames\n        else:\n            # load files\n            items = [self.__load__(filename) for filename in filenames]\n            # unzip images and masks\n            imgs, msks = zip(*items)\n            # create numpy batch\n            imgs = np.array(imgs)\n            msks = np.array(msks)\n            return imgs, msks\n\n    def on_epoch_end(self):\n        #triggered at start and end of each epoch,while shuffling of file names we will get a new order of epochs        if self.shuffle:\n            if self.shuffle:\n                random.shuffle(self.filenames)\n    \n    def __len__(self):\n        #'Denotes the number of batches per epoch'\n        if self.predict:\n            return int(np.ceil(len(self.filenames) / self.batch_size))\n        else:\n            return int(len(self.filenames) / self.batch_size)\n        \n    ","execution_count":null,"outputs":[]},{"metadata":{"id":"TMad5aVbQVrp","executionInfo":{"status":"ok","timestamp":1596218595818,"user_tz":-330,"elapsed":7403,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"6b2253da-7302-4902-ee2f-188b4cb5526d","trusted":true},"cell_type":"code","source":"#folder='/content/competitions/rsna-pneumonia-detection-challenge/stage_2_train_images/'\nfolder='/kaggle/input/rsna-pneumonia-detection-challenge/stage_2_train_images/'\nX_train=shuffleFileNames(folder,'Training')\nprint(len(X_train))\nX_Valid=shuffleFileNames(folder,'Validation')\nprint(len(X_Valid))\npneumonia_locations=getDictLocationFileName(data_combined)\ntrain_gen = dataGenerator(folder, X_train, pneumonia_locations=pneumonia_locations, batch_size=32, image_size=256, shuffle=True, augment=True, predict=False)\nvalid_gen = dataGenerator(folder, X_Valid, pneumonia_locations=pneumonia_locations, batch_size=32, image_size=256, shuffle=False, predict=False)\n","execution_count":null,"outputs":[]},{"metadata":{"id":"cbBWlr05QtvZ","executionInfo":{"status":"ok","timestamp":1596218466965,"user_tz":-330,"elapsed":866,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"trusted":true},"cell_type":"code","source":"def create_downsample(channels, inputs):\n    x = keras.layers.BatchNormalization(momentum=0.99)(inputs)\n    x = keras.layers.LeakyReLU(0)(x)\n    x = keras.layers.Conv2D(channels, 1, padding='same', use_bias=False)(x)\n    x = keras.layers.MaxPool2D(2)(x)\n    return x\n\ndef create_resblock(channels, inputs):\n    x = keras.layers.BatchNormalization(momentum=0.99)(inputs)\n    x = keras.layers.LeakyReLU(0)(x)\n    x = keras.layers.Conv2D(channels, 3, padding='same', use_bias=False)(x)\n    x = keras.layers.BatchNormalization(momentum=0.99)(x)\n    x = keras.layers.LeakyReLU(0)(x)\n    x = keras.layers.Conv2D(channels, 3, padding='same', use_bias=False)(x)\n    return keras.layers.add([x, inputs])\n\ndef create_network(input_size, channels, n_blocks=2, depth=4):\n    # input\n    inputs = keras.Input(shape=(input_size, input_size, 1))\n    x = keras.layers.Conv2D(channels, 3, padding='same', use_bias=False)(inputs)\n    # residual blocks\n    for d in range(depth):\n        channels = channels * 2\n        x = create_downsample(channels, x)\n        for b in range(n_blocks):\n            x = create_resblock(channels, x)\n    # output\n    x = keras.layers.BatchNormalization(momentum=0.99)(x)\n    x = keras.layers.LeakyReLU(0)(x)\n    x = keras.layers.Conv2D(1, 1, activation='sigmoid')(x)\n    outputs = keras.layers.UpSampling2D(2**depth)(x)\n    model = keras.Model(inputs=inputs, outputs=outputs)\n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_gen = dataGenerator(folder, X_train, pneumonia_locations=pneumonia_locations, batch_size=32, image_size=256, shuffle=True, augment=True, predict=False)\nvalid_gen = dataGenerator(folder, X_Valid, pneumonia_locations=pneumonia_locations, batch_size=32, image_size=256, shuffle=False, predict=False)\nmodelResNet = create_network(input_size=256, channels=32, n_blocks=2, depth=4)\nopt = keras.optimizers.Adam(learning_rate=0.01)\nmodelResNet.compile(optimizer=opt,loss=iou_bce_loss,metrics=['accuracy', mean_iou])\nmodelHist=modelResNet.fit(train_gen, validation_data=valid_gen, epochs=5)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(12,4))\nplt.subplot(131)\nplt.plot(modelHist.epoch, modelHist.history[\"loss\"], label=\"Train loss\")\nplt.plot(modelHist.epoch, modelHist.history[\"val_loss\"], label=\"Valid loss\")\nplt.legend()\nplt.subplot(132)\nplt.plot(modelHist.epoch, modelHist.history[\"accuracy\"], label=\"Train accuracy\")\nplt.plot(modelHist.epoch, modelHist.history[\"val_accuracy\"], label=\"Valid accuracy\")\nplt.legend()\nplt.subplot(133)\nplt.plot(modelHist.epoch, modelHist.history[\"mean_iou\"], label=\"Train iou\")\nplt.plot(modelHist.epoch, modelHist.history[\"val_mean_iou\"], label=\"Valid iou\")\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"modelHist.history","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.patches as patches\nfor imgs, msks in valid_gen:\n    # predict batch of images\n    preds = modelResNet.predict(imgs)\n    # create figure\n    f, axarr = plt.subplots(4, 8, figsize=(20,15))\n    axarr = axarr.ravel()\n    axidx = 0\n    # loop through batch\n    for img, msk, pred in zip(imgs, msks, preds):\n        # plot image\n        axarr[axidx].imshow(img[:, :, 0])\n        # threshold true mask\n        comp = msk[:, :, 0] > 0.5\n        # apply connected components\n        comp = measure.label(comp)\n        # apply bounding boxes\n        predictionString = ''\n        for region in measure.regionprops(comp):\n            # retrieve x, y, height and width\n            y, x, y2, x2 = region.bbox\n            height = y2 - y\n            width = x2 - x\n            axarr[axidx].add_patch(patches.Rectangle((x,y),width,height,linewidth=2,edgecolor='b',facecolor='none'))\n        # threshold predicted mask\n        comp = pred[:, :, 0] > 0.5\n        # apply connected components\n        comp = measure.label(comp)\n        # apply bounding boxes\n        predictionString = ''\n        for region in measure.regionprops(comp):\n            # retrieve x, y, height and width\n            y, x, y2, x2 = region.bbox\n            height = y2 - y\n            width = x2 - x\n            axarr[axidx].add_patch(patches.Rectangle((x,y),width,height,linewidth=2,edgecolor='r',facecolor='none'))\n        axidx += 1\n    plt.show()\n    # only plot one batch\n    break","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_folder='/kaggle/input/rsna-pneumonia-detection-challenge/stage_2_test_images/'\ntest_filenames = os.listdir(test_folder)\nprint('n test samples:', len(test_filenames))\n\n# create test generator with predict flag set to True\ntest_gen = dataGenerator(test_folder, test_filenames, None, batch_size=25, image_size=256, shuffle=False, predict=True)\n\n# create submission dictionary\nsubmission_dict = {}\n# loop through testset\nfor imgs, filenames in test_gen:\n    # predict batch of images\n    preds = modelResNet.predict(imgs)\n    # loop through batch\n    for pred, filename in zip(preds, filenames):\n        # resize predicted mask\n        pred = resize(pred, (1024, 1024), mode='reflect')\n        # threshold predicted mask\n        comp = pred[:, :, 0] > 0.5\n        # apply connected components\n        comp = measure.label(comp)\n        # apply bounding boxes\n        predictionString = ''\n        for region in measure.regionprops(comp):\n            # retrieve x, y, height and width\n            y, x, y2, x2 = region.bbox\n            height = y2 - y\n            width = x2 - x\n            # proxy for confidence score\n            conf = np.mean(pred[y:y+height, x:x+width])\n            # add to predictionString\n            predictionString += str(conf) + ' ' + str(x) + ' ' + str(y) + ' ' + str(width) + ' ' + str(height) + ' '\n        # add filename and predictionString to dictionary\n        filename = filename.split('.')[0]\n        submission_dict[filename] = predictionString\n    # stop if we've got them all\n    #if len(submission_dict) >= len(test_filenames):\n    if len(submission_dict) >= 100:\n        break","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(submission_dict.items())\nprint(submission_dict.keys())\nprint(plt.imshow(dcm.dcmread('/content/competitions/rsna-pneumonia-detection-challenge/stage_2_test_images/21ea7be5-b0a4-4d96-b56d-8288dd0292cd.dcm').pixel_array, cmap=plt.cm.bone) )\n","execution_count":null,"outputs":[]},{"metadata":{"id":"YOog6QH52wr0","executionInfo":{"status":"ok","timestamp":1596218654335,"user_tz":-330,"elapsed":830,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"trusted":true},"cell_type":"code","source":"EarlyStopping=tf.keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0.05, patience=2, verbose=0, mode='auto',baseline=None, restore_best_weights=True)\nfilepath=\"/kaggle/working/weights-improvement-{epoch:02d}-{val_loss:.2f}.hdf5\"\nModelCheckpoint = tf.keras.callbacks.ModelCheckpoint(filepath,monitor='val_loss',mode='min',save_best_only=True,verbose=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Increasing batch Size and checking model training performance and other parameters","execution_count":null},{"metadata":{"id":"tNyqnAdn0dDt","outputId":"6f1f2c27-c3ec-4767-e98d-e2ebf9a0a467","trusted":true},"cell_type":"code","source":"train_gen = dataGenerator(folder, X_train, pneumonia_locations=pneumonia_locations, batch_size=100, image_size=256, shuffle=True, augment=True, predict=False)\nvalid_gen = dataGenerator(folder, X_Valid, pneumonia_locations=pneumonia_locations, batch_size=100, image_size=256, shuffle=False, predict=False)\nmodel = create_network(input_size=256, channels=32, n_blocks=2, depth=4)\nopt = keras.optimizers.Adam(learning_rate=0.01)\nmodel.compile(optimizer=opt,loss=iou_bce_loss,metrics=['accuracy', mean_iou])\nlearning_rate = tf.keras.callbacks.LearningRateScheduler(cosine_annealing)\nmodelHistIncreaseBatchSize=model.fit(train_gen, validation_data=valid_gen, callbacks=[learning_rate,EarlyStopping,ModelCheckpoint],epochs=6)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"modelHistIncreaseBatchSize.history","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(12,4))\nplt.subplot(131)\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"loss\"], label=\"Train loss\")\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"val_loss\"], label=\"Valid loss\")\nplt.legend()\nplt.subplot(132)\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"accuracy\"], label=\"Train accuracy\")\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"val_accuracy\"], label=\"Valid accuracy\")\nplt.legend()\nplt.subplot(133)\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"mean_iou\"], label=\"Train iou\")\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"val_mean_iou\"], label=\"Valid iou\")\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%cd /kaggle/input/pneumoniadetection-v1\n!ls -lrt weights-improvement*","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Reducing input size and using model save weights","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"modelInputSize = create_network(input_size=128, channels=32, n_blocks=2, depth=4)\nopt = keras.optimizers.Adam(learning_rate=0.01)\nmodelInputSize.compile(optimizer=opt,loss=iou_bce_loss,metrics=['accuracy', mean_iou])\nmodelInputSize.load_weights('/kaggle/input/pneumoniadetection-v1/weights-improvement-02-0.63.hdf5')\nlearning_rate = tf.keras.callbacks.LearningRateScheduler(cosine_annealing)\nEarlyStopping=tf.keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0.05, patience=2, verbose=0, mode='auto',baseline=None, restore_best_weights=True)\nmodelHistDecreaseInputSize=modelInputSize.fit(train_gen, validation_data=valid_gen, callbacks=[learning_rate,EarlyStopping],epochs=6)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(modelInputSize.score())\nprint(modelResNet.score())\nprint(model.score())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(12,4))\nplt.subplot(131)\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"loss\"], label=\"Train loss\")\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"val_loss\"], label=\"Valid loss\")\nplt.legend()\nplt.subplot(132)\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"accuracy\"], label=\"Train accuracy\")\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"val_accuracy\"], label=\"Valid accuracy\")\nplt.legend()\nplt.subplot(133)\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"mean_iou\"], label=\"Train iou\")\nplt.plot(modelHistIncreaseBatchSize.epoch, modelHistIncreaseBatchSize.history[\"val_mean_iou\"], label=\"Valid iou\")\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"W9JreFpRDa9B","trusted":true},"cell_type":"code","source":"def map_iou(boxes_true, boxes_pred, scores, thresholds = [0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75]):\n    \"\"\"\n    Mean average precision at differnet intersection over union (IoU) threshold\n    \n    input:\n        boxes_true: Mx4 numpy array of ground true bounding boxes of one image. \n                    bbox format: (x1, y1, w, h)\n        boxes_pred: Nx4 numpy array of predicted bounding boxes of one image. \n                    bbox format: (x1, y1, w, h)\n        scores:     length N numpy array of scores associated with predicted bboxes\n        thresholds: IoU shresholds to evaluate mean average precision on\n    output: \n        map: mean average precision of the image\n    \"\"\"\n    \n    # According to the introduction, images with no ground truth bboxes will not be \n    # included in the map score unless there is a false positive detection (?)\n        \n    # return None if both are empty, don't count the image in final evaluation (?)\n    \n    if len(boxes_true) == 0 and len(boxes_pred) == 0:\n        return None\n    \n    assert boxes_true.shape[1] == 4 or boxes_pred.shape[1] == 4, \"boxes should be 2D arrays with shape[1]=4\"\n    if len(boxes_pred):\n        assert len(scores) == len(boxes_pred), \"boxes_pred and scores should be same length\"\n        # sort boxes_pred by scores in decreasing order\n        boxes_pred = boxes_pred[np.argsort(scores)[::-1], :]\n    \n    map_total = 0\n    \n    # loop over thresholds\n    for t in thresholds:\n        matched_bt = set()\n        tp, fn = 0, 0\n        for i, bt in enumerate(boxes_true):\n            matched = False\n            for j, bp in enumerate(boxes_pred):\n                miou = iou(bt, bp)\n                if miou >= t and not matched and j not in matched_bt:\n                    matched = True\n                    tp += 1 # bt is matched for the first time, count as TP\n                    matched_bt.add(j)\n            if not matched:\n                fn += 1 # bt has no match, count as FN\n                \n        fp = len(boxes_pred) - len(matched_bt) # FP is the bp that not matched to any bt\n        m = tp / (tp + fn + fp)\n        map_total += m\n    \n    return map_total / len(thresholds)","execution_count":null,"outputs":[]},{"metadata":{"id":"oWRZeivdDz5U","executionInfo":{"status":"ok","timestamp":1595874991731,"user_tz":-330,"elapsed":696876,"user":{"displayName":"Arijit Gupta","photoUrl":"","userId":"09801431714976450418"}},"outputId":"fea6b507-425d-4852-e3df-90b82f435878","trusted":true},"cell_type":"code","source":"test_folder='/kaggle/input/rsna-pneumonia-detection-challenge/stage_2_test_images/'\ntest_filenames = os.listdir(test_folder)\nprint('n test samples:', len(test_filenames))\n\n# create test generator with predict flag set to True\ntest_gen = dataGenerator(test_folder, test_filenames, None, batch_size=25, image_size=256, shuffle=False, predict=True)\nprob_thresholds = [0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5]\nnthresh = len(prob_thresholds)\ncount = 0\nns = nthresh*[0]\nnfps = nthresh*[0]\nntps = nthresh*[0]\noverall_maps = nthresh*[0.]\nfor imgs, filenames in test_gen:\n    # predict batch of images\n    preds = model.predict(imgs)\n    # loop through batch\n    for pred, filename in zip(preds, filenames):\n        count = count + 1\n        maxpred = np.max(pred)\n        # resize predicted mask\n        pred = resize(pred, (1024, 1024), mode='reflect')\n        # threshold predicted mask\n        boxes_preds = []\n        scoress = []\n        for thresh in prob_thresholds:\n            comp = pred[:, :, 0] > thresh\n            # apply connected components\n            comp = measure.label(comp)\n            # apply bounding boxes\n            boxes_pred = np.empty((0,4),int)\n            scores = np.empty((0))\n            for region in measure.regionprops(comp):\n                # retrieve x, y, height and width\n                y, x, y2, x2 = region.bbox\n                boxes_pred = np.append(boxes_pred, [[x, y, x2-x, y2-y]], axis=0)\n                # proxy for confidence score\n                conf = np.mean(pred[y:y2, x:x2])\n                scores = np.append( scores, conf )\n            boxes_preds = boxes_preds + [boxes_pred]\n            scoress = scoress + [scores]\n        boxes_true = np.empty((0,4),int)\n        fn = filename.split('.')[0]\n        # if image contains pneumonia\n        if fn in pneumonia_locations:\n            # loop through pneumonia\n            for location in pneumonia_locations[fn]:\n                x, y, w, h = location\n                boxes_true = np.append(boxes_true, [[x, y, w, h]], axis=0)\n        for i in range(nthresh):\n            if ( boxes_true.shape[0]==0 and boxes_preds[i].shape[0]>0 ):\n                ns[i] = ns[i] + 1\n                nfps[i] = nfps[i] + 1\n            elif ( boxes_true.shape[0]>0 ):\n                ns[i] = ns[i] + 1\n                contrib = map_iou( boxes_true, boxes_preds[i], scoress[i] ) \n                overall_maps[i] = overall_maps[i] + contrib\n                if ( boxes_preds[i].shape[0]>0 ):\n                    ntps[i] = ntps[i] + 1\n\n    # stop if we've got them all\n    if count >= len(test_filenames):\n        break","execution_count":null,"outputs":[]},{"metadata":{"id":"0uPv89QuklZu","trusted":true},"cell_type":"code","source":"print(plt.imshow(dcm.dcmread('/content/competitions/rsna-pneumonia-detection-challenge/stage_2_test_images/1b8d026f-3989-4cd6-a4c7-4ed8fb8887d0.dcm').pixel_array, cmap=plt.cm.bone) )","execution_count":null,"outputs":[]},{"metadata":{"id":"UIGkZWG5kNpu","trusted":true},"cell_type":"code","source":"print(plt.imshow(dcm.dcmread('/content/competitions/rsna-pneumonia-detection-challenge/stage_2_test_images/1e8359a0-6313-4908-9971-5682d02db185.dcm').pixel_array, cmap=plt.cm.bone) )","execution_count":null,"outputs":[]},{"metadata":{"id":"4ZSS1rOCv2T5"},"cell_type":"markdown","source":"Conclusion :\n\n1. The current kernel have a very high training time , need to focus on more  tuning.\n2. The loss function ","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}