{"cells":[{"metadata":{},"cell_type":"markdown","source":"For this project, I originally created a kernel that trained a CNN model (I originally used the LeNet architecture) on the data to try and predict the bounding box measurements as 5 features of the pixel data in each DICOM image. However, I came across two major problems that I could not seem to fix:One was that the training of the model quickly drained all the run time memory and threw an ResourceExhaustedError. Additionally, the a patient with pneumonia does not always have just one bounding box - a patient can have several different opacities which reflect pneumonia, and a new DICOM image with the same patientId and different bounding box measurements will be in the files for each opacity. To try and start again with something that could solve these problems, I looked at the most popular kernel, located at https://www.kaggle.com/jonnedtc/cnn-segmentation-connected-components/ for ideas. The approach of this kernel, which I followed conceptually, was to aggregate the different images of each patient into one, and combine the bounding boxes into one mask for each patient. A mask would be an equal size array of zeros, with 1's instead for items we choose to keep. The idea is that certain operations between the mask and image,such as AND, will annihilate values we don't want to consider. Then, we can compare the model predictions on normalized data to the mask for each patient."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport pydicom\nimport numpy as np\nimport pandas as pd\nfrom skimage import measure\nfrom skimage.transform import resize\n\nimport tensorflow as tf\nfrom tensorflow import keras","execution_count":2,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"After importing all the packages we need, the first step we can take is to analyze the data."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_labels = pd.read_csv('../input/stage_2_train_labels.csv')\nprint(train_labels.head())\nprint(pd.DataFrame(os.listdir('../input/stage_2_train_images/')).head())","execution_count":3,"outputs":[{"output_type":"stream","text":"                              patientId      x   ...    height  Target\n0  0004cfab-14fd-4e49-80ba-63a80b6bddd6    NaN   ...       NaN       0\n1  00313ee0-9eaa-42f4-b0ab-c148ed3241cd    NaN   ...       NaN       0\n2  00322d4d-1c29-4943-afc9-b6754be640eb    NaN   ...       NaN       0\n3  003d8fa0-6bf1-40ed-b54c-ac657f8495c5    NaN   ...       NaN       0\n4  00436515-870c-4b36-a041-de91049b9ab4  264.0   ...     379.0       1\n\n[5 rows x 6 columns]\n                                          0\n0  6b8884f1-abcc-4285-97e9-d70e867e1545.dcm\n1  94e5ea07-818c-4d2d-bf2b-022ddd979be3.dcm\n2  7918b5a8-0e82-43e9-ac13-70472f297d88.dcm\n3  c21157da-1f74-4966-ba4e-c144d8f5da2d.dcm\n4  a7c39046-12db-4454-a2d8-c9c624992b5d.dcm\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"So the columns of the labels are the patientId, followed by the location and dimensions of the bounding box, followed by the Target(a 1 or 0, where 1 indicates a patient with pneumonia.) The files are named after the patientId, so we can use patientIds to retrieve them. To build our masks, the source notebook has an interesting approach. A simple way to do this would be to create a mask for each patient and use the bounding boxes to build it, but unfortunately, the data is far too large to hold in memory all at once. Luckily, the Sequence class of keras contains generators that can modify for our specific purpose. We can create masks for our images as we load them into the model, but before that we can create a dictionary of all the coordinates to mask for each patient, which will fit into memory easily. "},{"metadata":{"trusted":true},"cell_type":"code","source":"pneum_locs = {}\nfor x in range(0,len(train_labels)):\n    row = train_labels.iloc[x]\n    patientId = row[0]\n    loc = row[1:3]\n\n    if row[4] == '1': #patient has pneumonia\n        location = [int(float(i))for i in loc] #data in labels is in string form, must convert\n        if(patientId in pneum_locs): #patient already in dictionary\n            pneum_locs[patientId].append(loc)\n        else:\n            pneum_locs[patientId] = [loc]","execution_count":4,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So now, pneum_locs contains, for each patient with pneumonia, the location and dimensions of the opacity bounding boxes. Now is the time to build our generator. The docs for keras.utils.Sequence(located at https://keras.io/utils/) mandate that every subclass must override the methods for retrieval and the data length of each batch. We will also override initialization, since we want to set a mode for when we use our generator to predict on test data, and set an image size to resize to (making the image smaller improves runtime). "},{"metadata":{"trusted":true},"cell_type":"code","source":"class datagenerator(keras.utils.Sequence):\n    def __init__(self,patientIds, p_locations = None, batch_size = 32,image_size = 256):\n        self.patientIds = patientIds\n        self.p_locations = p_locations\n        self.batch_size = batch_size\n        self.image_size = image_size\n      \n    def __load__(self,pID):#loads a file for retrieval \n        image = pydicom.read_file('../input/stage_2_train_images/%s.dcm' % pID).pixel_array\n        mask = np.zeros(image.shape)\n        image = resize(image,(self.image_size,self.image_size),mode = 'reflect')\n        mask = resize(mask, (self.image_size, self.image_size), mode='reflect') > 0.5\n        if pID in self.p_locations :\n            for loc in self.p_locations[patID]:\n                x,y,w,h = loc\n                mask[y:y+h,x:x+w] = 1\n        image = np.expand_dims(image,-1) #X = image\n        mask = np.expand_dims(mask,-1) #Y = mask\n\n        return image,mask\n    def __getitem__(self,index):#mandatory inheritance\n        pIDs = self.patientIds[index*self.batch_size:(index + 1)*self.batch_size]\n        images,masks = zip(*[self.__load__(patientId) for patientId in pIDs])\n        images = np.array(images)\n        masks = np.array(masks)\n        return images,masks\n    def __len__(self): #mandatory inheritance\n        return int(len(self.patientIds)/self.batch_size)\n\n        ","execution_count":40,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The source notebook created a model that used creates a CNN using two methods: One that creates a layer to downsample the data, and one that creates a residual block to feed directly to the output as a way to avoid losing information through too many backpropagations. I used the same model, but removed some of the layers to improve runtime."},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_downsample(channels, inputs):\n    x = keras.layers.BatchNormalization()(inputs)\n    x = keras.layers.LeakyReLU(0)(x)#LeakyReLU with alpha = 0 is identical to ReLU\n    x = keras.layers.Conv2D(channels, 1, padding='same')(x)\n    x = keras.layers.MaxPool2D(2)(x)\n    return x\n\ndef create_resblock(channels, inputs):\n    x = keras.layers.BatchNormalization()(inputs)\n    x = keras.layers.LeakyReLU(0)(x)\n    x = keras.layers.Conv2D(channels, 3, padding='same')(x)\n\n    return keras.layers.add([x, inputs])\n\ndef create_network(input_size, channels, n_blocks=2, depth=4): #creates a residual block layer\n    inputs = keras.Input(shape=(input_size, input_size, 1))\n    x = keras.layers.Conv2D(channels, 3, padding='same')(inputs)\n    for d in range(depth):\n        channels = channels * 2\n        x = create_downsample(channels, x)\n        for b in range(n_blocks):\n            x = create_resblock(channels, x)\n    # output\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.LeakyReLU(0)(x)\n    x = keras.layers.Conv2D(1, 1, activation='sigmoid')(x)\n    outputs = keras.layers.UpSampling2D(2**depth)(x) ##upsample data to counteract the first downsample\n    model = keras.Model(inputs=inputs, outputs=outputs)\n    return model","execution_count":6,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The source notebook provided a useful function for calculating the intersection-over-union loss. I used it as the loss for my model, but not the other two methods provided as they caused errors and their purpose was unclear. For the model fitting, the system gave me an error when using batch sizes greater than 1 and I could not find out why, so I stuck with that as it did not seem to affect training time too much. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def iou_loss(y_true, y_pred):\n    y_true = tf.reshape(y_true, [-1])\n    y_pred = tf.reshape(y_pred, [-1])\n    intersection = tf.reduce_sum(y_true * y_pred)\n    score = (intersection + 1.) / (tf.reduce_sum(y_true) + tf.reduce_sum(y_pred) - intersection + 1.)\n    return 1 - score\n\n\n\n\nmodel = create_network(input_size=256, channels=32, n_blocks=2, depth=4)\nmodel.compile(optimizer='adam',\n              loss=iou_loss,\n              metrics=['accuracy'])\nval_size = 3000 #about 10% of data used for validation\n\ntrain_IDs = train_labels['patientId'][val_size:]\nval_IDs= train_labels['patientId'][:val_size]\nvalidgen = datagenerator(val_IDs,pneum_locs,batch_size = 1,image_size = 256)\ntraingen = datagenerator(train_IDs, pneum_locs, batch_size=1, image_size=256)\nhist = model.fit_generator(traingen,epochs = 3,validation_data = validgen, workers = 4,use_multiprocessing = True)","execution_count":9,"outputs":[{"output_type":"stream","text":"WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nColocations handled automatically by placer.\nEpoch 1/5\n3000/3000 [==============================] - 275s 92ms/step - loss: 0.0000e+00 - acc: 1.0000\n27227/27227 [==============================] - 3038s 112ms/step - loss: 0.0020 - acc: 0.9997 - val_loss: 0.0000e+00 - val_acc: 1.0000\nEpoch 2/5\n3000/3000 [==============================] - 280s 93ms/step - loss: 0.0000e+00 - acc: 1.0000\n27227/27227 [==============================] - 2957s 109ms/step - loss: 0.0000e+00 - acc: 1.0000 - val_loss: 0.0000e+00 - val_acc: 1.0000\nEpoch 3/5\n3000/3000 [==============================] - 286s 95ms/step - loss: 0.0000e+00 - acc: 1.0000\n27227/27227 [==============================] - 3053s 112ms/step - loss: 0.0000e+00 - acc: 1.0000 - val_loss: 0.0000e+00 - val_acc: 1.0000\nEpoch 4/5\n3000/3000 [==============================] - 285s 95ms/step - loss: 0.0000e+00 - acc: 1.0000\n27227/27227 [==============================] - 3051s 112ms/step - loss: 0.0000e+00 - acc: 1.0000 - val_loss: 0.0000e+00 - val_acc: 1.0000\nEpoch 5/5\n3000/3000 [==============================] - 288s 96ms/step - loss: 0.0000e+00 - acc: 1.0000\n27227/27227 [==============================] - 3051s 112ms/step - loss: 0.0000e+00 - acc: 1.0000 - val_loss: 0.0000e+00 - val_acc: 1.0000\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"Finally, we test our model against the test data. I decided not to use another generator here, as we can just feed the model \"batches\" of 1 image at a time and finish in reasonable time as we are processing 3000 images as opposed to more than 30,000 five times. "},{"metadata":{"trusted":true},"cell_type":"code","source":"\ntest_IDs = pd.DataFrame(os.listdir('../input/stage_2_test_images/'))\nfor i in range (0, len(test_IDs)):\n    test_IDs[0][i]=test_IDs[0][i].split('.')[0]\ntest_IDs = test_IDs[0]\n\nsubmission = {}\nfor pID in test_IDs :\n    image = pydicom.read_file('../input/stage_2_test_images/%s.dcm' % pID).pixel_array\n    image = resize(image,(256,256),mode = 'reflect')\n    image = np.expand_dims(image,-1)\n    images = np.zeros((1,256,256,1))\n    images[0] = image\n    pred = model.predict(images)\n    predict = resize(np.squeeze(pred),(1024,1024), mode = 'reflect')\n    compute = predict[:,:] >0.5 #transforms values to 1s and 0s\n    compute = measure.label(compute)\n    predString = ''\n    for region in measure.regionprops(compute):\n        y,x,y2,x2 = region.bbox\n        confidence = np.mean(predict[y:y2,x:x2])\n        predictionString += str(conf) + ' ' + str(x) + ' ' + str(y) + ' ' + str(y2-y) + ' ' + str(x2 - x) + ' '\n    submission[pID]= predString\n    if(len(submission) >= len(test_IDs)): #loop exit control\n        break\n\nsubmit = pd.DataFrame.from_dict(submission,orient ='index')\nprint(\"%s predictions recorded.\" % len(submit))\nsubmit.index.names = ['patientId']\nsubmit.columns = ['PredictionString']\nsubmit.to_csv('submission.csv')\n","execution_count":39,"outputs":[{"output_type":"stream","text":"3000 predictions recorded.\n","name":"stdout"},{"output_type":"error","ename":"FileNotFoundError","evalue":"[Errno 2] No such file or directory: '../output/submission.csv'","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)","\u001b[0;32m<ipython-input-39-ab1ba8a20070>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     29\u001b[0m \u001b[0msubmit\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnames\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'patientId'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     30\u001b[0m \u001b[0msubmit\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumns\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'PredictionString'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 31\u001b[0;31m \u001b[0msubmit\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../output/submission.csv'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/pandas/core/frame.py\u001b[0m in \u001b[0;36mto_csv\u001b[0;34m(self, path_or_buf, sep, na_rep, float_format, columns, header, index, index_label, mode, encoding, compression, quoting, quotechar, line_terminator, chunksize, tupleize_cols, date_format, doublequote, escapechar, decimal)\u001b[0m\n\u001b[1;32m   1743\u001b[0m                                  \u001b[0mdoublequote\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdoublequote\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1744\u001b[0m                                  escapechar=escapechar, decimal=decimal)\n\u001b[0;32m-> 1745\u001b[0;31m         \u001b[0mformatter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msave\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1746\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1747\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mpath_or_buf\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/pandas/io/formats/csvs.py\u001b[0m in \u001b[0;36msave\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    154\u001b[0m             f, handles = _get_handle(self.path_or_buf, self.mode,\n\u001b[1;32m    155\u001b[0m                                      \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 156\u001b[0;31m                                      compression=self.compression)\n\u001b[0m\u001b[1;32m    157\u001b[0m             \u001b[0mclose\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    158\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/pandas/io/common.py\u001b[0m in \u001b[0;36m_get_handle\u001b[0;34m(path_or_buf, mode, encoding, compression, memory_map, is_text)\u001b[0m\n\u001b[1;32m    398\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    399\u001b[0m             \u001b[0;31m# Python 3 and encoding\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 400\u001b[0;31m             \u001b[0mf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath_or_buf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    401\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0mis_text\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    402\u001b[0m             \u001b[0;31m# Python 3 and no explicit encoding\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: '../output/submission.csv'"]}]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}