{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Part 3","metadata":{"_uuid":"aa8401d73c7a19e1a43fdd6a992ea9dcb60039a2"}},{"cell_type":"markdown","source":"1. Explain what is Mask R–CNN (5 points) \n\nMask-RNN is an improvement over Faster-RNN due to its ability to selectively ignore less important parts of the image and attend only the selective portions. Mask R-CNN is an extension of Faster R-CNN that adds semantic segmentation as an extra feature. The detection of all objects present in an image at the pixel level is known as semantic segmentation. Mask R-CNN accomplishes this by including an additional classifier branch that predicts a pixel level segmentation mask for each object instance in the image.\n\n\n**Mask-RNN generates:**\n\n- Class label and bounding-box with the help of to generate Region of Interests proposed by the Region proposal network <br>\n- Object mask using 2 more convolution layers and a binary mask classifier<br>\n\nMask-RCNN flows through the following steps of architecture:<br>\n \n1. The image is fed to a CNN to generate the feature maps<br>\n2. The Region Proposal Network uses another CNN to generate the multiple Region of Interest using a binary classifier <br>\n3. The RoI Align network outputs multiple bounding boxes rather than a single definite one and warps them into a fixed dimension<br>\n4. Warped features are then fed into fully connected layers to make object classification using a softmax layer. Using a regression model, the bounding boxes predicted are refined<br>\n5. Warped features are also fed into Mask classifier, which consists of two CNNs to output a binary mask for each RoI. To predict multiple objects or multiple instances of objects in an image, Mask R-CNN makes multiple predictions <br>\n6. Anchor boxes of the binary classifier (set of predefined bounding boxes of a certain height and width defined to capture the scale and aspect ratio of specific object classes) with IoU greater than 0.5 (high objectness score) are then selected using Non-Max suppression <br>","metadata":{}},{"cell_type":"markdown","source":"2. Explain U-Net (5 points)\n\nA CNN-based neural network called U-Net is primarily useful for semantic segmentation. Its architecture is made up of an encoder network and a decoder network.\n\nThe encoder is a classification network that uses convolution blocks followed by downsampling to convert the input image into feature representations at multiple levels (pre-trained networks such as VGG, ResNet, etc. can be used).\n\nTo obtain the classification output, the decoder semantically projects the encoder's learned discriminative features (lower resolution output) onto the pixel space (original input imagee resolution). Upsampling and concatenation are followed by regular convolution operations in the decoder.\n\nThe decoder's convolutional layer uses two 3x3 convolutions (unpadded convolutions) twice, with each application being followed by a  (ReLU) and a 2x2 max pooling operation with stride 2 for downsampling. Each step of downsampling doubles the number of feature channels.\n\nThe condensed feature map is resized to match the original size of the input image in the upsampling section of the decoder. Transposed convolution, upconvolution, and deconvolution are other names for upsampling. The simplest to the most advanced upsampling techniques include Nearest Neighbor, Bilinear Interpolation, and Transposed Convolution. We need a strong prior from earlier stages because upsampling is a sparse operation and will help to better depict the localization.\n\nFollowing the feature map's upsampling, there is a concatenation with the correspondingly cropped feature map from the contracting path, two 3x3 convolutions, each followed by a ReLU, an up-convolution (2x2 convolution) that reduces the number of feature channels, and two 3x3 convolutions. Because to the loss of border pixels in each convolution, cropping is required.\n\nEach of the 64-component feature vectors is mapped to the desired number of classes in the final layer using a 1x1 convolution.","metadata":{}},{"cell_type":"markdown","source":"3. What is intersection over union (2 points) \n\nIntersection over Union (IoU) calculates the intersection over the union of two bounding boxes - the bounding box of the ground truth and the bounding box of the object predicted. \n\nA complete overlap of the bounding boxes of the ground truth and object predicted will have IoU of 1 and no overlap will have IoU of 0.\n\nIn Mask R-CNN, Non-Max suppression is used to detect object instances whose bounding boxes have IoU >0.5","metadata":{}},{"cell_type":"markdown","source":"4. Download the images from the attached zip folders “Train Image” and “Test Image” in\nthe Dataset.\n5. Download the ‘train_df.csv’ (it contains the image id and the corresponding pixels\nwhere the ship is present (NaN when there is no ship)).","metadata":{"_uuid":"a6cd9d5ad61ffe3b8858769f20a5f9493f024a56"}},{"cell_type":"code","source":"BATCH_SIZE = 48\nEDGE_CROP = 16\nGAUSSIAN_NOISE = 0.1\nUPSAMPLE_MODE = 'SIMPLE'\nNET_SCALING = (1, 1)\nIMG_SCALING = (3, 3)\nVALID_IMG_COUNT = 900\nMAX_TRAIN_STEPS = 9\nMAX_TRAIN_EPOCHS = 99\nAUGMENT_BRIGHTNESS = False","metadata":{"_uuid":"301a5d939c566d1487a049bb2554d09b592b18b1","execution":{"iopub.status.busy":"2023-03-13T01:32:52.863553Z","iopub.execute_input":"2023-03-13T01:32:52.86385Z","iopub.status.idle":"2023-03-13T01:32:52.881436Z","shell.execute_reply.started":"2023-03-13T01:32:52.863788Z","shell.execute_reply":"2023-03-13T01:32:52.880715Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We have used the same ship dataset which was merged in one folder each of Test and Train Images for the ease of the access of paths.","metadata":{}},{"cell_type":"code","source":"import os\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom skimage.io import imread\nimport matplotlib.pyplot as plt\nfrom matplotlib.cm import get_cmap\nfrom skimage.segmentation import mark_boundaries\n#from skimage.util import montage2d as montage\nfrom skimage.morphology import binary_opening, disk, label\nimport gc; gc.enable() # memory is tight\n\nmontage_rgb = lambda x: np.stack([montage(x[:, :, :, i]) for i in range(x.shape[3])], -1)\nship_dir = '../input/airbus-ship-detection'\ntrain_image_dir = os.path.join(ship_dir, 'train_v2')\ntest_image_dir = os.path.join(ship_dir, 'test_v2')\n\ndef multi_rle_encode(img, **kwargs):\n    '''\n    Encode connected regions as separated masks\n    '''\n    labels = label(img)\n    if img.ndim > 2:\n        return [rle_encode(np.sum(labels==k, axis=2), **kwargs) for k in np.unique(labels[labels>0])]\n    else:\n        return [rle_encode(labels==k, **kwargs) for k in np.unique(labels[labels>0])]\n\n# ref: https://www.kaggle.com/paulorzp/run-length-encode-and-decode\ndef rle_encode(img, min_max_threshold=1e-3, max_mean_threshold=None):\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated\n    '''\n    if np.max(img) < min_max_threshold:\n        return '' ## no need to encode if it's all zeros\n    if max_mean_threshold and np.mean(img) > max_mean_threshold:\n        return '' ## ignore overfilled mask\n    pixels = img.T.flatten()\n    pixels = np.concatenate([[0], pixels, [0]])\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n\ndef rle_decode(mask_rle, shape=(768, 768)):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (height,width) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n    for lo, hi in zip(starts, ends):\n        img[lo:hi] = 1\n    return img.reshape(shape).T  # Needed to align to RLE direction\n\ndef masks_as_image(in_mask_list):\n    # Take the individual ship masks and create a single mask array for all ships\n    all_masks = np.zeros((768, 768), dtype = np.uint8)\n    for mask in in_mask_list:\n        if isinstance(mask, str):\n            all_masks |= rle_decode(mask)\n    return all_masks\n\ndef masks_as_color(in_mask_list):\n    # Take the individual ship masks and create a color mask array for each ships\n    all_masks = np.zeros((768, 768), dtype = np.float)\n    scale = lambda x: (len(in_mask_list)+x+1) / (len(in_mask_list)*2) ## scale the heatmap image to shift \n    for i,mask in enumerate(in_mask_list):\n        if isinstance(mask, str):\n            all_masks[:,:] += scale(i) * rle_decode(mask)\n    return all_masks","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-03-13T01:33:04.097955Z","iopub.execute_input":"2023-03-13T01:33:04.098297Z","iopub.status.idle":"2023-03-13T01:33:04.910803Z","shell.execute_reply.started":"2023-03-13T01:33:04.098217Z","shell.execute_reply":"2023-03-13T01:33:04.910009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"masks = pd.read_csv(os.path.join('/kaggle/input/shipdata/ShipData/', 'train_df.csv'))\nnot_empty = pd.notna(masks.EncodedPixels)\nprint(not_empty.sum(), 'masks in', masks[not_empty].ImageId.nunique(), 'images')\nprint((~not_empty).sum(), 'empty images in', masks.ImageId.nunique(), 'total images')\nmasks.head()","metadata":{"_uuid":"3ca7119188fbb4c6540d9df55f5833b55435287e","execution":{"iopub.status.busy":"2023-03-13T01:34:47.899608Z","iopub.execute_input":"2023-03-13T01:34:47.899892Z","iopub.status.idle":"2023-03-13T01:34:48.944101Z","shell.execute_reply.started":"2023-03-13T01:34:47.899838Z","shell.execute_reply":"2023-03-13T01:34:48.943356Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, (ax1, ax2, ax3, ax4) = plt.subplots(1, 4, figsize = (16, 5))\nrle_0 = masks.query('ImageId==\"00021ddc3.jpg\"')['EncodedPixels']\nimg_0 = masks_as_image(rle_0)\nax1.imshow(img_0)\nax1.set_title('Mask as image')\nrle_1 = multi_rle_encode(img_0)\nimg_1 = masks_as_image(rle_1)\nax2.imshow(img_1)\nax2.set_title('Re-encoded')\nimg_c = masks_as_color(rle_0)\nax3.imshow(img_c)\nax3.set_title('Masks in colors')\nimg_c = masks_as_color(rle_1)\nax4.imshow(img_c)\nax4.set_title('Re-encoded in colors')\nprint('Check Decoding->Encoding',\n      'RLE_0:', len(rle_0), '->',\n      'RLE_1:', len(rle_1))\nprint(np.sum(img_0 - img_1), 'error')","metadata":{"_uuid":"0081fd6f387abd7c05eb35f29575a2ee6ddc2236","execution":{"iopub.status.busy":"2023-03-13T01:34:52.770484Z","iopub.execute_input":"2023-03-13T01:34:52.770758Z","iopub.status.idle":"2023-03-13T01:34:53.654582Z","shell.execute_reply.started":"2023-03-13T01:34:52.770705Z","shell.execute_reply":"2023-03-13T01:34:53.65354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"6. Plot some of the random images from train set having 0, 1 and 2 ships (can figure out from the train_df.csv file) (1 points)\n7. Highlight the area having ship in the above images (using the pixel information from the train_df.csv) (5 points)","metadata":{}},{"cell_type":"code","source":"fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize = (30, 10))\nfig.suptitle('Images having 0,1 and 2 ships and their corressponding masks', fontsize=34, fontweight='bold')\nrgb_path_1 = os.path.join(train_image_dir, '00021ddc3.jpg')\nax1.imshow(imread(rgb_path_1))\nax1.set_title('Sample image with 0 ships', fontsize=22, fontweight='bold')\nrgb_path_2 = os.path.join(train_image_dir, '000303d4d.jpg')\nax2.imshow(imread(rgb_path_2))\nax2.set_title('Sample image with 1 ships', fontsize=22, fontweight='bold')\nrgb_path_3 = os.path.join(train_image_dir, '000811bb6.jpg')\nax3.imshow(imread(rgb_path_3))\nax3.set_title('Sample image with 2 ships', fontsize=22, fontweight='bold')\nfig, (ax4, ax5, ax6) = plt.subplots(1, 3, figsize = (30, 10))\nrle_0 = masks.query('ImageId==\"00021ddc3.jpg\"')['EncodedPixels']\nimg_0 = masks_as_image(rle_0)\nax4.imshow(img_0)\nax4.set_title('Area having ship in image with 0 ships', fontsize=22, fontweight='bold')\nrle_1 = masks.query('ImageId==\"000303d4d.jpg\"')['EncodedPixels']\nimg_1 = masks_as_image(rle_1)\nax5.imshow(img_1)\nax5.set_title('Area having ship in image with 1 ship', fontsize=22, fontweight='bold')\nrle_2 = masks.query('ImageId==\"000811bb6.jpg\"')['EncodedPixels']\nimg_2 = masks_as_image(rle_2)\nax6.imshow(img_2)\nax6.set_title('Area having ship in image with 2 ships', fontsize=22, fontweight='bold')","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:35:03.568512Z","iopub.execute_input":"2023-03-13T01:35:03.56882Z","iopub.status.idle":"2023-03-13T01:35:06.180266Z","shell.execute_reply.started":"2023-03-13T01:35:03.568759Z","shell.execute_reply":"2023-03-13T01:35:06.179652Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Plot some of the random test set images (2 points) ","metadata":{}},{"cell_type":"code","source":"fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize = (30, 10))\nfig.suptitle('Test images having 0,1 and 2 ships', fontsize=34, fontweight='bold')\n# rgb_path_1 = os.path.join(test_image_dir, '6a2514c6a.jpg')\nrgb_path_1 = os.path.join(test_image_dir, '008c483bb.jpg')\nax1.imshow(imread(rgb_path_1))\nax1.set_title('Sample test image with 0 ships', fontsize=22, fontweight='bold')\n# rgb_path_2 = os.path.join(test_image_dir, '6a0c625b8.jpg')\nrgb_path_2 = os.path.join(test_image_dir, '009c7f8ec.jpg')\nax2.imshow(imread(rgb_path_2))\nax2.set_title('Sample test image with 1 ships', fontsize=22, fontweight='bold')\n# rgb_path_3 = os.path.join(test_image_dir, '6aaebf1da.jpg')\nrgb_path_3 = os.path.join(test_image_dir, '02c88bc61.jpg')\nax3.imshow(imread(rgb_path_3))\nax3.set_title('Sample test image with 2 ships', fontsize=22, fontweight='bold')","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:35:56.789655Z","iopub.execute_input":"2023-03-13T01:35:56.790003Z","iopub.status.idle":"2023-03-13T01:35:58.640426Z","shell.execute_reply.started":"2023-03-13T01:35:56.789938Z","shell.execute_reply":"2023-03-13T01:35:58.639513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"masks['ships'] = masks['EncodedPixels'].map(lambda c_row: 1 if isinstance(c_row, str) else 0)\nunique_img_ids = masks.groupby('ImageId').agg({'ships': 'sum'}).reset_index()\nunique_img_ids['ships'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:38:29.277439Z","iopub.execute_input":"2023-03-13T01:38:29.277737Z","iopub.status.idle":"2023-03-13T01:38:29.497016Z","shell.execute_reply.started":"2023-03-13T01:38:29.277679Z","shell.execute_reply":"2023-03-13T01:38:29.496274Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unique_img_ids['has_ship'] = unique_img_ids['ships'].map(lambda x: 1.0 if x>0 else 0.0)\nunique_img_ids['has_ship_vec'] = unique_img_ids['has_ship'].map(lambda x: [x])\nunique_img_ids['has_ship'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:41:23.682564Z","iopub.execute_input":"2023-03-13T01:41:23.682856Z","iopub.status.idle":"2023-03-13T01:41:23.935862Z","shell.execute_reply.started":"2023-03-13T01:41:23.682799Z","shell.execute_reply":"2023-03-13T01:41:23.935183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unique_img_ids.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:41:29.025031Z","iopub.execute_input":"2023-03-13T01:41:29.025338Z","iopub.status.idle":"2023-03-13T01:41:29.042118Z","shell.execute_reply.started":"2023-03-13T01:41:29.02528Z","shell.execute_reply":"2023-03-13T01:41:29.041355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unique_img_ids['ships'].hist(bins=unique_img_ids['ships'].max())","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:41:44.322749Z","iopub.execute_input":"2023-03-13T01:41:44.323025Z","iopub.status.idle":"2023-03-13T01:41:44.572389Z","shell.execute_reply.started":"2023-03-13T01:41:44.32297Z","shell.execute_reply":"2023-03-13T01:41:44.571388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For effective prediction, the sample bias needs to be corrected because there are 140k+ images without ships, 20k+ images with one ship, and even fewer images with more than one ship.\n\nAs a result, we are limiting the amount of photos to 30k for each of the 0, 1, 2, 3, etc. number of ships.","metadata":{}},{"cell_type":"code","source":"#Undersample Empty Images\nSAMPLES_PER_GROUP = 40000\nbalanced_train_df = unique_img_ids.groupby('ships').apply(lambda x: x.sample(SAMPLES_PER_GROUP) if len(x) > SAMPLES_PER_GROUP else x)\nbalanced_train_df['ships'].hist(bins=balanced_train_df['ships'].max()+1)\nprint(balanced_train_df.shape[0], 'masks')","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:44:12.565795Z","iopub.execute_input":"2023-03-13T01:44:12.566074Z","iopub.status.idle":"2023-03-13T01:44:12.985114Z","shell.execute_reply.started":"2023-03-13T01:44:12.566019Z","shell.execute_reply":"2023-03-13T01:44:12.984298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Split into training and validation groups","metadata":{"_uuid":"40cb72e241c0c3d8bc245b4e3c663b4a835b0011"}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\ntrain_ids, valid_ids = train_test_split(balanced_train_df, \n                 test_size = 0.2, \n                 stratify = balanced_train_df['ships'])\ntrain_ids","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:52:20.97132Z","iopub.execute_input":"2023-03-13T01:52:20.971863Z","iopub.status.idle":"2023-03-13T01:52:21.490511Z","shell.execute_reply.started":"2023-03-13T01:52:20.971598Z","shell.execute_reply":"2023-03-13T01:52:21.489508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"masks['ships'] = masks['EncodedPixels'].map(lambda c_row: 1 if isinstance(c_row, str) else 0)\nunique_img_ids = masks.groupby('ImageId').agg({'ships': 'sum'}).reset_index()\nunique_img_ids['has_ship'] = unique_img_ids['ships'].map(lambda x: 1.0 if x>0 else 0.0)\nunique_img_ids['has_ship_vec'] = unique_img_ids['has_ship'].map(lambda x: [x])\nunique_img_ids['file_size_kb'] = unique_img_ids['ImageId'].map(lambda c_img_id: \n                                                               os.stat(os.path.join(train_image_dir, \n                                                                                    c_img_id)).st_size/1024)\nunique_img_ids = unique_img_ids[unique_img_ids['file_size_kb'] > 50] # keep only +50kb files\nunique_img_ids['file_size_kb'].hist()\nmasks.drop(['ships'], axis=1, inplace=True)\nunique_img_ids.sample(7)","metadata":{"_uuid":"c4f008bf6898518fd371de013418f936edaa09f8","execution":{"iopub.status.busy":"2023-03-13T01:45:43.089634Z","iopub.execute_input":"2023-03-13T01:45:43.089927Z","iopub.status.idle":"2023-03-13T01:52:04.405352Z","shell.execute_reply.started":"2023-03-13T01:45:43.089872Z","shell.execute_reply":"2023-03-13T01:52:04.404365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\ntrain_ids, valid_ids = train_test_split(balanced_train_df, \n                 test_size = 0.2, \n                 stratify = balanced_train_df['ships'])\ntrain_df = pd.merge(masks, train_ids)\nvalid_df = pd.merge(masks, valid_ids)\nprint(train_df.shape[0], 'training masks')\nprint(valid_df.shape[0], 'validation masks')","metadata":{"_uuid":"a26cd030942c2cd763c6fcd08b370f886c93ecdf","execution":{"iopub.status.busy":"2023-03-13T01:53:07.336561Z","iopub.execute_input":"2023-03-13T01:53:07.336881Z","iopub.status.idle":"2023-03-13T01:53:07.523385Z","shell.execute_reply.started":"2023-03-13T01:53:07.336818Z","shell.execute_reply":"2023-03-13T01:53:07.522713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-13T01:53:13.003113Z","iopub.execute_input":"2023-03-13T01:53:13.003429Z","iopub.status.idle":"2023-03-13T01:53:13.025441Z","shell.execute_reply.started":"2023-03-13T01:53:13.003373Z","shell.execute_reply":"2023-03-13T01:53:13.024472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def make_image_gen(in_df, batch_size = BATCH_SIZE):\n    all_batches = list(in_df.groupby('ImageId'))\n    out_rgb = []\n    out_mask = []\n    while True:\n        np.random.shuffle(all_batches)\n        for c_img_id, c_masks in all_batches:\n            rgb_path = os.path.join(train_image_dir, c_img_id)\n            c_img = imread(rgb_path)\n            c_mask = np.expand_dims(masks_as_image(c_masks['EncodedPixels'].values), -1)\n            if IMG_SCALING is not None:\n                c_img = c_img[::IMG_SCALING[0], ::IMG_SCALING[1]]\n                c_mask = c_mask[::IMG_SCALING[0], ::IMG_SCALING[1]]\n            out_rgb += [c_img]\n            out_mask += [c_mask]\n            if len(out_rgb)>=batch_size:\n                yield np.stack(out_rgb, 0)/255.0, np.stack(out_mask, 0)\n                out_rgb, out_mask=[], []","metadata":{"_uuid":"6181ac51577e5636995e38a9e29311cf47f513ca","execution":{"iopub.status.busy":"2023-03-13T01:53:29.458768Z","iopub.execute_input":"2023-03-13T01:53:29.459053Z","iopub.status.idle":"2023-03-13T01:53:29.465734Z","shell.execute_reply.started":"2023-03-13T01:53:29.458995Z","shell.execute_reply":"2023-03-13T01:53:29.464521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_gen = make_image_gen(train_df)\ntrain_x, train_y = next(train_gen)\nprint('Train x', train_x.shape, train_x.min(), train_x.max())\nprint('Train y', train_y.shape, train_y.min(), train_y.max())\n\nvalid_gen = make_image_gen(valid_df, 900)\nvalid_x, valid_y = next(valid_gen)\nprint('Validation x', valid_x.shape, valid_x.min(), valid_x.max())\nprint('Validation y', valid_y.shape, valid_y.min(), valid_y.max())","metadata":{"_uuid":"1983738da75b031f2bec8ba36db01c095e7c5d59","execution":{"iopub.status.busy":"2023-03-13T01:55:53.51834Z","iopub.execute_input":"2023-03-13T01:55:53.518676Z","iopub.status.idle":"2023-03-13T01:56:25.980064Z","shell.execute_reply.started":"2023-03-13T01:55:53.518626Z","shell.execute_reply":"2023-03-13T01:56:25.979372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from skimage.util import montage as montage2d\nmontage_rgb = lambda x: np.stack([montage2d(x[:, :, :, i]) for i in range(x.shape[3])], -1)\nfig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize = (30, 10))\nbatch_rgb = montage_rgb(train_x)\nbatch_seg = montage2d(train_y[:, :, :, 0])\nax1.imshow(batch_rgb)\nax1.set_title('Images')\nax2.imshow(batch_seg)\nax2.set_title('Segmentations')\nax3.imshow(mark_boundaries(batch_rgb, batch_seg.astype(int)))\nax3.set_title('Outlined Ships')\nfig.savefig('overview.png')","metadata":{"_uuid":"b4396cd28ddd2e4c8076fcb165e9b61e3baeeeb7","execution":{"iopub.status.busy":"2023-03-13T01:56:38.928854Z","iopub.execute_input":"2023-03-13T01:56:38.929138Z","iopub.status.idle":"2023-03-13T01:56:42.210719Z","shell.execute_reply.started":"2023-03-13T01:56:38.929084Z","shell.execute_reply":"2023-03-13T01:56:42.210091Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Data Augmentation","metadata":{"_uuid":"a8f65e7942816fb75b687a549dc1d5cc48d00e21"}},{"cell_type":"code","source":"from keras.preprocessing.image import ImageDataGenerator\ndg_args = dict(featurewise_center = False, \n                  samplewise_center = False,\n                  rotation_range = 45, \n                  width_shift_range = 0.1, \n                  height_shift_range = 0.1, \n                  shear_range = 0.01,\n                  zoom_range = [0.9, 1.25],  \n                  horizontal_flip = True, \n                  vertical_flip = True,\n                  fill_mode = 'reflect',\n                   data_format = 'channels_last')\n# brightness can be problematic since it seems to change the labels differently from the images \nif AUGMENT_BRIGHTNESS:\n    dg_args[' brightness_range'] = [0.5, 1.5]\nimage_gen = ImageDataGenerator(**dg_args)\n\nif AUGMENT_BRIGHTNESS:\n    dg_args.pop('brightness_range')\nlabel_gen = ImageDataGenerator(**dg_args)\n\ndef create_aug_gen(in_gen, seed = None):\n    np.random.seed(seed if seed is not None else np.random.choice(range(9999)))\n    for in_x, in_y in in_gen:\n        seed = np.random.choice(range(9999))\n        # keep the seeds syncronized otherwise the augmentation to the images is different from the masks\n        g_x = image_gen.flow(255*in_x, \n                             batch_size = in_x.shape[0], \n                             seed = seed, \n                             shuffle=True)\n        g_y = label_gen.flow(in_y, \n                             batch_size = in_x.shape[0], \n                             seed = seed, \n                             shuffle=True)\n\n        yield next(g_x)/255.0, next(g_y)","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","execution":{"iopub.status.busy":"2023-03-13T01:58:44.281221Z","iopub.execute_input":"2023-03-13T01:58:44.281572Z","iopub.status.idle":"2023-03-13T01:58:44.291192Z","shell.execute_reply.started":"2023-03-13T01:58:44.281508Z","shell.execute_reply":"2023-03-13T01:58:44.29045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cur_gen = create_aug_gen(train_gen)\nt_x, t_y = next(cur_gen)\nprint('x', t_x.shape, t_x.dtype, t_x.min(), t_x.max())\nprint('y', t_y.shape, t_y.dtype, t_y.min(), t_y.max())\n# only keep first 9 samples to examine in detail\nt_x = t_x[:9]\nt_y = t_y[:9]\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize = (20, 10))\nax1.imshow(montage_rgb(t_x), cmap='gray')\nax1.set_title('images')\nax2.imshow(montage2d(t_y[:, :, :, 0]), cmap='gray_r')\nax2.set_title('ships')","metadata":{"_uuid":"6122ccb9e58bfac6fa5e11c86121e78d9e5151b1","execution":{"iopub.status.busy":"2023-03-13T02:00:20.951759Z","iopub.execute_input":"2023-03-13T02:00:20.952069Z","iopub.status.idle":"2023-03-13T02:00:23.565352Z","shell.execute_reply.started":"2023-03-13T02:00:20.952022Z","shell.execute_reply":"2023-03-13T02:00:23.56439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gc.collect()","metadata":{"_uuid":"33300c4f03b6600da7b418f775d11d7ebf76a35a","execution":{"iopub.status.busy":"2023-03-13T02:00:26.645441Z","iopub.execute_input":"2023-03-13T02:00:26.645742Z","iopub.status.idle":"2023-03-13T02:00:27.496522Z","shell.execute_reply.started":"2023-03-13T02:00:26.645685Z","shell.execute_reply":"2023-03-13T02:00:27.495775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Building a Model","metadata":{"_uuid":"ba08494eb9736ec3556b7c879143cdcdea89febf"}},{"cell_type":"code","source":"from keras import models, layers\n# Build U-Net model\ndef upsample_conv(filters, kernel_size, strides, padding):\n    return layers.Conv2DTranspose(filters, kernel_size, strides=strides, padding=padding)\ndef upsample_simple(filters, kernel_size, strides, padding):\n    return layers.UpSampling2D(strides)\n\nif UPSAMPLE_MODE=='DECONV':\n    upsample=upsample_conv\nelse:\n    upsample=upsample_simple\n    \ninput_img = layers.Input(t_x.shape[1:], name = 'RGB_Input')\npp_in_layer = input_img\n\nif NET_SCALING is not None:\n    pp_in_layer = layers.AvgPool2D(NET_SCALING)(pp_in_layer)\n    \npp_in_layer = layers.GaussianNoise(GAUSSIAN_NOISE)(pp_in_layer)\npp_in_layer = layers.BatchNormalization()(pp_in_layer)\n\nc1 = layers.Conv2D(8, (3, 3), activation='relu', padding='same') (pp_in_layer)\nc1 = layers.Conv2D(8, (3, 3), activation='relu', padding='same') (c1)\np1 = layers.MaxPooling2D((2, 2)) (c1)\n\nc2 = layers.Conv2D(16, (3, 3), activation='relu', padding='same') (p1)\nc2 = layers.Conv2D(16, (3, 3), activation='relu', padding='same') (c2)\np2 = layers.MaxPooling2D((2, 2)) (c2)\n\nc3 = layers.Conv2D(32, (3, 3), activation='relu', padding='same') (p2)\nc3 = layers.Conv2D(32, (3, 3), activation='relu', padding='same') (c3)\np3 = layers.MaxPooling2D((2, 2)) (c3)\n\nc4 = layers.Conv2D(64, (3, 3), activation='relu', padding='same') (p3)\nc4 = layers.Conv2D(64, (3, 3), activation='relu', padding='same') (c4)\np4 = layers.MaxPooling2D(pool_size=(2, 2)) (c4)\n\n\nc5 = layers.Conv2D(128, (3, 3), activation='relu', padding='same') (p4)\nc5 = layers.Conv2D(128, (3, 3), activation='relu', padding='same') (c5)\n\nu6 = upsample(64, (2, 2), strides=(2, 2), padding='same') (c5)\nu6 = layers.concatenate([u6, c4])\nc6 = layers.Conv2D(64, (3, 3), activation='relu', padding='same') (u6)\nc6 = layers.Conv2D(64, (3, 3), activation='relu', padding='same') (c6)\n\nu7 = upsample(32, (2, 2), strides=(2, 2), padding='same') (c6)\nu7 = layers.concatenate([u7, c3])\nc7 = layers.Conv2D(32, (3, 3), activation='relu', padding='same') (u7)\nc7 = layers.Conv2D(32, (3, 3), activation='relu', padding='same') (c7)\n\nu8 = upsample(16, (2, 2), strides=(2, 2), padding='same') (c7)\nu8 = layers.concatenate([u8, c2])\nc8 = layers.Conv2D(16, (3, 3), activation='relu', padding='same') (u8)\nc8 = layers.Conv2D(16, (3, 3), activation='relu', padding='same') (c8)\n\nu9 = upsample(8, (2, 2), strides=(2, 2), padding='same') (c8)\nu9 = layers.concatenate([u9, c1], axis=3)\nc9 = layers.Conv2D(8, (3, 3), activation='relu', padding='same') (u9)\nc9 = layers.Conv2D(8, (3, 3), activation='relu', padding='same') (c9)\n\nd = layers.Conv2D(1, (1, 1), activation='sigmoid') (c9)\n# d = layers.Cropping2D((EDGE_CROP, EDGE_CROP))(d)\n# d = layers.ZeroPadding2D((EDGE_CROP, EDGE_CROP))(d)\nif NET_SCALING is not None:\n    d = layers.UpSampling2D(NET_SCALING)(d)\n\nseg_model = models.Model(inputs=[input_img], outputs=[d])\nseg_model.summary()","metadata":{"_uuid":"2687377309d3cbbab1197f4eccd2b50ab996f5a6","execution":{"iopub.status.busy":"2023-03-13T02:00:46.549935Z","iopub.execute_input":"2023-03-13T02:00:46.550224Z","iopub.status.idle":"2023-03-13T02:00:48.420931Z","shell.execute_reply.started":"2023-03-13T02:00:46.550163Z","shell.execute_reply":"2023-03-13T02:00:48.420137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import keras.backend as K\nfrom keras.optimizers import Adam\nfrom keras.losses import binary_crossentropy\n\n## intersection over union\ndef IoU(y_true, y_pred, eps=1e-6):\n    if np.max(y_true) == 0.0:\n        return IoU(1-y_true, 1-y_pred) ## empty image; calc IoU of zeros\n    intersection = K.sum(y_true * y_pred, axis=[1,2,3])\n    union = K.sum(y_true, axis=[1,2,3]) + K.sum(y_pred, axis=[1,2,3]) - intersection\n    return -K.mean( (intersection + eps) / (union + eps), axis=0)","metadata":{"_uuid":"1678069aa8013510264ba898291c6ae2dce88a76","execution":{"iopub.status.busy":"2023-03-13T02:01:15.528043Z","iopub.execute_input":"2023-03-13T02:01:15.528348Z","iopub.status.idle":"2023-03-13T02:01:15.533694Z","shell.execute_reply.started":"2023-03-13T02:01:15.52829Z","shell.execute_reply":"2023-03-13T02:01:15.532594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.callbacks import ModelCheckpoint, LearningRateScheduler, EarlyStopping, ReduceLROnPlateau\nweight_path=\"{}_weights.best.hdf5\".format('seg_model')\n\ncheckpoint = ModelCheckpoint(weight_path, monitor='val_loss', verbose=1, save_best_only=True, mode='min', save_weights_only=True)\n\nreduceLROnPlat = ReduceLROnPlateau(monitor='val_loss', factor=0.33,\n                                   patience=1, verbose=1, mode='min',\n                                   min_delta=0.0001, cooldown=0, min_lr=1e-8)\n\nearly = EarlyStopping(monitor=\"val_loss\", mode=\"min\", verbose=2,\n                      patience=20) # probably needs to be more patient, but kaggle time is limited\n\ncallbacks_list = [checkpoint, early, reduceLROnPlat]","metadata":{"_uuid":"7282d18de3aff1cee12ff89b7d511a391702814f","execution":{"iopub.status.busy":"2023-03-13T02:01:27.01027Z","iopub.execute_input":"2023-03-13T02:01:27.010612Z","iopub.status.idle":"2023-03-13T02:01:27.023268Z","shell.execute_reply.started":"2023-03-13T02:01:27.010557Z","shell.execute_reply":"2023-03-13T02:01:27.022517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def fit():\n    seg_model.compile(optimizer=Adam(1e-3, decay=1e-6), loss=IoU, metrics=['binary_accuracy'])\n    \n    step_count = min(MAX_TRAIN_STEPS, train_df.shape[0]//BATCH_SIZE)\n    aug_gen = create_aug_gen(make_image_gen(train_df))\n    loss_history = [seg_model.fit_generator(aug_gen,\n                                 steps_per_epoch=step_count,\n                                 epochs=MAX_TRAIN_EPOCHS,\n                                 validation_data=(valid_x, valid_y),\n                                 callbacks=callbacks_list,\n                                workers=1 # the generator is not very thread safe\n                                           )]\n    return loss_history\n\nwhile True:\n    loss_history = fit()\n    if np.min([mh.history['val_loss'] for mh in loss_history]) < -0.2:\n        break","metadata":{"_uuid":"5b67d808c0b8c7e28bff41e6d3858ff6f09dd626","execution":{"iopub.status.busy":"2023-03-13T02:01:38.95284Z","iopub.execute_input":"2023-03-13T02:01:38.953127Z","iopub.status.idle":"2023-03-13T02:10:59.226728Z","shell.execute_reply.started":"2023-03-13T02:01:38.95307Z","shell.execute_reply":"2023-03-13T02:10:59.223586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def show_loss(loss_history):\n    epochs = np.concatenate([mh.epoch for mh in loss_history])\n    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(22, 10))\n    \n    _ = ax1.plot(epochs, np.concatenate([mh.history['loss'] for mh in loss_history]), 'b-',\n                 epochs, np.concatenate([mh.history['val_loss'] for mh in loss_history]), 'r-')\n    ax1.legend(['Training', 'Validation'])\n    ax1.set_title('Loss')\n    \n    _ = ax2.plot(epochs, np.concatenate([mh.history['binary_accuracy'] for mh in loss_history]), 'b-',\n                 epochs, np.concatenate([mh.history['val_binary_accuracy'] for mh in loss_history]), 'r-')\n    ax2.legend(['Training', 'Validation'])\n    ax2.set_title('Binary Accuracy (%)')\n\nshow_loss(loss_history)","metadata":{"_uuid":"a168c8b1af446b800f6129104906003ededd61c4","execution":{"iopub.status.busy":"2023-03-13T02:12:32.681452Z","iopub.execute_input":"2023-03-13T02:12:32.681749Z","iopub.status.idle":"2023-03-13T02:12:33.263653Z","shell.execute_reply.started":"2023-03-13T02:12:32.681694Z","shell.execute_reply":"2023-03-13T02:12:33.262677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"seg_model.load_weights(weight_path)\nseg_model.save('seg_model.h5')","metadata":{"_uuid":"ce1167e9f09200f537e61f93f486168a13be1711","execution":{"iopub.status.busy":"2023-03-13T02:12:39.692826Z","iopub.execute_input":"2023-03-13T02:12:39.693138Z","iopub.status.idle":"2023-03-13T02:12:40.063479Z","shell.execute_reply.started":"2023-03-13T02:12:39.693077Z","shell.execute_reply":"2023-03-13T02:12:40.062671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_y = seg_model.predict(valid_x)\nprint(pred_y.shape, pred_y.min(axis=0).max(), pred_y.max(axis=0).min(), pred_y.mean())","metadata":{"_uuid":"275b411dc97a350aacaba46c8562efcf2658b1a7","execution":{"iopub.status.busy":"2023-03-13T02:12:43.080541Z","iopub.execute_input":"2023-03-13T02:12:43.080842Z","iopub.status.idle":"2023-03-13T02:12:46.52018Z","shell.execute_reply.started":"2023-03-13T02:12:43.080774Z","shell.execute_reply":"2023-03-13T02:12:46.519409Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 1, figsize = (6, 6))\nax.hist(pred_y.ravel(), np.linspace(0, 1, 20))\nax.set_xlim(0, 1)\nax.set_yscale('log', nonposy='clip')","metadata":{"execution":{"iopub.status.busy":"2023-03-13T02:13:23.67217Z","iopub.execute_input":"2023-03-13T02:13:23.672484Z","iopub.status.idle":"2023-03-13T02:13:25.059797Z","shell.execute_reply.started":"2023-03-13T02:13:23.672426Z","shell.execute_reply":"2023-03-13T02:13:25.058921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prepare Full Resolution Model\nHere we account for the scaling so everything can happen in the model itself","metadata":{"_uuid":"0018ab172d18936f8cc2c5df33d2f840dc16bf4f"}},{"cell_type":"code","source":"if IMG_SCALING is not None:\n    fullres_model = models.Sequential()\n    fullres_model.add(layers.AvgPool2D(IMG_SCALING, input_shape = (None, None, 3)))\n    fullres_model.add(seg_model)\n    fullres_model.add(layers.UpSampling2D(IMG_SCALING))\nelse:\n    fullres_model = seg_model\nfullres_model.save('fullres_model.h5')","metadata":{"_uuid":"17408f0ee8dc16149b8eff0447a1427ab3ed82ba","execution":{"iopub.status.busy":"2023-03-13T02:13:43.555896Z","iopub.execute_input":"2023-03-13T02:13:43.556193Z","iopub.status.idle":"2023-03-13T02:13:44.032975Z","shell.execute_reply.started":"2023-03-13T02:13:43.556128Z","shell.execute_reply":"2023-03-13T02:13:44.032265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualize predictions","metadata":{"_uuid":"17edb177402ae51651692511827a7e9d60646533"}},{"cell_type":"code","source":"def raw_prediction(img, path=test_image_dir):\n    c_img = imread(os.path.join(path, c_img_name))\n    c_img = np.expand_dims(c_img, 0)/255.0\n    cur_seg = fullres_model.predict(c_img)[0]\n    return cur_seg, c_img[0]\n\ndef smooth(cur_seg):\n    return binary_opening(cur_seg>0.99, np.expand_dims(disk(2), -1))\n\ndef predict(img, path=test_image_dir):\n    cur_seg, c_img = raw_prediction(img, path=path)\n    return smooth(cur_seg), c_img\n\n## Get a sample of each group of ship count\nsamples = valid_df.groupby('ships').apply(lambda x: x.sample(1))\nfig, m_axs = plt.subplots(samples.shape[0], 4, figsize = (15, samples.shape[0]*4))\n[c_ax.axis('off') for c_ax in m_axs.flatten()]\n\nfor (ax1, ax2, ax3, ax4), c_img_name in zip(m_axs, samples.ImageId.values):\n    first_seg, first_img = raw_prediction(c_img_name, train_image_dir)\n    ax1.imshow(first_img)\n    ax1.set_title('Image: ' + c_img_name)\n    ax2.imshow(first_seg[:, :, 0], cmap=get_cmap('jet'))\n    ax2.set_title('Model Prediction')\n    reencoded = masks_as_color(multi_rle_encode(smooth(first_seg)[:, :, 0]))\n    ax3.imshow(reencoded)\n    ax3.set_title('Prediction Masks')\n    ground_truth = masks_as_color(masks.query('ImageId==\"{}\"'.format(c_img_name))['EncodedPixels'])\n    ax4.imshow(ground_truth)\n    ax4.set_title('Ground Truth')\n    \nfig.savefig('validation.png')","metadata":{"_uuid":"e2c9ede3ab20bd7bfdd89c4fd18f09552cb4f5cb","execution":{"iopub.status.busy":"2023-03-13T02:13:58.32946Z","iopub.execute_input":"2023-03-13T02:13:58.329755Z","iopub.status.idle":"2023-03-13T02:14:05.343819Z","shell.execute_reply.started":"2023-03-13T02:13:58.329697Z","shell.execute_reply":"2023-03-13T02:14:05.342142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mIoU = IoU(num_classes=2, target_class_ids=[0, 1], sparse_y_true=True, sparse_y_pred=True, name='mean-IoU')\nIoU_results = []\nfor image, true_mask in test_dataset.take(TEST_LENGTH):\n    true_mask = true_mask.numpy().argmax(axis=-1)\n    pred_mask = predict(image)\n    mIoU.update_state(true_mask, pred_mask)\n    \n    iou = IoU(num_classes=2, target_class_ids=[0, 1], sparse_y_true=True, sparse_y_pred=True, name='mean-IoU')\n    iou.update_state(true_mask, pred_mask)\n    IoU_results.append(iou.result())\n\nplt.hist(IoU_results, bins=15)\nprint(mIoU.result())","metadata":{"execution":{"iopub.status.busy":"2023-03-13T02:30:15.322144Z","iopub.execute_input":"2023-03-13T02:30:15.322463Z","iopub.status.idle":"2023-03-13T02:30:15.340645Z","shell.execute_reply.started":"2023-03-13T02:30:15.322405Z","shell.execute_reply":"2023-03-13T02:30:15.339441Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Final submission","metadata":{"_uuid":"11a6c6615131ff8c317f95a5097b46565ef21121","trusted":true}},{"cell_type":"code","source":"test_paths = np.array(os.listdir(test_image_dir))\nprint(len(test_paths), 'test images found')","metadata":{"_uuid":"2671f602b571b70ad2bda613cbfad21c5fa5c160","execution":{"iopub.status.busy":"2023-03-13T02:14:33.912791Z","iopub.execute_input":"2023-03-13T02:14:33.913082Z","iopub.status.idle":"2023-03-13T02:14:34.849782Z","shell.execute_reply.started":"2023-03-13T02:14:33.913018Z","shell.execute_reply":"2023-03-13T02:14:34.849013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tqdm import tqdm_notebook\n\ndef pred_encode(img, **kwargs):\n    cur_seg, _ = predict(img)\n    cur_rles = multi_rle_encode(cur_seg, **kwargs)\n    return [[img, rle] for rle in cur_rles if rle is not None]\n\nout_pred_rows = []\nfor c_img_name in tqdm_notebook(test_paths[:30000]): ## only a subset as it takes too long to run\n    out_pred_rows += pred_encode(c_img_name, min_max_threshold=1.0)","metadata":{"_uuid":"11341f4037a3c44391877d35eb6704590c7e914e","execution":{"iopub.status.busy":"2023-03-13T02:14:45.248058Z","iopub.execute_input":"2023-03-13T02:14:45.248365Z","iopub.status.idle":"2023-03-13T02:28:24.457562Z","shell.execute_reply.started":"2023-03-13T02:14:45.248303Z","shell.execute_reply":"2023-03-13T02:28:24.456809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.DataFrame(out_pred_rows)\nsub.columns = ['ImageId', 'EncodedPixels']\nsub = sub[sub.EncodedPixels.notnull()]\nsub.head()","metadata":{"_uuid":"d569785624983fec2067b77f2d8d1fa1f1ac8da5","execution":{"iopub.status.busy":"2023-03-13T02:28:32.703371Z","iopub.execute_input":"2023-03-13T02:28:32.70366Z","iopub.status.idle":"2023-03-13T02:28:32.76247Z","shell.execute_reply.started":"2023-03-13T02:28:32.703607Z","shell.execute_reply":"2023-03-13T02:28:32.761403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## let's see what we got\nTOP_PREDICTIONS=5\nfig, m_axs = plt.subplots(TOP_PREDICTIONS, 2, figsize = (9, TOP_PREDICTIONS*5))\n[c_ax.axis('off') for c_ax in m_axs.flatten()]\n\nfor (ax1, ax2), c_img_name in zip(m_axs, sub.ImageId.unique()[:TOP_PREDICTIONS]):\n    c_img = imread(os.path.join(test_image_dir, c_img_name))\n    c_img = np.expand_dims(c_img, 0)/255.0\n    ax1.imshow(c_img[0])\n    ax1.set_title('Image: ' + c_img_name)\n    ax2.imshow(masks_as_color(sub.query('ImageId==\"{}\"'.format(c_img_name))['EncodedPixels']))\n    ax2.set_title('Prediction')","metadata":{"_uuid":"4b1be5a92a4fa7c5842757a7702ca1c3543c6f2c","execution":{"iopub.status.busy":"2023-03-13T02:29:01.522951Z","iopub.execute_input":"2023-03-13T02:29:01.523256Z","iopub.status.idle":"2023-03-13T02:29:02.03757Z","shell.execute_reply.started":"2023-03-13T02:29:01.52318Z","shell.execute_reply":"2023-03-13T02:29:02.036047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub1 = pd.read_csv('../input/airbus-ship-detection/sample_submission_v2.csv')\nsub1 = pd.DataFrame(np.setdiff1d(sub1['ImageId'].unique(), sub['ImageId'].unique(), assume_unique=True), columns=['ImageId'])\nsub1['EncodedPixels'] = None\nprint(len(sub1), len(sub))\n\nsub = pd.concat([sub, sub1])\nprint(len(sub))\nsub.to_csv('final_submission.csv', index=False)\nsub.head()","metadata":{"_uuid":"b67340ed5e046f323fba7cbc7e9af72b301dfd62","execution":{"iopub.status.busy":"2023-03-13T00:47:26.479409Z","iopub.execute_input":"2023-03-13T00:47:26.479714Z","iopub.status.idle":"2023-03-13T00:47:28.901272Z","shell.execute_reply.started":"2023-03-13T00:47:26.47966Z","shell.execute_reply":"2023-03-13T00:47:28.900571Z"},"trusted":true},"execution_count":null,"outputs":[]}]}