{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Project Overview\n\nFor this competition the majority of my time was spent on the inference. The initial cell segmentation utilized [this notebook.](https://www.kaggle.com/rdizzl3/hpa-segmentation-masks-no-internet) and associated datasets which are inputs for this work. Noting that the key to the segmentator is setting the scale factor to 0.25, padding to True which gave the cells the best masking. ","metadata":{}},{"cell_type":"markdown","source":"# Cell Segmentation\n\nFor the competition, I used the cell segmentator and then a modified version of the label_cell function from [this notebook].(https://www.kaggle.com/samusram/even-faster-hpa-cell-segmentation). This was combine with my OpenCV code to pull images for my different models.  See get_image_masks and create_cell_images functions for those details. ","metadata":{}},{"cell_type":"code","source":"!pip install -q \"../input/pycocotools/pycocotools-2.0-cp37-cp37m-linux_x86_64.whl\"\n!pip install -q \"../input/hpapytorchzoozip/pytorch_zoo-master\"\n!pip install -q \"../input/hpacellsegmentatormaster/HPA-Cell-Segmentation-master\"\n\nNUC_MODEL = '../input/hpacellsegmentatormodelweights/dpn_unet_nuclei_v1.pth'\nCELL_MODEL = '../input/hpacellsegmentatormodelweights/dpn_unet_cell_3ch_v1.pth'","metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nimport imageio\nimport torch\nfrom tqdm import tqdm\nimport time\nimport gc\n\nimport matplotlib.pyplot as plt\nimport cv2\n\nimport hpacellseg.cellsegmentator as cellseg\nsegmentator = cellseg.CellSegmentator(\n    NUC_MODEL,\n    CELL_MODEL,\n    scale_factor=0.25,\n    padding=True,\n    multi_channel_model=True\n)\n\n\n#from hpacellseg.utils import label_cell, label_nuclei\n\nfrom pycocotools import _mask as coco_mask\nimport typing as t\nimport zlib\nimport base64\n\nimport keras\nimport keras.backend as K\nimport tensorflow as tf\nfrom tensorflow.keras import backend, layers\nfrom keras.preprocessing.image import img_to_array\n\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-output":true,"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"With the cell segmentation processes taking up a significant amount of GPU, I used 2 different methods to download my classification models for the competition. Both can be found in the [keras docuementation](https://www.tensorflow.org/api_docs/python/tf/config/experimental/set_memory_growth). ","metadata":{}},{"cell_type":"code","source":"# Stop Tensorflow From Eating All The Memory\ngpus = tf.config.experimental.list_physical_devices('GPU')\nif gpus:\n        try:\n            # Currently, memory growth needs to be the same across GPUs\n            for gpu in gpus:\n                tf.config.experimental.set_memory_growth(gpu, True)\n            logical_gpus = tf.config.experimental.list_logical_devices('GPU')\n            print(len(gpus), \"... Physical GPUs,\", len(logical_gpus), \"Logical GPUs ...\\n\")\n        except RuntimeError as e:\n            # Memory growth must be set before GPUs have been initialized\n            print(e)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"gpus = tf.config.experimental.list_physical_devices('GPU')\nif gpus:\n    try:\n        tf.config.experimental.set_virtual_device_configuration(gpus[0], [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=1024)])\n    except RuntimeError as e:\n        print(e)","metadata":{"_kg_hide-input":true}},{"cell_type":"code","source":"#https://github.com/qubvel/efficientnet/blob/8984e988ecccd9c3a15be2e793991845619a8a26/efficientnet/model.py#L591\nclass FixedDropout(layers.Dropout):\n        def _get_noise_shape(self, inputs):\n            if self.noise_shape is None:\n                return self.noise_shape\n\n            symbolic_shape = backend.shape(inputs)\n            noise_shape = [symbolic_shape[axis] if shape is None else shape\n                           for axis, shape in enumerate(self.noise_shape)]\n            return tuple(noise_shape)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"raw","source":"My best performance was with a 4 model ensemble. The models each had a different viewpoint for the dataset. The first 2 looked at individual segmented tile cells for the training and used the weak classificaiton as the ground truth. Alone these models obtained about a .3 on the public leader board, but together around .375. \n\nOne of my biggest breakthroughs came after reading [this notebook](https://www.kaggle.com/h053473666/0-354-efnb7-classification-weights-0-4-0-6) modification by Alien. Although, I reimagined it with my function weighted_predictions rather than breaking apart the strings at the end. The second 2 models are the overall image being classified against the full cells images. \n\nFrom the names for each of the the models G stands for green only images, RGB is the red/green/blue channel and excludes the yellow mitocondria channel. RYB_G is a red/yellow/blue image with thte green protein added into the other channels to provide a highlighter effect for the models prediction. I also tried an ensemble of only RGB images but did not get a good  LB score.  The models presented below did get the highest LB score that i obtained in this competition. ","metadata":{}},{"cell_type":"code","source":"RGB_model = keras.models.load_model('../input/hpa-models-2021/ProteinModelRGB_rev_18.h5')\nG_model = keras.models.load_model('../input/hpa-models-2021/GreentileProteinModel_rev_2.h5')\nmulticellmodel = keras.models.load_model('../input/hpa-models-2021/Full_image_greenModelRev9.h5', custom_objects={'FixedDropout':FixedDropout(rate=0.5)})\nimg_type = 'g' #'rgb','g', 'ryb_g' per model above\nmulticell2model = keras.models.load_model('../input/hpa-models-2021/Full_image_RYB_GModelRev11.h5', custom_objects={'FixedDropout':FixedDropout(rate=0.5)})\nimg2_type = 'ryb_g' #'rgb','g', 'ryb_g' per model above","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import scipy.ndimage as ndi\nfrom skimage import filters, measure, segmentation, transform, util, morphology, feature\nfrom skimage.morphology import (binary_erosion, closing, disk,\n                                    remove_small_holes, remove_small_objects)\n\n\ndef label_cell(nuclei_pred, cell_pred):\n    \"\"\"Label the cells and the nuclei.\n    Keyword arguments:\n    nuclei_pred -- a 3D numpy array of a prediction from a nuclei image.\n    cell_pred -- a 3D numpy array of a prediction from a cell image.\n    Returns:\n    A tuple containing:\n    nuclei-label -- A nuclei mask data array.\n    cell-label  -- A cell mask data array.\n    0's in the data arrays indicate background while a continous\n    strech of a specific number indicates the area for a specific\n    cell.\n    The same value in cell mask and nuclei mask refers to the identical cell.\n    NOTE: The nuclei labeling from this function will be sligthly\n    different from the values in :func:`label_nuclei` as this version\n    will use information from the cell-predictions to make better\n    estimates.\n    \"\"\"\n    def __wsh(\n        mask_img,\n        threshold,\n        border_img,\n        seeds,\n        threshold_adjustment=0.35,\n        small_object_size_cutoff=10,\n    ):\n        img_copy = np.copy(mask_img)\n        m = seeds * border_img  # * dt\n        img_copy[m <= threshold + threshold_adjustment] = 0\n        img_copy[m > threshold + threshold_adjustment] = 1\n        img_copy = img_copy.astype(np.bool)\n        img_copy = remove_small_objects(img_copy, small_object_size_cutoff).astype(\n            np.uint8\n        )\n\n        mask_img[mask_img <= threshold] = 0\n        mask_img[mask_img > threshold] = 1\n        mask_img = mask_img.astype(np.bool)\n        mask_img = remove_small_holes(mask_img, 63)\n        mask_img = remove_small_objects(mask_img, 1).astype(np.uint8)\n        markers = ndi.label(img_copy, output=np.uint32)[0]\n        labeled_array = segmentation.watershed(\n            mask_img, markers, mask=mask_img, watershed_line=True\n        )\n        return labeled_array\n\n    nuclei_label = __wsh(\n        nuclei_pred[..., 2] / 255.0,\n        0.4,\n        1 - (nuclei_pred[..., 1] + cell_pred[..., 1]) / 255.0 > 0.05,\n        nuclei_pred[..., 2] / 255,\n        threshold_adjustment=-0.25,\n        small_object_size_cutoff=32,\n    )\n\n    # for hpa_image, to remove the small pseduo nuclei\n    nuclei_label = remove_small_objects(nuclei_label, 157)\n    nuclei_label = measure.label(nuclei_label)\n    # this is to remove the cell borders' signal from cell mask.\n    # could use np.logical_and with some revision, to replace this func.\n    # Tuned for segmentation hpa images\n    threshold_value = max(0.22, filters.threshold_otsu(cell_pred[..., 2] / 255) * 0.5)\n    # exclude the green area first\n    cell_region = np.multiply(\n        cell_pred[..., 2] / 255 > threshold_value,\n        np.invert(np.asarray(cell_pred[..., 1] / 255 > 0.05, dtype=np.int8)),\n    )\n    sk = np.asarray(cell_region, dtype=np.int8)\n    distance = np.clip(cell_pred[..., 2], 255 * threshold_value, cell_pred[..., 2])\n    cell_label = segmentation.watershed(-distance, nuclei_label, mask=sk)\n    cell_label = remove_small_objects(cell_label, 344).astype(np.uint8)\n    selem = disk(2)\n    cell_label = closing(cell_label, selem)\n    cell_label = __fill_holes(cell_label)\n    \n    # this part is to use green channel, and extend cell label to green channel\n    # benefit is to exclude cells clear on border but without nucleus\n    sk = np.asarray(\n        np.add(\n            np.asarray(cell_label > 0, dtype=np.int8),\n            np.asarray(cell_pred[..., 1] / 255 > 0.05, dtype=np.int8),\n        )\n        > 0,\n        dtype=np.int8,\n    )\n    cell_label = segmentation.watershed(-distance, cell_label, mask=sk)\n    cell_label = __fill_holes(cell_label)\n    cell_label = np.asarray(cell_label > 0, dtype=np.uint8)\n    cell_label = measure.label(cell_label)\n    cell_label = remove_small_objects(cell_label, 344)\n    cell_label = measure.label(cell_label)\n    cell_label = np.asarray(cell_label, dtype=np.uint16)\n\n    return nuclei_label, cell_label\n\ndef __fill_holes(image):\n    \"\"\"Fill_holes for labelled image, with a unique number.\"\"\"\n    boundaries = segmentation.find_boundaries(image)\n    image = np.multiply(image, np.invert(boundaries))\n    image = ndi.binary_fill_holes(image > 0)\n    image = ndi.label(image)[0]\n    return image","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def build_image_names(image_id: str) -> list:\n    # mt is the mitchondria\n    mt = f'/kaggle/input/hpa-single-cell-image-classification/test/{image_id}_red.png'\n    \n    # er is the endoplasmic reticulum\n    er = f'/kaggle/input/hpa-single-cell-image-classification/test/{image_id}_yellow.png'\n    \n    # nu is the nuclei\n    nu = f'/kaggle/input/hpa-single-cell-image-classification/test/{image_id}_blue.png'\n    \n    # high is the protein\n    high = f'/kaggle/input/hpa-single-cell-image-classification/test/{image_id}_green.png'\n    \n    return [mt], [er], [nu], [high], [[mt], [er], [nu]],\n\ndef grab_contours(cnts):\n    # if the length the contours tuple returned by cv2.findContours\n    # is '2' then we are using either OpenCV v2.4, v4-beta, or\n    # v4-official\n    if len(cnts) == 2:\n        cnts = cnts[0]\n\n    # if the length of the contours tuple is '3' then we are using\n    # either OpenCV v3, v4-pre, or v4-alpha\n    elif len(cnts) == 3:\n        cnts = cnts[1]\n\n    # otherwise OpenCV has changed their cv2.findContours return\n    # signature yet again and I have no idea WTH is going on\n    else:\n        raise Exception((\"Contours tuple must have length 2 or 3, \"\n            \"otherwise OpenCV changed their cv2.findContours return \"\n            \"signature yet again. Refer to OpenCV's documentation \"\n            \"in that case\"))\n\n    # return the actual contours array\n    return cnts\n\ndef create_cell_images(RGB, RYB_G, G, cell_masks, size):\n    def clipimgtosquare(group_img):\n        #cover and crop image/contour/cell to maximum size for model.\n        cnt_img = np.zeros_like(group_img)\n        cnt_img[cover == 255] = group_img[cover == 255]\n        cnt_img = cnt_img[y:y+h, x:x+w]\n        #resize to ratio of desired size\n        old_size = cnt_img.shape[:2] \n        ratio = float(size)/max(old_size)\n        new_size = tuple([int(x*ratio) for x in old_size])\n        resized = cv2.resize(cnt_img, (new_size[1], new_size[0]))\n        #Create padding for final square image at desired size\n        delta_w = size - new_size[1]\n        delta_h = size - new_size[0]\n        top, bottom = delta_h//2, delta_h-(delta_h//2)\n        left, right = delta_w//2, delta_w-(delta_w//2)\n        color = [0, 0, 0]\n        square = cv2.copyMakeBorder(resized, top, bottom, left, right, cv2.BORDER_CONSTANT, value=color)\n        square = img_to_array(square)\n        #square = np.expand_dims(square, axis=0)\n        return square\n\n    mask = cv2.convertScaleAbs(cell_masks)\n    cnts = grab_contours(cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE))\n    RGBs = []\n    RYB_Gs = []\n    Gs = []\n    for i in range(1,cell_masks.max()):\n        mask = cv2.convertScaleAbs(np.where(cell_masks==i, 1, 0))\n        c = grab_contours(cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE))\n        (x,y,w,h) = cv2.boundingRect(c[0])\n        # Create top cover where white is what we want, black otherwise\n        cover = np.zeros_like(RGB)\n        # Draw contour (white) over img blank(black) - all 3 channels\n        cv2.drawContours(cover, [c[0]], 0, (255,255,255), -1)\n        \n        rgb = clipimgtosquare(RGB)\n        ryb_g = clipimgtosquare(RYB_G)\n        g = clipimgtosquare(G)\n        \n        RGBs.append(rgb)\n        RYB_Gs.append(ryb_g)\n        Gs.append(g)\n        \n    return RGBs, RYB_Gs, Gs\n\n\ndef image_predictions(images, model, TTArepeat=0, batch_size=8):\n    labels = []\n    confidences = []\n    images = np.vstack(images)\n    confidence = model.predict(images,batch_size=batch_size)\n    if TTArepeat > 0:\n        TTApred = []\n        TTApred.append(confidence)\n        image = data_augmentation(images)\n        for i in range(TTArepeat):\n            image = data_augmentation(image)\n            TTApred.append(model.predict(image))\n        confidence = np.mean(TTApred,axis=0)\n            \n    confidences.append(confidence)\n    return confidences\n\n# Code obtained form competition main page:\n#https://www.kaggle.com/c/hpa-single-cell-image-classification/overview/evaluation\ndef create_cell_masks(mask):\n    \"\"\"Converts a binary mask into OID challenge encoding ascii text.\"\"\"\n    # check input mask --\n    if mask.dtype != np.bool:\n        raise ValueError(\n            \"encode_binary_mask expects a binary mask, received dtype == %s\" %\n            mask.dtype)\n\n    mask = np.squeeze(mask)\n    if len(mask.shape) != 2:\n        raise ValueError(\n            \"encode_binary_mask expects a 2d mask, received shape == %s\" %\n            mask.shape)\n\n    # convert input mask to expected COCO API input --\n    mask_to_encode = mask.reshape(mask.shape[0], mask.shape[1], 1)\n    mask_to_encode = mask_to_encode.astype(np.uint8)\n    mask_to_encode = np.asfortranarray(mask_to_encode)\n\n    # RLE encode mask --\n    encoded_mask = coco_mask.encode(mask_to_encode)[0][\"counts\"]\n\n    # compress and base64 encoding --\n    binary_str = zlib.compress(encoded_mask, zlib.Z_BEST_COMPRESSION)\n    base64_str = base64.b64encode(binary_str)\n    return base64_str\n\ndef get_image_masks(image_id):\n    mt, er, nu, high, images = build_image_names(image_id=image_id)\n    \n    # For nuclei\n    nuc_segmentations = segmentator.pred_nuclei(images[2])\n    \n    # For full cells\n    cell_segmentations = segmentator.pred_cells(images)\n    \n    # post-processing\n    nuclei_mask, cell_mask = label_cell(nuc_segmentations[0], cell_segmentations[0])\n\n    # Protein model image\n    blue = normalization(plt.imread(nu[0]))\n    green = normalization(plt.imread(high[0]))\n    red = normalization(plt.imread(mt[0]))\n    yellow = normalization(plt.imread(er[0]))\n    \n    RGB = np.dstack((red, green, blue))\n    RYB_G =img = np.dstack((red+green, yellow+green, blue+green))\n    G = np.stack((green,)*3, axis=-1)\n       \n    return RGB, RYB_G, G, cell_mask\n\ndef flatten_list_of_lists(l_o_l, to_string=False):\n    if not to_string:\n        return [item for sublist in l_o_l for item in sublist]\n    else:\n        return [str(item) for sublist in l_o_l for item in sublist]\n    \ndef image_prediction_string(confidences, masks, labelqty = 19, threshold = .00):\n    labels = []\n    probs = []\n    codes = []\n    predictionstring = []\n    for pred, mask in zip(confidences, masks):\n        neglabel = 1-pred.max()\n        for label in range(0,labelqty):\n            if pred[label]>threshold:\n                labels.append(label)\n                probs.append(pred[label])\n                codes.append(mask.decode('UTF-8'))\n        labels.append(labelqty)\n        probs.append(neglabel)\n        codes.append(mask.decode('UTF-8'))\n            \n    predictionstring = [\" \".join(flatten_list_of_lists(zip([label, pred, mask]), to_string=True)) for label, pred, mask in zip(labels, probs, codes)]\n    #print(\" \".join(predictionstring))\n    return (\" \".join(predictionstring))\n   \ndef data_aug_exp(img, modeltype, size):\n    images = [] \n    images.append(img)\n    image = tf.image.central_crop(img, central_fraction=.8)\n    image = tf.image.resize(image, [size,size])\n    image = tf.image.rot90(img, k=1)\n    images.append(image)\n    image = tf.image.central_crop(image, central_fraction=.8)\n    image = tf.image.resize(image, [size,size])\n    images.append(image)\n    image = tf.image.flip_left_right(img)\n    images.append(image)\n    image = tf.image.central_crop(image, central_fraction=.8)\n    image = tf.image.resize(image, [size,size])\n    images.append(image)\n    image = tf.image.flip_up_down(img)\n    images.append(image)\n    image = tf.image.central_crop(image, central_fraction=.8)\n    image = tf.image.resize(image, [size,size])\n    images.append(image)\n    #if modeltype == 'rgb':\n    #    image = tf.image.adjust_hue(img, 0.01)\n    #    images.append(image)\n    if modeltype == 'ryb_g':\n        image = tf.image.adjust_contrast(img, 0.51)\n        images.append(image)\n        image = tf.image.central_crop(image, central_fraction=.8)\n        image = tf.image.resize(image, [size,size])\n        images.append(image)\n        \n        \n        #image = tf.image.adjust_brightness(img, delta=0.2)\n        #images.append(image)    \n        #image = tf.image.adjust_saturation(img, 0.95)\n        #images.append(image)\n    images = tf.expand_dims(images, axis=0)\n    images = np.vstack(images)\n    return images\n\ndef image_predictions_exp(images, model, modeltype, TTArepeat=False, batch_size=8, size=128):\n    images = np.expand_dims(images, axis=0)\n    images = np.vstack(images)\n    confidence = model.predict(images,batch_size=batch_size)\n    if TTArepeat:\n        TTApred = []\n        aug_images = data_aug_exp(images,modeltype,size)\n        for img in aug_images:\n            TTApred.append(model.predict(img))\n        confidence = np.mean(TTApred,axis=0)\n    return confidence\n\ndef weighted_predictions(pred1, wt1, pred2=0, wt2=0, pred3=0, wt3=0):\n    new_pred = pred1*wt1+pred2*wt2+pred3*wt3\n    \n    return np.array(new_pred)\n\ndef normalization(array):\n    a = (array - array.min())/(array.max()-array.min())\n    return a\n\ndef add_neglabel(array, labelqty):\n    arraynew = []\n    maximum = array.max(axis = 1)\n    array = np.insert(array, labelqty, 1-maximum, axis = 1)\n    return array\n    \ndef print_cell(image, title, index):\n    plt.subplot(1,10,index)\n    plt.imshow(image)\n    plt.title(title)\n    plt.axis('off')\n    #plt.show()\n    ","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here are the different 'view points' that a supplied to each image for the model training and the inference prediction. Test time augmentation was important for accurate prediction of the images. the most important augmentation proved to be a random crop that acts like a zoom inot the features of the images. ","metadata":{}},{"cell_type":"code","source":"train_dir = '../input/hpa-single-cell-image-classification/test/'\nimage_id = '01a14326-67b8-43b0-ac7a-ba6dfb3c38ad'\nRGB, RYB_G, G, cell_masks = get_image_masks(image_id)\n\nRGBs, RYB_Gs, Gs = create_cell_images(RGB, RYB_G, G, cell_masks, 128)\n\nplt.figure(figsize=(25,4))\nprint_cell(cell_masks, 'masks', 1)\nprint_cell(RGB, 'RGB cells', 2)\nprint_cell(RYB_G, 'RYB_G cells', 3)\nprint_cell(G, 'Green cells', 4)\nplt.tight_layout()\nplt.show()\n\nrgb = data_aug_exp(RGBs, 'rgb', 128)\nryb_g = data_aug_exp(RYB_Gs, 'ryb_g', 128)\ng = data_aug_exp(Gs, 'g', 128)\ntitle = ''\n\ns=multicellmodel.input_shape[1]\nfull = tf.image.resize(RGB, [s,s])\nfull = data_aug_exp(full, img_type, s)\n\nfor j in range(2):#len(cells)):\n    i=0\n    plt.figure(figsize =(25,10))\n    for img in rgb:\n        i+=1\n        print_cell(img[j], title, i)\n    plt.tight_layout()\n    plt.show()\n    i=0\n    plt.figure(figsize =(25,10))\n    for img in rgb:\n        i+=1\n        print_cell(img[j][:,:,1], '', i)\n    plt.tight_layout()\n    plt.show()\n    i=0\n    plt.figure(figsize =(25,10))\n    for img in ryb_g:\n        i+=1\n        print_cell(img[j], title, i)\n    plt.tight_layout()\n    plt.show()\n    i=0\n    plt.figure(figsize =(25,10))\n    for img in ryb_g:\n        i+=1\n        print_cell(img[j][:,:,1], '', i)\n    plt.tight_layout()\n    plt.show()\n    i=0\n    plt.figure(figsize =(25,10))\n    for img in g:\n        i+=1\n        print_cell(img[j], title, i)\n    plt.tight_layout()\n    plt.show()\n    i=0\n    plt.figure(figsize =(25,10))\n    for img in g:\n        i+=1\n        print_cell(img[j][:,:,1], '', i)\n    plt.tight_layout()\n    plt.show()\n    i=0\n    plt.figure(figsize =(25,10))\n    for img in full:\n        i+=1\n        print_cell(img, title, i)\n    plt.tight_layout()\n    plt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To save overall time and GPU resources my inference only predicted 2 images before submission. ","metadata":{}},{"cell_type":"code","source":"start = time.time()\n\n#sub = pd.DataFrame(columns = ['ID', 'ImageWidth', 'ImageHeight', 'PredictionString'])\ntest_dir = '../input/hpa-single-cell-image-classification/test/'\ntest_images = os.listdir(test_dir)\n\nimages = [i.split(\"_\")[0] for i in test_images]\nnames = np.unique(images)\npublic = len(names)==559\nif public:\n    print('...only public testset...')\n    names = names[0:2]","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"multicellpredict = []\nmulticellpredmax = []\niterations = 1\nsize = 128\nimghs = []\nimgws = []\npredictionstrings = []\nfor image_id in tqdm(names):\n    RGB, RYB_G, G, cell_masks = get_image_masks(image_id)\n    RGBs, RYB_Gs, Gs = create_cell_images(RGB, RYB_G, G, cell_masks, size)\n    \n    RGBpred = image_predictions_exp(RGBs, RGB_model, 'rgb', TTArepeat=True, batch_size=1)\n    Gpred = image_predictions_exp(Gs, G_model, 'g', TTArepeat=True, batch_size=1)\n    confidences  = weighted_predictions(RGBpred, .5, Gpred, .5)\n       \n    if multicellmodel:\n        s = multicellmodel.input_shape[1]\n        img = G\n        TTA = True\n        img = tf.image.resize(img, [s,s])\n        img = np.expand_dims(img, axis=0)\n        if TTA:\n            img = data_aug_exp(img,img_type,s)[:,0]\n            multicellpredict = multicellmodel.predict(img)\n            multicellpredict  = np.mean(multicellpredict,axis=0)\n        else:\n            multicellpredict = multicellmodel.predict(img)[0]\n            \n        if multicell2model:\n            s = multicell2model.input_shape[1]\n            img = RYB_G\n            img = tf.image.resize(img, [s,s])\n            img = np.expand_dims(img, axis=0)\n            if TTA:\n                img = data_aug_exp(img,img2_type,s)[:,0]\n                multicell2predict = multicell2model.predict(img)\n                multicell2predict  = np.mean(multicell2predict,axis=0)\n            else:\n                multicell2predict = multicell2model.predict(img)[0]\n            multicellpredict = weighted_predictions(multicell2predict, .4, multicellpredict, .6)\n        #Using weights estimates\n        confidences = weighted_predictions(multicellpredict[:18], .45, confidences, .55)\n    \n    masks = [create_cell_masks(np.where(cell_masks==i, 1, 0).astype(np.bool)) for i in range(1,cell_masks.max())]\n    shape = RGB.shape\n    imghs.append(shape[0])\n    imgws.append(shape[1])\n    string = image_prediction_string(confidences, masks, labelqty = 18, threshold = .00)\n    predictionstrings.append(string)\n    iterations+=1\n    \n    ##print(string) ###debug only\n    \n    \n    \nsub=pd.DataFrame({'ID':names, 'ImageWidth':imghs , 'ImageHeight':imgws, 'PredictionString':predictionstrings})\n\nend = time.time()\nprint(f\"All {len(names)} images complete at {round((end-start)/60,1)} mins.\")","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub.head(8)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub.to_csv(\"/kaggle/working/submission.csv\", index=False)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]}]}