{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<font size=6>An interesting find</font>\n\n<b><font size=5>The threshold of confidence will determine the number of final masks.This will have a subtle impact on the final score.The default threshold for v8 is 0.25, and when I lower the threshold, the score goes up a bit</font><b>\n    \n<font color=red size=3>score: 0.371->0.385</font>\n \n    \n<font size=6 color=red>As a reminder</font>\n    \n<b><font size=4>For different weight files, the corresponding mask strategy, mask expansion parameters, and confidence thresholds will be different,<font color=red>Not all of them will result in the best score</font>, and the parameters I provide may not be the highest current full-weight score, which is a contest of luck and strength</font><b>","metadata":{}},{"cell_type":"markdown","source":"# introduce\n[I made adjustments based on this](https://www.kaggle.com/code/koushikcon/dummy-submission)\n\n**I design to use a mask image to record the mask data of each segmentation block and integrate the results of segmentation under different weights.\nAlthough the final score is higher than both alone, its inference time changed from *a few minutes* to about *2 hours***\n\n\n**Is there a better way to deduplicate when there are multiple models？**","metadata":{}},{"cell_type":"markdown","source":"<b><font size=3>I modified the previous idea to get the results of all the models before processing. The processing method is modified from overlapping direct discarding to deintersection, that is, accepting all results.With this method, I got a better score and the time changed from about<font color=red> 2 hours to 10 minutes</font><b>\n\n<font color=red size=3>score: 0.264->0.285</font>","metadata":{}},{"cell_type":"markdown","source":"<b><font size=4>I used SAM to correct the final mask result and get a better score</font><b>\n\n<font color=red size=4>The disadvantage is that the reasoning time has changed from ten minutes to one hour</font>\n    \n<font color=red size=3>score: 0.285->0.309</font>","metadata":{}},{"cell_type":"markdown","source":"<b><font size=5>I added a mask dilate,I have to admit that for this game, this method is the weapon to improve score</font><b>\n    \n<font color=red size=3>score: 0.309->0.371</font>","metadata":{}},{"cell_type":"code","source":"import sys\nsys.path.append(\"/kaggle/input/yolov8/ultralytics-main\")","metadata":{"execution":{"iopub.status.busy":"2023-08-01T06:54:12.878518Z","iopub.execute_input":"2023-08-01T06:54:12.878867Z","iopub.status.idle":"2023-08-01T06:54:12.892598Z","shell.execute_reply.started":"2023-08-01T06:54:12.878838Z","shell.execute_reply":"2023-08-01T06:54:12.891667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import shutil\nimport os\nimport pandas as pd\nimport numpy as np\nimport tifffile as tiff\nimport matplotlib.pyplot as plt\n\nfrom pathlib import Path\nfrom glob import glob\nfrom collections import defaultdict\nfrom sklearn.model_selection import train_test_split\nfrom tqdm import tqdm\nfrom IPython.display import Image as show_image\n\nimport ultralytics\nfrom ultralytics import YOLO\n\nimport torch\n\nultralytics.checks()","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:14.552028Z","iopub.execute_input":"2023-08-01T07:01:14.552458Z","iopub.status.idle":"2023-08-01T07:01:15.541135Z","shell.execute_reply.started":"2023-08-01T07:01:14.552426Z","shell.execute_reply":"2023-08-01T07:01:15.540216Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = 'cuda' if torch.cuda.is_available() else 'cpu'\n\nIMAGE_SIZE = 512\nBATCH_SIZE = 16\nEPOCHS = 10\n\nprint(device)","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:15.542810Z","iopub.execute_input":"2023-08-01T07:01:15.544034Z","iopub.status.idle":"2023-08-01T07:01:15.551811Z","shell.execute_reply.started":"2023-08-01T07:01:15.543999Z","shell.execute_reply":"2023-08-01T07:01:15.550935Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nHOME = os.getcwd()\n!mkdir /kaggle/working/packages\n!cp -r /kaggle/input/pycocotools/* /kaggle/working/packages\nos.chdir(\"/kaggle/working/packages/pycocotools-2.0.6/\")\n!python setup.py install\n!pip install . --no-index --find-links /kaggle/working/packages/\nos.chdir(\"/kaggle/working\")\n\nimport base64\nimport numpy as np\nfrom pycocotools import _mask as coco_mask\nimport typing as t\nimport zlib\nfrom PIL import Image\nimport cv2\nimport pandas as pd\nimport os\nfrom itertools import groupby\nfrom skimage.measure import label, regionprops\n\ndef encode_binary_mask(mask: np.ndarray) -> t.Text:\n    \"\"\"Converts a binary mask into OID challenge encoding ascii text.\"\"\"\n\n    # check input mask --\n    if mask.dtype != bool:\n        raise ValueError(\n            \"encode_binary_mask expects a binary mask, received dtype == %s\" %\n            mask.dtype)\n\n    mask = np.squeeze(mask)\n    if len(mask.shape) != 2:\n        raise ValueError(\n            \"encode_binary_mask expects a 2d mask, received shape == %s\" %\n            mask.shape)\n\n    # convert input mask to expected COCO API input --\n    mask_to_encode = mask.reshape(mask.shape[0], mask.shape[1], 1)\n    mask_to_encode = mask_to_encode.astype(np.uint8)\n    mask_to_encode = np.asfortranarray(mask_to_encode)\n\n    # RLE encode mask --\n    encoded_mask = coco_mask.encode(mask_to_encode)[0][\"counts\"]\n\n    # compress and base64 encoding --\n    binary_str = zlib.compress(encoded_mask, zlib.Z_BEST_COMPRESSION)\n    base64_str = base64.b64encode(binary_str)\n    return base64_str","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:15.553577Z","iopub.execute_input":"2023-08-01T07:01:15.553902Z","iopub.status.idle":"2023-08-01T07:01:54.912690Z","shell.execute_reply.started":"2023-08-01T07:01:15.553872Z","shell.execute_reply":"2023-08-01T07:01:54.911563Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# infer","metadata":{}},{"cell_type":"code","source":"# To determine if the current position has been detected\ndef count_change(old,new):\n    sum=0\n    for i in range(old.shape[0]):\n        for j in range(old.shape[1]):\n            if new[i,j] == True and old[i,j] == True:\n                return True\n    \n    return False","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:54.916305Z","iopub.execute_input":"2023-08-01T07:01:54.916689Z","iopub.status.idle":"2023-08-01T07:01:54.922318Z","shell.execute_reply.started":"2023-08-01T07:01:54.916650Z","shell.execute_reply":"2023-08-01T07:01:54.921449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Draw mask image\ndef draw_mask(array):\n    plt.figure()\n    image_array = np.array(array)\n    plt.imshow(image_array, cmap='binary')\n    plt.axis('off')\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:54.923665Z","iopub.execute_input":"2023-08-01T07:01:54.924218Z","iopub.status.idle":"2023-08-01T07:01:54.944095Z","shell.execute_reply.started":"2023-08-01T07:01:54.924184Z","shell.execute_reply":"2023-08-01T07:01:54.943199Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Segment the picture and record the mask result\ndef make_mask(model,im1,row, mask_all):\n    results = model.predict(source=im1)\n    boxe_data = results[0].boxes.data.cpu().numpy()\n    mask_data = results[0].masks.data\n    row[\"width\"] = results[0].orig_shape[0]\n    row[\"height\"] = results[0].orig_shape[1]\n\n    mask_data = mask_data == 1# multiple images are expected\n    mask_data = mask_data * 1\n    mask_data = mask_data.cpu().numpy()\n\n    mask_data = mask_data.astype(bool)\n    prediction_l = []\n    list_encode = []\n    conf_encode = []\n    \n    \n    \n    for i in range(boxe_data.shape[0]):\n        class_no = int(boxe_data[i][5])\n        class_conf = boxe_data[i][4]\n        class_conf = float(class_conf)\n        \n        if class_no !=0 :\n            continue\n            \n        if count_change(mask_all,mask_data[i,:,:]):\n            continue\n        \n        mask_all = mask_all | mask_data[i,:,:]\n        \n        sliceImage = mask_data[i,:,:]\n        coded_len = encode_binary_mask(sliceImage).decode('utf-8')\n                # if len(coded_len) < 90:\n        list_encode.append(coded_len)\n        conf_encode.append(class_conf)\n    \n    \n    for i in range(len(conf_encode)):\n        row[\"prediction_string\"] += '0 ' + str(conf_encode[i])+' '+ list_encode[i]+' '\n    \n    return mask_all","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:54.945482Z","iopub.execute_input":"2023-08-01T07:01:54.945988Z","iopub.status.idle":"2023-08-01T07:01:54.959436Z","shell.execute_reply.started":"2023-08-01T07:01:54.945952Z","shell.execute_reply":"2023-08-01T07:01:54.958545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# New approach","metadata":{}},{"cell_type":"code","source":"#Identify overlaps\ndef replace(mask_data,new_mask,num_mask):\n    for i in range(num_mask):\n        count = np.count_nonzero(new_mask | mask_data[i,:,:])\n        count1 = np.count_nonzero(new_mask & mask_data[i,:,:])\n        if count1 == 0:\n            continue\n        if (count1/count) > 0.65:\n            return i\n    return -1","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:54.960671Z","iopub.execute_input":"2023-08-01T07:01:54.961068Z","iopub.status.idle":"2023-08-01T07:01:54.969970Z","shell.execute_reply.started":"2023-08-01T07:01:54.961038Z","shell.execute_reply":"2023-08-01T07:01:54.969068Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the mask union\ndef remask(old_mask,new_mask):\n    return old_mask | new_mask","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:54.971533Z","iopub.execute_input":"2023-08-01T07:01:54.971971Z","iopub.status.idle":"2023-08-01T07:01:54.980243Z","shell.execute_reply.started":"2023-08-01T07:01:54.971941Z","shell.execute_reply":"2023-08-01T07:01:54.979251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_result(results):\n    mask_data = results[0].masks.data\n    mask_data = mask_data == 1# multiple images are expected\n    mask_data = mask_data * 1\n    mask_data = mask_data.cpu().numpy()\n    mask_data = mask_data.astype(bool)\n    return mask_data","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:54.982439Z","iopub.execute_input":"2023-08-01T07:01:54.983100Z","iopub.status.idle":"2023-08-01T07:01:54.990571Z","shell.execute_reply.started":"2023-08-01T07:01:54.983069Z","shell.execute_reply":"2023-08-01T07:01:54.989661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_change(num,mask_data,mask_data1,num_mask,conf,conf1):\n    for i in range(num):\n        change = replace(mask_data,mask_data1[i,:,:],num_mask)\n        if change >= 0 :\n            tmp = mask_data[change,:,:] & mask_data1[i,:,:]\n            kernel = np.ones(shape=(2, 2), dtype=np.uint8)\n            kernel1 = np.ones(shape=(1, 1), dtype=np.uint8)\n            tmp = cv2.dilate(tmp.astype(np.uint8), kernel, 1)\n            mask_data[change,:,:] = cv2.erode(mask_data[change,:,:].astype(np.uint8),kernel1,3)\n            mask_data[change,:,:] |= tmp.astype(np.bool_)\n            continue\n        else:\n#             if conf1[i] > 0.1:\n#                 mask_data = np.concatenate((mask_data,np.expand_dims(mask_data1[i,:,:], axis=0)), axis=0)\n#                 conf = np.concatenate((conf,np.expand_dims(conf1[i],axis=0)),axis=0)\n#                 num_mask+=1\n            continue\n    return num_mask,mask_data,conf","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:54.995409Z","iopub.execute_input":"2023-08-01T07:01:54.995727Z","iopub.status.idle":"2023-08-01T07:01:55.005340Z","shell.execute_reply.started":"2023-08-01T07:01:54.995704Z","shell.execute_reply":"2023-08-01T07:01:55.004471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_change_(boxe_data1,mask_data,mask_data1,num_mask,conf):\n    index = []\n    for i in range(boxe_data1.shape[0]):\n        change = replace(mask_data,mask_data1[i,:,:],num_mask)\n        if change >= 0 :\n            mask_data[change,:,:] = mask_data[change,:,:] & mask_data1[i,:,:]\n#             kernel = np.ones(shape=(2, 2), dtype=np.uint8)\n#             mask_data[change,:,:] = cv2.dilate(mask_data[change,:,:].astype(np.uint8), kernel, 2)\n            index.append(change)\n        else:\n#             if float(boxe_data1[i][4]) > 0.1:\n#                 mask_data = np.concatenate((mask_data,np.expand_dims(mask_data1[i,:,:], axis=0)), axis=0)\n#                 conf = np.concatenate((conf,np.expand_dims(float(boxe_data1[i][4]),axis=0)),axis=0)\n#                 num_mask+=1\n#                 index.append(i)\n            continue\n    return len(index),mask_data[index],conf[index]\n#     return num_mask,mask_data,conf","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:55.006877Z","iopub.execute_input":"2023-08-01T07:01:55.007202Z","iopub.status.idle":"2023-08-01T07:01:55.016598Z","shell.execute_reply.started":"2023-08-01T07:01:55.007173Z","shell.execute_reply":"2023-08-01T07:01:55.015561Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# SAM","metadata":{}},{"cell_type":"code","source":"def fine_tune(mask_sam,num_mask,mask_data):\n    for i in range(mask_sam.shape[0]):\n        for j in range(num_mask):\n            count = np.count_nonzero(mask_sam[i,:,:] | mask_data[j,:,:])\n            count1 = np.count_nonzero(mask_sam[i,:,:] & mask_data[j,:,:])\n            if count1 == 0:\n                continue\n            if (count1/count) > 0.5:\n                mask_data[j,:,:] = remask(mask_sam[i,:,:],mask_data[j,:,:])\n                break\n    return mask_data","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:55.018824Z","iopub.execute_input":"2023-08-01T07:01:55.019838Z","iopub.status.idle":"2023-08-01T07:01:55.030898Z","shell.execute_reply.started":"2023-08-01T07:01:55.019806Z","shell.execute_reply":"2023-08-01T07:01:55.030022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def new_mask(model,model1,model2,im1):\n    results = model.predict(source=im1,conf=0.001,iou=0.65,retina_masks=True,max_det=1000)\n    results1 = model1.predict(source=im1,conf=0.001,retina_masks=True)\n    results2 = model2.predict(source=im1,conf=0.001,retina_masks=True)\n    \n    mask_data = get_result(results)\n    mask_data1 = get_result(results1)\n    mask_data2 = get_result(results2)\n    \n    first_rs = results[0].boxes.data.cpu().numpy().shape[0]\n    num_mask = first_rs\n    second_rs = results1[0].boxes.data.cpu().numpy().shape[0]\n    num_mask1 = second_rs\n    \n    boxe_data = results[0].boxes.data.cpu().numpy()\n    boxe_data1 = results1[0].boxes.data.cpu().numpy()\n    boxe_data2 = results2[0].boxes.data.cpu().numpy()\n    \n    conf = np.empty(first_rs)\n    for i in range(first_rs):\n        class_conf = boxe_data[i][4]\n        class_conf = float(class_conf)\n        conf[i] = class_conf\n        \n    conf1 = np.empty(second_rs)\n    for i in range(second_rs):\n        class_conf = boxe_data1[i][4]\n        class_conf = float(class_conf)\n        conf1[i] = class_conf\n        \n    num_mask1,mask_data1,conf1 = get_change_(boxe_data2,mask_data1,mask_data2,num_mask1,conf1)\n    num_mask,mask_data,conf = get_change(num_mask1,mask_data,mask_data1,num_mask,conf,conf1)\n\n    \n    return mask_data,num_mask,conf","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-08-01T07:01:55.032365Z","iopub.execute_input":"2023-08-01T07:01:55.032883Z","iopub.status.idle":"2023-08-01T07:01:55.044385Z","shell.execute_reply.started":"2023-08-01T07:01:55.032853Z","shell.execute_reply":"2023-08-01T07:01:55.043343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = YOLO(\"/kaggle/input/yolov8x-seg/x-best.pt\") # 最好的模型\nmodel1 = YOLO(\"/kaggle/input/yolov8x-seg/x-100best.pt\") # 后两个为取交集的模型\nmodel2 = YOLO(\"/kaggle/input/yolov8x-seg/x-1024-34best.pt\")\nall_rows = []\nresults_csv = pd.DataFrame([], columns=[\"id\",\"height\",\"width\",\"prediction_string\"])\nresults_csv.set_index(\"id\")\n\nfor dirname, _, filenames in os.walk('/kaggle/input/hubmap-hacking-the-human-vasculature/test'):\n    for filename in filenames:\n        \n        im1 = Image.open(os.path.join(dirname, filename))\n        width, height = im1.size\n        \n        row = dict()\n        row[\"id\"] = filename[:-4]\n        row[\"height\"] = height\n        row[\"width\"] = width\n        row[\"prediction_string\"] = \"\"\n        try:\n            mask,num,conf = new_mask(model,model1,model2,im1)\n            for i in range(num):\n                sliceImage = mask[i,:,:]\n#                 kernel = np.ones(shape=(2, 2), dtype=np.uint8)\n#                 sliceImage = cv2.dilate(sliceImage.astype(np.uint8), kernel, 2)\n                coded_len = encode_binary_mask(sliceImage.astype(np.bool_)).decode('utf-8')\n                row[\"prediction_string\"] += '0 ' + str(conf[i])+' '+ coded_len+' '\n        except:\n            print(\"failed to some reason\")\n        \n        new_row = pd.DataFrame(row, index=[0])\n        results_csv = pd.concat([new_row, results_csv.loc[:]]).reset_index(drop=True)\n\nresults_csv.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:01:55.045954Z","iopub.execute_input":"2023-08-01T07:01:55.046394Z","iopub.status.idle":"2023-08-01T07:02:07.636142Z","shell.execute_reply.started":"2023-08-01T07:01:55.046365Z","shell.execute_reply":"2023-08-01T07:02:07.635178Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat submission.csv","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:02:07.637799Z","iopub.execute_input":"2023-08-01T07:02:07.638547Z","iopub.status.idle":"2023-08-01T07:02:08.670800Z","shell.execute_reply.started":"2023-08-01T07:02:07.638509Z","shell.execute_reply":"2023-08-01T07:02:08.669569Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mask_all = np.zeros((512,512),dtype=bool)\nfor i in range(num):\n#     print(i)\n#     draw_mask(mask[i,:,:])\n    mask_all |= mask[i,:,:]\ndraw_mask(mask_all)","metadata":{"execution":{"iopub.status.busy":"2023-08-01T07:02:08.672828Z","iopub.execute_input":"2023-08-01T07:02:08.673507Z","iopub.status.idle":"2023-08-01T07:02:08.811337Z","shell.execute_reply.started":"2023-08-01T07:02:08.673468Z","shell.execute_reply":"2023-08-01T07:02:08.810442Z"},"trusted":true},"execution_count":null,"outputs":[]}]}