{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":9988,"databundleVersionId":868324,"sourceType":"competition"}],"dockerImageVersionId":30588,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":null,"end_time":null,"environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2023-11-25T14:21:13.238420","version":"2.4.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Airbus Ship Detection Challenge\n\n<img src=\"https://upload.wikimedia.org/wikipedia/commons/5/5d/Airbus_Logo_2017.svg\" width=\"500\">\n\n\nThis notebook unveils a basic Mask RCNN model using customized FAIR Detectron2 model for training almost on the entire training dataset.\n\nSome modifications to the demo notebook have been performed to fit the competition problematics:\n- Non Max Suppression after inference set up with a threshold of 0.5\n- Validation loop during training every cfg_cv.TEST.EVAL_PERIOD = 500 iterations implemented inside used code [MyDetectron2](https://github.com/vintel38/MyDetectron2) to compute several AP metrics and get learning curves\n- Learning rate defined over the entire training frame before training beginning\n- Validation / Training subset split and n-fold training in order for first epoch then permutation over n-fold on next epochs\n- Code implemented before submission to avoid overlapping masks as required in Competition rules\n- Augmentation list can be passed during model building implemented in my custom [MyDetectron2](https://github.com/vintel38/MyDetectron2) code to fight overfitting  ","metadata":{"papermill":{"duration":0.009054,"end_time":"2023-11-25T14:21:16.695969","exception":false,"start_time":"2023-11-25T14:21:16.686915","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport random\nimport matplotlib.pyplot as plt\nfrom tqdm import tqdm\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nimport cv2\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":0.944399,"end_time":"2023-11-25T14:21:17.649128","exception":false,"start_time":"2023-11-25T14:21:16.704729","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T15:58:13.841141Z","iopub.execute_input":"2024-02-18T15:58:13.841414Z","iopub.status.idle":"2024-02-18T15:58:14.372145Z","shell.execute_reply.started":"2024-02-18T15:58:13.841388Z","shell.execute_reply":"2024-02-18T15:58:14.371116Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# EDA on satellite imagery ","metadata":{"papermill":{"duration":0.008123,"end_time":"2023-11-25T14:21:17.665936","exception":false,"start_time":"2023-11-25T14:21:17.657813","status":"completed"},"tags":[]}},{"cell_type":"code","source":"segs = pd.read_csv('/kaggle/input/airbus-ship-detection/train_ship_segmentations_v2.csv')\nsegs.head()","metadata":{"papermill":{"duration":1.152665,"end_time":"2023-11-25T14:21:18.826755","exception":false,"start_time":"2023-11-25T14:21:17.674090","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T15:58:14.373673Z","iopub.execute_input":"2024-02-18T15:58:14.374061Z","iopub.status.idle":"2024-02-18T15:58:15.437415Z","shell.execute_reply.started":"2024-02-18T15:58:14.374035Z","shell.execute_reply":"2024-02-18T15:58:15.436375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Annotations statistics of the dataset\n\nlst_img = os.listdir('/kaggle/input/airbus-ship-detection/train_v2')\nprint('There are {} images in this dataset'.format(len(lst_img)))\n\nimg_ships = segs.loc[segs['EncodedPixels'].notna(),'ImageId'].nunique()\nimg_woships = segs.loc[segs['EncodedPixels'].isna(),'ImageId'].nunique()\n\nprint('Number of images with ships    - {}   | {} %'.format(img_ships, round(img_ships / segs.ImageId.nunique() * 100)))\nprint('Number of images without ships - {}  | {} %'.format(img_woships, round(img_woships / segs.ImageId.nunique() * 100)))","metadata":{"papermill":{"duration":39.29156,"end_time":"2023-11-25T14:21:58.151644","exception":false,"start_time":"2023-11-25T14:21:18.860084","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T15:58:15.438583Z","iopub.execute_input":"2024-02-18T15:58:15.438870Z","iopub.status.idle":"2024-02-18T15:58:17.745721Z","shell.execute_reply.started":"2024-02-18T15:58:15.438845Z","shell.execute_reply":"2024-02-18T15:58:17.744552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 2, figsize=(10,5))\n\nax[0].pie([img_ships, img_woships], labels=['With Ships', 'No Ships'], autopct='%1.1f%%')\nax[0].set_title('Background images distribution')\n\nax[1].bar(range(1,16), segs.dropna().groupby('ImageId').count().EncodedPixels.value_counts())\nax[1].set_title('Number of ships per image distribution')\nax[1].set_xlabel('Nb of Ships')\nax[1].set_xticks([i for i in range(1, 16)])\nax[1].set_ylabel('Image count')\n\nfig.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-18T15:58:17.748683Z","iopub.execute_input":"2024-02-18T15:58:17.749919Z","iopub.status.idle":"2024-02-18T15:58:18.380497Z","shell.execute_reply.started":"2024-02-18T15:58:17.749865Z","shell.execute_reply":"2024-02-18T15:58:18.379503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Tool function to decode the RLE encoding annotations\n# ref.: https://www.kaggle.com/stainsby/fast-tested-rle\ndef rle_encode(img):\n    '''\n    img: numpy array, 1 - mask, 0 - background\n    Returns run length as string formated\n    '''\n    pixels = img.flatten()\n    pixels = np.concatenate([[0], pixels, [0]])\n    runs = np.where(pixels[1:] != pixels[:-1])[0] + 1\n    runs[1::2] -= runs[::2]\n    return ' '.join(str(x) for x in runs)\n \ndef rle_decode(mask_rle, shape):\n    '''\n    mask_rle: run-length as string formated (start length)\n    shape: (height,width) of array to return \n    Returns numpy array, 1 - mask, 0 - background\n\n    '''\n    s = mask_rle.split()\n    starts, lengths = [np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])]\n    starts -= 1\n    ends = starts + lengths\n    img = np.zeros(shape[0]*shape[1], dtype=np.uint8)\n    for lo, hi in zip(starts, ends):\n        img[lo:hi] = 1\n    return img.reshape(shape).T # T for transpose and get mask in order","metadata":{"papermill":{"duration":0.046823,"end_time":"2023-11-25T14:21:58.233097","exception":false,"start_time":"2023-11-25T14:21:58.186274","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T15:58:18.381891Z","iopub.execute_input":"2024-02-18T15:58:18.382256Z","iopub.status.idle":"2024-02-18T15:58:18.392788Z","shell.execute_reply.started":"2024-02-18T15:58:18.382223Z","shell.execute_reply":"2024-02-18T15:58:18.391935Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Check RLE encoding decoding is mirror\n\nmask = np.zeros((768, 768))\nmask = cv2.rectangle(mask, (10,10), (50, 50), 255, -1)\nplt.imshow(mask)\nplt.show()\n\nrle_mask = rle_encode(mask)\nrle_mask_rle = rle_decode(rle_mask, (768,768))\nplt.imshow(rle_mask_rle)\nplt.show()","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-01-27T17:17:17.987935Z","iopub.execute_input":"2024-01-27T17:17:17.988312Z","iopub.status.idle":"2024-01-27T17:17:18.429239Z","shell.execute_reply.started":"2024-01-27T17:17:17.988284Z","shell.execute_reply":"2024-01-27T17:17:18.428355Z"}}},{"cell_type":"code","source":"# Display images sample and their associated annotations\nsegs = pd.read_csv('/kaggle/input/airbus-ship-detection/train_ship_segmentations_v2.csv')\n\nidxs_random = random.sample(range(len(segs)), 5)\nfor idx in idxs_random:\n    seg_img, seg_bbox_rle = segs.loc[idx,:]\n    img = plt.imread('/kaggle/input/airbus-ship-detection/train_v2/'+seg_img)\n\n    if pd.isna(seg_bbox_rle):\n        fig, ax = plt.subplots(1, 2)\n        seg_bbox = []\n        ax[0].imshow(np.zeros(img.shape))\n    else:\n        # Trouver tous les masques de l'image\n        lst_seg = segs[segs['ImageId']==seg_img]\n        fig, ax = plt.subplots(1, len(lst_seg)+1, figsize=(15, 15*len(lst_seg)))\n        for i in range(len(lst_seg)):\n            #print('ok')\n            seg_bbox = rle_decode(lst_seg.loc[lst_seg.index[i], 'EncodedPixels'], img.shape[0:2])\n            ax[i].imshow(seg_bbox)\n\n    ax[-1].imshow(img)","metadata":{"papermill":{"duration":4.738454,"end_time":"2023-11-25T14:22:03.005791","exception":false,"start_time":"2023-11-25T14:21:58.267337","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T15:58:18.394153Z","iopub.execute_input":"2024-02-18T15:58:18.394530Z","iopub.status.idle":"2024-02-18T15:58:23.378464Z","shell.execute_reply.started":"2024-02-18T15:58:18.394495Z","shell.execute_reply":"2024-02-18T15:58:23.377105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Install [MyDetectron2 library](https://github.com/vintel38/MyDetectron2)","metadata":{"papermill":{"duration":0.043227,"end_time":"2023-11-25T14:22:03.093587","exception":false,"start_time":"2023-11-25T14:22:03.050360","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!python -m pip install pyyaml==5.1\n# Detectron2 has not released pre-built binaries for the latest pytorch (https://github.com/facebookresearch/detectron2/issues/4053)\n# so we install from source instead. This takes a few minutes.\n!python -m pip install 'git+https://github.com/vintel38/MyDetectron2.git'","metadata":{"papermill":{"duration":null,"end_time":null,"exception":false,"start_time":"2023-11-25T14:22:03.183080","status":"running"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T15:58:23.380287Z","iopub.execute_input":"2024-02-18T15:58:23.380874Z","iopub.status.idle":"2024-02-18T16:00:21.191850Z","shell.execute_reply.started":"2024-02-18T15:58:23.380833Z","shell.execute_reply":"2024-02-18T16:00:21.190580Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch, detectron2\n!nvcc --version\nTORCH_VERSION = \".\".join(torch.__version__.split(\".\")[:2])\nCUDA_VERSION = torch.__version__.split(\"+\")[-1]\nprint(\"torch: \", TORCH_VERSION, \"; cuda: \", CUDA_VERSION)\nprint(\"detectron2:\", detectron2.__version__)","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T16:00:21.193497Z","iopub.execute_input":"2024-02-18T16:00:21.193832Z","iopub.status.idle":"2024-02-18T16:00:23.688267Z","shell.execute_reply.started":"2024-02-18T16:00:21.193804Z","shell.execute_reply":"2024-02-18T16:00:23.687066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Some basic setup:\n# Setup detectron2 logger\nimport detectron2\nfrom detectron2.utils.logger import setup_logger\nsetup_logger()\n\n# import some common libraries\nimport numpy as np\nimport os, json, cv2, random\n# from google.colab.patches import cv2_imshow \n# https://github.com/jupyter/notebook/issues/3935\n\n# import some common detectron2 utilities\nfrom detectron2 import model_zoo\nfrom detectron2.engine import DefaultPredictor\nfrom detectron2.config import get_cfg\nfrom detectron2.utils.visualizer import Visualizer\nfrom detectron2.data import MetadataCatalog, DatasetCatalog","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T16:00:23.690326Z","iopub.execute_input":"2024-02-18T16:00:23.691748Z","iopub.status.idle":"2024-02-18T16:00:24.263097Z","shell.execute_reply.started":"2024-02-18T16:00:23.691705Z","shell.execute_reply":"2024-02-18T16:00:24.262341Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Train split dry run\nds_train = segs.sample(50).copy()\nds_train.dropna(how='any', inplace=True)","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T16:00:24.266270Z","iopub.execute_input":"2024-02-18T16:00:24.266557Z","iopub.status.idle":"2024-02-18T16:00:24.280963Z","shell.execute_reply.started":"2024-02-18T16:00:24.266533Z","shell.execute_reply":"2024-02-18T16:00:24.280087Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in DatasetCatalog.list():\n    if i.startswith('ship'):\n        DatasetCatalog.remove(i)","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T16:00:24.282260Z","iopub.execute_input":"2024-02-18T16:00:24.282574Z","iopub.status.idle":"2024-02-18T16:00:24.293817Z","shell.execute_reply.started":"2024-02-18T16:00:24.282550Z","shell.execute_reply":"2024-02-18T16:00:24.292946Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Custom DataLoader for the Airbus Ship Detection Challenge\n# if your dataset is in COCO format, this cell can be replaced by the following three lines:\n# from detectron2.data.datasets import register_coco_instances\n# register_coco_instances(\"my_dataset_train\", {}, \"json_annotation_train.json\", \"path/to/image/dir\")\n# register_coco_instances(\"my_dataset_val\", {}, \"json_annotation_val.json\", \"path/to/image/dir\")\n\nfrom detectron2.structures import BoxMode\n\ndef get_ship_dicts(img_dir, ds_annot):\n\n    ds = ds_annot.copy()\n    dataset_dicts = []\n    while len(ds.index)>0:\n\n        # take the first annotations\n        idx = ds.index[0]\n        # and join all its annotations with the same name\n        annos = ds.loc[ds['ImageId']==ds.loc[idx, 'ImageId'],:]\n\n        record = {}\n        filename = os.path.join(img_dir, annos.loc[idx,\"ImageId\"])\n        height, width = cv2.imread(filename).shape[:2]\n\n        record[\"file_name\"] = filename\n        record[\"image_id\"] = len(dataset_dicts)\n        record[\"height\"] = height\n        record[\"width\"] = width\n\n        objs = []\n        for idx_annos in annos.index:\n\n            seg_img, seg_bbox_rle = annos.loc[idx_annos, :]\n            img = plt.imread('/kaggle/input/airbus-ship-detection/train_v2/'+seg_img)\n            if not str(seg_bbox_rle) == str('nan'):\n                seg_bbox = rle_decode(seg_bbox_rle, img.shape[0:2])\n                (contours,_) = cv2.findContours(seg_bbox, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n                # findContours can produce akward results : contour with only one or two points\n                # which cannot be considered as proper contour\n                obj={}\n                for contour in contours:\n                    if len(contour)<4:\n                        continue # avoid considering non useful contour and crash Detectron2\n                    rect = cv2.boundingRect(contour) # x,y,w,h\n                    obj = { # https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html\n                        \"bbox\": list(rect),\n                        \"bbox_mode\": BoxMode.XYWH_ABS,\n                        \"segmentation\": [[int(coord) for elt in contour for coord in list(elt[0])]],\n                            # int() to avoid serialization problem in cocoevaluator definition mask conversion : np.int32 is not suitable\n                        \"category_id\": 0,\n                    }\n                    objs.append(obj)\n        record[\"annotations\"] = objs\n        dataset_dicts.append(record)\n\n        # ds drop the annotations treated\n        ds.drop(index = annos.index, inplace=True)\n\n    return dataset_dicts\n\nd=''\nDatasetCatalog.register(\"ship_train\", lambda d=d: get_ship_dicts(\"/kaggle/input/airbus-ship-detection/train_v2/\", ds_train))\nMetadataCatalog.get(\"ship_train\").set(thing_classes=[\"ship\"])\nDatasetCatalog.register(\"ship_test\", lambda d=d: get_ship_dicts(\"/kaggle/input/airbus-ship-detection/train_v2/\", ds_val))\nMetadataCatalog.get(\"ship_test\").set(thing_classes=[\"ship\"])\nship_metadata = MetadataCatalog.get(\"ship_train\")","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T16:00:24.295018Z","iopub.execute_input":"2024-02-18T16:00:24.295250Z","iopub.status.idle":"2024-02-18T16:00:24.309102Z","shell.execute_reply.started":"2024-02-18T16:00:24.295230Z","shell.execute_reply":"2024-02-18T16:00:24.308200Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Test if annotations correctly computed\ndataset_dicts = get_ship_dicts(\"/kaggle/input/airbus-ship-detection/train_v2/\", ds_train)","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T16:00:24.310299Z","iopub.execute_input":"2024-02-18T16:00:24.310892Z","iopub.status.idle":"2024-02-18T16:00:24.760379Z","shell.execute_reply.started":"2024-02-18T16:00:24.310851Z","shell.execute_reply":"2024-02-18T16:00:24.759360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for d in random.sample(dataset_dicts, 5):\n    img = cv2.imread(d[\"file_name\"])\n    visualizer = Visualizer(img[:, :, ::-1], metadata=ship_metadata, scale=0.5)\n    # print(d)\n    out = visualizer.draw_dataset_dict(d)\n    plt.imshow(out.get_image()[:, :, ::-1])\n    plt.show()","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T16:00:24.761726Z","iopub.execute_input":"2024-02-18T16:00:24.762497Z","iopub.status.idle":"2024-02-18T16:00:26.939602Z","shell.execute_reply.started":"2024-02-18T16:00:24.762459Z","shell.execute_reply":"2024-02-18T16:00:26.938582Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# N-fold training phase on entire dataset \n\nImplementation choices: \n- It can be shown by hyperparameter tuning that higher NMS threshold cfg_cv.MODEL.ROI_HEADS.NMS_THRESH_TEST improves model performance but 0.5 reasonably seems a good threshold\n- [MyDetectron2](https://github.com/vintel38/MyDetectron2) library is a customized version of original FAIR Detectron2 library with evaluation loop and augmentations list added in the code. This allows every cfg.TEST.EVAL_PERIOD to compute several AP metrics by COCOEvaluator to watch the model progress during training. \n- It has been observed that COCOEvaluator requires the same evaluation dataset at each evaluation steps of training phase of a same model (this also applies to several trainer for a model) cf ship_test_coco_format.json file use. Consequently, the evaluation dataset is extracted from overall available data before any training process and put aside before split the remaining annotations in n-fold. \n- The use of checkpoint in Detectron2 saves and reuse training metrics but also the optimizer settings and the learning rate scheduler. The two later elements can be hardly modified during training process. So the scheduler has to be overwritten each time the training process is loading a new fold. ","metadata":{}},{"cell_type":"code","source":"# Split training and testing subset\nsegs = pd.read_csv('/kaggle/input/airbus-ship-detection/train_ship_segmentations_v2.csv')\n# Dans un premier temps on ne met pas d'images background dans le dataset d'entraînement\nnan_segs_idx = segs.loc[:,'EncodedPixels'].isna() # removing all empty row of folds\nsegs.drop(index = nan_segs_idx[nan_segs_idx].index, inplace=True)\nsegs.reset_index(drop=True, inplace=True)\n\n# on retire 300 images et leurs annotations associées de la base de données de segmentation pour construire le dataset de validation\ntest_size = 300\ntest_segs = segs.sample(n=test_size)\nds_val = pd.DataFrame({})\nfor seg in test_segs['ImageId']:\n    ds_tmp = segs.loc[segs['ImageId']==seg,:]\n    ds_val = pd.concat([ds_val, ds_tmp])\n    segs.drop(index=ds_tmp.index, inplace=True)\nds_val.reset_index(drop=True, inplace=True)\nsegs.reset_index(drop=True, inplace=True)\n\n# On divise les batch en fonction du nb de training iteration qu'on veut faire et du nb d'image par batch\nfolds = dict({})\nn_iter = 2000\nnb_batch = 4\n\ni=0\nwhile True:\n    if (i+1)*n_iter*nb_batch > len(segs):\n        folds['fold_'+str(i)] = segs.loc[[idx for idx in range(i*n_iter*nb_batch,len(segs))],:]\n        print(len(folds['fold_'+str(i)]))\n        n_fold=i\n        break\n    folds['fold_'+str(i)] = segs.loc[[idx for idx in range(i*n_iter*nb_batch,(i+1)*n_iter*nb_batch)],:]\n    print(len(folds['fold_'+str(i)]))\n    i=i+1","metadata":{"execution":{"iopub.status.busy":"2024-02-18T16:00:26.940985Z","iopub.execute_input":"2024-02-18T16:00:26.941338Z","iopub.status.idle":"2024-02-18T16:00:33.255711Z","shell.execute_reply.started":"2024-02-18T16:00:26.941311Z","shell.execute_reply":"2024-02-18T16:00:33.254768Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from detectron2.engine import CustomTrainer\nimport detectron2.data.transforms as T\nimport os\n\n# 1ST EPOCH : TRAIN ON EVERY SEGMENTATIONS AVAILABLE IN THE TRAINING DATASETS\nfor i in range(n_fold): # which folds are being process in this loop,\n\n    # reinitialize the dataset for iteration\n    for ctg in DatasetCatalog.list():\n        if ctg.startswith('ship'):\n            DatasetCatalog.remove(ctg)\n\n    # create the fold and register it\n    ds_train_cvi = folds['fold_'+str((i))]\n\n    d=''\n    DatasetCatalog.register(\"ship_train_cvi\", lambda d=d: get_ship_dicts(\"/kaggle/input/airbus-ship-detection/train_v2\", ds_train_cvi))\n    MetadataCatalog.get(\"ship_train_cvi\").set(thing_classes=[\"ship\"])\n    DatasetCatalog.register(\"ship_test_cvi\", lambda d=d: get_ship_dicts(\"/kaggle/input/airbus-ship-detection/train_v2\", ds_val))\n    MetadataCatalog.get(\"ship_test_cvi\").set(thing_classes=[\"ship\"])\n    ship_metadata = MetadataCatalog.get(\"ship_train_cvi\")\n\n    # train the neural network with usual config in resume = True style\n    if i==0: # initialisation\n        cfg_cv = get_cfg()\n        cfg_cv.merge_from_file(model_zoo.get_config_file(\"COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml\"))\n        cfg_cv.DATASETS.TRAIN = (\"ship_train_cvi\",)\n        cfg_cv.DATASETS.TEST = (\"ship_test_cvi\",)\n        cfg_cv.DATALOADER.NUM_WORKERS = 2\n        cfg_cv.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(\"COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml\")  # Let training initialize from model zoo\n        cfg_cv.SOLVER.IMS_PER_BATCH = 4\n        cfg_cv.SOLVER.BASE_LR = 0.001  # pick a good LR\n        cfg_cv.SOLVER.MAX_ITER = 2001\n        cfg_cv.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 128   # faster, and good enough for this toy dataset (default: 512)\n        cfg_cv.MODEL.ROI_HEADS.NUM_CLASSES = 1  # only has one class (roof). (see https://detectron2.readthedocs.io/tutorials/datasets.html#update-the-config-for-new-datasets)\n\n        # Non Max Suppression https://detectron2.readthedocs.io/en/latest/modules/config.html#config-references\n        cfg_cv.MODEL.ROI_HEADS.NMS_THRESH_TEST = 0.5\n\n        # VALIDATION STEPS DURING TRAINING LOOPS\n        cfg_cv.TEST.EVAL_PERIOD = 500\n        # https://eidos-ai.medium.com/training-on-detectron2-with-a-validation-set-and-plot-loss-on-it-to-avoid-overfitting-6449418fbf4e\n        \n        # Augmentations lists during training passed to build_detection_train_loader\n        cfg_cv.SOLVER.AUG_LIST = [T.RandomBrightness(0.8, 1.8),\n                                  T.RandomContrast(0.6, 1.3),\n                                  T.RandomFlip(prob=0.4, horizontal=False, vertical=True),\n                                  T.RandomFlip(prob=0.4, horizontal=True, vertical=False),\n                                  T.MinIoURandomCrop()]\n        \n        # LEARNING RATE\n        cfg_cv.SOLVER.LR_SCHEDULER_NAME = \"WarmupMultiStepLR\"\n        cfg_cv.SOLVER.STEPS = [2000, 4000, 6000, 8000, 10000]\n        cfg_cv.SOLVER.GAMMA = 0.9\n        cfg_cv.SOLVER.WARMUP_ITERS = 200\n\n        os.makedirs(cfg_cv.OUTPUT_DIR, exist_ok=True)\n        trainer_cv = CustomTrainer(cfg_cv)\n        trainer_cv.resume_or_load(resume=False)\n        trainer_cv.train()\n    else:\n        # New Trainer for each iteration\n        cfg_cv.MODEL.WEIGHTS = \"/kaggle/working/model_final.pth\"\n        cfg_cv.SOLVER.BASE_LR = max(0.001-(i-1)*2e-4, 5e-4)\n        cfg_cv.SOLVER.MAX_ITER = cfg_cv.SOLVER.MAX_ITER + 2000\n        cfg_cv.SOLVER.WARMUP_ITERS = 0\n\n        trainer_cv = CustomTrainer(cfg_cv)\n        trainer_cv.resume_or_load(resume=True)\n        trainer_cv.train()","metadata":{"execution":{"iopub.status.busy":"2024-02-18T16:00:33.257197Z","iopub.execute_input":"2024-02-18T16:00:33.257833Z","iopub.status.idle":"2024-02-18T16:26:46.100663Z","shell.execute_reply.started":"2024-02-18T16:00:33.257796Z","shell.execute_reply":"2024-02-18T16:26:46.099490Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# NTH EPOCH : TRAINING ON PERMUTATIONS OF THE FOLDS CREATED ABOVE\nfrom itertools import permutations\nimport random\n\nn_epoch = 1\n\nperm = list(permutations(range(n_fold)))\n\nfor _ in range(n_epoch):\n    \n    ds = segs.copy() # remaining segmentations used for training phase\n    fold_order = perm[random.randint(0, len(perm))]\n    \n    for idx in fold_order:\n        \n        for ctg in DatasetCatalog.list():\n            if ctg.startswith('ship_train'):\n                DatasetCatalog.remove(ctg)\n        \n        d=''\n        DatasetCatalog.register(\"ship_train_cvi\", lambda d=d: get_ship_dicts(\"/kaggle/input/airbus-ship-detection/train_v2\", folds['fold_'+str((idx))]))\n        MetadataCatalog.get(\"ship_train_cvi\").set(thing_classes=[\"ship\"])\n        # DatasetCatalog.register(\"ship_test_cvi\", lambda d=d: get_ship_dicts(\"/kaggle/input/airbus-ship-detection/train_v2\", ds_val))\n        # MetadataCatalog.get(\"ship_test_cvi\").set(thing_classes=[\"ship\"])\n        ship_metadata = MetadataCatalog.get(\"ship_train_cvi\")\n        \n        # New Trainer for each iteration\n        cfg_cv.MODEL.WEIGHTS = \"/kaggle/working/model_final.pth\"\n        # cfg_cv.SOLVER.BASE_LR = max(0.001-(i-1)*2e-4, 5e-4)\n        cfg_cv.SOLVER.MAX_ITER = cfg_cv.SOLVER.MAX_ITER + 2000\n        cfg_cv.SOLVER.WARMUP_ITERS = 0\n\n        trainer_cv = CustomTrainer(cfg_cv)\n        trainer_cv.resume_or_load(resume=True)\n        trainer_cv.train()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Look at training curves in tensorboard:\n%load_ext tensorboard\n%tensorboard --logdir /content/output","metadata":{}},{"cell_type":"markdown","source":"# Submission","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[]}},{"cell_type":"code","source":"import numpy as np\nimport tensorflow as tf\nimport matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2024-02-18T16:26:46.102405Z","iopub.execute_input":"2024-02-18T16:26:46.103436Z","iopub.status.idle":"2024-02-18T16:26:46.109311Z","shell.execute_reply.started":"2024-02-18T16:26:46.103400Z","shell.execute_reply":"2024-02-18T16:26:46.108182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# https://www.kaggle.com/code/awsaf49/sartorius-fix-overlap?scriptVersionId=77621407&cellId=5\ndef check_overlap(msk):\n    msk = msk.astype(bool).astype(np.uint8)\n    return np.any(np.sum(msk, axis=-1)>1)\n\nfrom operator import itemgetter\ndef fix_overlap(msks):\n    \"\"\"\n    Args:\n        mask: multi-channel mask, each channel is an instance of cell, shape:(520,704,None)\n    Returns:\n        multi-channel mask with non-overlapping values, shape:(520,704,None)\n    \"\"\"\n    msks = np.array(msks)\n    msk = np.zeros(msks.shape)\n    # sorting masks based on their filled areas\n    areas = [[(msks[:,:,i] != 0).sum().sum(), i] for i in range(msks.shape[2])]\n    # https://stackoverflow.com/questions/4174941/how-to-sort-a-list-of-lists-by-a-specific-index-of-the-inner-list\n    sort = [elt[1] for elt in sorted(areas, key=itemgetter(0))]\n    for i in range(msks.shape[2]):\n        msk[:,:,i] = msks[:,:,sort[-(i+1)]]\n    \n    msk = np.pad(msk, [[0,0],[0,0],[1,0]]) # add a whole empty mask at the beginning of the msk list\n    ins_len = msk.shape[-1]\n    msk = np.argmax(msk,axis=-1) # keep pixels of first masks in priority (implying masks are ordered by size)\n    # justifying the first mask being padded for representing background pixels as principal px\n    msk = tf.keras.utils.to_categorical(msk, num_classes=ins_len) # mask data formatting\n    msk = msk[...,1:] # removing first padded mask\n    msk = msk[...,np.any(msk, axis=(0,1))] # delete mask that could be empty at this point (have lost all their px)\n    return msk","metadata":{"execution":{"iopub.status.busy":"2024-02-18T16:26:46.110745Z","iopub.execute_input":"2024-02-18T16:26:46.111178Z","iopub.status.idle":"2024-02-18T16:26:46.127056Z","shell.execute_reply.started":"2024-02-18T16:26:46.111129Z","shell.execute_reply":"2024-02-18T16:26:46.125947Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Inference should use the config with parameters that are used in training\n# cfg now already contains everything we've set previously. We changed it a little bit for inference:\ncfg_cv.MODEL.WEIGHTS = os.path.join(cfg_cv.OUTPUT_DIR, \"model_final.pth\")  # path to the model we just trained\ncfg_cv.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5   # set a custom testing threshold\ncfg_cv.MODEL.ROI_HEADS.NMS_THRESH_TEST = 0.5\npredictor_cv = DefaultPredictor(cfg_cv)","metadata":{"execution":{"iopub.status.busy":"2024-02-18T16:26:46.128268Z","iopub.execute_input":"2024-02-18T16:26:46.128715Z","iopub.status.idle":"2024-02-18T16:26:47.205640Z","shell.execute_reply.started":"2024-02-18T16:26:46.128679Z","shell.execute_reply":"2024-02-18T16:26:47.204565Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lst_test = os.listdir('/kaggle/input/airbus-ship-detection/test_v2')\nsubmit = []\n\nfor img_name in tqdm(lst_test):\n    img = cv2.imread(os.path.join('/kaggle/input/airbus-ship-detection/test_v2', img_name))\n    img_width, img_height = img.shape[0:2]\n    outputs = predictor_cv(img)\n    predictions = outputs['instances'].to(\"cpu\")\n    if len(predictions)==0:\n        submit.append([img_name, float('nan')])\n    else:\n        mask = np.zeros((img_width, img_height, len(predictions)))\n        for idx in range(len(predictions)):\n            mask[:,:,idx] = predictions[idx].pred_masks.numpy()[0] # en s'inspirant du script de départ\n            \n        mask_fixed = fix_overlap(mask)\n        # fix overlap can potentially delete entire mask if they completely overlap with other mask\n        # so its length can change\n        \n        for i in range(mask_fixed.shape[-1]): \n            encoded_px = rle_encode(mask_fixed[:,:,i])\n            submit.append([img_name, encoded_px])\n\n\nsubmit_pd = pd.DataFrame(data=submit, columns=['ImageId', 'EncodedPixels'])\nsubmit_pd.to_csv('/kaggle/working/submission.csv', index=False)","metadata":{"papermill":{"duration":null,"end_time":null,"exception":null,"start_time":null,"status":"pending"},"tags":[],"execution":{"iopub.status.busy":"2024-02-18T16:26:47.207226Z","iopub.execute_input":"2024-02-18T16:26:47.208124Z","iopub.status.idle":"2024-02-18T16:49:20.176773Z","shell.execute_reply.started":"2024-02-18T16:26:47.208084Z","shell.execute_reply":"2024-02-18T16:49:20.175380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Optimization ideas\n\n- Do more (or better) epochs\n- Change Mask RCNN backbone (ResNet50 is known to have poor performance wrt current SOTA instance segmentation)\n- Implement Soft Max Suppression instead of NMS","metadata":{}},{"cell_type":"markdown","source":"# Notebook results\n\nRun\n23813.1s - GPU P100\n\nPrivate Score\n0.72538\n\nPublic Score\n0.47119","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}