{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **SIIM COVID-19 Detectron2 Training**","metadata":{"papermill":{"duration":0.022818,"end_time":"2021-08-10T14:42:17.504623","exception":false,"start_time":"2021-08-10T14:42:17.481805","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Inferance ,EDA and Dataset \n- [SIIM COVID-19 Detectron2 Inferance](https://www.kaggle.com/ammarnassanalhajali/siim-covid19-detectron2-inferance)\n- [SIIM-FISABIO-RSNA COVID-19 Detection-EDA](https://www.kaggle.com/ammarnassanalhajali/siim-fisabio-rsna-covid-19-detection-eda)\n- [SIIM-COVID-19 Detection Training Labels (Dataset)](https://www.kaggle.com/ammarnassanalhajali/siimcovid19-detection-training-label)\n","metadata":{"papermill":{"duration":0.020801,"end_time":"2021-08-10T14:42:17.546515","exception":false,"start_time":"2021-08-10T14:42:17.525714","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### Hi kagglers, This is `training` notebook using `Detectron2`.\n\n> #### Thanks:\n> - https://www.kaggle.com/xhlulu/siim-covid19-resized-to-256px-jpg\n\n\n### Please if this kernel is useful, <font color='red'>please upvote !!</font>","metadata":{"papermill":{"duration":0.022909,"end_time":"2021-08-10T14:42:17.593058","exception":false,"start_time":"2021-08-10T14:42:17.570149","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Detectron2\nDetectron2 is Facebook AI Research's next generation software system that implements state-of-the-art object detection algorithms. It is a ground-up rewrite of the previous version, Detectron, and it originates from maskrcnn-benchmark.","metadata":{"papermill":{"duration":0.028442,"end_time":"2021-08-10T14:42:17.64227","exception":false,"start_time":"2021-08-10T14:42:17.613828","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Installation\n* detectron2 is not pre-installed in this kaggle docker, so let's install it.\n* we need to know CUDA and pytorch version to install correct detectron2.","metadata":{"papermill":{"duration":0.031146,"end_time":"2021-08-10T14:42:17.711918","exception":false,"start_time":"2021-08-10T14:42:17.680772","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!nvidia-smi","metadata":{"papermill":{"duration":0.799508,"end_time":"2021-08-10T14:42:18.533171","exception":false,"start_time":"2021-08-10T14:42:17.733663","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:15.11317Z","iopub.execute_input":"2021-08-13T09:30:15.113721Z","iopub.status.idle":"2021-08-13T09:30:15.796675Z","shell.execute_reply.started":"2021-08-13T09:30:15.113605Z","shell.execute_reply":"2021-08-13T09:30:15.795727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!nvcc --version","metadata":{"papermill":{"duration":0.713009,"end_time":"2021-08-10T14:42:19.267843","exception":false,"start_time":"2021-08-10T14:42:18.554834","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:15.798108Z","iopub.execute_input":"2021-08-13T09:30:15.798397Z","iopub.status.idle":"2021-08-13T09:30:16.437027Z","shell.execute_reply.started":"2021-08-13T09:30:15.798368Z","shell.execute_reply":"2021-08-13T09:30:16.436137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch, torchvision\nprint(torch.__version__, torch.cuda.is_available())","metadata":{"papermill":{"duration":1.570835,"end_time":"2021-08-10T14:42:20.860038","exception":false,"start_time":"2021-08-10T14:42:19.289203","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:16.438664Z","iopub.execute_input":"2021-08-13T09:30:16.438921Z","iopub.status.idle":"2021-08-13T09:30:17.830088Z","shell.execute_reply.started":"2021-08-13T09:30:16.438893Z","shell.execute_reply":"2021-08-13T09:30:17.828718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* It seems CUDA=11.0 and torch==1.7.0 is used in this kaggle docker image.\n* See installation for details. https://detectron2.readthedocs.io/en/latest/tutorials/install.html","metadata":{"papermill":{"duration":0.02229,"end_time":"2021-08-10T14:42:20.904571","exception":false,"start_time":"2021-08-10T14:42:20.882281","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Install Pre-Built Detectron2","metadata":{"papermill":{"duration":0.022759,"end_time":"2021-08-10T14:42:20.952515","exception":false,"start_time":"2021-08-10T14:42:20.929756","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!pip install detectron2 -f \\\n  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html","metadata":{"_kg_hide-output":true,"papermill":{"duration":33.42711,"end_time":"2021-08-10T14:42:54.401459","exception":false,"start_time":"2021-08-10T14:42:20.974349","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:17.833757Z","iopub.execute_input":"2021-08-13T09:30:17.834112Z","iopub.status.idle":"2021-08-13T09:30:50.848681Z","shell.execute_reply.started":"2021-08-13T09:30:17.834075Z","shell.execute_reply":"2021-08-13T09:30:50.847678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Import Libraries","metadata":{"papermill":{"duration":0.036463,"end_time":"2021-08-10T14:42:54.475535","exception":false,"start_time":"2021-08-10T14:42:54.439072","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\nimport numpy as np \nimport pandas as pd \nfrom datetime import datetime\nimport time\nfrom tqdm import tqdm_notebook as tqdm # progress bar\nimport matplotlib.pyplot as plt\n\nimport os, json, cv2, random\nimport skimage.io as io\nimport copy\nimport pickle\nfrom pathlib import Path\nfrom typing import Optional\nfrom tqdm import tqdm\n\n# torch\nimport torch\n\n\n\n# Albumenatations\nimport albumentations as A\nfrom albumentations.pytorch.transforms import ToTensorV2\n\n#from pycocotools.coco import COCO\nfrom sklearn.model_selection import StratifiedKFold\n\n# glob\nfrom glob import glob\n\n# numba\nimport numba\nfrom numba import jit\n\nimport warnings\nwarnings.filterwarnings('ignore') #Ignore \"future\" warnings and Data-Frame-Slicing warnings.\n\n\n# detectron2\nfrom detectron2.structures import BoxMode\nfrom detectron2 import model_zoo\nfrom detectron2.config import get_cfg\nfrom detectron2.data import DatasetCatalog, MetadataCatalog\nfrom detectron2.engine import DefaultPredictor, DefaultTrainer, launch\nfrom detectron2.evaluation import COCOEvaluator\nfrom detectron2.structures import BoxMode\nfrom detectron2.utils.visualizer import ColorMode\nfrom detectron2.utils.logger import setup_logger\nfrom detectron2.utils.visualizer import Visualizer\n\nfrom detectron2.data import DatasetCatalog, MetadataCatalog, build_detection_test_loader, build_detection_train_loader\nfrom detectron2.data import detection_utils as utils\n\n\nfrom detectron2.data import DatasetCatalog, MetadataCatalog, build_detection_test_loader, build_detection_train_loader\nfrom detectron2.data import detection_utils as utils\nimport detectron2.data.transforms as T\nfrom detectron2.evaluation import COCOEvaluator, inference_on_dataset\n\nsetup_logger()","metadata":{"papermill":{"duration":2.918113,"end_time":"2021-08-10T14:42:57.430093","exception":false,"start_time":"2021-08-10T14:42:54.51198","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:50.855128Z","iopub.execute_input":"2021-08-13T09:30:50.85577Z","iopub.status.idle":"2021-08-13T09:30:53.69089Z","shell.execute_reply.started":"2021-08-13T09:30:50.855727Z","shell.execute_reply":"2021-08-13T09:30:53.690084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Loading","metadata":{"papermill":{"duration":0.036891,"end_time":"2021-08-10T14:42:57.506602","exception":false,"start_time":"2021-08-10T14:42:57.469711","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# --- Read data ---\nimgdir = \"../input/siim-covid19-resized-384512-and-640px/SIIM-COVID19-Resized/img_sz_640\" # Orginial Size\n# Read in the data CSV files\ntrain_df = pd.read_csv(\"../input/siimcovid19-detection-training-label/train_image_df.csv\")\nlen(train_df)","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.156897,"end_time":"2021-08-10T14:42:57.701116","exception":false,"start_time":"2021-08-10T14:42:57.544219","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:53.692205Z","iopub.execute_input":"2021-08-13T09:30:53.692563Z","iopub.status.idle":"2021-08-13T09:30:53.813118Z","shell.execute_reply.started":"2021-08-13T09:30:53.692526Z","shell.execute_reply":"2021-08-13T09:30:53.812207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# configs","metadata":{"papermill":{"duration":0.038116,"end_time":"2021-08-10T14:42:57.776572","exception":false,"start_time":"2021-08-10T14:42:57.738456","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# --- configs ---\nthing_classes = [\n    \"atypical\",\n    \"indeterminate\",\n    \"negative\",\n    \"typical\"\n]\n\ndebug=False\nsplit_mode=\"valid20\" # Or  valid20 all_train\n\n\ncategory_name_to_id = {class_name: index for index, class_name in enumerate(thing_classes)}\ncategory_name_to_id","metadata":{"papermill":{"duration":0.045683,"end_time":"2021-08-10T14:42:57.858952","exception":false,"start_time":"2021-08-10T14:42:57.813269","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:53.814378Z","iopub.execute_input":"2021-08-13T09:30:53.814743Z","iopub.status.idle":"2021-08-13T09:30:53.822784Z","shell.execute_reply.started":"2021-08-13T09:30:53.814706Z","shell.execute_reply":"2021-08-13T09:30:53.821628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data preparation\n* `detectron2` provides high-level API for training custom dataset.\n\nTo define custom dataset, we need to create **list of dict** (`dataset_dicts`) where each dict contains following:\n\n - file_name: file name of the image.\n - image_id: id of the image, index is used here.\n - height: height of the image.\n - width: width of the image.\n - annotation: This is the ground truth annotation data for object detection, which contains following\n     - bbox: bounding box pixel location with shape (n_boxes, 4)\n     - bbox_mode: `BoxMode.XYXY_ABS` is used here, meaning that absolute value of (x_min, y_min, x_max, y_max) annotation is used in the `bbox`.\n     - category_id: class label id for each bounding box, with shape (n_boxes,)\n\n`get_COVID19_data_dicts` is for train dataset preparation and `get_COVID19_data_dicts_test` is for test dataset preparation.","metadata":{"papermill":{"duration":0.03692,"end_time":"2021-08-10T14:42:57.933048","exception":false,"start_time":"2021-08-10T14:42:57.896128","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from glob import glob\n\ndef get_COVID19_data_dicts(\n    imgdir: Path,\n    train_df: pd.DataFrame,\n    use_cache: bool = True,\n    target_indices: Optional[np.ndarray] = None,\n    debug: bool = False,\n    data_type:str=\"train\"\n   \n):\n\n    cache_path = Path(\".\") / f\"dataset_dicts_cache_{data_type}.pkl\"\n    if not use_cache or not cache_path.exists():\n        print(\"Creating data...\")\n        df_meta = pd.read_csv(\"../input/siim-covid19-resized-384512-and-640px/SIIM-COVID19-Resized/img_sz_640/meta_sz_640.csv\")\n        train_meta=df_meta[df_meta.split==\"train\"]\n        if debug:\n            train_meta = train_meta.iloc[:100]  # For debug....\n\n        # Load 1 image to get image size.\n        image_id = train_meta.iloc[0,0]\n        #image_path = str(imgdir / \"train\" / f\"{image_id}.jpg\")\n        image_path = str(f'../input/siim-covid19-resized-384512-and-640px/SIIM-COVID19-Resized/img_sz_640/train/{image_id}.jpg')\n        image = cv2.imread(image_path)\n        resized_height, resized_width, ch = image.shape\n        print(f\"image shape: {image.shape}\")\n\n        dataset_dicts = []\n        for index, train_meta_row in tqdm(train_meta.iterrows(), total=len(train_meta)):\n            record = {}\n            image_id, height, width,s = train_meta_row.values\n            #filename = str(imgdir / \"train\" / f\"{image_id}.jpg\")\n            filename = str(f'../input/siim-covid19-resized-384512-and-640px/SIIM-COVID19-Resized/img_sz_640/train/{image_id}.jpg')\n            record[\"file_name\"] = filename\n            record[\"image_id\"] = image_id\n            record[\"height\"] = resized_height\n            record[\"width\"] = resized_width\n            objs = []\n            for index2, row in train_df.query(\"id == @image_id\").iterrows():\n                # print(row)\n                # print(row[\"class_name\"])\n                # class_name = row[\"class_name\"]\n                class_id = row[\"integer_label\"]\n                if class_id == 2: # NO class\n                    # It is \"No finding\"\n \n                    # Use this No finding class with the bbox covering all image area.\n                    bbox_resized = [0, 0, resized_width, resized_height]\n                    #bbox_resized = [50, 50, 200, 200]\n                    obj = {\n                        \"bbox\": bbox_resized,\n                        \"bbox_mode\": BoxMode.XYXY_ABS,\n                        \"category_id\": class_id,\n                    }\n                    objs.append(obj)\n                else:\n                    #bbox_original = [int(row[\"x_min\"]), int(row[\"y_min\"]), int(row[\"x_max\"]), int(row[\"y_max\"])]\n                    h_ratio = resized_height / height\n                    w_ratio = resized_width / width\n                    bbox_resized = [\n                        float(row[\"x_min\"]) * w_ratio,\n                        float(row[\"y_min\"]) * h_ratio,\n                        float(row[\"x_max\"]) * w_ratio,\n                        float(row[\"y_max\"]) * h_ratio,\n                    ]\n                    obj = {\n                        \"bbox\": bbox_resized,# bbox_original, #bbox_resized#\n                        \"bbox_mode\": BoxMode.XYXY_ABS,\n                        \"category_id\": class_id,\n                    }\n                    objs.append(obj)\n            record[\"annotations\"] = objs\n            dataset_dicts.append(record)\n        with open(cache_path, mode=\"wb\") as f:\n            pickle.dump(dataset_dicts, f)\n\n    print(f\"Load from cache {cache_path}\")\n    with open(cache_path, mode=\"rb\") as f:\n        dataset_dicts = pickle.load(f)\n    if target_indices is not None:\n        dataset_dicts = [dataset_dicts[i] for i in target_indices]\n    return dataset_dicts\n\n\ndef get_COVID19_data_dicts_test(\n    imgdir: Path, test_meta: pd.DataFrame, use_cache: bool = True, debug: bool = False,\n):\n    debug_str = f\"_debug{int(debug)}\"\n    cache_path = Path(\".\") / f\"dataset_dicts_cache_test.pkl\"\n    if not use_cache or not cache_path.exists():\n        print(\"Creating data...\")\n        # test_meta = pd.read_csv(imgdir / \"test_meta.csv\")\n        df_meta = pd.read_csv(\"../input/siim-covid19-resized-384512-and-640px/SIIM-COVID19-Resized/img_sz_640/meta_sz_640.csv\")\n        test_meta=df_meta[df_meta.split==\"test\"]\n        if debug:\n            test_meta = test_meta.iloc[:100]  # For debug....\n        # Load 1 image to get image size.\n        image_id = test_meta.iloc[0,0]\n        #image_path = str(imgdir / \"test\" / f\"{image_id}.jpg\")\n        image_path = str(f'../input/siim-covid19-resized-384512-and-640px/SIIM-COVID19-Resized/img_sz_640/test/{image_id}.jpg')\n        #image = cv2.imread(image_path)\n        #resized_height, resized_width, ch = image.shape\n        #print(f\"image shape: {image.shape}\")\n\n        dataset_dicts = []\n        for index, test_meta_row in tqdm(test_meta.iterrows(), total=len(test_meta)):\n            record = {}\n\n            image_id, height, width,s = test_meta_row.values\n            #filename = str(imgdir / \"test\" / f\"{image_id}.jpg\")\n            filename = str(f'../input/siim-covid19-resized-384512-and-640px/SIIM-COVID19-Resized/img_sz_640/test/{image_id}.jpg')\n            record[\"file_name\"] = filename\n            # record[\"image_id\"] = index\n            record[\"image_id\"] = image_id\n            record[\"height\"] = height\n            record[\"width\"] = width\n            # objs = []\n            # record[\"annotations\"] = objs\n            dataset_dicts.append(record)\n        with open(cache_path, mode=\"wb\") as f:\n            pickle.dump(dataset_dicts, f)\n\n    #print(f\"Load from cache {cache_path}\")\n    with open(cache_path, mode=\"rb\") as f:\n        dataset_dicts = pickle.load(f)\n    return dataset_dicts","metadata":{"papermill":{"duration":0.061066,"end_time":"2021-08-10T14:42:58.031025","exception":false,"start_time":"2021-08-10T14:42:57.969959","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:53.82743Z","iopub.execute_input":"2021-08-13T09:30:53.827969Z","iopub.status.idle":"2021-08-13T09:30:53.856829Z","shell.execute_reply.started":"2021-08-13T09:30:53.827927Z","shell.execute_reply":"2021-08-13T09:30:53.855816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if split_mode == \"all_train\":\n    DatasetCatalog.register(\n        \"COVID19_data_train\",\n        lambda: get_COVID19_data_dicts(\n            imgdir,\n            train_df,\n            debug=debug,\n            data_type=\"train\"\n        ),\n    )\n    MetadataCatalog.get(\"COVID19_data_train\").set(thing_classes=thing_classes)\n    \n    \n    dataset_dicts_train = DatasetCatalog.get(\"COVID19_data_train\")\n    metadata_dicts_train = MetadataCatalog.get(\"COVID19_data_train\")\n    \n    \nelif split_mode == \"valid20\":\n\n    n_dataset = len(\n        get_COVID19_data_dicts(\n            imgdir, train_df, debug=debug,data_type=\"All\"\n        )\n    )\n    n_train = int(n_dataset * 0.90)\n    print(\"n_dataset\", n_dataset, \"n_train\", n_train)\n    rs = np.random.RandomState(42)\n    inds = rs.permutation(n_dataset)\n    train_inds, valid_inds = inds[:n_train], inds[n_train:]\n\n    DatasetCatalog.register(\n        \"COVID19_data_train\",\n        lambda: get_COVID19_data_dicts(\n            imgdir,\n            train_df,\n            target_indices=train_inds,\n            debug=debug,\n            data_type=\"train\"\n        ),\n    )\n    MetadataCatalog.get(\"COVID19_data_train\").set(thing_classes=thing_classes)\n    \n\n    DatasetCatalog.register(\n        \"COVID19_data_valid\",\n        lambda: get_COVID19_data_dicts(\n            imgdir,\n            train_df,\n            target_indices=valid_inds,\n            debug=debug,\n            data_type=\"val\"\n            ),\n        )\n    MetadataCatalog.get(\"COVID19_data_valid\").set(thing_classes=thing_classes)\n    \n    dataset_dicts_train = DatasetCatalog.get(\"COVID19_data_train\")\n    metadata_dicts_train = MetadataCatalog.get(\"COVID19_data_train\")\n\n    dataset_dicts_valid = DatasetCatalog.get(\"COVID19_data_valid\")\n    metadata_dicts_valid = MetadataCatalog.get(\"COVID19_data_valid\")\n    \nelse:\n    raise ValueError(f\"[ERROR] Unexpected value split_mode={split_mode}\")","metadata":{"papermill":{"duration":56.994767,"end_time":"2021-08-10T14:43:55.063291","exception":false,"start_time":"2021-08-10T14:42:58.068524","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:30:53.866615Z","iopub.execute_input":"2021-08-13T09:30:53.867031Z","iopub.status.idle":"2021-08-13T09:31:48.956091Z","shell.execute_reply.started":"2021-08-13T09:30:53.866985Z","shell.execute_reply":"2021-08-13T09:31:48.955125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataset_dicts_train[0]","metadata":{"papermill":{"duration":0.183499,"end_time":"2021-08-10T14:43:55.424763","exception":false,"start_time":"2021-08-10T14:43:55.241264","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:31:48.957895Z","iopub.execute_input":"2021-08-13T09:31:48.95853Z","iopub.status.idle":"2021-08-13T09:31:48.965868Z","shell.execute_reply.started":"2021-08-13T09:31:48.958481Z","shell.execute_reply":"2021-08-13T09:31:48.964806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df=train_df[train_df.id==\"4e95d8ca7f2f\"]\ndf\n","metadata":{"papermill":{"duration":0.207405,"end_time":"2021-08-10T14:43:55.809003","exception":false,"start_time":"2021-08-10T14:43:55.601598","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:31:48.96747Z","iopub.execute_input":"2021-08-13T09:31:48.968193Z","iopub.status.idle":"2021-08-13T09:31:49.008043Z","shell.execute_reply.started":"2021-08-13T09:31:48.968149Z","shell.execute_reply":"2021-08-13T09:31:49.006991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"data_vis\"></a>\n# Data Visualization\n\nIt's also very easy to visualize prepared training dataset with `detectron2`.<br/>\nIt provides `Visualizer` class, we can use it to draw an image with bounding box as following.","metadata":{"papermill":{"duration":0.176063,"end_time":"2021-08-10T14:43:56.161841","exception":false,"start_time":"2021-08-10T14:43:55.985778","status":"completed"},"tags":[]}},{"cell_type":"code","source":"fig, ax = plt.subplots(2, 4, figsize =(20,10))\nindices=[ax[0][0],ax[1][0],ax[0][1],ax[1][1],ax[0][2],ax[1][2],ax[0][3],ax[1][3]]\ni=-1\nfor d in random.sample(dataset_dicts_train, 8):\n    i=i+1    \n    img = cv2.imread(d[\"file_name\"])\n    v = Visualizer(img[:, :, ::-1],\n                   metadata=metadata_dicts_train, \n                   scale=0.5, \n                   instance_mode=ColorMode.IMAGE_BW   # remove the colors of unsegmented pixels. This option is only available for segmentation models\n    )\n    out = v.draw_dataset_dict(d)\n    indices[i].grid(False)\n    indices[i].axis('off')\n    indices[i].imshow(out.get_image()[:, :, ::-1])","metadata":{"_kg_hide-input":true,"papermill":{"duration":1.329909,"end_time":"2021-08-10T14:43:57.66906","exception":false,"start_time":"2021-08-10T14:43:56.339151","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:31:49.009454Z","iopub.execute_input":"2021-08-13T09:31:49.009962Z","iopub.status.idle":"2021-08-13T09:31:50.287749Z","shell.execute_reply.started":"2021-08-13T09:31:49.009923Z","shell.execute_reply":"2021-08-13T09:31:50.286849Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Augmentation\nThe dataset is transformed by changing the brighness and flipping the image with 50% probability.","metadata":{"papermill":{"duration":0.185933,"end_time":"2021-08-10T14:43:58.041543","exception":false,"start_time":"2021-08-10T14:43:57.85561","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def custom_mapper(dataset_dict):\n    \n    dataset_dict = copy.deepcopy(dataset_dict)\n    image = utils.read_image(dataset_dict[\"file_name\"], format=\"BGR\")\n    transform_list = [#T.Resize((640,640)),\n                      T.RandomBrightness(0.8, 1.2),\n                      T.RandomFlip(prob=0.5, horizontal=False, vertical=True),\n                      T.RandomFlip(prob=0.5, horizontal=True, vertical=False)\n                      ]\n    image, transforms = T.apply_transform_gens(transform_list, image)\n    dataset_dict[\"image\"] = torch.as_tensor(image.transpose(2, 0, 1).astype(\"float32\"))\n\n    annos = [\n        utils.transform_instance_annotations(obj, transforms, image.shape[:2])\n        for obj in dataset_dict.pop(\"annotations\")\n        if obj.get(\"iscrowd\", 0) == 0\n    ]\n    instances = utils.annotations_to_instances(annos, image.shape[:2])\n    dataset_dict[\"instances\"] = utils.filter_empty_instances(instances)\n    return dataset_dict\nclass AugTrainer(DefaultTrainer):\n    @classmethod\n    def build_train_loader(cls, cfg):\n        return build_detection_train_loader(cfg, mapper=custom_mapper)","metadata":{"papermill":{"duration":0.198604,"end_time":"2021-08-10T14:43:58.428604","exception":false,"start_time":"2021-08-10T14:43:58.23","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:31:50.289075Z","iopub.execute_input":"2021-08-13T09:31:50.289501Z","iopub.status.idle":"2021-08-13T09:31:50.301712Z","shell.execute_reply.started":"2021-08-13T09:31:50.28946Z","shell.execute_reply":"2021-08-13T09:31:50.300705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\ntorch.cuda.empty_cache()","metadata":{"papermill":{"duration":0.325004,"end_time":"2021-08-10T14:43:58.939057","exception":false,"start_time":"2021-08-10T14:43:58.614053","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:31:50.30344Z","iopub.execute_input":"2021-08-13T09:31:50.30419Z","iopub.status.idle":"2021-08-13T09:31:50.312355Z","shell.execute_reply.started":"2021-08-13T09:31:50.304149Z","shell.execute_reply":"2021-08-13T09:31:50.311432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import gc\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"papermill":{"duration":0.349003,"end_time":"2021-08-10T14:43:59.58334","exception":false,"start_time":"2021-08-10T14:43:59.234337","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:31:50.313811Z","iopub.execute_input":"2021-08-13T09:31:50.314535Z","iopub.status.idle":"2021-08-13T09:31:50.476333Z","shell.execute_reply.started":"2021-08-13T09:31:50.314497Z","shell.execute_reply":"2021-08-13T09:31:50.475323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{"papermill":{"duration":0.18641,"end_time":"2021-08-10T14:43:59.959331","exception":false,"start_time":"2021-08-10T14:43:59.772921","status":"completed"},"tags":[]}},{"cell_type":"code","source":"cfg = get_cfg()\n#config_name = \"COCO-Detection/faster_rcnn_X_101_32x8d_FPN_3x.yaml\" #1\n#config_name = \"COCO-Detection/faster_rcnn_R_101_DC5_3x.yaml\"      #2\nconfig_name = \"COCO-Detection/faster_rcnn_R_101_C4_3x.yaml\"      #3\n\n\ncfg.merge_from_file(model_zoo.get_config_file(config_name))\ncfg.DATASETS.TRAIN = (\"COVID19_data_train\",)\n\nif split_mode == \"all_train\":\n    cfg.DATASETS.TEST = ()\nelse:\n    cfg.DATASETS.TEST = (\"COVID19_data_valid\",)\n    cfg.TEST.EVAL_PERIOD = 1000\n\ncfg.DATALOADER.NUM_WORKERS = 4\n#cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(config_name)\ncfg.MODEL.WEIGHTS=\"../input/1siim-covid19-detectron2-weights/output/model_final.pth\"\n\n\ncfg.SOLVER.IMS_PER_BATCH = 1\ncfg.SOLVER.BASE_LR = 0.00025\n\n\ncfg.SOLVER.WARMUP_ITERS = 1000\ncfg.SOLVER.MAX_ITER = 20000 #adjust up if val mAP is still rising, adjust down if overfit\n#cfg.SOLVER.STEPS = (100, 500) # must be less than  MAX_ITER \n#cfg.SOLVER.GAMMA = 0.05\n\n\ncfg.SOLVER.CHECKPOINT_PERIOD = 1000  # Small value=Frequent save need a lot of storage.\ncfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 512\ncfg.MODEL.ROI_HEADS.NUM_CLASSES = 4\n\n\nos.makedirs(cfg.OUTPUT_DIR, exist_ok=True)\n\n\n#Training using custom trainer defined above\ntrainer = AugTrainer(cfg) \n#trainer = DefaultTrainer(cfg) \ntrainer.resume_or_load(resume=False)\ntrainer.train()\n\n# Evaluator","metadata":{"_kg_hide-output":true,"papermill":{"duration":7253.926705,"end_time":"2021-08-10T16:44:54.072767","exception":false,"start_time":"2021-08-10T14:44:00.146062","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-08-13T09:31:50.477798Z","iopub.execute_input":"2021-08-13T09:31:50.478359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* Famouns dataset's evaluator is already implemented in detectron2.\n* For example, many kinds of AP (Average Precision) is calculted in COCOEvaluator.\n* COCOEvaluator only calculates AP with IoU from 0.50 to 0.95","metadata":{"papermill":{"duration":0.407285,"end_time":"2021-08-10T16:44:54.890609","exception":false,"start_time":"2021-08-10T16:44:54.483324","status":"completed"},"tags":[]}},{"cell_type":"code","source":"evaluator = COCOEvaluator(\"COVID19_data_valid\", cfg, False, output_dir=\"./output/\")\n#cfg.MODEL.WEIGHTS=\"./output/model_final.pth\"\n#cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.001   # set a custom testing threshold\nval_loader = build_detection_test_loader(cfg, \"COVID19_data_valid\")\ninference_on_dataset(trainer.model, val_loader, evaluator)","metadata":{"_kg_hide-input":true,"papermill":{"duration":94.44404,"end_time":"2021-08-10T16:46:29.744665","exception":false,"start_time":"2021-08-10T16:44:55.300625","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nmetrics_df = pd.read_json(\"./output/metrics.json\", orient=\"records\", lines=True)\nmdf = metrics_df.sort_values(\"iteration\")\nmdf.head(10).T","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.705343,"end_time":"2021-08-10T16:46:30.903154","exception":false,"start_time":"2021-08-10T16:46:30.197811","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 1. Loss curve\nfig, ax = plt.subplots()\n\nmdf1 = mdf[~mdf[\"total_loss\"].isna()]\nax.plot(mdf1[\"iteration\"], mdf1[\"total_loss\"], c=\"C0\", label=\"train\")\nif \"validation_loss\" in mdf.columns:\n    mdf2 = mdf[~mdf[\"validation_loss\"].isna()]\n    ax.plot(mdf2[\"iteration\"], mdf2[\"validation_loss\"], c=\"C1\", label=\"validation\")\n\n# ax.set_ylim([0, 0.5])\nax.legend()\nax.set_title(\"Loss curve\")\nplt.show()","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.590874,"end_time":"2021-08-10T16:46:31.938786","exception":false,"start_time":"2021-08-10T16:46:31.347912","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 1. Loss curve\nfig, ax = plt.subplots()\n\nmdf1 = mdf[~mdf[\"fast_rcnn/cls_accuracy\"].isna()]\nax.plot(mdf1[\"iteration\"], mdf1[\"fast_rcnn/cls_accuracy\"], c=\"C0\", label=\"train\")\n# ax.set_ylim([0, 0.5])\nax.legend()\nax.set_title(\"Accuracy curve\")\nplt.show()","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.564971,"end_time":"2021-08-10T16:46:32.929412","exception":false,"start_time":"2021-08-10T16:46:32.364441","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# References\n1. https://www.kaggle.com/ammarnassanalhajali/training-detectron2-for-blood-cells-detection\n1. https://www.kaggle.com/corochann/vinbigdata-detectron2-train\n","metadata":{"papermill":{"duration":0.422375,"end_time":"2021-08-10T16:46:33.782947","exception":false,"start_time":"2021-08-10T16:46:33.360572","status":"completed"},"tags":[]}}]}