{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://i.ibb.co/LP8YffP/lung-nb2-short.jpg)\n\n

Image Source: https://www.hopkinsmedicine.org/health/

\n\n

SIIM COVID-19 MMDetection+CascadeRCNN+Weight&Bias

","metadata":{}},{"cell_type":"markdown","source":"Overview\n\n  ✅  MMDetection and Weights & Biases Setup
\n  ✅  Preparation of MMDetection Config
\n  ✅  Weights & Biases Integration
\n  ✅  Training and Evaluation
\n  ✅  Experiment Tracking and Logging with Weights & Biases
\n  ✅  Storing Model Files as Artifacts
\n  ✅  Inference
\n  ✅  Interactive Analysis & Visualization
\n\n
\n\n### Version Notes:\nv5: Model `cascade_rcnn_r50_fpn_20e` | mAP 0.490\n\nv5: Model `cascade_rcnn_x101_32x4d_fpn_1x` | added more augmentations | 12 epochs | mAP 0.530+\n\n
\n\nMMDetection\n\nMMDetection is an open-source toolbox based on PyTorch for Object Detection and Segmentation tasks. The toolbox supports over **50+ baselines**.\n\n🧫 Features:\n\n

Modular Design: The framework is split into different components and we can easily construct a customized object detection framework by combining different modules.

\n\n

Support of multiple frameworks out of box: The toolbox directly supports popular and contemporary detection frameworks, e.g. Faster RCNN, Mask RCNN, RetinaNet, etc.

\n\n

High efficiency: All basic bbox and mask operations run on GPUs. The training speed is faster than or comparable to other codebases, including Detectron2, maskrcnn-benchmark and SimpleDet.

\n\n

State of the art: The toolbox stems from the codebase developed by the MMDet team, who won COCO Detection Challenge in 2018.

\n\n\nSupported Backbones:\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ResNet (CVPR'2016)RegNet (CVPR'2020)
ResNeXt (CVPR'2017)Res2Net (TPAMI'2020)
VGG (ICLR'2015)ResNeSt (ArXiv'2020)
HRNet (CVPR'2019)
\n\nSupported Methods:\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
RPN (NeurIPS'2015)NAS-FPN (CVPR'2019)Generalized Focal Loss (NeurIPS'2020)
Fast R-CNN (ICCV'2015)ATSS (CVPR'2020)CornerNet (ECCV'2018)
Faster R-CNN (NeurIPS'2015)FSAF (CVPR'2019)Side-Aware Boundary Localization (ECCV'2020)
Mask R-CNN (ICCV'2017)PAFPN (CVPR'2018)YOLOv3 (ArXiv'2018)
Cascade R-CNN (CVPR'2018)Dynamic R-CNN (ECCV'2020)PAA (ECCV'2020)
Cascade Mask R-CNN (CVPR'2018)PointRend (CVPR'2020)YOLACT (ICCV'2019)
SSD (ECCV'2016)CARAFE (ICCV'2019)CentripetalNet (CVPR'2020)
RetinaNet (ICCV'2017)DCNv2 (CVPR'2019)VFNet (ArXix'2020)
GHM (AAAI'2019)Group Normalization (ECCV'2018)DETR (ECCV'2020)
Mask Scoring R-CNN (CVPR'2019)Weight Standardization (ArXiv'2019)Deformable DETR (ICLR'2021)
Double-Head R-CNN (CVPR'2020)OHEM (CVPR'2016)CascadeRPN (NeurIPS'2019)
Hybrid Task Cascade (CVPR'2019)Soft-NMS (ICCV'2017)SCNet (AAAI'2021)
Libra R-CNN (CVPR'2019)Generalized Attention (ICCV'2019)AutoAssign (ArXix'2020)
Guided Anchoring (CVPR'2019)GCNet (ICCVW'2019)YOLOF (CVPR'2021)
FCOS (ICCV'2019)Mixed Precision (FP16) Training (ArXiv'2017)Seasaw Loss (CVPR'2021)
RepPoints (ICCV'2019)InstaBoost (ICCV'2019)CenterNet (CVPR'2019)
Foveabox (TIP'2020)GRoIE (ICPR'2020)
FreeAnchor (NeurIPS'2019)DetectoRS (ArXix'2020)
\n\n
\nNever Lose Track of ML Models with Weights & Biases \n\n\nWeights & Biases helps us build better models faster with a central dashboard for machine learning projects. It helps to log the hyperparameters, output metrics and images from runs, then visualize and compare results with ease from anywhere.\n\nIt helps a lot in managing and tracking experiments during rapid prototyping and development by enabling advanced monitoring and visualization of ML processes with minimal integration effort.\n\n**As Kaggle notebooks do not allow running aditional scripts in parallel or hosting pages for logging (which is required in Tensorboard or other IPython tools), Weights & Biases seems to be the most viable option for monitoring training live, while it is in progress.**\n\n**It also provides free SaaS hosting and 100 GB storage for a single member.**\n\n
\n\n![](https://i.ibb.co/ZhpRGJP/gif1.gif)\n\nHighlights:\n\n

Train Anywhere: The W&B dashboard is centralized - whether you're training on a local machine, lab cluster, or spot instances in the cloud, all of your results get logged to a single place.

\n\n

Visualize Seamlessly: Add W&B's lightweight integration to your existing ML code and quickly get live metrics, terminal logs, and system stats streamed to the centralized dashboard.\n

\n\n

Stay Organized: W&B logs data into powerful, querably tables that you can search, filter, sort, and group. This makes it easy to compare thousands of different models and find the best performing model for different tasks.

\n\n

Collaborate in Real-time: Explain how your model works, show graphs of how model versions improved, discuss bugs, and demonstrate progress towards milestones.

\n\n

Designed for all use cases: Never lose track of another ML project.

\n\n

Logged Automatically: System metrics such as the CPU and GPU utilization, network, etc are shown in the System tab on the run page. For the GPU, these are fetched with nvidia-smi. The stdout and stderr from the running shell, are picked up and shown in the logs tab on the run page.

\n\n
\n📌 Useful Resources\n\n

Notebook on processing annotations and generating COCO Dataset:

\n\nSIIM Covid-19 Resize & Process + Coco Dataset🌀💹\n\nhttps://www.kaggle.com/sreevishnudamodaran/siim-covid-19-resize-process-coco-dataset\n\n
\n

The images and the COCO Annotations used in this notebook:

\n\nSIIM covid19 512 images and metadata\n\nhttps://www.kaggle.com/sreevishnudamodaran/siim-covid19-512-images-and-metadata\n\nSIIM Covid-19 COCO 512x512 GroupKFold\n\nhttps://www.kaggle.com/sreevishnudamodaran/siim-covid19-coco-512x512-groupkfold\n\nI will update these datasets if needed, when the issues in the discussions regarding the dataset are resolved by the organizers.\n\n
\n🎆 References:\n\n- **Some amazing notebooks on Weights and Biases from [@ayuraj](https://www.kaggle.com/ayuraj)**\n - https://www.kaggle.com/ayuraj/visualize-bounding-boxes-interactively\n - https://www.kaggle.com/ayuraj/train-covid-19-detection-using-yolov5\n\n\n\n- https://www.kaggle.com/sreevishnudamodaran/vinbigdata-fusing-bboxes-coco-dataset\n\n
\n
\"Ask
\n
\n
\"Ask
","metadata":{}},{"cell_type":"markdown","source":"📐 Setup MMDetection","metadata":{}},{"cell_type":"code","source":"%%bash\n# Check nvcc version\nnvcc -V\necho\n# Check GCC version\ngcc --version\necho\n# Check the version of torch and cuda packages\npip list | grep \"torch\\|cuda\"","metadata":{"id":"320cDWMGgsVb","outputId":"86ab66f9-6812-4ba0-80e4-de882ba8cca1","execution":{"iopub.status.busy":"2021-06-01T11:25:42.986525Z","iopub.execute_input":"2021-06-01T11:25:42.986932Z","iopub.status.idle":"2021-06-01T11:25:44.99882Z","shell.execute_reply.started":"2021-06-01T11:25:42.98685Z","shell.execute_reply":"2021-06-01T11:25:44.997919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install mmcv-full==1.3.8 -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html","metadata":{"id":"RG3ZA4TYhFUx","outputId":"fd3c8d2b-da23-43aa-a8b1-4dab01cfdfcc","_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-01T11:25:45.00078Z","iopub.execute_input":"2021-06-01T11:25:45.001145Z","iopub.status.idle":"2021-06-01T11:25:59.173563Z","shell.execute_reply.started":"2021-06-01T11:25:45.001101Z","shell.execute_reply":"2021-06-01T11:25:59.172542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!rm -rf mmdetection\n!git clone https://github.com/open-mmlab/mmdetection.git\n!cd mmdetection && pip install -e .\n\n!pip install Pillow==7.0.0","metadata":{"id":"iJshzji7hHop","outputId":"7893d0dc-9fb0-46d2-c2b0-d95834125ff6","_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-01T11:25:59.17738Z","iopub.execute_input":"2021-06-01T11:25:59.177665Z","iopub.status.idle":"2021-06-01T11:26:29.673984Z","shell.execute_reply.started":"2021-06-01T11:25:59.177633Z","shell.execute_reply":"2021-06-01T11:26:29.672957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Setup Weights & Biases","metadata":{}},{"cell_type":"code","source":"!pip install wandb --upgrade","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-01T11:26:29.677949Z","iopub.execute_input":"2021-06-01T11:26:29.678215Z","iopub.status.idle":"2021-06-01T11:26:39.773144Z","shell.execute_reply.started":"2021-06-01T11:26:29.678183Z","shell.execute_reply":"2021-06-01T11:26:39.772075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nTo connect the Kaggle Notebook and log in to Weights & Biases, we need to create an API key:\n\n1. New users can sign up for a Free Weights & Biases account for Research and Personal use from the https://wandb.ai/site page. Sign up process takes around 1-2 minutes.\n2. Now get the API key from https://wandb.ai/authorize.\n\nLogin to Weights & Biases from the notebook with the API key by using any of the two methods below:\n\n* Interative:\n 1. Run a cell with wandb.login(). It will ask for the API key, which can be copied and pasted to authenticate.\n\n* Kaggle Secrets:\n 1. The recommended way to use the API key is to use Kaggle Secrets to store the API key. From the top Menu on the Notebook Editor, click on 'Add-ons' and then, select 'Secrets'.\n 2. Select 'Add a new secret' and provide 'wandb_key' for label and it's value as the API key obtained from the previous steps.","metadata":{}},{"cell_type":"code","source":"import wandb\nfrom kaggle_secrets import UserSecretsClient\n\n# I have saved my API token with \"wandb_api\" as Label. \n# If you use some other Label make sure to change the same below. \nuser_secrets = UserSecretsClient()\nsecret_value_0 = user_secrets.get_secret(\"fssf\")\n\nwandb.login(key=secret_value_0)","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:26:39.776718Z","iopub.execute_input":"2021-06-01T11:26:39.777013Z","iopub.status.idle":"2021-06-01T11:26:41.50944Z","shell.execute_reply.started":"2021-06-01T11:26:39.776981Z","shell.execute_reply":"2021-06-01T11:26:41.508454Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Imports and Seed Everything","metadata":{}},{"cell_type":"code","source":"import sys\nsys.path.insert(0, \"./mmdetection\")\n\nimport os\n# Check Pytorch installation\nimport torch, torchvision\nprint(torch.__version__, torch.cuda.is_available())\n\n# Check mmcv installation\nfrom mmcv.ops import get_compiling_cuda_version, get_compiler_version\nprint(get_compiling_cuda_version())\nprint(get_compiler_version())\n\n# Check MMDetection installation\nfrom mmdet.apis import set_random_seed\n\n# Imports\nimport mmdet\nfrom mmdet.apis import set_random_seed\nfrom mmdet.datasets import build_dataset\nfrom mmdet.models import build_detector\nfrom mmdet.apis import train_detector\n\nimport random\nimport numpy as np\nfrom pathlib import Path","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.616015Z","iopub.execute_input":"2021-06-01T11:57:37.616434Z","iopub.status.idle":"2021-06-01T11:57:37.624796Z","shell.execute_reply.started":"2021-06-01T11:57:37.616381Z","shell.execute_reply":"2021-06-01T11:57:37.623858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"global_seed = 111\n\ndef set_seed(seed=global_seed):\n \"\"\"Sets the random seeds.\"\"\"\n set_random_seed(seed, deterministic=False)\n random.seed(seed)\n np.random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.backends.cudnn.deterministic = True\n torch.backends.cudnn.benchmark = False\n os.environ['PYTHONHASHSEED'] = str(seed)\n \nset_seed()","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.644692Z","iopub.execute_input":"2021-06-01T11:57:37.644958Z","iopub.status.idle":"2021-06-01T11:57:37.652951Z","shell.execute_reply.started":"2021-06-01T11:57:37.644932Z","shell.execute_reply":"2021-06-01T11:57:37.652087Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"🔨 Prepare the MMDetection Config","metadata":{}},{"cell_type":"code","source":"from mmcv import Config\n\n# cfg = Config.fromfile('/kaggle/working/mmdetection/configs/vfnet/vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco.py')\n# cfg = Config.fromfile(\"/kaggle/working/mmdetection/configs/vfnet/vfnet_r50_fpn_mstrain_2x_coco.py\")\n# cfg = Config.fromfile(\"/kaggle/working/mmdetection/configs/gfl/gfl_r50_fpn_mstrain_2x_coco.py\")\n# baseline_cfg_path = \"/kaggle/working/mmdetection/configs/cascade_rcnn/cascade_rcnn_r50_fpn_20e_coco.py\"\nbaseline_cfg_path = \"/kaggle/working/mmdetection/configs/cascade_rcnn/cascade_rcnn_x101_32x4d_fpn_1x_coco.py\"\ncfg = Config.fromfile(baseline_cfg_path)","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.675013Z","iopub.execute_input":"2021-06-01T11:57:37.675338Z","iopub.status.idle":"2021-06-01T11:57:37.700707Z","shell.execute_reply.started":"2021-06-01T11:57:37.675308Z","shell.execute_reply":"2021-06-01T11:57:37.699879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"General Training Settings\n","metadata":{}},{"cell_type":"code","source":"# model_name = 'vfnet_r50_fpn'\n# model_name = 'cascade_rcnn_r50_fpn'\nmodel_name = 'cascade_rcnn_x101_32x4d_fpn_1x'\nfold = 0\njob = 4\n\n# Folder to store model logs and weight files\njob_folder = f'/kaggle/working/job{job}_{model_name}_fold{fold}'\ncfg.work_dir = job_folder\n\n# Change the wnd username and project name below\nwnb_username = 'sreevishnu-damodaran'\nwnb_project_name = 'siim-covid19-1'\n\n# Set seed thus the results are more reproducible\ncfg.seed = global_seed\n\nif not os.path.exists(job_folder):\n os.makedirs(job_folder)\n\nprint(\"Job folder:\", job_folder)","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.71792Z","iopub.execute_input":"2021-06-01T11:57:37.718207Z","iopub.status.idle":"2021-06-01T11:57:37.72432Z","shell.execute_reply.started":"2021-06-01T11:57:37.718179Z","shell.execute_reply":"2021-06-01T11:57:37.723491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set the number of classes\nfor head in cfg.model.roi_head.bbox_head:\n head.num_classes = 1\n# cfg.model.roi_head.bbox_head.num_classes = 1\n# cfg.model.bbox_head.num_classes = 1\n\n# cfg.gpu_ids = range(1)\ncfg.gpu_ids = [0]\n\n# Setting pretrained model in the init_cfg which is required \n# for transfer learning as per the latest MMdetection update\ncfg.model.backbone.init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')\ncfg.model.backbone.init_cfg=dict(type='Pretrained', checkpoint='open-mmlab://resnext101_32x4d')\ncfg.model.pop('pretrained', None)\n\ncfg.runner.max_epochs = 12 # Epochs for the runner that runs the workflow \ncfg.total_epochs = 12\n\n# Learning rate of optimizers. The LR is divided by 8 since the config file is originally for 8 GPUs\ncfg.optimizer.lr = 0.02/8\n\n## Learning rate scheduler config used to register LrUpdater hook\ncfg.lr_config = dict(\n policy='CosineAnnealing', # The policy of scheduler, also support CosineAnnealing, Cyclic, etc. Refer to details of supported LrUpdater from https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/lr_updater.py#L9.\n by_epoch=False,\n warmup='linear', # The warmup policy, also support `exp` and `constant`.\n warmup_iters=500, # The number of iterations for warmup\n warmup_ratio=0.001, # The ratio of the starting learning rate used for warmup\n min_lr=1e-07)\n\n# config to register logger hook\ncfg.log_config.interval = 20 # Interval to print the log\n\n# Config to set the checkpoint hook, Refer to https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/hooks/checkpoint.py for implementation.\ncfg.checkpoint_config.interval = 1 # The save interval is 1","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.755092Z","iopub.execute_input":"2021-06-01T11:57:37.755405Z","iopub.status.idle":"2021-06-01T11:57:37.764174Z","shell.execute_reply.started":"2021-06-01T11:57:37.755356Z","shell.execute_reply":"2021-06-01T11:57:37.763289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Configure Datasets for Training and Evaluation","metadata":{}},{"cell_type":"code","source":"cfg.dataset_type = 'CocoDataset' # Dataset type, this will be used to define the dataset\ncfg.classes = (\"Covid_Abnormality\",)\n\ncfg.data.train.img_prefix = '/kaggle/input/siim-covid19-512-images-and-metadata/train' # Prefix of image path\ncfg.data.train.classes = cfg.classes\ncfg.data.train.ann_file = f'/kaggle/input/siim-covid19-coco-512x512-groupkfold/train_annotations_fold{fold}.json'\ncfg.data.train.type='CocoDataset'\n\ncfg.data.val.img_prefix = '/kaggle/input/siim-covid19-512-images-and-metadata/train' # Prefix of image path\ncfg.data.val.classes = cfg.classes\ncfg.data.val.ann_file = f'/kaggle/input/siim-covid19-coco-512x512-groupkfold/val_annotations_fold{fold}.json'\ncfg.data.val.type='CocoDataset'\n\ncfg.data.test.img_prefix = '/kaggle/input/siim-covid19-512-images-and-metadata/train' # Prefix of image path\ncfg.data.test.classes = cfg.classes\ncfg.data.test.ann_file = f'/kaggle/input/siim-covid19-coco-512x512-groupkfold/val_annotations_fold{fold}.json'\ncfg.data.test.type='CocoDataset'\n\ncfg.data.samples_per_gpu = 4 # Batch size of a single GPU used in testing\ncfg.data.workers_per_gpu = 2 # Worker to pre-fetch data for each single GPU","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.798425Z","iopub.execute_input":"2021-06-01T11:57:37.798712Z","iopub.status.idle":"2021-06-01T11:57:37.805766Z","shell.execute_reply.started":"2021-06-01T11:57:37.798687Z","shell.execute_reply":"2021-06-01T11:57:37.804482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Setting Metric for Evaluation","metadata":{}},{"cell_type":"code","source":"# The config to build the evaluation hook, refer to https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/evaluation/eval_hooks.py#L7 for more details.\ncfg.evaluation.metric = 'bbox' # Metrics used during evaluation\n\n# Set the epoch intervel to perform evaluation\ncfg.evaluation.interval = 1\n\n# Set the iou threshold of the mAP calculation during evaluation\ncfg.evaluation.iou_thrs = [0.5]\n\n# cfg.evaluation.save_best='bbox_mAP_50'","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.810203Z","iopub.execute_input":"2021-06-01T11:57:37.810509Z","iopub.status.idle":"2021-06-01T11:57:37.820103Z","shell.execute_reply.started":"2021-06-01T11:57:37.810481Z","shell.execute_reply":"2021-06-01T11:57:37.819226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Prepare the Pre-processing & Augmentation Pipelines","metadata":{}},{"cell_type":"code","source":"albu_train_transforms = [\n dict(type='ShiftScaleRotate', shift_limit=0.0625,\n scale_limit=0.15, rotate_limit=15, p=0.4),\n dict(type='RandomBrightnessContrast', brightness_limit=0.2,\n contrast_limit=0.2, p=0.5),\n dict(type='IAAAffine', shear=(-10.0, 10.0), p=0.4),\n dict(type='MixUp', p=0.2, lambd=0.3),\n dict(type=\"Blur\", p=1.0, blur_limit=7),\n dict(type='CLAHE', p=0.5),\n dict(type='Equalize', mode='cv', p=0.4),\n dict(\n type=\"OneOf\",\n transforms=[\n dict(type=\"GaussianBlur\", p=1.0, blur_limit=7),\n dict(type=\"MedianBlur\", p=1.0, blur_limit=7),\n ],\n p=0.4,\n ),\n \n# dict(type='MixUp', p=0.2, lambd=0.5),\n# dict(type='RandomRotate90', p=0.5),\n# dict(type='CLAHE', p=0.5),\n# dict(type='InvertImg', p=0.5),\n# dict(type='Equalize', mode='cv', p=0.4),\n# dict(type='MedianBlur', blur_limit=3, p=0.1)\n ]\n\n\ncfg.train_pipeline = [\n dict(type='LoadImageFromFile'),\n dict(type='LoadAnnotations', with_bbox=True, with_mask=True),\n dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),\n dict(type='RandomFlip', flip_ratio=0.5),\n dict(\n type='Albu',\n transforms=albu_train_transforms,\n bbox_params=dict(\n type='BboxParams',\n format='pascal_voc',\n label_fields=['gt_labels'],\n min_visibility=0.0,\n filter_lost_elements=True),\n keymap=dict(img='image', gt_bboxes='bboxes'),\n update_pad_shape=False,\n skip_img_without_anno=True),\n dict(\n type='Normalize',\n mean=[123.675, 116.28, 103.53],\n std=[58.395, 57.12, 57.375],\n to_rgb=True),\n dict(type='Pad', size_divisor=32),\n dict(type='DefaultFormatBundle'),\n dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks'])\n]\ncfg.test_pipeline = [\n dict(type='LoadImageFromFile'),\n dict(\n type='MultiScaleFlipAug',\n img_scale=(1333, 800),\n flip=False,\n transforms=[\n dict(type='Resize', keep_ratio=True),\n dict(type='RandomFlip'),\n dict(\n type='Normalize',\n mean=[123.675, 116.28, 103.53],\n std=[58.395, 57.12, 57.375],\n to_rgb=True),\n dict(type='Pad', size_divisor=32),\n dict(type='ImageToTensor', keys=['img']),\n dict(type='Collect', keys=['img'])\n ])\n]","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.841304Z","iopub.execute_input":"2021-06-01T11:57:37.841678Z","iopub.status.idle":"2021-06-01T11:57:37.854088Z","shell.execute_reply.started":"2021-06-01T11:57:37.841646Z","shell.execute_reply":"2021-06-01T11:57:37.852857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Weights & Biases Integration for Experiment Tracking and Logging","metadata":{}},{"cell_type":"code","source":"## 4, 8\n# cfg.log_level = 'DEBUG'\ncfg.log_config.hooks = [dict(type='TextLoggerHook'),\n dict(type='WandbLoggerHook',\n init_kwargs=dict(project=wnb_project_name,\n name=f'exp-{model_name}-fold{fold}-job{job}',\n entity=wnb_username))\n ]","metadata":{"execution":{"iopub.status.busy":"2021-06-01T11:57:37.859739Z","iopub.execute_input":"2021-06-01T11:57:37.860024Z","iopub.status.idle":"2021-06-01T11:57:37.868441Z","shell.execute_reply.started":"2021-06-01T11:57:37.859997Z","shell.execute_reply":"2021-06-01T11:57:37.867246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Save Config File","metadata":{}},{"cell_type":"code","source":"cfg_path = f'{job_folder}/job{job}_{Path(baseline_cfg_path).name}'\nprint(cfg_path)\n\n# Save config file for inference later\ncfg.dump(cfg_path)\nprint(f'Config:\\n{cfg.pretty_text}')","metadata":{"id":"9C8s78L_hY2P","outputId":"5c532360-7684-42e1-bb2b-e59377576c2e","_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-06-01T11:57:37.878317Z","iopub.execute_input":"2021-06-01T11:57:37.87865Z","iopub.status.idle":"2021-06-01T11:57:40.378905Z","shell.execute_reply.started":"2021-06-01T11:57:37.878621Z","shell.execute_reply":"2021-06-01T11:57:40.377935Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"🚀 Build Dataset and Start Training","metadata":{}},{"cell_type":"code","source":"model = build_detector(cfg.model,\n train_cfg=cfg.get('train_cfg'),\n test_cfg=cfg.get('test_cfg'))\nmodel.init_weights()","metadata":{"id":"o6u_0gZuhcUy","outputId":"557284a4-a687-474a-aa4d-2abbb8fb403c","execution":{"iopub.status.busy":"2021-06-01T11:57:40.382169Z","iopub.execute_input":"2021-06-01T11:57:40.382481Z","iopub.status.idle":"2021-06-01T11:57:47.686161Z","shell.execute_reply.started":"2021-06-01T11:57:40.382449Z","shell.execute_reply":"2021-06-01T11:57:47.685205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"datasets = [build_dataset(cfg.data.train)]","metadata":{"id":"om_JbWv9heIQ","outputId":"0f264828-5b49-4a30-b035-9635424cd2d1","execution":{"iopub.status.busy":"2021-06-01T11:57:47.68811Z","iopub.execute_input":"2021-06-01T11:57:47.688463Z","iopub.status.idle":"2021-06-01T11:57:47.789265Z","shell.execute_reply.started":"2021-06-01T11:57:47.688427Z","shell.execute_reply":"2021-06-01T11:57:47.788324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"📌 Visit the Run page link in the cell below Ex: \"https://wandb.ai/sreevishnu-damodaran/siim-covid19-1/runs/\" as soon as the training starts to see the metrics live seamlessly in the Weights & Biases Dashboard.\n \nThe projects page https://wandb.ai/sreevishnu-damodaran/siim-covid19-1 compares it with other training jobs.","metadata":{}},{"cell_type":"code","source":"train_detector(model, datasets[0], cfg, distributed=False, validate=True)","metadata":{"id":"anIjmmhVhgKE","outputId":"9a97104b-5d08-4aa7-ce9c-be376e789ea5","execution":{"iopub.status.busy":"2021-06-01T11:57:47.79089Z","iopub.execute_input":"2021-06-01T11:57:47.7914Z","iopub.status.idle":"2021-06-01T13:08:32.336081Z","shell.execute_reply.started":"2021-06-01T11:57:47.791343Z","shell.execute_reply":"2021-06-01T13:08:32.334858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n\n📌 Please visit the Weights and Biases Dashboard (https://wandb.ai/sreevishnu-damodaran/siim-covid19-1) to see the current training progress and the results of some experiments which I ran previously.","metadata":{}},{"cell_type":"markdown","source":"Save Model Files in WnB as Artifacts\n\n
\n\n![](https://i.ibb.co/vhdzH4j/gif3.gif)","metadata":{}},{"cell_type":"code","source":"# Get the best epoch number\nfrom collections import defaultdict\n\nlog_file = f'{job_folder}/None.log.json'\n\n# Source: mmdetection/tools/analysis_tools/analyze_logs.py \ndef load_json_logs(json_logs):\n # load and convert json_logs to log_dict, key is epoch, value is a sub dict\n # keys of sub dict is different metrics, e.g. memory, bbox_mAP\n # value of sub dict is a list of corresponding values of all iterations\n log_dicts = [dict() for _ in json_logs]\n for json_log, log_dict in zip(json_logs, log_dicts):\n with open(json_log, 'r') as log_file:\n for line in log_file:\n log = json.loads(line.strip())\n # skip lines without `epoch` field\n if 'epoch' not in log:\n continue\n epoch = log.pop('epoch')\n if epoch not in log_dict:\n log_dict[epoch] = defaultdict(list)\n for k, v in log.items():\n log_dict[epoch][k].append(v)\n return log_dicts\n\nlog_dict = load_json_logs([log_file])\n# [(print(inner['bbox_mAP']) for inner in item) for item in log_dict]\n# [print(item) for item in log_dict[0]]\nbest_epoch = np.argmax([item['bbox_mAP'][0] for item in log_dict[0].values()])+1\nbest_epoch","metadata":{"execution":{"iopub.status.busy":"2021-07-16T20:13:19.834786Z","iopub.execute_input":"2021-07-16T20:13:19.835119Z","iopub.status.idle":"2021-07-16T20:13:19.856441Z","shell.execute_reply.started":"2021-07-16T20:13:19.835086Z","shell.execute_reply":"2021-07-16T20:13:19.855516Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_files = [f'{job_folder}/epoch_{best_epoch}.pth',\n cfg_path\n ]\n\n# Create a new wnb run for saving models as artifacts\nrun = wandb.init(project=wnb_project_name,\n name=f'models_files_{model_name}_fold{fold}_job{job}',\n entity=wnb_username,\n group='Artifact',\n job_type='model-files')\n\nartifact = wandb.Artifact(f'models_files_{model_name}_fold{fold}_job{job}', type='model')\n\nfor model_file in model_files:\n artifact.add_file(model_file)\n\nrun.log_artifact(artifact)\nrun.finish()","metadata":{"execution":{"iopub.status.busy":"2021-06-01T13:08:32.340664Z","iopub.execute_input":"2021-06-01T13:08:32.342823Z","iopub.status.idle":"2021-06-01T13:08:53.218949Z","shell.execute_reply.started":"2021-06-01T13:08:32.342746Z","shell.execute_reply":"2021-06-01T13:08:53.217832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"📰 Inference and Visualize Output","metadata":{}},{"cell_type":"code","source":"import mmcv\nfrom mmdet.models import build_detector\nfrom mmcv.runner import load_checkpoint\nfrom mmcv.parallel import MMDataParallel\nfrom mmdet.datasets import build_dataloader, build_dataset\nfrom mmdet.apis import single_gpu_test\nfrom mmdet.apis import init_detector, inference_detector, show_result_pyplot\n\nimport matplotlib.pyplot as plt\nimport pandas as pd\nfrom pathlib import Path\nimport cv2\nimport json","metadata":{"execution":{"iopub.status.busy":"2021-06-01T13:08:53.220548Z","iopub.execute_input":"2021-06-01T13:08:53.220908Z","iopub.status.idle":"2021-06-01T13:08:53.226675Z","shell.execute_reply.started":"2021-06-01T13:08:53.220868Z","shell.execute_reply":"2021-06-01T13:08:53.225717Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(\"../input/siim-covid19-coco-512x512-groupkfold/val_annotations_fold0.json\") as f:\n val_ann = json.load(f)\nimagepaths = [item['file_name'] for item in val_ann['images'][:9]]\n\ndf_annotations = pd.read_csv('../input/siim-covid19-512-images-and-metadata/df_train_processed_meta.csv')","metadata":{"execution":{"iopub.status.busy":"2021-06-01T13:08:53.228173Z","iopub.execute_input":"2021-06-01T13:08:53.228554Z","iopub.status.idle":"2021-06-01T13:08:53.284857Z","shell.execute_reply.started":"2021-06-01T13:08:53.228517Z","shell.execute_reply":"2021-06-01T13:08:53.284042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def draw_bbox(image,\n box,\n label,\n color,\n label_size = 0.5,\n alpha_box = 0.3,\n alpha_label = 0.6):\n \n overlay_bbox = image.copy()\n overlay_label = image.copy()\n output = image.copy()\n\n text_width, text_height = cv2.getTextSize(label.upper(),\n cv2.FONT_HERSHEY_SIMPLEX, label_size, 1)[0]\n cv2.rectangle(overlay_bbox, (box[0], box[1]), (box[2], box[3]),\n color, -1)\n cv2.addWeighted(overlay_bbox, alpha_box, output, 1-alpha_box, 0, output)\n \n cv2.rectangle(overlay_label, (box[0], box[1]-7-text_height),\n (box[0]+text_width+2, box[1]), (0, 0, 0), -1)\n cv2.addWeighted(overlay_label, alpha_label, output, 1-alpha_label, 0, output)\n output = cv2.rectangle(output, (box[0], box[1]), (box[2], box[3]),\n color, 2)\n cv2.putText(output, label.upper(), (box[0], box[1]-5),\n cv2.FONT_HERSHEY_SIMPLEX, label_size, (255, 255, 255), 1, cv2.LINE_AA)\n return output","metadata":{"execution":{"iopub.status.busy":"2021-06-01T13:08:53.288056Z","iopub.execute_input":"2021-06-01T13:08:53.28841Z","iopub.status.idle":"2021-06-01T13:08:53.299619Z","shell.execute_reply.started":"2021-06-01T13:08:53.288357Z","shell.execute_reply":"2021-06-01T13:08:53.298665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"checkpoint = f'{job_folder}/epoch_{best_epoch}.pth'\n\nprint(\"Loading weights from:\", checkpoint)\ncfg = Config.fromfile(cfg_path)\nmodel = init_detector(cfg, checkpoint, device='cuda:0')","metadata":{"execution":{"iopub.status.busy":"2021-06-01T13:08:53.301264Z","iopub.execute_input":"2021-06-01T13:08:53.301573Z","iopub.status.idle":"2021-06-01T13:08:54.519893Z","shell.execute_reply.started":"2021-06-01T13:08:53.301546Z","shell.execute_reply":"2021-06-01T13:08:54.51902Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"new_size = (512, 512)\nimgs_path = \"/kaggle/input/siim-covid19-512-images-and-metadata/train\"\nthreshold = 0.45\n\nfig, axes = plt.subplots(3,3, figsize=(19,21))\nfig.subplots_adjust(hspace=0.2, wspace=0.2)\naxes = axes.ravel()\n\nresults_list = []\n\nfor idx, img_id in enumerate(imagepaths):\n img_path = os.path.join(imgs_path, img_id)\n img = cv2.imread(img_path)\n result = inference_detector(model, img_path)\n results_filtered = result[0][result[0][:, 4]>threshold]\n bboxes = results_filtered[:, :4]\n scores = results_filtered[:, 4] \n results_list.append(result[0])\n \n for box in bboxes:\n img = draw_bbox(img, list(np.int_(box)), \"Covid_Abnormality\",\n (255, 243, 0))\n\n axes[idx].imshow(img, cmap='gray')\n axes[idx].set_title(img_id, size=18, pad=30)\n axes[idx].set_xticklabels([])\n axes[idx].set_yticklabels([])","metadata":{"execution":{"iopub.status.busy":"2021-06-01T13:08:54.524063Z","iopub.execute_input":"2021-06-01T13:08:54.526014Z","iopub.status.idle":"2021-06-01T13:08:57.110328Z","shell.execute_reply.started":"2021-06-01T13:08:54.525971Z","shell.execute_reply":"2021-06-01T13:08:57.109364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Interactively Visualize & Analyze Output in Dashboard","metadata":{}},{"cell_type":"code","source":"run = wandb.init(project=wnb_project_name,\n name=f'images-{model_name}-fold{fold}-job{job}',\n job_type='images')\n\nclass_id_to_label = {\n 1: \"pred_covid_abnormality\",\n 2: \"GT_covid_abnormality\"\n}\n\nwnb_images = []\n\nfor img_id, result in zip(imagepaths, results_list):\n \n bboxes = result[:, :4]\n scores = result[:, 4]\n ann_dict = {\"predictions\":{\n \"box_data\":[],\n \"class_labels\": class_id_to_label\n },\n \"ground_truth\":{\n \"box_data\":[],\n \"class_labels\": class_id_to_label\n }\n }\n\n for box, score in zip(bboxes, scores):\n single_data = {\n # one box expressed in the default relative/fractional domain\n \"position\": {\n \"minX\": round(float(box[0])/512, 3),\n \"maxX\": round(float(box[2])/512, 3),\n \"minY\": round(float(box[1])/512, 3),\n \"maxY\": round(float(box[3])/512, 3),\n },\n \"class_id\" : 1,\n \"box_caption\": class_id_to_label[1],\n \"scores\" : {\n \"confidence\": float(score),\n }\n }\n ann_dict[\"predictions\"][\"box_data\"].append(single_data)\n\n image_annotations = df_annotations[df_annotations.id==img_id.strip('.png')]\n\n for idxx, row in image_annotations[['xmin', 'ymin', 'xmax', 'ymax']].iterrows():\n single_data = {\n # one box expressed in the default relative/fractional domain\n \"position\": {\n \"minX\": round(float(row[0])/512, 3),\n \"maxX\": round(float(row[2])/512, 3),\n \"minY\": round(float(row[1])/512, 3),\n \"maxY\": round(float(row[3])/512, 3),\n },\n \"class_id\" : 2,\n \"box_caption\": class_id_to_label[2],\n \"scores\" : {\n \"confidence\": 1.0,\n }\n }\n ann_dict[\"ground_truth\"][\"box_data\"].append(single_data)\n\n image = cv2.imread(os.path.join(imgs_path, img_id))\n wnb_images.append(wandb.Image(image, boxes=ann_dict))\n \nwandb.log({f'images-{model_name}-fold{fold}-job{job}': wnb_images})\n\nrun.finish()\nrun","metadata":{"execution":{"iopub.status.busy":"2021-06-01T13:08:57.111645Z","iopub.execute_input":"2021-06-01T13:08:57.111952Z","iopub.status.idle":"2021-06-01T13:09:07.633874Z","shell.execute_reply.started":"2021-06-01T13:08:57.111919Z","shell.execute_reply":"2021-06-01T13:09:07.632878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"🚩 Click on the ⚙️ icon to change the bbox threshold\n\n🚩 Visit the training job run page and the images run page to see the interactive visualizations of training metrics and images in detail\n\n![](https://i.ibb.co/c3DFpmt/gif2.gif)\n","metadata":{}},{"cell_type":"code","source":"!rm -rf mmdetection/","metadata":{"execution":{"iopub.status.busy":"2021-06-01T13:09:07.635452Z","iopub.execute_input":"2021-06-01T13:09:07.635865Z","iopub.status.idle":"2021-06-01T13:09:07.842737Z","shell.execute_reply.started":"2021-06-01T13:09:07.63582Z","shell.execute_reply":"2021-06-01T13:09:07.841587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Additional Resources\n\n  🔖  [MMDetection Documentation](https://mmdetection.readthedocs.io/en/latest/)\n\n  🔖  [MDetection Github Repository](https://github.com/open-mmlab/mmdetection)\n\n  🔖  [Weights & Biases Documentation](https://docs.wandb.ai/)\n\n  🔖  [Weights & Biases Github Repository](https://github.com/wandb/examples)\n","metadata":{}},{"cell_type":"markdown","source":"

Let me know if you have any suggestions!

\n\n

THANKS!

","metadata":{}}]}