{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🐠 Reef - Pytorch Starter - FasterRCNN Train\n\n## A self-contained, simple, pure pytorch 🔥 Faster R-CNN implementation with `LB=0.413`\n\n![](https://storage.googleapis.com/kaggle-competitions/kaggle/31703/logos/header.png)\n\n#### FasterR-CNN is one of the SOTA models for Object detection.\n\n### In this notebook we present a simple solution using a pure pytorch Faster R-CNN with pretrained weights, and finetuning it for few epochs.\n\nIt is an adapted version of [this notebook](https://www.kaggle.com/pestipeti/pytorch-starter-fasterrcnn-train) mentioned in [this comment](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/290016).\n\n## You can find the [inference notebook here](https://www.kaggle.com/julian3833/coral-reef-pytorch-fasterrcnn-infer-0-xxx).\n\n## Details: \n- FasterRCNN from torchvision\n- Use Resnet50 backbone\n\n**Update**: Added simple train/validation split in this version, using the \"subsequence\" split of this notebook: [🐠 Reef - CV strategy: subsequences!](https://www.kaggle.com/julian3833/reef-cv-strategy-subsequences)\n\nStill dropping all the images with no objects, as the model doesn't support them out-of-the-box. The other starters are removing the empty images as well, so it might be a general condition of Object Detection. I'm quite noob in the field to be honest.\n\n# Please, _DO_ upvote if you find this useful!!\n\n\n&nbsp;\n&nbsp;\n&nbsp;\n\n#### Changelog\n\n| Version | Description| Dataset| Best LB |\n| --- | ----| --- | --- |\n| [**V8**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-293?scriptVersionId=80517118)  | 2 epochs - Save last epoch | [coral-reef-pytorch-starter-fasterrcnn-weights](https://www.kaggle.com/julian3833/coral-reef-pytorch-starter-fasterrcnn-weights)| `0.293`|\n| [**V16**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-293?scriptVersionId=80601095) | 4 epochs - Save all epochs | [reef-starter-torch-fasterrcnn-4e](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-4e)| `0.361` |\n| [**V17**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-369?scriptVersionId=80604402) | Add **validation**. 95-5 split. 8 epochs, keeping track of validation loss. | [reef-starter-torch-fasterrcnn-8e](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-8e)| `0.369` |\n| [**V19**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-369?scriptVersionId=80610403) | 12 epochs, lower LR | [reef-starter-torch-fasterrcnn-12e](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-12e)| `0.413` |\n| [**V24**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-369) | V19 with 90-10 train-validation split. Tidy up code. Add Flip. Correct problem with augmentations. | -- | `??` |\n\n\n","metadata":{}},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"# Very few imports. This is a pure torch solution!\nimport cv2\nimport time\n\nimport numpy as np\nimport pandas as pd\nfrom matplotlib import pyplot as plt\n\nimport albumentations as A\nfrom albumentations.pytorch.transforms import ToTensorV2\n\nimport torch\nimport torchvision\nfrom torch.utils.data import DataLoader, Dataset\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor\nfrom torchvision.models.detection import FasterRCNN","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","execution":{"iopub.status.busy":"2022-01-16T09:59:29.691229Z","iopub.execute_input":"2022-01-16T09:59:29.691571Z","iopub.status.idle":"2022-01-16T09:59:32.095943Z","shell.execute_reply.started":"2022-01-16T09:59:29.691518Z","shell.execute_reply":"2022-01-16T09:59:32.095018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Constants","metadata":{}},{"cell_type":"code","source":"DEVICE = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')\nBASE_DIR = \"../input/tensorflow-great-barrier-reef/train_images/\"\n\nNUM_EPOCHS = 12","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:34:26.268136Z","iopub.execute_input":"2022-01-15T12:34:26.268552Z","iopub.status.idle":"2022-01-15T12:34:26.274683Z","shell.execute_reply.started":"2022-01-15T12:34:26.268476Z","shell.execute_reply":"2022-01-15T12:34:26.273149Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load `df`\n\n### See: [🐠 Reef - CV strategy: subsequences!](https://www.kaggle.com/julian3833/reef-cv-strategy-subsequences)","metadata":{}},{"cell_type":"code","source":"import ast\ndf = pd.read_csv(\"../input/reef-cv-strategy-subsequences-dataframes/train-validation-split/train-0.1.csv\")\n\n# string으로 되어있는 annotation을 list of dictionaris로 변환 (사실 train dataset 만들때 미리 해둠)\ndf['annotations'] = df['annotations'].apply(ast.literal_eval)\n\n# Create the image path for the row\ndf['image_path'] = \"video_\" + df['video_id'].astype(str) + \"/\" + df['video_frame'].astype(str) + \".jpg\"\n\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-16T10:03:24.835843Z","iopub.execute_input":"2022-01-16T10:03:24.836216Z","iopub.status.idle":"2022-01-16T10:03:25.406646Z","shell.execute_reply.started":"2022-01-16T10:03:24.836158Z","shell.execute_reply":"2022-01-16T10:03:25.405904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# is_Train = True -> df_train / False -> df_val\ndf_train, df_val = df[df['is_train']], df[~df['is_train']]","metadata":{"execution":{"iopub.status.busy":"2022-01-16T10:03:29.956989Z","iopub.execute_input":"2022-01-16T10:03:29.957334Z","iopub.status.idle":"2022-01-16T10:03:29.967465Z","shell.execute_reply.started":"2022-01-16T10:03:29.957279Z","shell.execute_reply":"2022-01-16T10:03:29.966493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train[df_train.annotations.str.len()== 0]","metadata":{"execution":{"iopub.status.busy":"2022-01-16T10:03:32.627639Z","iopub.execute_input":"2022-01-16T10:03:32.628009Z","iopub.status.idle":"2022-01-16T10:03:32.672540Z","shell.execute_reply.started":"2022-01-16T10:03:32.627940Z","shell.execute_reply":"2022-01-16T10:03:32.671826Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# The model doesn't support images with no annotations\n# It raises an error that suggest that it just doesn't support them:\n# ValueError: No ground-truth boxes available for one of the images during training\n# I'm dropping those images for now\n# https://discuss.pytorch.org/t/fasterrcnn-images-with-no-objects-present-cause-an-error/117974/3\n\n# annotations가 없으면 모델이 에러나기 때문에 annotations의 길이가 0이상인 것만 남깁니다.\ndf_train = df_train[df_train.annotations.str.len() > 0 ].reset_index(drop=True)\ndf_val = df_val[df_val.annotations.str.len() > 0 ].reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2022-01-16T10:03:41.619732Z","iopub.execute_input":"2022-01-16T10:03:41.620088Z","iopub.status.idle":"2022-01-16T10:03:41.634642Z","shell.execute_reply.started":"2022-01-16T10:03:41.620039Z","shell.execute_reply":"2022-01-16T10:03:41.633613Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.shape[0], df_val.shape[0]","metadata":{"execution":{"iopub.status.busy":"2022-01-16T10:03:57.639064Z","iopub.execute_input":"2022-01-16T10:03:57.639401Z","iopub.status.idle":"2022-01-16T10:03:57.644952Z","shell.execute_reply.started":"2022-01-16T10:03:57.639349Z","shell.execute_reply":"2022-01-16T10:03:57.644020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-16T10:04:00.879030Z","iopub.execute_input":"2022-01-16T10:04:00.879363Z","iopub.status.idle":"2022-01-16T10:04:00.896758Z","shell.execute_reply.started":"2022-01-16T10:04:00.879313Z","shell.execute_reply":"2022-01-16T10:04:00.895774Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"row = df_train.iloc[0]\nboxes = pd.DataFrame(row['annotations'], columns=['x', 'y', 'width', 'height']).astype(float).values\nboxes[:, 2] = boxes[:, 0] + boxes[:, 2] # x_max\nboxes[:, 3] = boxes[:, 1] + boxes[:, 3] # y_max\nprint(\"[x_min, y_min, x_max, y_max]\",boxes) \nbox_outside_image = ((boxes[:, 0] < 0).any() or (boxes[:, 1] < 0).any() \n                             or (boxes[:, 2] > 1280).any() or (boxes[:, 3] > 720).any())\nprint(box_outside_image)","metadata":{"execution":{"iopub.status.busy":"2022-01-16T10:08:14.054829Z","iopub.execute_input":"2022-01-16T10:08:14.055218Z","iopub.status.idle":"2022-01-16T10:08:14.067639Z","shell.execute_reply.started":"2022-01-16T10:08:14.055164Z","shell.execute_reply":"2022-01-16T10:08:14.066776Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Dataset class","metadata":{}},{"cell_type":"code","source":"class ReefDataset:\n\n    def __init__(self, df, transforms=None):\n        self.df = df\n        self.transforms = transforms\n\n    def can_augment(self, boxes):\n        \"\"\" Check if bounding boxes are OK to augment\n        augmentation이 가능한지 확인 하는 함수입니다. annotation이 image의 영역 밖으로 나가지 않도록 합니다.\n        \n        For example: image_id 1-490 has a bounding box that is partially outside of the image\n        It breaks albumentation\n        Here we check the margins are within the image to make sure the augmentation can be applied\n        \"\"\"\n        \n        box_outside_image = ((boxes[:, 0] < 0).any() or (boxes[:, 1] < 0).any() \n                             or (boxes[:, 2] > 1280).any() or (boxes[:, 3] > 720).any())\n        return not box_outside_image\n\n    def get_boxes(self, row):\n        \"\"\"\n        3D foramt의 bboxes를 반환합니다.\n        Returns the bboxes for a given row as a 3D matrix with format [x_min, y_min, x_max, y_max]\n        \"\"\"\n        \n        boxes = pd.DataFrame(row['annotations'], columns=['x', 'y', 'width', 'height']).astype(float).values\n        \n        # Change from [x_min, y_min, w, h] to [x_min, y_min, x_max, y_max]\n        boxes[:, 2] = boxes[:, 0] + boxes[:, 2] # x_max\n        boxes[:, 3] = boxes[:, 1] + boxes[:, 3] # y_max\n        return boxes\n    \n    def get_image(self, row):\n        \"\"\"Gets the image for a given row\"\"\"\n        \n        image = cv2.imread(f'{BASE_DIR}/{row[\"image_path\"]}', cv2.IMREAD_COLOR)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n        image /= 255.0 # normalization\n        return image\n    \n    def __getitem__(self, i):\n\n        row = self.df.iloc[i]\n        image = self.get_image(row)\n        boxes = self.get_boxes(row)\n        \n        n_boxes = boxes.shape[0]\n        \n        # Calculate the area\n        area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0]) # (x_max - x_min) * (y_max - y_min)\n        \n        \n        target = {\n            'boxes': torch.as_tensor(boxes, dtype=torch.float32),\n            'area': torch.as_tensor(area, dtype=torch.float32),\n            \n            'image_id': torch.tensor([i]),\n            \n            # There is only one class\n            'labels': torch.ones((n_boxes,), dtype=torch.int64),\n            \n            # Suppose all instances are not crowd\n            'iscrowd': torch.zeros((n_boxes,), dtype=torch.int64)            \n        }\n\n        if self.transforms and self.can_augment(boxes): #transform이 있고 augmentation이 가능하다면...\n            sample = {\n                'image': image,\n                'bboxes': target['boxes'],\n                'labels': target['labels']\n            }\n            sample = self.transforms(**sample)\n            image = sample['image']\n            \n            if n_boxes > 0: # 타겟이 여러 개일 경우 stack함수를 이용해 쌓아올림 map함수로 tensor 변환\n                target['boxes'] = torch.stack(tuple(map(torch.tensor, zip(*sample['bboxes'])))).permute(1, 0)\n        else: # augmentation이 안된다면 그대로 tensor로 변환\n            image = ToTensorV2(p=1.0)(image=image)['image']\n\n        return image, target\n\n    def __len__(self):\n        return len(self.df)","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:42:58.283053Z","iopub.execute_input":"2022-01-15T12:42:58.283506Z","iopub.status.idle":"2022-01-15T12:42:58.308771Z","shell.execute_reply.started":"2022-01-15T12:42:58.283444Z","shell.execute_reply":"2022-01-15T12:42:58.307418Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Augmentations","metadata":{}},{"cell_type":"code","source":"# train시에만 augmentation 진행\ndef get_train_transform():\n    return A.Compose([\n        A.Flip(0.5),\n        ToTensorV2(p=1.0)\n    ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})\n\n\ndef get_valid_transform():\n    return A.Compose([\n        ToTensorV2(p=1.0)\n    ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:43:01.825124Z","iopub.execute_input":"2022-01-15T12:43:01.825565Z","iopub.status.idle":"2022-01-15T12:43:01.833279Z","shell.execute_reply.started":"2022-01-15T12:43:01.825488Z","shell.execute_reply":"2022-01-15T12:43:01.831601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Define datasets\nds_train = ReefDataset(df_train, get_train_transform())\nds_val = ReefDataset(df_val, get_valid_transform())","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:43:10.855344Z","iopub.execute_input":"2022-01-15T12:43:10.855765Z","iopub.status.idle":"2022-01-15T12:43:10.8636Z","shell.execute_reply.started":"2022-01-15T12:43:10.855699Z","shell.execute_reply":"2022-01-15T12:43:10.862421Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Check one sample","metadata":{}},{"cell_type":"code","source":"# Let's get an interesting one ;)\ndf_train[df_train.annotations.str.len() > 12].head()","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:43:17.944822Z","iopub.execute_input":"2022-01-15T12:43:17.945231Z","iopub.status.idle":"2022-01-15T12:43:18.002954Z","shell.execute_reply.started":"2022-01-15T12:43:17.945168Z","shell.execute_reply":"2022-01-15T12:43:18.001561Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image, targets = ds_train[2200]\nimage","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:43:26.231973Z","iopub.execute_input":"2022-01-15T12:43:26.232414Z","iopub.status.idle":"2022-01-15T12:43:26.42288Z","shell.execute_reply.started":"2022-01-15T12:43:26.232335Z","shell.execute_reply":"2022-01-15T12:43:26.421883Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"targets","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:43:29.855531Z","iopub.execute_input":"2022-01-15T12:43:29.856051Z","iopub.status.idle":"2022-01-15T12:43:29.869746Z","shell.execute_reply.started":"2022-01-15T12:43:29.855991Z","shell.execute_reply":"2022-01-15T12:43:29.868199Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"boxes = targets['boxes'].cpu().numpy().astype(np.int32)\nimg = image.permute(1,2,0).cpu().numpy()\nfig, ax = plt.subplots(1, 1, figsize=(16, 8))\n\nfor box in boxes:\n    cv2.rectangle(img,\n                  (box[0], box[1]),\n                  (box[2], box[3]),\n                  (220, 0, 0), 3)\n    \nax.set_axis_off()\nax.imshow(img);","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:43:39.226535Z","iopub.execute_input":"2022-01-15T12:43:39.227011Z","iopub.status.idle":"2022-01-15T12:43:39.744167Z","shell.execute_reply.started":"2022-01-15T12:43:39.226938Z","shell.execute_reply":"2022-01-15T12:43:39.742779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## DataLoaders","metadata":{}},{"cell_type":"code","source":"def collate_fn(batch):\n    return tuple(zip(*batch))\n\ndl_train = DataLoader(ds_train, batch_size=8, shuffle=False, num_workers=4, collate_fn=collate_fn)\ndl_val = DataLoader(ds_val, batch_size=8, shuffle=False, num_workers=4, collate_fn=collate_fn)","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:43:59.369194Z","iopub.execute_input":"2022-01-15T12:43:59.369632Z","iopub.status.idle":"2022-01-15T12:43:59.376999Z","shell.execute_reply.started":"2022-01-15T12:43:59.369547Z","shell.execute_reply":"2022-01-15T12:43:59.375738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Create the model","metadata":{}},{"cell_type":"code","source":"def get_model():\n    # load a model; pre-trained on COCO\n    model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\n\n    num_classes = 2  # 1 class (starfish) + background\n\n    # get number of input features for the classifier\n    in_features = model.roi_heads.box_predictor.cls_score.in_features\n\n    # replace the pre-trained head with a new one\n    model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)\n\n    model.to(DEVICE)\n    return model\n\nmodel = get_model()","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:44:08.000794Z","iopub.execute_input":"2022-01-15T12:44:08.001183Z","iopub.status.idle":"2022-01-15T12:44:14.125502Z","shell.execute_reply.started":"2022-01-15T12:44:08.001107Z","shell.execute_reply":"2022-01-15T12:44:14.124493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train","metadata":{}},{"cell_type":"code","source":"params = [p for p in model.parameters() if p.requires_grad]\noptimizer = torch.optim.SGD(params, lr=0.0025, momentum=0.9, weight_decay=0.0005)\n# lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=3, gamma=0.1)\nlr_scheduler = None\n\nn_batches, n_batches_val = len(dl_train), len(dl_val)\nvalidation_losses = []\n\n\nfor epoch in range(NUM_EPOCHS):\n    time_start = time.time()\n    loss_accum = 0\n    \n    for batch_idx, (images, targets) in enumerate(dl_train, 1):\n         \n        images = list(image.to(DEVICE) for image in images)\n        targets = [{k: v.to(DEVICE) for k, v in t.items()} for t in targets]\n\n        # Predict\n        loss_dict = model(images, targets)\n        losses = sum(loss for loss in loss_dict.values())\n        loss_value = losses.item()\n\n        loss_accum += loss_value\n\n        # Back-prop\n        optimizer.zero_grad()\n        losses.backward()\n        optimizer.step()\n\n    \n    # update the learning rate\n    if lr_scheduler is not None:\n        lr_scheduler.step()\n\n    # Validation \n    val_loss_accum = 0\n        \n    # Validation \n    with torch.no_grad():\n        for batch_idx, (images, targets) in enumerate(dl_val, 1):\n            images = list(image.to(DEVICE) for image in images)\n            targets = [{k: v.to(DEVICE) for k, v in t.items()} for t in targets]\n            \n            val_loss_dict = model(images, targets)\n            val_batch_loss = sum(loss for loss in val_loss_dict.values())\n            val_loss_accum += val_batch_loss.item()\n    \n    # Logging\n    val_loss = val_loss_accum / n_batches_val\n    train_loss = loss_accum / n_batches\n    validation_losses.append(val_loss)\n    \n    # Save model\n    chk_name = f'fasterrcnn_resnet50_fpn-e{epoch}.bin'\n    torch.save(model.state_dict(), chk_name)\n    \n    \n    elapsed = time.time() - time_start\n    \n    print(f\"[Epoch {epoch+1:2d} / {NUM_EPOCHS:2d}] Train loss: {train_loss:.3f}. Val loss: {val_loss:.3f} --> {chk_name}  [{elapsed:.0f} secs]\")   ","metadata":{"execution":{"iopub.status.busy":"2022-01-15T12:44:38.84575Z","iopub.execute_input":"2022-01-15T12:44:38.846217Z","iopub.status.idle":"2022-01-15T17:05:32.204859Z","shell.execute_reply.started":"2022-01-15T12:44:38.846125Z","shell.execute_reply":"2022-01-15T17:05:32.20361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"validation_losses","metadata":{"execution":{"iopub.status.busy":"2022-01-15T17:05:32.20943Z","iopub.execute_input":"2022-01-15T17:05:32.209833Z","iopub.status.idle":"2022-01-15T17:05:32.217568Z","shell.execute_reply.started":"2022-01-15T17:05:32.209749Z","shell.execute_reply":"2022-01-15T17:05:32.21622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.argmin(validation_losses)","metadata":{"execution":{"iopub.status.busy":"2022-01-15T17:05:32.219567Z","iopub.execute_input":"2022-01-15T17:05:32.220519Z","iopub.status.idle":"2022-01-15T17:05:32.2342Z","shell.execute_reply.started":"2022-01-15T17:05:32.220359Z","shell.execute_reply":"2022-01-15T17:05:32.232818Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Check result","metadata":{}},{"cell_type":"code","source":"idx = 0\n\nimages, targets = next(iter(dl_val))\nimages = list(img.to(DEVICE) for img in images)\ntargets = [{k: v.to(DEVICE) for k, v in t.items()} for t in targets]\n\nboxes = targets[idx]['boxes'].cpu().numpy().astype(np.int32)\nsample = images[idx].permute(1,2,0).cpu().numpy()\n\nmodel.eval()\n\noutputs = model(images)\noutputs = [{k: v.detach().cpu().numpy() for k, v in t.items()} for t in outputs]","metadata":{"execution":{"iopub.status.busy":"2022-01-15T17:05:32.236365Z","iopub.execute_input":"2022-01-15T17:05:32.237155Z","iopub.status.idle":"2022-01-15T17:05:35.215769Z","shell.execute_reply.started":"2022-01-15T17:05:32.236864Z","shell.execute_reply":"2022-01-15T17:05:35.214491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 1, figsize=(16, 8))\n\n# Red for ground truth\nfor box in boxes:\n    cv2.rectangle(sample,\n                  (box[0], box[1]),\n                  (box[2], box[3]),\n                  (220, 0, 0), 3)\n\n    \n# Green for predictions\n# Print the first 5\nfor box in outputs[idx]['boxes'][:5]:\n    cv2.rectangle(sample,\n                  (box[0], box[1]),\n                  (box[2], box[3]),\n                  (0, 220, 0), 3)\n\nax.set_axis_off()\nax.imshow(sample);","metadata":{"execution":{"iopub.status.busy":"2022-01-15T17:05:35.219063Z","iopub.execute_input":"2022-01-15T17:05:35.219563Z","iopub.status.idle":"2022-01-15T17:05:35.887107Z","shell.execute_reply.started":"2022-01-15T17:05:35.219491Z","shell.execute_reply":"2022-01-15T17:05:35.885797Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Please, _DO_ upvote if you found it useful!","metadata":{}}]}