{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# SIIM-FISABIO-RSNA COVID-19 Detection with Faster RCNN\n\n![](https://storage.googleapis.com/kaggle-competitions/kaggle/26680/logos/header.png)\n\n\n<h3 style=\"background-color:red;\">Work in progress: Upvote and support</h3>","metadata":{"papermill":{"duration":0.022943,"end_time":"2021-05-20T09:49:20.26441","exception":false,"start_time":"2021-05-20T09:49:20.241467","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"<div class=\"alert alert-block alert-info\">  \n<h1><strong>Introduction</strong></h1>\n    <p>Five times more deadly than the flu, COVID-19 causes significant morbidity and mortality. Like other pneumonias, pulmonary infection with COVID-19 results in inflammation and fluid in the lungs. COVID-19 looks very similar to other viral and bacterial pneumonias on chest radiographs, which makes it difficult to diagnose. Your computer vision model to detect and localize COVID-19 would help doctors provide a quick and confident diagnosis. As a result, patients could get the right treatment before the most severe effects of the virus take hold.\n\nCurrently, COVID-19 can be diagnosed via polymerase chain reaction to detect genetic material from the virus or chest radiograph. However, it can take a few hours and sometimes days before the molecular test results are back. By contrast, chest radiographs can be obtained in minutes. While guidelines exist to help radiologists differentiate COVID-19 from other types of infection, their assessments vary. In addition, non-radiologists could be supported with better localization of the disease, such as with a visual bounding box.\n\nAs the leading healthcare organization in their field, the Society for Imaging Informatics in Medicine (SIIM)'s mission is to advance medical imaging informatics through education, research, and innovation. SIIM has partnered with the Foundation for the Promotion of Health and Biomedical Research of Valencia Region (FISABIO), Medical Imaging Databank of the Valencia Region (BIMCV) and the Radiological Society of North America (RSNA) for this competition.\n\nIn this competition, you’ll identify and localize COVID-19 abnormalities on chest radiographs. In particular, you'll categorize the radiographs as negative for pneumonia or typical, indeterminate, or atypical for COVID-19. You and your model will work with imaging data and annotations from a group of radiologists.\n\nIf successful, you'll help radiologists diagnose the millions of COVID-19 patients more confidently and quickly. This will also enable doctors to see the extent of the disease and help them make decisions regarding treatment. Depending upon severity, affected patients may need hospitalization, admission into an intensive care unit, or supportive therapies like mechanical ventilation. As a result of better diagnosis, more patients will quickly receive the best care for their condition, which could mitigate the most severe effects of the virus.\n </p>\n    <br>\n        <hr>\n      <b>Code Requirements: </b> \n    <hr>\n<ul>\n   This is a Code Competition\n\nSubmissions to this competition must be made through Notebooks. In order for the \"Submit\" button to be active after a commit, the following conditions must be met:\n\n    CPU Notebook <= 9 hours run-time\n    GPU Notebook <= 9 hours run-time\n    Internet access disabled\n    Freely & publicly available external data is allowed, including pre-trained models\n    Submission file must be named submission.csv\n\n</ul>\n\n\n<hr>\n<b>Prizes: </b> \n<hr>\n<ul>\n<p> Leaderboard Prizes: Awarded on the basis of private leaderboard rank. Only selected submissions will be ranked on the private leaderboard.\n\n* 1st Place - $30,000\n\n* 2nd Place - $20,000\n    \n* 3rd Place - $10,000\n\n* 4th Place - $8,000\n    \n* 5th Place - $7,000\n\n* 6th - 10th Places - $5,000 each\n\nStudent Team Prize: Awarded to the top-scoring eligible Student Team on the basis of private leaderboard rank. To be fulfilled by HP's Fulfillment Agency\n\nOne HP ZBook Studio G8 data science workstation will be awarded to each member of the winning Student Team. Each ZBook is valued at $6,000. \n</ul>\n<hr>\n\n \n<hr>\n<b>Source of Data: </b> \n<hr> \n <a href=\"https://www.kaggle.com/c/siim-covid19-detection\">https://www.kaggle.com/c/siim-covid19-detection</a>\n   \n</div>","metadata":{"execution":{"iopub.status.busy":"2021-05-24T09:54:30.946191Z","iopub.execute_input":"2021-05-24T09:54:30.946554Z","iopub.status.idle":"2021-05-24T09:54:30.958579Z","shell.execute_reply.started":"2021-05-24T09:54:30.946523Z","shell.execute_reply":"2021-05-24T09:54:30.957229Z"}}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"## Libraries and Path","metadata":{"papermill":{"duration":0.021449,"end_time":"2021-05-20T09:49:20.307797","exception":false,"start_time":"2021-05-20T09:49:20.286348","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!conda install gdcm -c conda-forge -y\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nimport pydicom\nimport glob\nfrom tqdm.notebook import tqdm\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nimport matplotlib.pyplot as plt\nfrom skimage import exposure\n#import gdcm\nimport cv2\nimport warnings\nfrom fastai.vision.all import *\nfrom fastai.medical.imaging import *\nwarnings.filterwarnings('ignore')\ndataset_path = Path('../input/siim-covid19-detection')\nimport vtk\n# numba\nimport numba\nfrom numba import jit\nfrom vtk.util import numpy_support\n\nreader = vtk.vtkDICOMImageReader()","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":65.749573,"end_time":"2021-05-20T09:50:26.078998","exception":false,"start_time":"2021-05-20T09:49:20.329425","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:14:50.824745Z","iopub.execute_input":"2021-05-26T05:14:50.82511Z","iopub.status.idle":"2021-05-26T05:15:56.072339Z","shell.execute_reply.started":"2021-05-26T05:14:50.825055Z","shell.execute_reply":"2021-05-26T05:15:56.071463Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Support Functions","metadata":{"papermill":{"duration":0.083608,"end_time":"2021-05-20T09:50:26.251611","exception":false,"start_time":"2021-05-20T09:50:26.168003","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Thanks https://www.kaggle.com/tanlikesmath/siim-covid-19-detection-a-simple-eda\ndef dicom2array(path, voi_lut=True, fix_monochrome=True):\n    # Original from: https://www.kaggle.com/raddar/convert-dicom-to-np-array-the-correct-way\n    dicom = pydicom.read_file(path)\n    \n    # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to \n    # \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n    else:\n        data = dicom.pixel_array\n               \n    # depending on this value, X-ray may look inverted - fix that:\n    if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n        \n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n        \n    return data\n        \n    \ndef plot_img(img, size=(7, 7), is_rgb=True, title=\"\", cmap='gray'):\n    plt.figure(figsize=size)\n    plt.imshow(img, cmap=cmap)\n    plt.suptitle(title)\n    plt.show()\n\n\ndef plot_imgs(imgs, cols=4, size=7, is_rgb=True, title=\"\", cmap='gray', img_size=(500,500)):\n    rows = len(imgs)//cols + 1\n    fig = plt.figure(figsize=(cols*size, rows*size))\n    for i, img in enumerate(imgs):\n        if img_size is not None:\n            img = cv2.resize(img, img_size)\n        fig.add_subplot(rows, cols, i+1)\n        plt.imshow(img, cmap=cmap)\n    plt.suptitle(title)\n    plt.show()\n    \n\n\ndef image_path(row):\n    study_path = dataset_path/'train'/row.StudyInstanceUID\n    for i in get_dicom_files(study_path):\n        if row.id.split('_')[0] == i.stem: return i \n        \n\n\n\nclass Config:\n    n_folds: int = 5\n    seed: int = 2021\n    num_classes: int = 2 \n    img_size: int = 256\n    fold_num: int = 0\n    device: str = 'cuda:0'\n\n\n\ndef seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = True\n    \nseed_everything(Config.seed)\n\n","metadata":{"papermill":{"duration":0.096376,"end_time":"2021-05-20T09:50:26.431706","exception":false,"start_time":"2021-05-20T09:50:26.33533","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:15:56.073851Z","iopub.execute_input":"2021-05-26T05:15:56.074212Z","iopub.status.idle":"2021-05-26T05:15:56.088222Z","shell.execute_reply.started":"2021-05-26T05:15:56.074175Z","shell.execute_reply":"2021-05-26T05:15:56.087244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dicom_paths = get_dicom_files(dataset_path/'train')\nimgs = [dicom2array(path) for path in dicom_paths[:4]]\nplot_imgs(imgs)","metadata":{"papermill":{"duration":27.197869,"end_time":"2021-05-20T09:50:53.71252","exception":false,"start_time":"2021-05-20T09:50:26.514651","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:16:40.092437Z","iopub.execute_input":"2021-05-26T05:16:40.092801Z","iopub.status.idle":"2021-05-26T05:16:54.374813Z","shell.execute_reply.started":"2021-05-26T05:16:40.092771Z","shell.execute_reply":"2021-05-26T05:16:54.374025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Train Data Overview","metadata":{"papermill":{"duration":0.092457,"end_time":"2021-05-20T09:50:53.898463","exception":false,"start_time":"2021-05-20T09:50:53.806006","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train_image_df = pd.read_csv(dataset_path/'train_image_level.csv')\n\n\n# Thanks https://www.kaggle.com/tanlikesmath/siim-covid-19-detection-a-simple-eda\ntrain_image_df['class'] = train_image_df.label.apply(lambda x: x.split()[0])\n\n\ntrain_image_df['x_min'] = train_image_df.label.apply(lambda x: float(x.split()[2]))\ntrain_image_df['y_min'] = train_image_df.label.apply(lambda x: float(x.split()[3]))\ntrain_image_df['x_max'] = train_image_df.label.apply(lambda x: float(x.split()[4]))\ntrain_image_df['y_max'] = train_image_df.label.apply(lambda x: float(x.split()[5]))\n\n\n\ndef get_bbox_area(row):\n    return (row['x_max']-row['x_min'])*(row['y_max']-row['y_min'])\n\n\ntrain_image_df['bbox_area'] = train_image_df.apply(get_bbox_area, axis=1)\n\ntrain_image_df['image_path'] = train_image_df.apply(image_path, axis=1)\n\ntrain_image_df['bbox_area'].hist()","metadata":{"papermill":{"duration":6.866456,"end_time":"2021-05-20T09:51:00.857709","exception":false,"start_time":"2021-05-20T09:50:53.991253","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:17:41.154104Z","iopub.execute_input":"2021-05-26T05:17:41.154436Z","iopub.status.idle":"2021-05-26T05:17:46.073079Z","shell.execute_reply.started":"2021-05-26T05:17:41.154409Z","shell.execute_reply":"2021-05-26T05:17:46.072296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_image_df.head()","metadata":{"papermill":{"duration":0.13709,"end_time":"2021-05-20T09:51:01.088705","exception":false,"start_time":"2021-05-20T09:51:00.951615","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:17:46.075922Z","iopub.execute_input":"2021-05-26T05:17:46.076203Z","iopub.status.idle":"2021-05-26T05:17:46.097982Z","shell.execute_reply.started":"2021-05-26T05:17:46.076177Z","shell.execute_reply":"2021-05-26T05:17:46.096869Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Train Data Visualization","metadata":{"papermill":{"duration":0.093946,"end_time":"2021-05-20T09:51:01.625367","exception":false,"start_time":"2021-05-20T09:51:01.531421","status":"completed"},"tags":[]}},{"cell_type":"code","source":"imgs = []\nimage_paths = train_image_df['image_path'].values\nclass_ids = train_image_df['class']\n\n# map label_id to specify color\nlabel2color = {class_id:[random.randint(0,255) for i in range(3)] for class_id in class_ids}\nthickness = 3\nscale = 5\n\n\nfor i in range(8):\n    image_path = random.choice(image_paths)\n    print(image_path)\n    img = dicom2array(str(image_path))\n    img = cv2.resize(img, None, fx=1/scale, fy=1/scale)\n    img = np.stack([img, img, img], axis=-1)\n    \n    box = train_image_df.loc[train_image_df['image_path'] == image_path, ['x_min', 'y_min', 'x_max', 'y_max']].values[0]/scale\n    label = train_image_df.loc[train_image_df['image_path'] == image_path, ['class']].values[0][0]\n    \n    color = label2color[label]\n    img = cv2.rectangle(\n        img,\n        (int(box[0]), int(box[1])),\n        (int(box[2]), int(box[3])),\n        color, thickness)\n    \n    img = cv2.resize(img, (500,500))\n    imgs.append(img)\n    \nplot_imgs(imgs, cmap=None)","metadata":{"papermill":{"duration":6.393403,"end_time":"2021-05-20T09:51:08.113409","exception":false,"start_time":"2021-05-20T09:51:01.720006","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:17:46.099791Z","iopub.execute_input":"2021-05-26T05:17:46.100157Z","iopub.status.idle":"2021-05-26T05:17:51.650481Z","shell.execute_reply.started":"2021-05-26T05:17:46.100121Z","shell.execute_reply":"2021-05-26T05:17:51.644515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model\n\nLets import Faster RCNN from timm\n\n\n## Architecture\n\nThe architecture of Faster R-CNN is complex because it has several moving parts. We’ll start with a high level overview, and then go over the details for each of the components.\n\nIt all starts with an image, from which we want to obtain:\n\n    a list of bounding boxes.\n    a label assigned to each bounding box.\n    a probability for each label and bounding box.\n    \n![](https://tryolabs.com/blog/images/blog/post-images/2018-01-18-faster-rcnn/fasterrcnn-architecture.b9035cba.png)    \n\n\nThe input images are represented as Height×Width×Depth\\mathit{Height} \\times \\mathit{Width} \\times \\mathit{Depth}Height×Width×Depth tensors (multidimensional arrays), which are passed through a pre-trained CNN up until an intermediate layer, ending up with a convolutional feature map. We use this as a feature extractor for the next part.\n\nThis technique is very commonly used in the context of Transfer Learning, especially for training a classifier on a small dataset using the weights of a network trained on a bigger dataset. We’ll take a deeper look at this in the following sections.\n\nNext, we have what is called a Region Proposal Network (RPN, for short). Using the features that the CNN computed, it is used to find up to a predefined number of regions (bounding boxes), which may contain objects.\n\nProbably the hardest issue with using Deep Learning (DL) for object detection is generating a variable-length list of bounding boxes. When modeling deep neural networks, the last block is usually a fixed sized tensor output (except when using Recurrent Neural Networks, but that is for another post). For example, in image classification, the output is a (N,)(N,)(N,) shaped tensor, with NNN being the number of classes, where each scalar in location iii contains the probability of that image being labeli\\mathit{label}_ilabel​i​​.\n\nThe variable-length problem is solved in the RPN by using anchors: fixed sized reference bounding boxes which are placed uniformly throughout the original image. Instead of having to detect where objects are, we model the problem into two parts. For every anchor, we ask:\n\n    Does this anchor contain a relevant object?\n    How would we adjust this anchor to better fit the relevant object?\n\nThis is probably getting confusing, but fear not, we’ll dive into this below.\n\nAfter having a list of possible relevant objects and their locations in the original image, it becomes a more straightforward problem to solve. Using the features extracted by the CNN and the bounding boxes with relevant objects, we apply Region of Interest (RoI) Pooling and extract those features which would correspond to the relevant objects into a new tensor.\n\nFinally, comes the R-CNN module, which uses that information to:\n\n    Classify the content in the bounding box (or discard it, using “background” as a label).\n    Adjust the bounding box coordinates (so it better fits the object).\n\nObviously, some major bits of information are missing, but that’s basically the general idea of how Faster R-CNN works. Next, we’ll go over the details on both the architecture and loss/training for each of the components\n\n\nhttps://www.alegion.com/faster-r-cnn - For further read","metadata":{"papermill":{"duration":0.114055,"end_time":"2021-05-20T09:51:08.343567","exception":false,"start_time":"2021-05-20T09:51:08.229512","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from IPython.display import HTML\nHTML('<iframe src=https://arxiv.org/pdf/1506.01497.pdf width=600 height=650></iframe>')","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.124373,"end_time":"2021-05-20T09:51:08.582232","exception":false,"start_time":"2021-05-20T09:51:08.457859","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:17:51.65218Z","iopub.execute_input":"2021-05-26T05:17:51.6525Z","iopub.status.idle":"2021-05-26T05:17:51.657607Z","shell.execute_reply.started":"2021-05-26T05:17:51.652467Z","shell.execute_reply":"2021-05-26T05:17:51.65689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Libraries for Pytorch","metadata":{"papermill":{"duration":0.114299,"end_time":"2021-05-20T09:51:08.811181","exception":false,"start_time":"2021-05-20T09:51:08.696882","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\nimport pandas as pd\nimport numpy as np\nimport cv2\nimport os\nimport re\nimport pydicom\nimport warnings\n\nfrom PIL import Image\n\nimport albumentations as A\nfrom albumentations.pytorch.transforms import ToTensorV2\nfrom albumentations.core.transforms_interface import ImageOnlyTransform\n\nimport torch\nimport torchvision\n\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor\nfrom torchvision.models.detection import FasterRCNN\nfrom torchvision.models.detection.rpn import AnchorGenerator\n\nfrom torch.utils.data import DataLoader, Dataset\nfrom torch.utils.data.sampler import SequentialSampler\nfrom pydicom import dcmread\n\nfrom matplotlib import pyplot as plt\nimport random\npaddingSize= 0\n    ","metadata":{"papermill":{"duration":0.74259,"end_time":"2021-05-20T09:51:09.667563","exception":false,"start_time":"2021-05-20T09:51:08.924973","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:23:30.456951Z","iopub.execute_input":"2021-05-26T05:23:30.457339Z","iopub.status.idle":"2021-05-26T05:23:30.465407Z","shell.execute_reply.started":"2021-05-26T05:23:30.457307Z","shell.execute_reply":"2021-05-26T05:23:30.462917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Train and Validation Split","metadata":{"papermill":{"duration":0.114085,"end_time":"2021-05-20T09:51:10.14128","exception":false,"start_time":"2021-05-20T09:51:10.027195","status":"completed"},"tags":[]}},{"cell_type":"code","source":"image_ids = train_image_df['id'].unique()\nvalid_ids = image_ids[-5000:]# Tran and Validation Split \ntrain_ids = image_ids[:-5000]\n\n\nvalid_df = train_image_df[train_image_df['id'].isin(valid_ids)]\ntrain_df = train_image_df[train_image_df['id'].isin(train_ids)]\n\ntrain_df[\"class_id\"] = [1]*len(train_df)\nvalid_df[\"class_id\"] = [1]*len(valid_df)\nprint(len(train_image_df))\nprint(train_df.shape)\ntrain_df.head()","metadata":{"papermill":{"duration":0.14646,"end_time":"2021-05-20T09:51:10.402634","exception":false,"start_time":"2021-05-20T09:51:10.256174","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:23:30.910529Z","iopub.execute_input":"2021-05-26T05:23:30.910833Z","iopub.status.idle":"2021-05-26T05:23:30.942541Z","shell.execute_reply.started":"2021-05-26T05:23:30.910804Z","shell.execute_reply":"2021-05-26T05:23:30.94156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Defineing DataLoader","metadata":{"papermill":{"duration":0.115497,"end_time":"2021-05-20T09:51:10.634131","exception":false,"start_time":"2021-05-20T09:51:10.518634","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\n\nclass COVIDTrainDataLoader(Dataset): #Class to load Training Data\n    \n    def __init__(self, dataframe, transforms=None,stat = 'Train'):\n        super().__init__()\n        \n        self.image_ids = dataframe[\"id\"].unique()\n        \n        self.df = dataframe\n        self.transforms = transforms\n        self.stat = stat\n        \n    def __getitem__(self, index):\n        if self.stat == 'Train':\n            \n            image_id = self.image_ids[index]\n            \n            records = self.df[(self.df['id'] == image_id)]\n            records = records.reset_index(drop=True)\n            image = dicom2array(self.df[(self.df['id'] == image_id)]['image_path'].values[0])#dcmread\n\n            #image = ds.pixel_array\n           \n            '''if \"PhotometricInterpretation\" in dicom:\n                if dicom.PhotometricInterpretation == \"MONOCHROME1\":\n                    image = np.amax(image) - image'''\n\n            intercept =  0.0\n            slope =1.0\n\n            if slope != 1:\n                image = slope * image.astype(np.float64)\n                image = image.astype(np.int16)\n\n            \n            image += np.int16(intercept)        \n\n            image = np.stack([image, image, image])\n            image = image.astype('float32')\n            image = image - image.min()\n            image = image / image.max()\n            image = image * 255.0\n            image = image.transpose(1,2,0)\n\n            if records.loc[0, \"class_id\"] == 0:\n                records = records.loc[[0], :]\n\n            boxes = records[['x_min', 'y_min', 'x_max', 'y_max']].values\n            area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0])\n            area = torch.as_tensor(area, dtype=torch.float32)\n            labels = torch.tensor(records[\"class_id\"].values, dtype=torch.int64)\n\n            # suppose all instances are not crowd\n            iscrowd = torch.zeros((records.shape[0],), dtype=torch.int64)\n\n            target = {}\n            target['boxes'] = boxes\n            target['labels'] = labels\n            target['id'] = torch.tensor([index])\n            target['area'] = area\n            target['iscrowd'] = iscrowd\n\n            if self.transforms:\n                sample = {\n                    'image': image,\n                    'bboxes': target['boxes'],\n                    'labels': labels\n                }\n                sample = self.transforms(**sample)\n                image = sample['image']\n\n                target['boxes'] = torch.tensor(sample['bboxes'])\n\n            if target[\"boxes\"].shape[0] == 0:\n                # Albumentation cuts the target (class 14, 1x1px in the corner)\n                target[\"boxes\"] = torch.from_numpy(np.array([[0.0, 0.0, 1.0, 1.0]]))\n                target[\"area\"] = torch.tensor([1.0], dtype=torch.float32)\n                target[\"labels\"] = torch.tensor([0], dtype=torch.int64)\n            \n            return image, target, image_ids\n        \n        else:\n                   \n            image_id = self.image_ids[index]\n            records = self.df[(self.df['id'] == image_id)]\n            records = records.reset_index(drop=True)\n\n            image = dicom2array(self.df[(self.df['id'] == image_id)]['image_path'].values[0])#dcmread\n\n            #image = ds.pixel_array\n\n            intercept =  0.0\n            slope = 1.0\n\n            if slope != 1:\n                image = slope * image.astype(np.float64)\n                image = image.astype(np.int16)\n\n            image += np.int16(intercept)        \n\n            image = np.stack([image, image, image])\n            image = image.astype('float32')\n            image = image - image.min()\n            image = image / image.max()\n            image = image * 255.0\n            image = image.transpose(1,2,0)\n\n            if self.transforms:\n                sample = {\n                    'image': image,\n                }\n                sample = self.transforms(**sample)\n                image = sample['image']\n\n            return image, image_id\n    \n    def __len__(self):\n        return self.image_ids.shape[0]\n\n","metadata":{"papermill":{"duration":0.139097,"end_time":"2021-05-20T09:51:10.888196","exception":false,"start_time":"2021-05-20T09:51:10.749099","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:23:31.459246Z","iopub.execute_input":"2021-05-26T05:23:31.459545Z","iopub.status.idle":"2021-05-26T05:23:31.479178Z","shell.execute_reply.started":"2021-05-26T05:23:31.459518Z","shell.execute_reply":"2021-05-26T05:23:31.478221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Augmentation with Albumentations","metadata":{"papermill":{"duration":0.121814,"end_time":"2021-05-20T09:51:11.151616","exception":false,"start_time":"2021-05-20T09:51:11.029802","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Albumentations\ndef get_train_transform():\n    return A.Compose([\n        A.ShiftScaleRotate(scale_limit=0.1, rotate_limit=45, p=0.25),\n        A.LongestMaxSize(max_size=800, p=1.0),\n        A.Normalize(mean=(0, 0, 0), std=(1, 1, 1), max_pixel_value=255.0, p=1.0),\n        ToTensorV2(p=1.0)\n    ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})\n\ndef get_valid_transform():\n    return A.Compose([\n        A.Normalize(mean=(0, 0, 0), std=(1, 1, 1), max_pixel_value=255.0, p=1.0),\n        ToTensorV2(p=1.0)\n    ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})\n\ndef get_test_transform():\n    return A.Compose([\n        A.Normalize(mean=(0, 0, 0), std=(1, 1, 1), max_pixel_value=255.0, p=1.0),\n        ToTensorV2(p=1.0)\n    ])","metadata":{"papermill":{"duration":0.127032,"end_time":"2021-05-20T09:51:11.395154","exception":false,"start_time":"2021-05-20T09:51:11.268122","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:23:32.194283Z","iopub.execute_input":"2021-05-26T05:23:32.194618Z","iopub.status.idle":"2021-05-26T05:23:32.202378Z","shell.execute_reply.started":"2021-05-26T05:23:32.194589Z","shell.execute_reply":"2021-05-26T05:23:32.201431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Metrics","metadata":{}},{"cell_type":"code","source":"# Thanks https://www.kaggle.com/pestipeti/competition-metric-details-script\n\n\n@jit(nopython=True)\ndef calculate_iou(gt, pr, form='pascal_voc') -> float:\n    \"\"\"Calculates the Intersection over Union.\n\n    Args:\n        gt: (np.ndarray[Union[int, float]]) coordinates of the ground-truth box\n        pr: (np.ndarray[Union[int, float]]) coordinates of the prdected box\n        form: (str) gt/pred coordinates format\n            - pascal_voc: [xmin, ymin, xmax, ymax]\n            - coco: [xmin, ymin, w, h]\n    Returns:\n        (float) Intersection over union (0.0 <= iou <= 1.0)\n    \"\"\"\n    if form == 'coco':\n        gt = gt.copy()\n        pr = pr.copy()\n\n        gt[2] = gt[0] + gt[2]\n        gt[3] = gt[1] + gt[3]\n        pr[2] = pr[0] + pr[2]\n        pr[3] = pr[1] + pr[3]\n\n    # Calculate overlap area\n    dx = min(gt[2], pr[2]) - max(gt[0], pr[0]) + 1\n    \n    if dx < 0:\n        return 0.0\n    \n    dy = min(gt[3], pr[3]) - max(gt[1], pr[1]) + 1\n\n    if dy < 0:\n        return 0.0\n\n    overlap_area = dx * dy\n\n    # Calculate union area\n    union_area = (\n            (gt[2] - gt[0] + 1) * (gt[3] - gt[1] + 1) +\n            (pr[2] - pr[0] + 1) * (pr[3] - pr[1] + 1) -\n            overlap_area\n    )\n\n    return overlap_area / union_area\n\n@jit(nopython=True)\ndef find_best_match(gts, pred, pred_idx, threshold = 0.5, form = 'pascal_voc', ious=None) -> int:\n    \"\"\"Returns the index of the 'best match' between the\n    ground-truth boxes and the prediction. The 'best match'\n    is the highest IoU. (0.0 IoUs are ignored).\n\n    Args:\n        gts: (List[List[Union[int, float]]]) Coordinates of the available ground-truth boxes\n        pred: (List[Union[int, float]]) Coordinates of the predicted box\n        pred_idx: (int) Index of the current predicted box\n        threshold: (float) Threshold\n        form: (str) Format of the coordinates\n        ious: (np.ndarray) len(gts) x len(preds) matrix for storing calculated ious.\n\n    Return:\n        (int) Index of the best match GT box (-1 if no match above threshold)\n    \"\"\"\n    best_match_iou = -np.inf\n    best_match_idx = -1\n\n    for gt_idx in range(len(gts)):\n        \n        if gts[gt_idx][0] < 0:\n            # Already matched GT-box\n            continue\n        \n        iou = -1 if ious is None else ious[gt_idx][pred_idx]\n\n        if iou < 0:\n            iou = calculate_iou(gts[gt_idx], pred, form=form)\n            \n            if ious is not None:\n                ious[gt_idx][pred_idx] = iou\n\n        if iou < threshold:\n            continue\n\n        if iou > best_match_iou:\n            best_match_iou = iou\n            best_match_idx = gt_idx\n\n    return best_match_idx\n\n@jit(nopython=True)\ndef calculate_precision(gts, preds, threshold = 0.5, form = 'coco', ious=None) -> float:\n    \"\"\"Calculates precision for GT - prediction pairs at one threshold.\n\n    Args:\n        gts: (List[List[Union[int, float]]]) Coordinates of the available ground-truth boxes\n        preds: (List[List[Union[int, float]]]) Coordinates of the predicted boxes,\n               sorted by confidence value (descending)\n        threshold: (float) Threshold\n        form: (str) Format of the coordinates\n        ious: (np.ndarray) len(gts) x len(preds) matrix for storing calculated ious.\n\n    Return:\n        (float) Precision\n    \"\"\"\n    n = len(preds)\n    tp = 0\n    fp = 0\n    \n    # for pred_idx, pred in enumerate(preds_sorted):\n    for pred_idx in range(n):\n\n        best_match_gt_idx = find_best_match(gts, preds[pred_idx], pred_idx,\n                                            threshold=threshold, form=form, ious=ious)\n\n        if best_match_gt_idx >= 0:\n            # True positive: The predicted box matches a gt box with an IoU above the threshold.\n            tp += 1\n            # Remove the matched GT box\n            gts[best_match_gt_idx] = -1\n\n        else:\n            # No match\n            # False positive: indicates a predicted box had no associated gt box.\n            fp += 1\n\n    # False negative: indicates a gt box had no associated predicted box.\n    fn = (gts.sum(axis=1) > 0).sum()\n\n    return tp / (tp + fp + fn)\n\n\n@jit(nopython=True)\ndef calculate_image_precision(gts, preds, thresholds = (0.5, ), form = 'coco') -> float:\n    \"\"\"Calculates image precision.\n       The mean average precision at different intersection over union (IoU) thresholds.\n\n    Args:\n        gts: (List[List[Union[int, float]]]) Coordinates of the available ground-truth boxes\n        preds: (List[List[Union[int, float]]]) Coordinates of the predicted boxes,\n               sorted by confidence value (descending)\n        thresholds: (float) Different thresholds\n        form: (str) Format of the coordinates\n\n    Return:\n        (float) Precision\n    \"\"\"\n    n_threshold = len(thresholds)\n    image_precision = 0.0\n    \n    ious = np.ones((len(gts), len(preds))) * -1\n    # ious = None\n\n    for threshold in thresholds:\n        precision_at_threshold = calculate_precision(gts.copy(), preds, threshold=threshold,\n                                                     form=form, ious=ious)\n        image_precision += precision_at_threshold / n_threshold\n\n    return image_precision\n\niou_thresholds = [0.5]\n\nclass EvalMeter(object):\n    \"\"\"Computes and stores the average and current value\"\"\"\n    def __init__(self):\n        self.reset()\n\n    def reset(self):\n        self.image_precision = 0\n        self.avg = 0\n        self.sum = 0\n        self.count = 0\n\n    def update(self, gt_boxes, pred_boxes, n=1):       \n        \"\"\" pred_boxes : need to be sorted.\"\"\"\n        \n        self.image_precision = calculate_image_precision(pred_boxes,\n                                                         gt_boxes,\n                                                         thresholds=iou_thresholds,\n                                                         form='pascal_voc')\n        self.count += n\n        self.sum += self.image_precision * n\n        self.avg = self.sum / self.count","metadata":{"execution":{"iopub.status.busy":"2021-05-26T05:23:33.790584Z","iopub.execute_input":"2021-05-26T05:23:33.790904Z","iopub.status.idle":"2021-05-26T05:23:33.811515Z","shell.execute_reply.started":"2021-05-26T05:23:33.790873Z","shell.execute_reply":"2021-05-26T05:23:33.810679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\n# get number of input features for the classifier\nin_features = model.roi_heads.box_predictor.cls_score.in_features\n\n# replace the pre-trained head with a new one\nmodel.roi_heads.box_predictor = FastRCNNPredictor(in_features, 2)","metadata":{"papermill":{"duration":9.194822,"end_time":"2021-05-20T09:51:20.705841","exception":false,"start_time":"2021-05-20T09:51:11.511019","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:23:35.107486Z","iopub.execute_input":"2021-05-26T05:23:35.108139Z","iopub.status.idle":"2021-05-26T05:23:35.842063Z","shell.execute_reply.started":"2021-05-26T05:23:35.108093Z","shell.execute_reply":"2021-05-26T05:23:35.841212Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Set Train And Validation Data Loader","metadata":{"papermill":{"duration":0.116996,"end_time":"2021-05-20T09:51:20.94035","exception":false,"start_time":"2021-05-20T09:51:20.823354","status":"completed"},"tags":[]}},{"cell_type":"code","source":"device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')\n\ndef collate_fn(batch):\n    return tuple(zip(*batch))\n\ntrain_dataset = COVIDTrainDataLoader(train_df, get_train_transform())\nvalid_dataset = COVIDTrainDataLoader(valid_df, get_valid_transform())\n\n\n# split the dataset in train and test set\nindices = torch.randperm(len(train_dataset)).tolist()\n# Create train and validate data loader\ntrain_data_loader = DataLoader(\n    train_dataset,\n    batch_size=2,\n    shuffle=True,\n    num_workers=4,\n    collate_fn=collate_fn\n)\n\nvalid_data_loader = DataLoader(\n    valid_dataset,\n    batch_size=2,\n    shuffle=False,\n    num_workers=4,\n    collate_fn=collate_fn\n)","metadata":{"papermill":{"duration":0.132103,"end_time":"2021-05-20T09:51:21.197893","exception":false,"start_time":"2021-05-20T09:51:21.06579","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:23:37.350494Z","iopub.execute_input":"2021-05-26T05:23:37.350829Z","iopub.status.idle":"2021-05-26T05:23:37.360213Z","shell.execute_reply.started":"2021-05-26T05:23:37.3508Z","shell.execute_reply":"2021-05-26T05:23:37.359125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Averager:\n    def __init__(self):\n        self.current_total = 0.0\n        self.iterations = 0.0\n\n    def send(self, value):\n        self.current_total += value\n        self.iterations += 1\n\n    @property\n    def value(self):\n        if self.iterations == 0:\n            return 0\n        else:\n            return 1.0 * self.current_total / self.iterations\n\n    def reset(self):\n        self.current_total = 0.0\n        self.iterations = 0.0","metadata":{"papermill":{"duration":0.1297,"end_time":"2021-05-20T09:51:21.450767","exception":false,"start_time":"2021-05-20T09:51:21.321067","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:23:38.532013Z","iopub.execute_input":"2021-05-26T05:23:38.532352Z","iopub.status.idle":"2021-05-26T05:23:38.53842Z","shell.execute_reply.started":"2021-05-26T05:23:38.532321Z","shell.execute_reply":"2021-05-26T05:23:38.537433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.to(device)\nparams = [p for p in model.parameters() if p.requires_grad]\noptimizer = torch.optim.SGD(params, lr=0.005, momentum=0.9, weight_decay=0.0005)\nlr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=4, gamma=0.1)\n\nnum_epochs =  40 #low epoch to save GPU time","metadata":{"papermill":{"duration":4.542403,"end_time":"2021-05-20T09:51:26.109425","exception":false,"start_time":"2021-05-20T09:51:21.567022","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:23:39.553729Z","iopub.execute_input":"2021-05-26T05:23:39.554067Z","iopub.status.idle":"2021-05-26T05:23:39.61481Z","shell.execute_reply.started":"2021-05-26T05:23:39.554036Z","shell.execute_reply":"2021-05-26T05:23:39.614056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Train Loop","metadata":{"papermill":{"duration":0.11941,"end_time":"2021-05-20T09:51:26.346382","exception":false,"start_time":"2021-05-20T09:51:26.226972","status":"completed"},"tags":[]}},{"cell_type":"code","source":"loss_hist = Averager()\nitr = 1\nlossHistoryiter = []\nlossHistoryepoch = []\n\nimport time\nstart = time.time()\n\nfor epoch in range(num_epochs):\n    loss_hist.reset()\n    \n    for images, targets, image_ids in train_data_loader:\n        \n        images = list(image.to(device) for image in images)\n        targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n        loss_dict = model(images, targets)  \n        \n        losses = sum(loss for loss in loss_dict.values())\n        loss_value = losses.item()\n\n        loss_hist.send(loss_value)\n        lossHistoryiter.append(loss_value)\n        optimizer.zero_grad()\n        losses.backward()\n        optimizer.step()\n\n        if itr % 50 == 0:\n            print(f\"Iteration #{itr} loss: {loss_value}\")\n\n        itr += 1\n    \n    # update the learning rate\n    if lr_scheduler is not None:\n        lr_scheduler.step()\n    lossHistoryepoch.append(loss_hist.value)\n    print(f\"Epoch #{epoch} loss: {loss_hist.value}\")   \n    \nend = time.time()\nhours, rem = divmod(end-start, 3600)\nminutes, seconds = divmod(rem, 60)\nprint(\"Time taken to Train the model :{:0>2}:{:0>2}:{:05.2f}\".format(int(hours),int(minutes),seconds))","metadata":{"execution":{"iopub.status.busy":"2021-05-26T05:23:40.902112Z","iopub.execute_input":"2021-05-26T05:23:40.902456Z","iopub.status.idle":"2021-05-26T05:24:55.212266Z","shell.execute_reply.started":"2021-05-26T05:23:40.902428Z","shell.execute_reply":"2021-05-26T05:24:55.210276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objects as go\n\nx = [i for i in range(num_epochs)]\ny = lossHistoryepoch\nfig = go.Figure()\nfig.add_trace(go.Scatter(x=x,y=y,\n                    mode='lines',\n                    name='lines'))\n\nfig.update_layout(title='Loss vs Epochs',\n                   xaxis_title='Epochs',\n                   yaxis_title='Loss')\nfig.show()","metadata":{"papermill":{"duration":0.23304,"end_time":"2021-05-20T09:59:38.128918","exception":false,"start_time":"2021-05-20T09:59:37.895878","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:03.266678Z","iopub.execute_input":"2021-05-26T05:25:03.267036Z","iopub.status.idle":"2021-05-26T05:25:03.371203Z","shell.execute_reply.started":"2021-05-26T05:25:03.266983Z","shell.execute_reply":"2021-05-26T05:25:03.370267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PATH = \"Covid_FRCNN.pt\"\ntorch.save(model, PATH)","metadata":{"execution":{"iopub.status.busy":"2021-05-26T05:25:05.082828Z","iopub.execute_input":"2021-05-26T05:25:05.083248Z","iopub.status.idle":"2021-05-26T05:25:05.456668Z","shell.execute_reply.started":"2021-05-26T05:25:05.083215Z","shell.execute_reply":"2021-05-26T05:25:05.455772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Test Data","metadata":{"papermill":{"duration":0.123239,"end_time":"2021-05-20T09:59:38.377126","exception":false,"start_time":"2021-05-20T09:59:38.253887","status":"completed"},"tags":[]}},{"cell_type":"code","source":"test_df = pd.read_csv('../input/siim-covid19-detection/sample_submission.csv')\n\ndef study_path(row):\n    study_path = dataset_path/'test'/row.id.split(\"_\")[0]\n    for i in get_dicom_files(study_path):\n        return i \n        \ntest_df['image_path'] = test_df.apply(study_path, axis=1)\ntest_df[\"image_path\"] = test_df[\"image_path\"].apply(str)\ntest_df.head()","metadata":{"papermill":{"duration":5.410875,"end_time":"2021-05-20T09:59:43.910829","exception":false,"start_time":"2021-05-20T09:59:38.499954","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:06.768744Z","iopub.execute_input":"2021-05-26T05:25:06.769201Z","iopub.status.idle":"2021-05-26T05:25:09.320593Z","shell.execute_reply.started":"2021-05-26T05:25:06.769157Z","shell.execute_reply":"2021-05-26T05:25:09.319775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def image_file_path(row):\n    s = row.id.split(\"_\")[0]\n\n    try:\n        return test_df.loc[test_df[\"image_path\"].str.contains(s)][\"image_path\"].values[0]\n    except:\n        print(s)\n        return \"\"","metadata":{"papermill":{"duration":0.132775,"end_time":"2021-05-20T09:59:44.167938","exception":false,"start_time":"2021-05-20T09:59:44.035163","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:09.322007Z","iopub.execute_input":"2021-05-26T05:25:09.322352Z","iopub.status.idle":"2021-05-26T05:25:09.329127Z","shell.execute_reply.started":"2021-05-26T05:25:09.322316Z","shell.execute_reply":"2021-05-26T05:25:09.328303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df[test_df.image_path == \"None\"].apply(image_file_path,axis = 1)\nNa_test_df = test_df[test_df.image_path == \"\"]\ntest_df = test_df[test_df.image_path != \"\"]","metadata":{"papermill":{"duration":2.129781,"end_time":"2021-05-20T09:59:46.421908","exception":false,"start_time":"2021-05-20T09:59:44.292127","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:09.331104Z","iopub.execute_input":"2021-05-26T05:25:09.331474Z","iopub.status.idle":"2021-05-26T05:25:11.823513Z","shell.execute_reply.started":"2021-05-26T05:25:09.331435Z","shell.execute_reply":"2021-05-26T05:25:11.822704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cpu_device = torch.device(\"cpu\")","metadata":{"papermill":{"duration":0.136717,"end_time":"2021-05-20T09:59:46.685559","exception":false,"start_time":"2021-05-20T09:59:46.548842","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:11.824899Z","iopub.execute_input":"2021-05-26T05:25:11.825177Z","iopub.status.idle":"2021-05-26T05:25:11.835872Z","shell.execute_reply.started":"2021-05-26T05:25:11.825152Z","shell.execute_reply":"2021-05-26T05:25:11.835129Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_dataset = COVIDTrainDataLoader(test_df, get_test_transform(),\"Test\")\n\ntest_data_loader = DataLoader(\n    test_dataset,\n    batch_size=8,\n    shuffle=False,\n    num_workers=1,\n    drop_last=False,\n    collate_fn=collate_fn\n)","metadata":{"papermill":{"duration":0.134236,"end_time":"2021-05-20T09:59:46.946527","exception":false,"start_time":"2021-05-20T09:59:46.812291","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:11.83715Z","iopub.execute_input":"2021-05-26T05:25:11.837503Z","iopub.status.idle":"2021-05-26T05:25:11.844835Z","shell.execute_reply.started":"2021-05-26T05:25:11.837468Z","shell.execute_reply":"2021-05-26T05:25:11.843999Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def format_prediction_string(labels, boxes, scores):\n    pred_strings = []\n    for j in zip(labels, scores, boxes):\n        pred_strings.append(\"{0} {1:.4f} {2} {3} {4} {5}\".format(\n            j[0], j[1], j[2][0], j[2][1], j[2][2], j[2][3]))\n\n    return \" \".join(pred_strings)","metadata":{"papermill":{"duration":0.136377,"end_time":"2021-05-20T09:59:47.209903","exception":false,"start_time":"2021-05-20T09:59:47.073526","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:11.846181Z","iopub.execute_input":"2021-05-26T05:25:11.84657Z","iopub.status.idle":"2021-05-26T05:25:11.853459Z","shell.execute_reply.started":"2021-05-26T05:25:11.846496Z","shell.execute_reply":"2021-05-26T05:25:11.852325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Test Data","metadata":{"papermill":{"duration":0.125865,"end_time":"2021-05-20T09:59:47.461461","exception":false,"start_time":"2021-05-20T09:59:47.335596","status":"completed"},"tags":[]}},{"cell_type":"code","source":"images, image_ids = next(iter(test_data_loader))\nimages = list(image.to(device) for image in images)\n\nfor number in random.sample([1,2,3],3):\n  img = images[number].permute(1,2,0).cpu().numpy()\n  #labels= targets[number]['labels'].cpu().numpy().astype(np.int32)\n  fig, ax = plt.subplots(1, 1, figsize=(16, 8))\n  ax.set_axis_off()\n  ax.imshow(img)","metadata":{"papermill":{"duration":17.510001,"end_time":"2021-05-20T10:00:05.097809","exception":false,"start_time":"2021-05-20T09:59:47.587808","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:11.855661Z","iopub.execute_input":"2021-05-26T05:25:11.85619Z","iopub.status.idle":"2021-05-26T05:25:29.008952Z","shell.execute_reply.started":"2021-05-26T05:25:11.856155Z","shell.execute_reply":"2021-05-26T05:25:29.008082Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sample Prediction","metadata":{"papermill":{"duration":0.137922,"end_time":"2021-05-20T10:00:05.376316","exception":false,"start_time":"2021-05-20T10:00:05.238394","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\nmodel.eval()\nimages = list(img.to(device) for img in images)\n\noutputs = model(images)\noutputs = [{k: v.to(cpu_device) for k, v in t.items()} for t in outputs]\n\n\nboxes = outputs[2]['boxes'].cpu().detach().numpy().astype(np.int32)\nimg = images[2].permute(1,2,0).cpu().detach().numpy()\nlabels= outputs[2]['labels'].cpu().detach().numpy().astype(np.int32)\nscore = outputs[2]['scores']\n\nfig, ax = plt.subplots(1, 1, figsize=(16, 8))\n\nimg = cv2.cvtColor(np.float32(img), cv2.COLOR_RGB2BGR)\nfor i in range(len(boxes)):\n  img = cv2.rectangle(img,(boxes[i][0]+paddingSize,boxes[i][1]+paddingSize),(boxes[i][2]+paddingSize,boxes[i][3]+paddingSize),(255,0,0),20)\n  #img = cv2.putText(img, label_to_name(labels[i]), (int(boxes[i][0]), int(boxes[i][1])), cv2.FONT_HERSHEY_TRIPLEX,3, (255,0,0), 3, cv2.LINE_AA)\n\nax.set_axis_off()\nax.imshow(img)\n\n","metadata":{"papermill":{"duration":3.735536,"end_time":"2021-05-20T10:00:09.250304","exception":false,"start_time":"2021-05-20T10:00:05.514768","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2021-05-26T05:25:29.01305Z","iopub.execute_input":"2021-05-26T05:25:29.014984Z","iopub.status.idle":"2021-05-26T05:25:32.457557Z","shell.execute_reply.started":"2021-05-26T05:25:29.01494Z","shell.execute_reply":"2021-05-26T05:25:32.456732Z"},"trusted":true},"execution_count":null,"outputs":[]}]}