{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🐠 Reef - Pytorch FasterRCNN Infer\n\n## A self-contained, simple, pure pytorch 🔥 FasterR-CNN implementation with `LB=0.416`\n\n![](https://storage.googleapis.com/kaggle-competitions/kaggle/31703/logos/header.png)\n\n#### FasterR-CNN is one of the SOTA models for Object detection.\n\n#### In this notebook we present a simple solution using a pure pytorch Faster R-CNN with pretrained weights, and finetuning it for few epochs.\n\n\nIs is an adapted version of [this notebook](https://www.kaggle.com/pestipeti/pytorch-starter-fasterrcnn-inference) mentioned in [this comment](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/290016)\n\n## You can find the [training notebook here](https://www.kaggle.com/julian3833/coral-reef-pytorch-starter-fasterrcnn-train)\n\n\n# Please, _DO_ upvote if you find it useful or interesting!!\n\n \n \n \n \n\n---\n\n### Changelog\n\n\n|Best| Version | Description|  Weights| LB |\n|---| --- | ----| --- | --- |\n|| [**V5**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-293?scriptVersionId=80517131)  | 2 epochs. `detection_threshold=0.5` | [coral-reef-pytorch-starter-fasterrcnn-weights](https://www.kaggle.com/julian3833/coral-reef-pytorch-starter-fasterrcnn-weights) | `0.201`|\n|| [**V6**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-293?scriptVersionId=80518365)  | `detection_threshold=0.66` | [coral-reef-pytorch-starter-fasterrcnn-weights](https://www.kaggle.com/julian3833/coral-reef-pytorch-starter-fasterrcnn-weights) |`0.243`|\n|| [**V7**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-293?scriptVersionId=80520095)  | `detection_threshold=0.85` |[coral-reef-pytorch-starter-fasterrcnn-weights](https://www.kaggle.com/julian3833/coral-reef-pytorch-starter-fasterrcnn-weights) | `0.292`|\n|| [**V8**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-293?scriptVersionId=80534807)  | `detection_threshold=0.9` |  [coral-reef-pytorch-starter-fasterrcnn-weights](https://www.kaggle.com/julian3833/coral-reef-pytorch-starter-fasterrcnn-weights)|`0.293`|\n|| [**V10**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-293)  | V8 + 4 epochs | [reef-starter-torch-fasterrcnn-4e](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-4e) | `0.361`|\n|| [**V11**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-361?scriptVersionId=80610091)  | Train 8 epochs with validation. Pick best epoch (6th epoch ) | [reef-starter-torch-fasterrcnn-8e](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-8e) |`0.343`|\n|| [**V12**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-361?scriptVersionId=80622626)  | V11 + `detection_threshold=0.66` | [reef-starter-torch-fasterrcnn-8e](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-8e) |  `0.369`|\n|| [**V13**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-413?scriptVersionId=80624811)  |Best of 12 epochs, lower LR (epoch=5th) + `detection_threshold=0.66` | [reef-starter-torch-fasterrcnn-12e](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-12e) |  `0.413` |\n|_Best_| [**V14**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-416?scriptVersionId=80690903)  | Best of 12 epochs. Split 90-10. Augmentations.  (epoch=8th) + `detection_threshold=0.66` | [reef-starter-torch-fasterrcnn-12e-v2](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-12e-v2) |  `0.416` |\n|| [**V18**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-416?scriptVersionId=80783030)  | V14 + `DETECTION_THRESHOLD=0.35`. Tidy-up. | [reef-starter-torch-fasterrcnn-12e-v2](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-12e-v2) |  `0.384` |\n|| [**V28**](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-infer-lb-0-416?scriptVersionId=80783030)  | V14's train with last epoch (12th) | [reef-starter-torch-fasterrcnn-12e-v2](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-12e-v2) |  `??` |\n\n\n---","metadata":{"_kg_hide-input":false}},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"# Very few imports... pure pytorch!\nimport numpy as np\nfrom albumentations.pytorch.transforms import ToTensorV2\n\nimport torch\nimport torchvision\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-12-09T00:07:16.351838Z","iopub.execute_input":"2021-12-09T00:07:16.352181Z","iopub.status.idle":"2021-12-09T00:07:18.935908Z","shell.execute_reply.started":"2021-12-09T00:07:16.352128Z","shell.execute_reply":"2021-12-09T00:07:18.935093Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Constants","metadata":{}},{"cell_type":"code","source":"DEVICE = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')\n\nWEIGHTS_FILE = \"../input/reef-starter-torch-fasterrcnn-12e-v2/fasterrcnn_resnet50_fpn-e7.bin\"\n\nDETECTION_THRESHOLD = 0.66","metadata":{"execution":{"iopub.status.busy":"2021-12-09T00:07:18.937845Z","iopub.execute_input":"2021-12-09T00:07:18.938213Z","iopub.status.idle":"2021-12-09T00:07:18.962253Z","shell.execute_reply.started":"2021-12-09T00:07:18.938164Z","shell.execute_reply":"2021-12-09T00:07:18.961337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model","metadata":{}},{"cell_type":"code","source":"def get_model():\n    model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=False, pretrained_backbone=False)\n    num_classes = 2  # 1 class (starfish) + background\n\n    # get number of input features for the classifier\n    in_features = model.roi_heads.box_predictor.cls_score.in_features\n\n    # replace the pre-trained head with a new one\n    model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)\n\n    # Load the trained weights\n    model.load_state_dict(torch.load(WEIGHTS_FILE, map_location=DEVICE))\n    model.eval()\n\n    model = model.to(DEVICE)\n    return model\n\nmodel = get_model()","metadata":{"execution":{"iopub.status.busy":"2021-12-09T00:07:18.964087Z","iopub.execute_input":"2021-12-09T00:07:18.964648Z","iopub.status.idle":"2021-12-09T00:07:26.480021Z","shell.execute_reply.started":"2021-12-09T00:07:18.964595Z","shell.execute_reply":"2021-12-09T00:07:26.479201Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict functions","metadata":{}},{"cell_type":"code","source":"def format_prediction_string(boxes, scores):\n    # Format as specified in the evaluation page\n    pred_strings = []\n    for j in zip(scores, boxes):\n        pred_strings.append(\"{0:.10f} {1} {2} {3} {4}\".format(j[0], j[1][0], j[1][1], j[1][2], j[1][3]))\n    return \" \".join(pred_strings)\n\n\ndef predict(model, pixel_array):\n    # Predictions for a single image\n    \n    # Apply all the transformations that are required\n    pixel_array = pixel_array.astype(np.float32) / 255.\n    tensor_img = ToTensorV2(p=1.0)(image=pixel_array)['image'].unsqueeze(0)\n    \n    # Get predictions\n    with torch.no_grad():\n        outputs = model(tensor_img.to(DEVICE))[0]\n    \n    # Move predictions to cpu and numpy\n    boxes = outputs['boxes'].data.cpu().numpy()\n    scores = outputs['scores'].data.cpu().numpy()\n    \n    # Filter predictions with low score\n    boxes = boxes[scores >= DETECTION_THRESHOLD].astype(np.int32)\n    scores = scores[scores >= DETECTION_THRESHOLD]\n    \n    # Go back from x_min, y_min, x_max, y_max to x_min, y_min, w, h\n    boxes[:, 2] = boxes[:, 2] - boxes[:, 0]\n    boxes[:, 3] = boxes[:, 3] - boxes[:, 1]\n  \n    # Format results as requested in the Evaluation tab\n    return format_prediction_string(boxes, scores)","metadata":{"execution":{"iopub.status.busy":"2021-12-09T00:07:26.481366Z","iopub.execute_input":"2021-12-09T00:07:26.481745Z","iopub.status.idle":"2021-12-09T00:07:26.494278Z","shell.execute_reply.started":"2021-12-09T00:07:26.481694Z","shell.execute_reply":"2021-12-09T00:07:26.493021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submit\n\n(See: [Great Barrier Reef API Tutorial](https://www.kaggle.com/sohier/great-barrier-reef-api-tutorial))","metadata":{}},{"cell_type":"code","source":"import greatbarrierreef\nenv = greatbarrierreef.make_env()\niter_test = env.iter_test() \nfor (pixel_array, df_pred) in iter_test:\n    # Predictions\n    df_pred['annotations'] = predict(model, pixel_array)\n    env.predict(df_pred)","metadata":{"execution":{"iopub.status.busy":"2021-12-09T00:07:26.4963Z","iopub.execute_input":"2021-12-09T00:07:26.496859Z","iopub.status.idle":"2021-12-09T00:07:27.85991Z","shell.execute_reply.started":"2021-12-09T00:07:26.496806Z","shell.execute_reply":"2021-12-09T00:07:27.859077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Please, _DO_ upvote if you find it useful or interesting!!","metadata":{}}]}