{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Inference with a trained fastai/PyTorch model\nThis notebook is paired with a [training notebook](https://www.kaggle.com/bjoernholzhauer/fastai-how-to-set-up-efficientnet-b4-0-945-lb) that trains a fastai/PyTorch model using transfer learning on an EfficientNet-B4 pre-trained on ImageNet. This follows the strategy of training your model in one notebook (that can have the internet on) and then to run in the inference in another notebook (that can have internet off, as needed for submission). I borrowed a lot of the set-up from [a notebook by Zach Mueller in another competition](https://www.kaggle.com/muellerzr/fastai-abhishek-inference).\n\nFurther improvements could include taking into account the correlation of outputs - at the moment, I ignore that the different diagnoses can sometimes just not co-occur or do more often co-occur."},{"metadata":{},"cell_type":"markdown","source":"# Loading packages and defining augmentations\n\nI'm probably re-defining too many things from the training notebook here, but it does little harm."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import torch\ntorch.cuda.is_available()\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"%cd ../input/efficientnet-pytorch/EfficientNet-PyTorch/EfficientNet-PyTorch-master\nfrom efficientnet_pytorch import EfficientNet\n%cd -\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from fastai.vision.all import *\nimport albumentations\nimport re","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class AlbumentationsTransform(RandTransform):\n    \"A transform handler for multiple `Albumentation` transforms\"\n    split_idx,order=None,2\n    def __init__(self, train_aug, valid_aug): store_attr()\n    \n    def before_call(self, b, split_idx):\n        self.idx = split_idx\n    \n    def encodes(self, img: PILImage):\n        if self.idx == 0:\n            aug_img = self.train_aug(image=np.array(img))['image']\n        else:\n            aug_img = self.valid_aug(image=np.array(img))['image']\n        return PILImage.create(aug_img)\n    \n#def get_x(row): return data_path/row['image_id']\n#def get_y(row): return row['label']\n\nclass MyModel(Module):\n    def __init__(self, num_classes):\n\n        self.effnet = EfficientNet.from_pretrained(\"efficientnet-b4\")\n        self.dropout = nn.Dropout(0.1)\n        self.out = nn.Linear(1792, num_classes)\n\n    def forward(self, image):\n        batch_size, _, _, _ = image.shape\n\n        x = self.effnet.extract_features(image)\n        x = F.adaptive_avg_pool2d(x, 1).reshape(batch_size, -1)\n        outputs = self.out(self.dropout(x))\n        return outputs\n    \nclass AlbumentationsTransform(RandTransform):\n    \"A transform handler for multiple `Albumentation` transforms\"\n    split_idx,order=None,2\n    def __init__(self, train_aug, valid_aug): store_attr()\n    \n    def before_call(self, b, split_idx):\n        self.idx = split_idx\n    \n    def encodes(self, img: PILImage):\n        if self.idx == 0:\n            aug_img = self.train_aug(image=np.array(img))['image']\n        else:\n            aug_img = self.valid_aug(image=np.array(img))['image']\n        return PILImage.create(aug_img)    \n    \ndef get_train_aug(): return albumentations.Compose([            \n            albumentations.RandomResizedCrop(380,380, scale=(0.85, 1.0)),\n            albumentations.ShiftScaleRotate(shift_limit=0.025, scale_limit=0.1, rotate_limit=10, p=0.5),\n            albumentations.HueSaturationValue(\n                hue_shift_limit=0.2, \n                sat_shift_limit=0.2, \n                val_shift_limit=0.2, \n                p=0.5\n            ),\n            albumentations.RandomBrightnessContrast(\n                brightness_limit=(-0.1,0.1),\n                contrast_limit=(-0.1, 0.1), \n                p=0.5\n            ),\n            albumentations.CoarseDropout(p=0.5),\n            albumentations.Cutout(p=0.5)\n])    \n    \ndef get_valid_aug(): return albumentations.Compose([\n    albumentations.Resize(385,385),\n    albumentations.CenterCrop(380,380, p=1.)    \n], p=1.)\n\nitem_tfms = AlbumentationsTransform(get_train_aug(), get_valid_aug())    \n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load the pre-trained model\n\nNow we use the `load_learner` function in fastai to load the pre-trained model. During training, we used mixed precision (i.e. both 16-bit and 32-bit floating-point in the model) to make the training run faster and use less memory, but for inference we switch back to completely 32-bit"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = load_learner(Path('../input/fastai-first-try/baseline-b4'), \n                     cpu=False)\nlearn.to_native_fp32()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Set-up data loading\n\nThe only tricky bit here is that for the training data I had specified a path of `../input/ranzcr-clip-catheter-line-classification/train`, but here I want to use images from `../input/ranzcr-clip-catheter-line-classification/test`, so I pre-pend `../test/` before the image names, so that the column in the dataframe becomes e.g. `../test/1.2.826.0.1.3680043.8.498.10003659706701445041816900371598078663` and the dataloader will then turn this into `../input/ranzcr-clip-catheter-line-classification/train/../test/1.2.826.0.1.3680043.8.498.10003659706701445041816900371598078663.jpg`. Clearly, I set this up a bit stupidly and you can easily make this more elegant."},{"metadata":{"trusted":true},"cell_type":"code","source":"path = Path(\"../input\")\ndata_path = path/'ranzcr-clip-catheter-line-classification/test'\ndf = pd.DataFrame({'StudyInstanceUID': ['../test/' + re.sub(r'.jpg', '', file) for file in os.listdir('../input/ranzcr-clip-catheter-line-classification/test')]})\ndf\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_dl = learn.dls.test_dl(df)\ntest_dl.show_batch(figsize=(12,12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Get predictions using test-time-augmentation (TTA)\n\nNow we create augmented versions of each image in the test set, do predictions for each of these version (=[test-time-augmentation](https://docs.fast.ai/learner.html#TTA)) using the training augmentations and then also using the less \"aggressive\" test dataset augmentations, then we average the predicted probabilities with a ratio 75:25 ratio."},{"metadata":{"trusted":true},"cell_type":"code","source":"preds, _ = learn.tta(dl=test_dl, n=15, beta=0.25)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Create submission.csv\nNote, that here I'm taking out the '../test/' string I added above to facilitate data loading."},{"metadata":{"trusted":true},"cell_type":"code","source":"submit = pd.DataFrame(preds, columns=['ETT - Abnormal', 'ETT - Borderline',\n       'ETT - Normal', 'NGT - Abnormal', 'NGT - Borderline',\n       'NGT - Incompletely Imaged', 'NGT - Normal', 'CVC - Abnormal',\n       'CVC - Borderline', 'CVC - Normal', 'Swan Ganz Catheter Present'])\n\nsubmit.insert(0, 'StudyInstanceUID', [re.sub( '../test/', '', astring) for astring in df.StudyInstanceUID])\n\nsubmit\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submit.to_csv('submission.csv',\n              index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}