{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Using fastai for the RANZCR CLiP - Catheter and Line Position Challenge\n\nIn this notebook, I had a look how easily one can adapt code from the Cassava leaf disease classification challenge to this one. This is heavily borrowing from Zach Mueller's work, where he shared a [fastai notebook](https://www.kaggle.com/muellerzr/recreating-abhishek-s-tez-with-fastai) that implemented [ideas of Abishek Thakur](https://www.kaggle.com/abhishek/tez-faster-and-easier-training-for-leaf-detection?scriptVersionId=47408263) and his [tez package](https://github.com/abhishekkrthakur/tez) in fastai.\n\nRight off the bat, this notebook trains a model that then gave me a LB score of 0.945.\n\nHere I'm showing how to adapt that code, in particular how to set-up the target. As my back-bone pre-trained model, I use EfficientNet-B4. Since we install the `efficientnet-pytorch` package I download the pre-trained weights, we leave the internet on in this notebook, but then leave it off in the inference notebook we use for submission."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"!pip install efficientnet-pytorch -qqq","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Image augmentation\n\nThe first thing to do, is to set up augmentations. I use albumentations for that. Looking at the images (see some examples further on in the notbook), I had the impression that I would not want to use augmentations that did not show most of the upper torso. That's why I picked `albumentations.RandomResizedCrop(380,380, scale=(0.85, 1.0))` (i.e. I keep 85 to 100% of the original image). I also limited the amount of rotation to what I saw happening naturally in the images, and kept some coarse dropout + cutout. Clearly, this needs a lot more experimentation."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"from fastai.vision.all import *\nimport albumentations # Data Augmentation\nfrom efficientnet_pytorch import EfficientNet # The Model\n\nclass AlbumentationsTransform(DisplayedTransform):\n    def __init__(self, aug): self.aug = aug\n    def encodes(self, img: PILImage):\n        aug_img = self.aug(image=np.array(img))['image']\n        return PILImage.create(aug_img)\n    \n    \nclass AlbumentationsTransform(RandTransform):\n    \"A transform handler for multiple `Albumentation` transforms\"\n    split_idx,order=None,2\n    def __init__(self, train_aug, valid_aug): store_attr()\n    \n    def before_call(self, b, split_idx):\n        self.idx = split_idx\n    \n    def encodes(self, img: PILImage):\n        if self.idx == 0:\n            aug_img = self.train_aug(image=np.array(img))['image']\n        else:\n            aug_img = self.valid_aug(image=np.array(img))['image']\n        return PILImage.create(aug_img)    \n    \ndef get_train_aug(): return albumentations.Compose([            \n            albumentations.RandomResizedCrop(380,380, scale=(0.85, 1.0)),\n            albumentations.ShiftScaleRotate(shift_limit=0.025, scale_limit=0.1, rotate_limit=10, p=0.5),\n            albumentations.HueSaturationValue(\n                hue_shift_limit=0.2, \n                sat_shift_limit=0.2, \n                val_shift_limit=0.2, \n                p=0.5\n            ),\n            albumentations.RandomBrightnessContrast(\n                brightness_limit=(-0.1,0.1),\n                contrast_limit=(-0.1, 0.1), \n                p=0.5\n            ),\n            albumentations.CoarseDropout(p=0.5),\n            albumentations.Cutout(p=0.5)\n])    \n    \ndef get_valid_aug(): return albumentations.Compose([\n    albumentations.Resize(385,385),\n    albumentations.CenterCrop(380,380, p=1.)    \n], p=1.)\n\nitem_tfms = AlbumentationsTransform(get_train_aug(), get_valid_aug())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Setting up the data block\n\nThis is what took me the longest to adapt. Now that I see the syntax, it seems straightforward enough, but it took me a while to realize that I had to use `encoded=True` and `vocab` in the `MultiCategoryBlock`. As you can see, there's no proper cross-validation here, which is of course something you'd want to add. Setting this up helped me understand better what each bit does:\n* `blocks`: Defining what data we are reading in and what it means. Here's it's images and tabular data indicating multi-label labels that are to be encoded with the names of the targets in this competition.\n* `splitter`: splits into training and validation data\n* `get_x` and `get_y`: these get the inputs and outputs. The `ColReader` function is a pretty convenient utility function for setting this up.\n* Finally, we specify transformations (i.e. augmentations)."},{"metadata":{"trusted":true},"cell_type":"code","source":"set_seed(538)\n\npath = Path(\"../input\")\ndata_path = path/'ranzcr-clip-catheter-line-classification'\ndf = pd.read_csv(data_path/'train.csv')\nblocks = (ImageBlock, MultiCategoryBlock)\nsplitter = RandomSplitter(valid_pct=0.2, seed=999)\n\n\nblock = DataBlock(blocks=(ImageBlock, \n                          MultiCategoryBlock(encoded=True, vocab=['ETT - Abnormal', 'ETT - Borderline','ETT - Normal', 'NGT - Abnormal', 'NGT - Borderline',\n                                     'NGT - Incompletely Imaged', 'NGT - Normal', 'CVC - Abnormal',\n                                     'CVC - Borderline', 'CVC - Normal', 'Swan Ganz Catheter Present'])),                   \n                   splitter=splitter,\n                   get_x=ColReader(0, pref='../input/ranzcr-clip-catheter-line-classification/train/', suff='.jpg'),\n                   get_y=ColReader(['ETT - Abnormal', 'ETT - Borderline','ETT - Normal', 'NGT - Abnormal', 'NGT - Borderline',\n                                     'NGT - Incompletely Imaged', 'NGT - Normal', 'CVC - Abnormal',\n                                     'CVC - Borderline', 'CVC - Normal', 'Swan Ganz Catheter Present']),\n                   item_tfms=item_tfms,\n                  batch_tfms=[Normalize.from_stats(*imagenet_stats)])\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here is how data gets loaded. As we can see our augmentations did not fundamentally change what the pictures look like, so this is probably the kind of augmentation that we would expect to help, but which can probably be improved upon."},{"metadata":{"trusted":true},"cell_type":"code","source":"dls = block.dataloaders(df, bs=32, val_bs=64)\ndls.show_batch(figsize=(12,12))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Setting up a model\n\nNow we define a PyTorch model, without doing anything very fancy. We're going from a pre-trained EfficientNet-B4 to predicting for the number of classes we have."},{"metadata":{"trusted":true},"cell_type":"code","source":"class MyModel(Module):\n    def __init__(self, num_classes):\n\n        self.effnet = EfficientNet.from_pretrained(\"efficientnet-b4\")\n        self.dropout = nn.Dropout(0.1)\n        self.out = nn.Linear(1792, num_classes)\n\n    def forward(self, image):\n        batch_size, _, _, _ = image.shape\n\n        x = self.effnet.extract_features(image)\n        x = F.adaptive_avg_pool2d(x, 1).reshape(batch_size, -1)\n        outputs = self.out(self.dropout(x))\n        return outputs\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since we set up the dataloader properly, it knows the number of classes (11)."},{"metadata":{"trusted":true},"cell_type":"code","source":"net = MyModel(dls.c)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We setup a fastai learner. I use mixed precision (fp16). As a loss function, I'm using binary cross-entroy loss on each target, but given the class imbalance, one might imagine that something else like label smoothing or focal loss might be worth trying. I monitored accuracy here, but really should also monitor the ROC."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = Learner(dls, \n                net, \n                loss_func=BCEWithLogitsLossFlat(),\n                metrics=[accuracy_multi]).to_native_fp16()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Training the model\nNow we pick a maximum learning rate and then train using [cosine annealing with warm restarts](https://paperswithcode.com/paper/sgdr-stochastic-gradient-descent-with-warm). There's plenty of other things to try here like fine tuning after initially freezing the backbone, or e.g. freezing the batchnorm layers. \n\nFinally, we same the trained model so we can use it in another notebook."},{"metadata":{"trusted":true},"cell_type":"code","source":"lr_min, lr_steep = learn.lr_find()\nprint(f'steep: {lr_steep}, min: {lr_min}')\nprint( (lr_min+lr_steep)/2 )\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_flat_cos(10, 3e-3, pct_start=0.0) # We could also do things like: cbs=[EarlyStoppingCallback(patience=3),SaveModelCallback()])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.export('baseline-b4')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}