{"cells":[{"metadata":{},"cell_type":"markdown","source":"This is my first Kaggle Kernel so I am always open to feedback please upvote if you enjoy, hopefully it will be the first kernel of many.\n\n\n# Iterating RANZCR Baselines in a Flash\n\n\n## What is Flash?\n\n![](https://miro.medium.com/max/1400/1*zF-Uy9kX-Fe38_NjE8hhSw.png)\nPyTorch Lightning Flash is a collection of tasks for fast prototyping, baselining and fine-tuning scalable Deep Learning models, built on PyTorch Lightning. This Kernel shows to go from simple baseline defaults < 15 lines of code to fine-tuning more complex state of the art models with complex augmenations using flash.\n\n\nCheck out this post describing [Flash](https://medium.com/pytorch/introducing-lightning-flash-the-fastest-way-to-get-started-with-deep-learning-202f196b3b98) and the repo on [GitHub](https://github.com/PyTorchLightning/lightning-flash) for more info.\n\n","attachments":{}},{"metadata":{},"cell_type":"markdown","source":"## Getting Started With a Baseline\n\nLet's get started by creating a Flash baseline for the Ranzcr challenge with about 15 lines of code using the default Resnet 18 configuration.\n\n\n### Install Flash"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install git+https://github.com/aribornstein/lightning-flash.git@flash_multilabel_clf_aug","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nfrom flash.data import labels_from_csv\nfrom flash.vision import ImageClassificationData\nfrom flash.vision import ImageClassifier\nfrom flash import Trainer\nfrom torch.nn import functional as F\n\n\nroot = '../input/ranzcr-clip-catheter-line-classification'\ncolumns = ['ETT - Abnormal', 'ETT - Borderline', 'ETT - Normal',\n           'NGT - Abnormal','NGT - Borderline','NGT - Incompletely Imaged','NGT - Normal',\n           'CVC - Abnormal', 'CVC - Borderline', 'CVC - Normal', 'Swan Ganz Catheter Present']\n\n# get train dataloader\ndata = ImageClassificationData.from_filepaths(\n    train_filepaths=os.path.join(root, 'train'),\n    train_labels= labels_from_csv(os.path.join(root, 'train.csv'),'StudyInstanceUID', representation='onehot', feature_cols=columns),\n    valid_split=0.10)\n\nmodel = ImageClassifier(multilabel=True, num_classes=len(columns), loss_fn=F.binary_cross_entropy_with_logits)\n\n## Fine Tune With 1 GPU\ntrainer = Trainer(gpus=1, max_epochs=1)\ntrainer.finetune(model, data, strategy='no_freeze')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### That's all we needed to do to get a baseline model. Not bad for less than 15 lines of code. Now granted this model isn't getting on a leader board anytime soon but we can do much better let's play around with some complex state of the art backbones using the [Timm package](https://github.com/rwightman/pytorch-image-models).\n"},{"metadata":{},"cell_type":"markdown","source":"## State of the Art with Timm Resnet200d and ViT"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install timm","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import timm\nimport torch\nfrom pytorch_lightning.callbacks.early_stopping import EarlyStopping\nimport pytorch_lightning as pl\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Custom Resnet200d Backbone"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = timm.create_model('resnet200d', pretrained=True)\nmodel.global_pool = torch.nn.Identity()\nmodel.fc = torch.nn.Identity()\npooling = torch.nn.AdaptiveAvgPool2d(1)\nbackbone = (model, model.num_features)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = ImageClassifier(backbone=backbone, # use resnet200d backbone\n                        optimizer = torch.optim.Adam, # Use Adam instead of SGD\n                        loss_fn=F.binary_cross_entropy_with_logits,\n                        multilabel=True,\n                        num_classes=len(columns))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## Fine Tune With 1 GPU\ntrainer = Trainer(\n    gpus=1, \n    max_epochs=1,\n    auto_lr_find=True,# auto find learning rate     \n    precision=16, # use 16 bit precision  \n    auto_scale_batch_size='binsearch', # maximize batch size to fit device memory\n    callbacks=[EarlyStopping(monitor='val_binary_cross_entropy_with_logits')] # early stopping\n)\n\ntrainer.finetune(model, data, strategy='freeze')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Custom ViT Backbone"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = timm.create_model('vit_base_patch16_224', pretrained=True)\nbackbone = (model, model.num_features)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = ImageClassifier(backbone=backbone, # use resnet200d backbone\n                        optimizer = torch.optim.Adam, # Use Adam instead of SGD\n                        loss_fn=F.binary_cross_entropy_with_logits,\n                        multilabel=True,\n                        num_classes=len(columns))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## Fine Tune With 1 GPU\ntrainer = Trainer(\n    gpus=1, \n    max_epochs=1,\n    auto_lr_find=True,# auto find learning rate     \n    precision=16, # use 16 bit precision  \n    auto_scale_batch_size='binsearch', # maximize batch size to fit device memory\n    callbacks=[EarlyStopping(monitor='val_binary_cross_entropy_with_logits')] # early stopping\n)\n\ntrainer.finetune(model, data, strategy='freeze')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Adding Data Augmenation with [Albumentations](https://github.com/albumentations-team/albumentations)\n![](https://camo.githubusercontent.com/3bb6e4bb500d96ad7bb4e4047af22a63ddf3242a894adf55ebffd3e184e4d113/68747470733a2f2f686162726173746f726167652e6f72672f776562742f62642f6e652f72762f62646e6572763563746b75646d73617a6e687734637273646669772e6a706567)\n","attachments":{}},{"metadata":{},"cell_type":"markdown","source":"Now we are making progress let's add some data augmentations using the Albumentations library insipred by this [repo by VietHoang1710](https://github.com/VietHoang1710/RANZCR_2021)"},{"metadata":{"trusted":true},"cell_type":"code","source":"image_size = 700\ntrain_transform = albumentations.Compose(\n            [\n                albumentations.RandomResizedCrop(height=image_size, width=image_size, scale=(0.9, 1), p=1),\n                albumentations.HorizontalFlip(p=0.5),\n                albumentations.ShiftScaleRotate(p=0.5),\n                albumentations.HueSaturationValue(hue_shift_limit=10, sat_shift_limit=10, val_shift_limit=10, p=0.7),\n                albumentations.RandomBrightnessContrast(\n                    brightness_limit=(-0.2, 0.2), contrast_limit=(-0.2, 0.2), p=0.7\n                ),\n                albumentations.CLAHE(clip_limit=(1, 4), p=0.5),\n                albumentations.OneOf(\n                    [\n                        albumentations.OpticalDistortion(distort_limit=1.0),\n                        albumentations.GridDistortion(num_steps=5, distort_limit=1.0),\n                        albumentations.ElasticTransform(alpha=3),\n                    ],\n                    p=0.2,\n                ),\n                albumentations.OneOf(\n                    [\n                        albumentations.GaussNoise(var_limit=[10, 50]),\n                        albumentations.GaussianBlur(),\n                        albumentations.MotionBlur(),\n                        albumentations.MedianBlur(),\n                    ],\n                    p=0.2,\n                ),\n                albumentations.OneOf(\n                    [\n                        albumentations.JpegCompression(),\n                        albumentations.Downscale(scale_min=0.1, scale_max=0.15),\n                    ],\n                    p=0.2,\n                ),\n                albumentations.Resize(height=image_size, width=image_size),\n                albumentations.IAAPiecewiseAffine(p=0.2),\n                albumentations.IAASharpen(p=0.2),\n                albumentations.Cutout(\n                    max_h_size=int(image_size * 0.1),\n                    max_w_size=int(image_size * 0.1),\n                    num_holes=5,\n                    p=0.5,\n                ),\n                albumentations.Normalize(),\n            ]\n)\n\ntransform_val = albumentations.Compose(\n    [\n        albumentations.Resize(height=image_size, width=image_size),\n        albumentations.Normalize(),\n    ]\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# get train dataloader with transforms \ndata = ImageClassificationData.from_filepaths(\n    train_transform = train_transform,\n    valid_transform = transform_val,\n    batch_size=32,\n    train_filepaths=os.path.join(root, 'train'),\n    train_labels=train_labels,\n    valid_split=0.10,\n    num_workers=4\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## Fine Tune With 1 GPU\ntrainer = Trainer(\n    gpus=1, \n    max_epochs=1,\n    auto_lr_find=True,# auto find learning rate     \n    precision=16, # use 16 bit precision  \n    auto_scale_batch_size='binsearch', # maximize batch size to fit device memory\n    callbacks=[EarlyStopping(monitor='val_binary_cross_entropy_with_logits')] # early stopping\n)\n\ntrainer.finetune(model, data, strategy='freeze')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Predict\n\nNow lets run our model on the test set. Future versions of Flash will make this process even cleaner."},{"metadata":{"trusted":true},"cell_type":"code","source":"from tqdm.notebook import tqdm \nimport pandas as pd \n\n# list of files to predict\ntest_path = os.path.join(root, 'test')\ntest_files = os.listdir(test_path)\ntest_files = [os.path.join(test_path, x) for x in test_files]\n\n# make the predictions in batches\n\npreds = []\nbatch_size = 64\nprint(\"Predicting\")\nfor i in tqdm(range(0, len(test_files), batch_size)):\n    end_i = min(i + batch_size, len(test_files))\n    batch_file_paths = test_files[i: end_i]\n    batch_preds = clf.predict(batch_file_paths)\n    preds.extend(batch_preds)\n    \n# read the names of the test files\ntest_file_dir = os.path.join(root, 'test')\ntest_file_names = os.listdir(test_file_dir)\ntest_file_names = [os.path.splitext(x)[0] for x in test_file_names]\npred_csv = pd.DataFrame(test_file_names, columns=['StudyInstanceUID'])\npred_df = pd.DataFrame(preds, columns=columns)\npred_csv = pd.concat([pred_csv, pred_df], axis=1)\npred_csv.to_csv(os.path.join(output, 'submission.csv'), index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Conclusions\n\nHopefully you now see how easy flash makes it to baseline and iterate image classificaiton tasks powered by PyTorch Lightning under the hood.\n\nI want to give a huge thanks to the authors of Flash, Timm and Albumentations you should star each of these repos to show your support and thank you a huge shout out to these kernels that helped insipire me please up vote them.\n\n- Sin's https://www.kaggle.com/underwearfitting/resnet200d-public-benchmark-2xtta-lb0-965/data?scriptVersionId=51087772\n- Ammarali32's https://www.kaggle.com/ammarali32/resnet200d-inference-single-model-lb-96-5/data\n- AshishGupta https://www.kaggle.com/roydatascience/resnet200d-public-benchmark-inference-model\n\nData Analysis\n\n- [https://www.kaggle.com/isaienkov/ranzcr-clip-data-understanding](https://www.kaggle.com/isaienkov/ranzcr-clip-data-understanding)\n- [https://www.kaggle.com/amitalexander/ranzcr-clip-exploratory-data-analysis](https://www.kaggle.com/amitalexander/ranzcr-clip-exploratory-data-analysis)\n- [https://www.kaggle.com/parthdhameliya77/ranzcr-clip-eda-class-imbalance-patient-overlap](https://www.kaggle.com/parthdhameliya77/ranzcr-clip-eda-class-imbalance-patient-overlap)\n\nData Preprocessing \n\n- Get rid of patient overlap between test and validation data [https://www.kaggle.com/parthdhameliya77/ranzcr-clip-eda-class-imbalance-patient-overlap](https://www.kaggle.com/parthdhameliya77/ranzcr-clip-eda-class-imbalance-patient-overlap)\n- Making images more clear [https://www.kaggle.com/aryaman1999/clearing-the-fog-outta-those-catheters](https://www.kaggle.com/aryaman1999/clearing-the-fog-outta-those-catheters)\n- Segmenting out the catheter lines - [https://www.kaggle.com/ryches/segmentation-model](https://www.kaggle.com/ryches/segmentation-model)\n- Make sure folds are stratified [https://www.kaggle.com/virilo/ranzcr-clip-stratified-kfold-to-team-up-v3/](https://www.kaggle.com/virilo/ranzcr-clip-stratified-kfold-to-team-up-v3/)\n\n"},{"metadata":{},"cell_type":"markdown","source":"## About the Author\n\nAaron (Ari) Bornstein is an AI researcher with a passion for history, engaging with new technologies and computational medicine. As Head of Developer Advocacy at Grid.ai, he collaborates with the Machine Learning Community to solve real-world problems with game-changing technologies that are then documented, open-sourced, and shared with the rest of the world.\n"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}