{"cells":[{"metadata":{},"cell_type":"markdown","source":"**BEFORE YOU COPY AND EDIT NOTEBOOK, PLEASE SUPPORT AND UPVOTE**\n\n# RANZCR CLiP - a simple EDA and fastai starter\n\nIn this competition, we are asked to apply machine learning in order to automatically detect malpositioned catheters and lines based on X-ray images of patients (a more important problem as COVID-19 cases continue to rise). Quick feedback on catheter and line placement could help clinicians better treat patients. The competition is hosted by the Royal Australian and New Zealand College of Radiologists (RANZCR), which is a not-for-profit professional organisation for clinical radiologists and radiation oncologists in Australia, New Zealand, and Singapore.\n\nIn this kernel, I will present a quick 'n dirty EDA and fastai starter.\n\n## A look at the data\nLet's start out by setting up our environment by installing and importing the required modules and setting a random seed:"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install ../input/pretrainedmodels/pretrainedmodels-0.7.4/pretrainedmodels-0.7.4/ > /dev/null\nimport pretrainedmodels","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport os\nimport pandas as pd\nfrom fastai.vision.all import *\nfrom fastai.metrics import *","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"set_seed(999)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's check what is available to us:"},{"metadata":{"trusted":true},"cell_type":"code","source":"dataset_path = Path('../input/ranzcr-clip-catheter-line-classification')\nos.listdir(dataset_path)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see that we have our train csv file with the train image names and labels, the sample submission csv with the test image names, and the train and test image folders. We also have the images in tfrecords format which is useful for quick loading of images, especially for TensorFlow and TPUs. We won't use this for today though.\n\nLet's check the train csv file:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv(dataset_path/'train.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's do some quick processing of the image filenames to make it easier to access:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df['path'] = train_df['StudyInstanceUID'].map(lambda x:str(dataset_path/'train'/x)+'.jpg')\ntrain_df = train_df.drop(columns=['StudyInstanceUID'])\ntrain_df = train_df.sample(frac=1).reset_index(drop=True) #shuffle dataframe\ntrain_df.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Okay let's check how many images are available in the training dataset:"},{"metadata":{"trusted":true},"cell_type":"code","source":"len_df = len(train_df)\nprint(f\"There are {len_df} images\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We have >30,000 images! Hopefully, we can develop a highly-predictive, robust, and generalizable model with this dataset.\nLet's check the distribution of the different classes:"},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nplt.bar(list(range(11)), train_df.sum(axis=0).values[:11])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There are 11 classes present:\n\n> ETT - Abnormal - endotracheal tube placement abnormal\n>\n> ETT - Borderline - endotracheal tube placement borderline abnormal\n> \n> ETT - Normal - endotracheal tube placement normal\n> \n> NGT - Abnormal - nasogastric tube placement abnormal\n> \n> NGT - Borderline - nasogastric tube placement borderline abnormal\n> \n> NGT - Incompletely Imaged - nasogastric tube placement inconclusive due to imaging\n> \n> NGT - Normal - nasogastric tube placement borderline normal\n> \n> CVC - Abnormal - central venous catheter placement abnormal\n> \n> CVC - Borderline - central venous catheter placement borderline abnormal\n> \n> CVC - Normal - central venous catheter placement normal\n> \n> Swan Ganz Catheter Present\n> \n\nThis is a multi-label classification problem, so a single image could have multiple labels.\n\nThere are a lot of normal (class 9) and borderline (class 8) images. Maybe oversampling or weighted loss may help to deal with this imbalance.\n\nLet's check an example image to see what it looks like:"},{"metadata":{"trusted":true},"cell_type":"code","source":"from PIL import Image\n\nim = Image.open(train_df['path'][1])\nwidth, height = im.size\nprint(width,height) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"im","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data loading\n\nAfter my quick 'n dirty EDA, let's load the data into fastai as DataLoaders objects.\n\nFirst, let's define the item transforms, and the batch transforms."},{"metadata":{"trusted":true},"cell_type":"code","source":"item_tfms = RandomResizedCrop(512, min_scale=0.75, ratio=(1.,1.))\nbatch_tfms = [*aug_transforms(size=256, max_warp=0), Normalize.from_stats(*imagenet_stats)]\nbs=128","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can use the DataBlock API to create DataLoaders for this task.\n\n`blocks` --> indicates the type of data that our DataLoader returns. Here, we want to return a tuple of the image (`ImageBlock`) and the multi-label target (`MultiCategoryBlock`). Here, I also make sure to pass in the names of the labels as `vocab`.\n\n`splitter` --> Here we define how we want to split our data into training and validation subsets. I am doing random splitting for now.\n\n`get_x` --> Here we tell fastai how to obtain the input images. `ColReader(12)` indicates that the image path can be found in `train_df.columns[12]`\n\n`get_y` --> Here we tell fastai how to obtain the targets. `ColReader(list(range(11)))` indicates that the target can be found in `train_df.columns[:11]`.\n\n`item_tfms` --> Here we tell fastai about any transforms we need to do for each image in the dataset (ex: resizing).\n\n`batch_tfms` --> Here we define any transforms that can take place on a batch of images (ex: many augmentations)."},{"metadata":{"trusted":true},"cell_type":"code","source":"label_names = list(train_df.columns[:11])\nranzcr = DataBlock(blocks=(ImageBlock, MultiCategoryBlock(encoded=True, vocab=label_names)),\n                   splitter = RandomSplitter(seed=999),\n                   get_x = ColReader(12),\n                   get_y = ColReader(list(range(11))),\n                   item_tfms = item_tfms,\n                   batch_tfms = batch_tfms\n                  )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Note how the defined DataBlock doesn't actually take the dataset DataFrame. To create the DataLoader, we now pass in `train_df` to the DataBlock. To confirm successful dataloader creation, we can use the `show_batch` command, which shows a subset of the batch:"},{"metadata":{"trusted":true},"cell_type":"code","source":"dls = ranzcr.dataloaders(train_df)\ndls.show_batch()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Model training:\n\nLet's train a simple SE-ResNext50 model. Since this competition doesn't allow internet access, I have added the pretrained weights as a dataset, and the below code cell will allow PyTorch to find the file:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Making pretrained weights work without needing to find the default filename\nif not os.path.exists('/root/.cache/torch/hub/checkpoints/'):\n        os.makedirs('/root/.cache/torch/hub/checkpoints/')\n!cp '../input/pytorch-se-resnext/se_resnext50_32x4d-a260b3a4.pth' '/root/.cache/torch/hub/checkpoints/se_resnext50_32x4d-a260b3a4.pth'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's define our model:"},{"metadata":{"trusted":true},"cell_type":"code","source":"m = pretrainedmodels.se_resnext50_32x4d(pretrained='imagenet')\nchildren = list(m.children())\nhead = nn.Sequential(nn.AdaptiveAvgPool2d(1), Flatten(), \n                                  nn.Linear(children[-1].in_features,11))\nmodel = nn.Sequential(nn.Sequential(*children[:-2]), head)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In fastai, the trainer class is the `Learner`, which takes in the data, model, optimizer, loss function, etc. and allows you to train models, make predictions, etc.\n\nLet's create our `Learner` with our data, model, and also enable mixed precision:"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = Learner(dls,model,splitter=default_split, metrics = [accuracy_multi]).to_native_fp16()\nlearn.freeze()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We are now provided with a Learner object which has a frozen model (only the weights of the head of the model can be updated). In order to train a model, we need to find the most optimal learning rate, which can be done with fastai's learning rate finder:"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As shown above, the optimal learning rate for training the frozen model is where the loss is decreasing most rapidly: around ~3e-2. We can use the `fine_tune` function to train a pretrained model with this given learning rate:"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fine_tune(9,base_lr=2e-1,cbs=[SaveModelCallback()])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.recorder.plot_loss()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = learn.to_native_fp32()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.save('fine-tune')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We plotted the loss, put the model back to fp32, and now we can export the model if we want to use later (i.e. for an inference kernel):\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.export()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Inference\n\nIt's very simple to perform inference with fastai. The `dls.test_dl` function allows you to create test dataloader using the same pipeline defined earlier."},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_df = pd.read_csv(dataset_path/'sample_submission.csv')\nsample_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"_sample_df = sample_df.copy()\n_sample_df['PatientID'] = 'None'\n_sample_df['path'] = _sample_df['StudyInstanceUID'].map(lambda x:str(dataset_path/'test'/x)+'.jpg')\n_sample_df = _sample_df.drop(columns=['StudyInstanceUID'])\ntest_dl = dls.test_dl(_sample_df)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can easily confirm that the test_dl is correct:\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_dl.show_batch()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's pass the dataloader to the model and get predictions."},{"metadata":{"trusted":true},"cell_type":"code","source":"preds, _ = learn.get_preds(dl=test_dl)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's make a submission with these predictions!\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df = sample_df\nfor i in range(len(submission_df)):\n    for j in range(len(label_names)):\n        submission_df.iloc[i, j+1] = preds[i][j].numpy().astype(np.float32)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df.to_csv(f'submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now, **WE ARE DONE!**\n\nIf you enjoyed this kernel, please give it an upvote. If you have any questions or suggestions, please leave a comment!"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}