{"cells":[{"metadata":{},"cell_type":"markdown","source":"<div style=\"margin-left:50px;margin-right:50px;\">\n<center><div style=\"background:url(https://www.teahub.io/photos/full/19-197024_backgrounds-medical-hd-best-free-medicine-wallpapers-medicine.jpg); border-radius:8px;padding:20px; color:white;\"><h1 style=\"font-size: 40px;\">DATA EXPLANATION VIDEO</h1></div><br>\n    <p style=\"font-size:20px\">Did I actually sit down to discuss this competetion with myself? You bet I did.<br><br><a hreaf=\"https://www.linkedin.com/posts/vyom-bhatia-40ba79181_computervision-activity-6747848578720419840-wMNc\">Check it out here on LinkedIn.</a></p></div>\n   "},{"metadata":{},"cell_type":"markdown","source":"<div style=\"margin-left:50px;margin-right:50px;\">\n<center><div style=\"background:url(https://www.teahub.io/photos/full/19-197024_backgrounds-medical-hd-best-free-medicine-wallpapers-medicine.jpg); border-radius:8px;padding:20px; color:white;\"><h1 style=\"font-size: 40px;\">TIME TO CODE</h1></div><br>\n<p style=\"font-size:20px;margin-left:50px;margin-right:50px;\">The point is simple, we need a model to solve the problem, let's build one. <br>\nWe will be using the latest <a href=\"https://paperswithcode.com/paper/bottleneck-transformers-for-visual\">Bottleneck Transformers</a>!<br><br>Also,<br><br>I have my semesters in 15 days and I haven't really studied anything, just busy building models ;) <br>I have no idea what I am going to do :D</p></center></div>"},{"metadata":{},"cell_type":"markdown","source":"# Importing Libraries + Data"},{"metadata":{},"cell_type":"markdown","source":"### Installing the Bottleneck Transformer:"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install bottleneck-transformer-pytorch","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Importing Stuff:"},{"metadata":{"trusted":true},"cell_type":"code","source":"import torch\nfrom torch import nn\nfrom torchvision.models import resnet50\n\nfrom bottleneck_transformer_pytorch import BottleStack\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"from fastai.vision.all import *\n\nimport pandas as pd","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Reading the CSV with annotations."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"data = pd.read_csv(\"../input/ranzcr-clip-catheter-line-classification/train_annotations.csv\")\ndata.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Forming the Dataloader"},{"metadata":{},"cell_type":"markdown","source":"### Defining the functions to grab those files."},{"metadata":{"trusted":true},"cell_type":"code","source":"path = \"../input/ranzcr-clip-catheter-line-classification\"\n\ndef get_x(r):\n    return path + \"/\" + 'train' + \"/\" + r['StudyInstanceUID'] + \".jpg\"\n\ndef get_y(r):\n    return r['label']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Finally setting up the Dataloader for FastAI.\n### Well below you can see we got a lot of hyperparmeter tuning. Hit me up on [LinkedIn](https://www.linkedin.com/in/vyom-bhatia-40ba79181/) on [Instagram](https://instagram.com/vyombhatia) if you got any questions."},{"metadata":{"trusted":true},"cell_type":"code","source":"def getblock(size=256,batch=3, data_df=data):\n    block = DataBlock(blocks=(ImageBlock, CategoryBlock),\n                      splitter=RandomSplitter(seed=42),\n                      get_x=get_x, # Using the function above.\n                      get_y=get_y, # Using the function above.\n                      item_tfms = RandomResizedCrop(256), # Resizing images to 400 pixels.\n                      \n                      # Data Augmentation time:\n                      batch_tfms = [*aug_transforms(mult=1.0, do_flip=False,\n                                                   flip_vert=False, max_rotate=0,\n                                                   min_zoom=1.0, max_zoom=1.0,\n                                                   pad_mode=\"zeros\",\n                                                   xtra_tfms=None,max_warp=0,\n                                                   p_affine=0, p_lighting=0),\n                                    Normalize.from_stats(*imagenet_stats)])\n    \n    return block.dataloaders(data_df, bs=batch)\n                      \n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dataloader = getblock()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Visualizing"},{"metadata":{},"cell_type":"markdown","source":"### Lets see how the data augmentations is doing:"},{"metadata":{"trusted":true},"cell_type":"code","source":"dataloader.show_batch()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Building the model"},{"metadata":{},"cell_type":"markdown","source":"### Easy Peasy grab a resnet50 on top of the BotNet and train the model using transfer learning:"},{"metadata":{"trusted":true},"cell_type":"code","source":"layer = BottleStack(\n    dim = 256,\n    fmap_size = 64,        # set specifically for imagenet's 224 x 224\n    dim_out = 2048,\n    proj_factor = 4,\n    downsample = True,\n    heads = 4,\n    dim_head = 128,\n    rel_pos_emb = True,\n    activation = nn.ReLU()\n)\n\nresnet = resnet50()\n\n# model surgery\n\nbackbone = list(resnet.children())\n\nmodel = nn.Sequential(\n    *backbone[:5],\n    layer,\n    nn.AdaptiveAvgPool2d((1, 1)),\n    nn.Flatten(1),\n    nn.Linear(2048, 1000)\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = Learner(dataloader, model, metrics = accuracy)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<div style=\"margin-left:50px;margin-right:50px;\">\n<center><div style=\"background:url(https://www.teahub.io/photos/full/19-197024_backgrounds-medical-hd-best-free-medicine-wallpapers-medicine.jpg); border-radius:8px;padding:20px; color:white;\"><h1 style=\"font-size: 40px;\">REMINDER</h1></div><br>\n<p style=\"font-size:20px;margin-left:50px;margin-right:50px;\">If you are planning to fork this, upvote and link me! :D</p></center></div>"},{"metadata":{},"cell_type":"markdown","source":"# Learning Rate Finder"},{"metadata":{},"cell_type":"markdown","source":"### I absolutely love this function from the FastAI library. \n### Why? It finds you the best Learning Rate."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fine_tune(2, 0.0006918309954926372)\n# The above LR was suggested by the lr_find.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Yes Karen, I know that the accuracy sucks with BotNet and my current hyperparameters for data augmentation. But this was just a little experimenting with the BotNet.\n\n### A little background about me?\n\n### I am an 18-year-old business major, Fresher at Delhi University. Thank you. Love from India."},{"metadata":{},"cell_type":"markdown","source":"<center><p style=\"font-size:20px;margin-left:50px;margin-right:50px;\">\n☁☁🌰🌰🌰☁☁<br>\n☁☁👂👀👂☁☁<br>\n☁☁👍👃👍☁☁<br>\n☁☁👍〰👍☁☁<br>\n☁🔵🔵🔵🔵🔵☁<br>\n🔵🔵🔴💛🔴🔵🔵<br>\n🔵☁🔵⁣🔴🔵☁🔵<br>\n🔵☁🔵🔵🔵☁🔵<br>\n👇☁🔴🔴🔴☁👇<br>\n☁☁🔵🔴🔵☁☁<br>\n☁☁🔵☁🔵☁☁<br>\n☁☁🔵☁🔵☁☁<br>\n☁☁👢☁👢☁☁<br>\n\n Random Superman Image with Emojis</p></center>"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}