{"cells":[{"metadata":{"_uuid":"4dc0035d0a4e1fea87fdf14584947f62597a4c0f"},"cell_type":"markdown","source":"## HISTOPATHOLOGIC CANCER DETECTION\n> ***Identify metastatic tissue in histopathologic scans of lymph node sections***\n\n---\n> ### INTRODUCTION OF COMPETITION (Histopathologic Cancer Detection)\n> <div class=\"competition-overview__content\"><div><div class=\"markdown-converter__text--rendered\"><p><img src=\"https://storage.googleapis.com/kaggle-media/competitions/playground/Microscope\" alt=\"Microscope\" width=\"350\" style=\"float: right;\">\n> In this competition, you must create an algorithm to identify metastatic cancer in small image patches taken from larger digital pathology scans. The data for this competition is a slightly modified version of the PatchCamelyon (PCam) <a href=\"https://github.com/basveeling/pcam\" rel=\"nofollow\">benchmark dataset</a> (the original PCam dataset contains duplicate images due to its probabilistic sampling, however, the version presented on Kaggle does not contain duplicates).\n> </p><p>PCam is highly interesting for both its size, simplicity to get started on, and approachability. In the authors' words:</p>\n> <p></p><blockquote> [PCam] packs the clinically-relevant task of metastasis detection into a straight-forward binary image classification task, akin to CIFAR-10 and MNIST. Models can easily be trained on a single GPU in a couple hours, and achieve competitive scores in the Camelyon16 tasks of tumor detection and whole-slide image diagnosis. Furthermore, the balance between task-difficulty and tractability makes it a prime suspect for fundamental machine learning research on topics as active learning, model uncertainty, and explainability. </blockquote><p></p>\n> #### Solution which we have given in the pytorch with resnet 101.\n\n> ### Evaluation\n> <div class=\"competition-overview__content\"><div><div class=\"markdown-converter__text--rendered\"><p>Submissions are evaluated on <a href=\"http://en.wikipedia.org/wiki/Receiver_operating_characteristic\" rel=\"nofollow\">area under the ROC curve</a> between the predicted probability and the observed target.</p>\n> <h4>Submission File</h4>\n> <p>For each <code>id</code> in the test set, you must predict a probability that center 32x32px region of a patch contains at least one pixel of tumor tissue. The file should contain a header and have the following format:</p>\n> <pre><code>id,label\n0b2ea2a822ad23fdb1b5dd26653da899fbd2c0d5,0\n95596b92e5066c5c52466c90b69ff089b39f2737,0\n248e6738860e2ebcf6258cdc1f32f299e0c76914,0\netc.\n>  </code></pre></div></div></div>\n\n> ## Outline of the Notebook\n> 1. [***Load Library***](#1)\n> 1. [***Class Distribution***](#2)\n> 1. [***Data Visulization***](#3)\n> 1. [***Normalize Images***](#4)\n> 1. [***Prepare data loaders***](#5)\n> 1. [***Model Training***](#6)\n> 1. [***Predication***](#7)\n> 1. [***Submission***](#8)\n\n---"},{"metadata":{"_uuid":"84d99ef441ab5e591d21728a62c3b0defaa08786"},"cell_type":"markdown","source":"## 1.Load Library <a id=\"1\"></a>"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# libraries\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport os\nimport cv2\nimport matplotlib.pyplot as plt\nimport matplotlib.style as style\nstyle.use('fivethirtyeight')\n\n%matplotlib inline\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import roc_auc_score\nimport torch\nfrom torch.utils.data import TensorDataset, DataLoader,Dataset\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torchvision\nimport torchvision.transforms as transforms\nimport torch.optim as optim\nfrom torch.optim import lr_scheduler\nimport time \nimport tqdm\nfrom PIL import Image\ntrain_on_gpu = True\nfrom torch.utils.data.sampler import SubsetRandomSampler\nfrom torch.optim.lr_scheduler import StepLR, ReduceLROnPlateau, CosineAnnealingLR\n\n\n\n# five_thirty_eight = [\n#     \"#30a2da\",\n#     \"#fc4f30\",\n#     \"#e5ae38\",\n#     \"#6d904f\",\n#     \"#8b8b8b\",\n# ]\n\n# sns.set_palette(five_thirty_eight)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"labels = pd.read_csv('../input/train_labels.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1789395babff7e8730ad644289f94f7e0a97ec17"},"cell_type":"code","source":"print(f'{len(os.listdir(\"../input/train\"))} pictures in train.')\nprint(f'{len(os.listdir(\"../input/test\"))} pictures in test.')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"15c28e9970c6e2f5b1be90f4378ea0b304bc08b2"},"cell_type":"markdown","source":"## 2.Class Distribution <a id=\"2\"></a>"},{"metadata":{"trusted":true,"_uuid":"3182c4c5f2b714521800feab2f226491bef43ecf"},"cell_type":"code","source":"sns.countplot('label',data=labels).set_title(\"Class Distribution\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"edfc8ef33efac653dc5881c8827ba48089695352"},"cell_type":"markdown","source":"## 3. Data Visulization <a id=\"3\"></a>"},{"metadata":{"trusted":true,"_uuid":"e5b740694848c1553ee67ab450cf9e0eaa88ae3b"},"cell_type":"code","source":"fig = plt.figure(figsize=(25, 4))\n# display 20 images\ntrain_imgs = os.listdir(\"../input/train\")\nfor idx, img in enumerate(np.random.choice(train_imgs, 20)):\n    ax = fig.add_subplot(2, 20//2, idx+1, xticks=[], yticks=[])\n    im = Image.open(\"../input/train/\" + img)\n    plt.imshow(im)\n    lab = labels.loc[labels['id'] == img.split('.')[0], 'label'].values[0]\n    ax.set_title(f'Label: {lab}')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"733dd2f48a6a906fa33f6607ba8f03e445423e94"},"cell_type":"markdown","source":"# 4.Normalize Images <a id=\"4\"></a>"},{"metadata":{"trusted":true,"_uuid":"a02480e11c4e31452dda44c2633bdefd7fe80f1a"},"cell_type":"code","source":"data_transforms = transforms.Compose([\n    #transforms.CenterCrop(32),\n    transforms.Grayscale(num_output_channels=3),\n    #transforms.RandomRotation(degrees=160,expand=True),\n    transforms.Pad(64, padding_mode='reflect'),\n    transforms.RandomHorizontalFlip(),\n    transforms.RandomVerticalFlip(),\n    transforms.ToTensor(),\n    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n    ])\ndata_transforms_test = transforms.Compose([\n    #transforms.CenterCrop(32),\n    transforms.Grayscale(num_output_channels=3),\n    #transforms.RandomRotation(degrees=160,expand=True),\n    transforms.Pad(64, padding_mode='reflect'),\n    transforms.ToTensor(),\n    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n    ])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"24449ffff4732eb5c76468099601bdd3cb049aa0"},"cell_type":"code","source":"# indices for validation\ntr, val = train_test_split(labels.label, stratify=labels.label, test_size=0.1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"902b090fea736d93953ee602cbc75f3f3b99c5c4"},"cell_type":"code","source":"# dictionary with labels and ids of train data\nimg_class_dict = {k:v for k, v in zip(labels.id, labels.label)}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e9fcff484380f8882db1699dc3815b6538607b12"},"cell_type":"code","source":"class CancerDataset(Dataset):\n    def __init__(self, datafolder, datatype='train', transform = transforms.Compose([transforms.CenterCrop(32),transforms.ToTensor()]), labels_dict={}):\n        self.datafolder = datafolder\n        self.datatype = datatype\n        self.image_files_list = [s for s in os.listdir(datafolder)]\n        self.transform = transform\n        self.labels_dict = labels_dict\n        if self.datatype == 'train':\n            self.labels = [labels_dict[i.split('.')[0]] for i in self.image_files_list]\n        else:\n            self.labels = [0 for _ in range(len(self.image_files_list))]\n\n    def __len__(self):\n        return len(self.image_files_list)\n\n    def __getitem__(self, idx):\n        img_name = os.path.join(self.datafolder, self.image_files_list[idx])\n        image = Image.open(img_name)\n        image = self.transform(image)\n        img_name_short = self.image_files_list[idx].split('.')[0]\n\n        if self.datatype == 'train':\n            label = self.labels_dict[img_name_short]\n        else:\n            label = 0\n        return image, label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"261a99e836ea413a966f5fce6cec32f78a21b30e"},"cell_type":"code","source":"%%time\n# Load train data \ndataset = CancerDataset(datafolder='../input/train/', datatype='train', transform=data_transforms, labels_dict=img_class_dict)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4918c71166742673f8cdff1d2fc7d7449afa5dab"},"cell_type":"code","source":"%%time\n# Load test data \ntest_set = CancerDataset(datafolder='../input/test/', datatype='test', transform=data_transforms_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"07d0441f83ec529e6d3f5cb32aaf2e2311f1ef8c"},"cell_type":"code","source":"dataset = CancerDataset(datafolder='../input/train/', datatype='train', transform=data_transforms, labels_dict=img_class_dict)\ntest_set = CancerDataset(datafolder='../input/test/', datatype='test', transform=data_transforms_test)\ntrain_sampler = SubsetRandomSampler(list(tr.index))\nvalid_sampler = SubsetRandomSampler(list(val.index))\nbatch_size = 512\nnum_workers = 0","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0cea7e685c4317c3dc32ef08c48b7533af06930f"},"cell_type":"markdown","source":"## 5.Prepare data loaders <a id=\"5\"></a>"},{"metadata":{"trusted":true,"_uuid":"814d194807e403b5b04da310eaaf76546fac084a"},"cell_type":"code","source":"# prepare data loaders (combine dataset and sampler)\ntrain_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, sampler=train_sampler, num_workers=num_workers)\nvalid_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, sampler=valid_sampler, num_workers=num_workers)\ntest_loader = torch.utils.data.DataLoader(test_set, batch_size=batch_size, num_workers=num_workers)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"45c5bf2a0b910b0e347ebb1e32852d3bfe47e633"},"cell_type":"code","source":"model_conv = torchvision.models.resnet101(pretrained=True)\nfor i, param in model_conv.named_parameters():\n    param.requires_grad = False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ee37923851c53f52a1e60e03b22825182f315b16"},"cell_type":"code","source":"model_conv","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"787b84812778165184f0c2e511db55c027ce6caa"},"cell_type":"code","source":"num_ftrs = model_conv.fc.in_features\nmodel_conv.fc = nn.Linear(num_ftrs, 2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1a75ef6b6c50652b0e2d5c61508ec231abd67f1a"},"cell_type":"code","source":"model_conv.cuda()\ncriterion = nn.BCEWithLogitsLoss()\n\n# specify optimizer (stochastic gradient descent) and learning rate = 0.001\noptimizer = optim.SGD(model_conv.fc.parameters(), lr=10**-3, momentum=0.9)\n#scheduler = CyclicLR(optimizer, base_lr=lr, max_lr=0.01, step_size=5, mode='triangular2')\nexp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=5, gamma=0.1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"416ff1eafd122714fd11d865b40517fb32d67fa2"},"cell_type":"code","source":"valid_loss_min = np.Inf\npatience = 10\n# current number of epochs, where validation loss didn't increase\np = 0\n# whether training should be stopped\nstop = False\n\n# number of epochs to train the model\nn_epochs = 5\nfor epoch in range(1, n_epochs+1):\n    print(time.ctime(), 'Epoch:', epoch)\n\n    train_loss = []\n    exp_lr_scheduler.step()\n    train_auc = []\n\n    for batch_i, (data, target) in enumerate(train_loader):\n\n        data, target = data.cuda(), target.cuda()\n\n        optimizer.zero_grad()\n        output = model_conv(data)\n        loss = criterion(output[:,1], target.float())\n        train_loss.append(loss.item())\n        \n        a = target.data.cpu().numpy()\n        b = output[:,-1].detach().cpu().numpy()\n        train_auc.append(roc_auc_score(a, b))\n\n        loss.backward()\n        optimizer.step()\n    \n    model_conv.eval()\n    val_loss = []\n    val_auc = []\n    for batch_i, (data, target) in enumerate(valid_loader):\n        data, target = data.cuda(), target.cuda()\n        output = model_conv(data)\n\n        loss = criterion(output[:,1], target.float())\n\n        val_loss.append(loss.item()) \n        a = target.data.cpu().numpy()\n        b = output[:,-1].detach().cpu().numpy()\n        val_auc.append(roc_auc_score(a, b))\n\n    print(f'Epoch {epoch}, train loss: {np.mean(train_loss):.4f}, valid loss: {np.mean(val_loss):.4f}, train auc: {np.mean(train_auc):.4f}, valid acc: {np.mean(val_auc):.4f}')\n    \n    valid_loss = np.mean(val_loss)\n    if valid_loss <= valid_loss_min:\n        print('Validation loss decreased ({:.6f} --> {:.6f}).  Saving model ...'.format(\n        valid_loss_min,\n        valid_loss))\n        torch.save(model_conv.state_dict(), 'model.pt')\n        valid_loss_min = valid_loss\n        p = 0\n\n    # check if validation loss didn't improve\n    if valid_loss > valid_loss_min:\n        p += 1\n        print(f'{p} epochs of increasing val loss')\n        if p > patience:\n            print('Stopping training')\n            stop = True\n            break        \n            \n    if stop:\n        break","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"13de452f1c56c7c9dbdaa31038e52da6f6f010fd"},"cell_type":"code","source":"model_conv.eval()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7a54d94e3aeb67f9583dd59b52eeb0ec40349461"},"cell_type":"code","source":"preds = []\nfor batch_i, (data, target) in enumerate(test_loader):\n    data, target = data.cuda(), target.cuda()\n    output = model_conv(data)\n\n    pr = output[:,1].detach().cpu().numpy()\n    for i in pr:\n        preds.append(i)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dccd7ee24aa842c537a22c892d85fa57b011760d"},"cell_type":"code","source":"test_preds = pd.DataFrame({'imgs': test_set.image_files_list, 'preds': preds})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"fb324377a876764135620f99050a953dd4c02200"},"cell_type":"code","source":"test_preds['imgs'] = test_preds['imgs'].apply(lambda x: x.split('.')[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"23c3d03828266742e697b97472f0f30ae52fdef4"},"cell_type":"code","source":"sub = pd.read_csv('../input/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b21de332768766ae85c91ef62234d26ea1b4fe3b"},"cell_type":"code","source":"sub = pd.merge(sub, test_preds, left_on='id', right_on='imgs')\nsub = sub[['id', 'preds']]\nsub.columns = ['id', 'label']\nsub.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"df85e82faaaf185e346583115c3f761d5e5aae56"},"cell_type":"code","source":"sub.to_csv('ResNet.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}