{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Data Description\nIn this dataset, you are provided with a large number of small pathology images to classify. Files are named with an image id. The train_labels.csv file provides the ground truth for the images in the train folder. You are predicting the labels for the images in the test folder. A positive label indicates that the center 32x32px region of a patch contains at least one pixel of tumor tissue. Tumor tissue in the outer region of the patch does not influence the label. This outer region is provided to enable fully-convolutional models that do not use zero-padding, to ensure consistent behavior when applied to a whole-slide image.\n\nThe original PCam dataset contains duplicate images due to its probabilistic sampling, however, the version presented on Kaggle does not contain duplicates. We have otherwise maintained the same data and splits as the PCam benchmark."},{"metadata":{},"cell_type":"markdown","source":"## I will cover the following recipes:\n* Exploring the dataset\n* Creating a custom dataset\n* Splitting the dataset\n* Transforming the data\n* Creating dataloaders\n* Building the classification model\n* Defining the loss function\n* Defining the optimizer\n* Training and evaluation of the model\n* Deploying the model\n* Model inference on test data"},{"metadata":{},"cell_type":"markdown","source":"# Exploring the dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.pylab as plt\nfrom PIL import Image, ImageDraw\nimport numpy as np\nimport os","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path2csv ='../input/histopathologic-cancer-detection/train_labels.csv'\nlabels_df = pd.read_csv(path2csv)\nlabels_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels_df['label'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%matplotlib inline\nplt.hist(labels_df['label'])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**let's visualize a few images that have a positive label. A positive label shows that\nthe center 32 x 32 region of an image contains at least one pixel of tumor tissue**"},{"metadata":{"trusted":true},"cell_type":"code","source":"# get the ids for malignant images\nmalignantIds = labels_df.loc[labels_df['label']==1]['id'].values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"malignantIds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define the path to data:\npath2train = '../input/histopathologic-cancer-detection/train'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# show images in grayscale, if you want color change it to True\ncolor=False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# e set the figure sizes:\nplt.rcParams['figure.figsize'] = (10.0,10.0)\nplt.subplots_adjust(wspace=0, hspace=0)\nnrows,ncols=3,3","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### display the images"},{"metadata":{"trusted":true},"cell_type":"code","source":"for i,id_ in enumerate(malignantIds[:nrows*ncols]):\n    print(id_)\n    full_filenames = os.path.join(path2train , id_ +'.tif')\n    # load image\n    img = Image.open(full_filenames)\n    # draw a 32*32 rectangle\n    draw = ImageDraw.Draw(img)\n    draw.rectangle(((32, 32), (64, 64)),outline=\"green\")\n    plt.subplot(3, 3, i+1)\n    if color is True:\n        plt.imshow(np.array(img))\n    else:\n        plt.imshow(np.array(img)[:,:,0],cmap=\"gray\")\n    plt.axis('off')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#See what the path looks like\nfull_filenames","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels_df.loc[labels_df['label']==0]['id'].values","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Creating a custom dataset\ncreate a custom Dataset class by subclassing the PyTorch Dataset class."},{"metadata":{"trusted":true},"cell_type":"code","source":"import torch\nfrom torch.utils.data import Dataset\nimport pandas as pd\nimport torchvision.transforms as transforms\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fix torch random seed\ntorch.manual_seed(0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class histoCancerDataset(Dataset):\n    def __init__(self, data_dir, transform,data_type=\"train\"):\n    \n        # path to images\n        path2data=os.path.join(data_dir,data_type)\n\n        # get list of images\n        filenames = os.listdir(path2data)\n\n        # get the full path to images\n        self.full_filenames = [os.path.join(path2data, f) for f in filenames]\n\n        # labels are in a csv file named train_labels.csv\n        path2csvLabels=os.path.join(data_dir,\"train_labels.csv\")\n        labels_df=pd.read_csv(path2csvLabels)\n\n        # set data frame index to id\n        labels_df.set_index(\"id\", inplace=True)\n\n        # obtain labels from data frame\n        self.labels = [labels_df.loc[filename[:-4]].values[0] for filename in filenames]\n\n        self.transform = transform\n        \n    def __len__(self):\n        #return the size of dataset\n        return len(self.full_filenames)\n    \n    def __getitem__(self, idx):\n        # open image, apply transforms and return with label\n        image = Image.open(self.full_filenames[idx])  \n        image = self.transform(image)\n        return image, self.labels[idx]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import torchvision.transforms as transforms\ndata_transformer = transforms.Compose([transforms.ToTensor()])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_dir = \"../input/histopathologic-cancer-detection\"\nhisto_dataset = histoCancerDataset(data_dir, data_transformer, \"train\")\nprint(len(histo_dataset))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# load an image\nimg,label=histo_dataset[9]\nprint(img.shape,torch.min(img),torch.max(img))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Splitting the dataset\n\n**We need to provide a validation dataset to track the model's performance during training.\nWe use 20% of histo_dataset as the validation dataset and use the rest as the training\ndataset.**"},{"metadata":{},"cell_type":"markdown","source":"### Let's split histo_dataset:"},{"metadata":{"trusted":true},"cell_type":"code","source":"from torch.utils.data import random_split\n\nlen_histo=len(histo_dataset)\nlen_train=int(0.8*len_histo)\nlen_val=len_histo-len_train\n\ntrain_ds,val_ds=random_split(histo_dataset,[len_train,len_val])\n\nprint(\"train dataset length:\", len(train_ds))\nprint(\"validation dataset length:\", len(val_ds))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#get an image from the training dataset:\nfor x,y in train_ds:\n    print(x.shape,y)\n    break","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# get an image from the validation dataset:\nfor x,y in val_ds:\n    print(x.shape,y)\n    break","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Let's display a few samples from train_ds"},{"metadata":{"trusted":true},"cell_type":"code","source":"import torch.utils\nimport numpy as np \nnp.random.seed(0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Define a helper function to show an image:"},{"metadata":{"trusted":true},"cell_type":"code","source":"from torchvision import utils\nimport numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline\nnp.random.seed(0)\n\n\ndef show(img,y,color=False):\n    # convert tensor to numpy array\n    npimg = img.numpy()\n   \n    # Convert to H*W*C shape\n    npimg_tr=np.transpose(npimg, (1,2,0))\n    \n    if color==False:\n        npimg_tr=npimg_tr[:,:,0]\n        plt.imshow(npimg_tr,interpolation='nearest',cmap=\"gray\")\n    else:\n        # display images\n        plt.imshow(npimg_tr,interpolation='nearest')\n    plt.title(\"label: \"+str(y))\n\ngrid_size=4\nrnd_inds=np.random.randint(0,len(train_ds),grid_size)\nprint(\"image indices:\",rnd_inds)\n\nx_grid_train=[train_ds[i][0] for i in rnd_inds]\ny_grid_train=[train_ds[i][1] for i in rnd_inds]\n\nx_grid_train=utils.make_grid(x_grid_train, nrow=4, padding=2)\nprint(x_grid_train.shape)\n\nplt.rcParams['figure.figsize'] = (10.0, 5)\nshow(x_grid_train,y_grid_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"grid_size=4\nrnd_inds=np.random.randint(0,len(val_ds),grid_size)\nprint(\"image indices:\",rnd_inds)\nx_grid_val=[val_ds[i][0] for i in range(grid_size)]\ny_grid_val=[val_ds[i][1] for i in range(grid_size)]\nx_grid_val=utils.make_grid(x_grid_val, nrow=4, padding=2)\nprint(x_grid_val.shape)\nshow(x_grid_val,y_grid_val)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"# Transforming the data\n**we will define a few image transformations and then update the dataset\ntransformation function**"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_transformer = transforms.Compose([\n transforms.RandomHorizontalFlip(p=0.5),\n transforms.RandomVerticalFlip(p=0.5),\n transforms.RandomRotation(45),\ntransforms.RandomResizedCrop(96,scale=(0.8,1.0),ratio=(1.0,1.0)),\n transforms.ToTensor()])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"**For the validation dataset, we don't need any augmentation. So, we only convert\nthe images into tensors in the transforms function:**"},{"metadata":{"trusted":true},"cell_type":"code","source":"val_transformer = transforms.Compose([transforms.ToTensor()])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# overwrite the transform functions\ntrain_ds.transform=train_transformer\nval_ds.transform=val_transformer","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"# Creating dataloaders"},{"metadata":{"trusted":true},"cell_type":"code","source":"#let's define two dataloaders for the datasets:\nfrom torch.utils.data import DataLoader\ntrain_dl = DataLoader(train_ds, batch_size=32, shuffle=True)\nval_dl = DataLoader(val_ds, batch_size=64, shuffle=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# extract a batch from training data\nfor x,y in train_dl:\n    print(x.shape)\n    print(y.shape)\n    break","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# get a data batch from the validation dataloader\nfor x,y in val_dl:\n    print(x.shape)\n    print(y.shape)\n    break\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"# Building the classification model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# get labels for validation dataset\ny_val=[y for _,y in val_ds]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#define a function to calculate the classification accuracy\n\ndef accuracy(labels, out):\n    return np.sum(out==labels)/float(len(labels))\n\n#calculate a dumb baseline for all-zero predictions:\nacc_all_zeros = accuracy(y_val,np.zeros_like(y_val))\nprint(\"accuracy all zero prediction: %.2f\" %acc_all_zeros)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"acc_all_one = accuracy(y_val,np.ones_like(y_val))\nprint(\"accuracy all ones prediction: %.2f\" %acc_all_one)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**let's calculate a dumb baseline for random predictions:**\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"acc_random=accuracy(y_val,np.random.randint(2,size=len(y_val)))\nprint(\"accuracy random prediction: %.2f\" %acc_random)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**we developed findConv2DOutShape to automatically compute the output size of\na CNN and pooling layer. The inputs to this function are:**\n* H_in: an integer representing the height of input data\n* W_in: an integer representing the width of input data\n* conv: an object of the CNN layer\n* pool: an integer representing the pooling size and default to 2\n\nThe function receives the input size, H_in, W_in, and conv layer and provides the output\nsize, H_out, W_out. The formula to compute the output size is given in the following link:\nhttps://pytorch.org/docs/stable/nn.html\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"import torch.nn as nn\n#define the helper function\n\ndef findConv2dOutShape(H_in,W_in,conv,pool =2):\n    kernel_size=conv.kernel_size\n    stride=conv.stride\n    padding=conv.padding\n    dilation=conv.dilation\n    \n    # Ref: https://pytorch.org/docs/stable/nn.html\n    H_out=np.floor((H_in+2*padding[0]-dilation[0]*(kernel_size[0]-1)-1)/stride[0]+1)\n    W_out=np.floor((W_in+2*padding[1]-dilation[1]*(kernel_size[1]-1)-1)/stride[1]+1)\n    \n    if pool:\n        H_out/=pool\n        W_out/=pool\n        \n    return int(H_out),int(W_out)\n\n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# example\nconv1 = nn.Conv2d(3, 8, kernel_size=3)\nh,w=findConv2dOutShape(96,96,conv1)\nprint(h,w)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### now let's implement the CNN model.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"import torch.nn.functional as F\nimport torch.nn as nn\nclass Net (nn.Module):\n    def __init__(self,params):\n        super(Net,self). __init__()\n        C_in,H_in,W_in=params[\"input_shape\"]\n        init_f=params[\"initial_filters\"]\n        num_fc1=params[\"num_fc1\"]\n        num_classes=params[\"num_classes\"]\n        self.dropout_rate=params[\"dropout_rate\"]\n        \n        \n        self.conv1 = nn.Conv2d(C_in, init_f, kernel_size=3)\n        h,w=findConv2dOutShape(H_in,W_in,self.conv1)\n        \n        self.conv2 = nn.Conv2d(init_f, 2*init_f, kernel_size=3)\n        h,w=findConv2dOutShape(h,w,self.conv2)\n        \n        self.conv3 = nn.Conv2d(2*init_f, 4*init_f, kernel_size=3)\n        h,w=findConv2dOutShape(h,w,self.conv3)\n        \n        self.conv4 = nn.Conv2d(4*init_f, 8*init_f, kernel_size=3)\n        h,w=findConv2dOutShape(h,w,self.conv4)\n        \n        # compute the flatten size\n        self.num_flatten=h*w*8*init_f\n        self.fc1 = nn.Linear(self.num_flatten, num_fc1)\n        self.fc2 = nn.Linear(num_fc1, num_classes)\n        \n        \n    def forward(self, x):\n        x = F.relu(self.conv1(x))\n        x = F.max_pool2d(x, 2, 2)\n        x = F.relu(self.conv2(x))\n        x = F.max_pool2d(x, 2, 2)\n        x = F.relu(self.conv3(x))\n        x = F.max_pool2d(x, 2, 2)\n        x = F.relu(self.conv4(x))\n        x = F.max_pool2d(x, 2, 2)\n        x = x.view(-1, self.num_flatten)\n        x = F.relu(self.fc1(x))\n        x=F.dropout(x, self.dropout_rate, training= self.training)\n        x = self.fc2(x)\n        return F.log_softmax(x, dim=1)\n\n\n# dict to define model parameters\nparams_model={\"input_shape\": (3,96,96),\"initial_filters\": 8,\"num_fc1\": 100\n              ,\"dropout_rate\": 0.25, \"num_classes\": 2}\n        \n# create model\ncnn_model = Net(params_model)\n       ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"torch.cuda.is_available() ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# move model to cuda/gpu device\nif torch.cuda.is_available():\n    device = torch.device(\"cuda\")\n    cnn_model=cnn_model.to(device)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(cnn_model.parameters)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(next(cnn_model.parameters()).device)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pip install torchsummary","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from torchsummary import summary\nsummary(cnn_model, input_size=(3, 96, 96),device=device.type)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Defining the loss function"},{"metadata":{"trusted":true},"cell_type":"code","source":"loss_func = nn.NLLLoss(reduction=\"sum\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# use the loss in an example:\n\n# fixed random seed\ntorch.manual_seed(0)\n\nn,c=8,2\ny = torch.randn(n, c, requires_grad=True)\nls_F = nn.LogSoftmax(dim=1)\ny_out=ls_F(y)\nprint(y_out.shape)\n\ntarget = torch.randint(c,size=(n,))\nprint(target.shape)\n\nloss = loss_func(y_out, target)\nprint(loss.item())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"loss.backward()\nprint (y.data)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Defining the optimizer\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"cnn_model.parameters()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# let's define an object of the Adam optimizer with a learning rate of 3e-4:\n\nfrom torch import optim\nopt = optim.Adam(cnn_model.parameters(), lr=3e-4)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# We can read the current value of the learning rate using the following function:\ndef get_lr(opt):\n    for param_group in opt.param_groups:\n        return param_group['lr']\n\ncurrent_lr=get_lr(opt)\nprint('current lr={}'.format(current_lr))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from torch.optim.lr_scheduler import ReduceLROnPlateau\n\n# define learning rate scheduler\nlr_scheduler = ReduceLROnPlateau(opt, mode='min',factor=0.5, patience=20,verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for i in range(100):\n    lr_scheduler.step(1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Training and Evaluation\n**first, let's develop a helper function to count the number of correct predictions\nper data batch:**"},{"metadata":{"trusted":true},"cell_type":"code","source":"def metrics_batch(output, target):\n    # get output class\n    pred = output.argmax(dim=1, keepdim=True)\n    \n    # compare output class with target class\n    corrects=pred.eq(target.view_as(pred)).sum().item()\n    return corrects","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def loss_batch(loss_func, output, target, opt=None):\n    \n    # get loss \n    loss = loss_func(output, target)\n    \n    # get performance metric\n    metric_b = metrics_batch(output,target)\n    \n    if opt is not None:\n        opt.zero_grad()\n        loss.backward()\n        opt.step()\n\n    return loss.item(), metric_b\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# define device as a global variable\ndevice = torch.device(\"cuda\")\n\ndef loss_epoch(model,loss_func,dataset_dl,sanity_check=False,opt=None):\n    running_loss=0.0\n    running_metric=0.0\n    len_data=len(dataset_dl.dataset)\n\n    for xb, yb in dataset_dl:\n        # move batch to device\n        xb=xb.to(device)\n        yb=yb.to(device)\n        \n        # get model output\n        output=model(xb)\n        \n        # get loss per batch\n        loss_b,metric_b=loss_batch(loss_func, output, yb, opt)\n        \n        # update running loss\n        running_loss+=loss_b\n        \n        # update running metric\n        if metric_b is not None:\n            running_metric+=metric_b\n\n        # break the loop in case of sanity check\n        if sanity_check is True:\n            break\n    \n    # average loss value\n    loss=running_loss/float(len_data)\n    \n    # average metric value\n    metric=running_metric/float(len_data)\n    \n    return loss, metric","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def train_val(model, params):\n    # extract model parameters\n    num_epochs=params[\"num_epochs\"]\n    loss_func=params[\"loss_func\"]\n    opt=params[\"optimizer\"]\n    train_dl=params[\"train_dl\"]\n    val_dl=params[\"val_dl\"]\n    sanity_check=params[\"sanity_check\"]\n    lr_scheduler=params[\"lr_scheduler\"]\n    path2weights=params[\"path2weights\"]\n    \n    # history of loss values in each epoch\n    loss_history={\n        \"train\": [],\n        \"val\": [],\n    }\n    \n    # histroy of metric values in each epoch\n    metric_history={\n        \"train\": [],\n        \"val\": [],\n    }\n    \n    # a deep copy of weights for the best performing model\n    best_model_wts = copy.deepcopy(model.state_dict())\n    \n    # initialize best loss to a large value\n    best_loss=float('inf')\n    \n    # main loop\n    for epoch in range(num_epochs):\n        \n        # get current learning rate\n        current_lr=get_lr(opt)\n        print('Epoch {}/{}, current lr={}'.format(epoch, num_epochs - 1, current_lr))\n        \n        # train model on training dataset\n        model.train()\n        train_loss, train_metric=loss_epoch(model,loss_func,train_dl,sanity_check,opt)\n\n        # collect loss and metric for training dataset\n        loss_history[\"train\"].append(train_loss)\n        metric_history[\"train\"].append(train_metric)\n        \n        # evaluate model on validation dataset    \n        model.eval()\n        with torch.no_grad():\n            val_loss, val_metric=loss_epoch(model,loss_func,val_dl,sanity_check)\n        \n       \n        # store best model\n        if val_loss < best_loss:\n            best_loss = val_loss\n            best_model_wts = copy.deepcopy(model.state_dict())\n            \n            # store weights into a local file\n            torch.save(model.state_dict(), path2weights)\n            print(\"Copied best model weights!\")\n        \n        # collect loss and metric for validation dataset\n        loss_history[\"val\"].append(val_loss)\n        metric_history[\"val\"].append(val_metric)\n        \n        # learning rate schedule\n        lr_scheduler.step(val_loss)\n        if current_lr != get_lr(opt):\n            print(\"Loading best model weights!\")\n            model.load_state_dict(best_model_wts) \n\n        print(\"train loss: %.6f, dev loss: %.6f, accuracy: %.2f\" %(train_loss,val_loss,100*val_metric))\n        print(\"-\"*10) \n\n    # load best model weights\n    model.load_state_dict(best_model_wts)\n        \n    return model, loss_history, metric_history","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import copy\n\nloss_func = nn.NLLLoss(reduction=\"sum\")\nopt = optim.Adam(cnn_model.parameters(), lr=3e-4)\nlr_scheduler = ReduceLROnPlateau(opt, mode='min',factor=0.5, patience=20,verbose=1)\n\nparams_train={\n \"num_epochs\": 100,\n \"optimizer\": opt,\n \"loss_func\": loss_func,\n \"train_dl\": train_dl,\n \"val_dl\": val_dl,\n \"sanity_check\": True,\n \"lr_scheduler\": lr_scheduler,\n \"path2weights\": \"weights.pt\",\n}\n\n# train and validate the model\ncnn_model,loss_hist,metric_hist=train_val(cnn_model,params_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Train-Validation Progress\nnum_epochs=params_train[\"num_epochs\"]\n\n# plot loss progress\nplt.title(\"Train-Val Loss\")\nplt.plot(range(1,num_epochs+1),loss_hist[\"train\"],label=\"train\")\nplt.plot(range(1,num_epochs+1),loss_hist[\"val\"],label=\"val\")\nplt.ylabel(\"Loss\")\nplt.xlabel(\"Training Epochs\")\nplt.legend()\nplt.show()\n\n# plot accuracy progress\nplt.title(\"Train-Val Accuracy\")\nplt.plot(range(1,num_epochs+1),metric_hist[\"train\"],label=\"train\")\nplt.plot(range(1,num_epochs+1),metric_hist[\"val\"],label=\"val\")\nplt.ylabel(\"Accuracy\")\nplt.xlabel(\"Training Epochs\")\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import copy\n\nloss_func = nn.NLLLoss(reduction=\"sum\")\nopt = optim.Adam(cnn_model.parameters(), lr=3e-4)\nlr_scheduler = ReduceLROnPlateau(opt, mode='min',factor=0.5, patience=20,verbose=1)\n\nparams_train={\n \"num_epochs\": 2,\n \"optimizer\": opt,\n \"loss_func\": loss_func,\n \"train_dl\": train_dl,\n \"val_dl\": val_dl,\n \"sanity_check\": False,\n \"lr_scheduler\": lr_scheduler,\n \"path2weights\": \"weights.pt\",\n}\n\n# train and validate the model\ncnn_model,loss_hist,metric_hist=train_val(cnn_model,params_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Train-Validation Progress\nnum_epochs=params_train[\"num_epochs\"]\n\n# plot loss progress\nplt.title(\"Train-Val Loss\")\nplt.plot(range(1,num_epochs+1),loss_hist[\"train\"],label=\"train\")\nplt.plot(range(1,num_epochs+1),loss_hist[\"val\"],label=\"val\")\nplt.ylabel(\"Loss\")\nplt.xlabel(\"Training Epochs\")\nplt.legend()\nplt.show()\n\n# plot accuracy progress\nplt.title(\"Train-Val Accuracy\")\nplt.plot(range(1,num_epochs+1),metric_hist[\"train\"],label=\"train\")\nplt.plot(range(1,num_epochs+1),metric_hist[\"val\"],label=\"val\")\nplt.ylabel(\"Accuracy\")\nplt.xlabel(\"Training Epochs\")\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# # Turn off gradients\n#cnn_model.eval()\n#\n#preds = []\n#for batch_i, (data, target) in enumerate(valid_loader):\n#    data, target = data.cuda(), target.cuda()\n#    output = cnn_model(data)\n#    if(batch_i==0):\n#        print(data.shape, target.shape)\n#    pr = output.detach().cpu().numpy()\n#    for i in pr:\n#        preds.append(i)\n#\n## # Create Submission file        \n#sample_sub['label'] = preds ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}