{"cells":[{"metadata":{},"cell_type":"markdown","source":"#                                      **SIIM Melanoma Competition**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"This competition is an interesting and important one from the pov of improvments in medical image diagnosis we can have using modern and state of the art deep learning models and computer vision. Melanoma, as the competyition overview states,is responsible for 75% of skin cancer deaths, despite being the least common skin cancer and the American Cancer Society estimates over 100,000 new melanoma cases will be diagnosed in 2020. Current AI techniques have not been much succesful but the hosts have expected the large pool of experienced and efficient machine learning and data scientists on Kaggle will be able to provide models that have better results and help in early diagnosis.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## **Version 8** --  Resnext50_32x4d model used on 224x224 uncropped image. [LB - 0.87]\n\n## **Version 15** -- Efficientnet b2 model used on uncropped 224x224 image. [LB - 0.892]\n\n## **Version 24** -- Efficientnet b3 model used on square centre cropped 224x224 image posted by Chris Deotte. [LB-- 0.910] \n\n## **Version 28** -- Changed to using Focal loss as loss function.\n\nStarted experimenting with higher image sizes. Training on gpu was very slow and frustrating,had to move to tpus.Given I have never used tpus before either in pytorch or tf, Abhisek Thakur's accelerator-power-hour [workshop](https://www.youtube.com/watch?v=DEuvGh4ZwaY&feature=youtu.be) on kaggle was very helpful for me to change my code to tpu version. Thanks to Chris for making so many important discussions in this competition and sharing the datasets and to Abhisek for his videos on youtube and public notebooks.\n\n## **Version 40** -- Efficientnet b4 on 384x384 centre cropped images by Chris Deotte on TPU. [LB--9205]","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Dataset and discussion links**\n\n224x224 square centre cropped by Chris Deotte - [link](https://www.kaggle.com/cdeotte/jpeg-melanoma-256x256?select=test.csv)\n\nPytorch or Tensorflow jpegs discussion post by Chris Deotte - [link](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/164910)\n\nmelanoma merged external data 512x512 jpeg by Alex Shonenkov - [link](https://www.kaggle.com/shonenkov/melanoma-merged-external-data-512x512-jpeg)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Setting up the environment**","execution_count":null},{"metadata":{"_kg_hide-output":true,"trusted":true,"_kg_hide-input":false},"cell_type":"code","source":"#!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py\n#!python pytorch-xla-env-setup.py --version nightly --apt-packages libomp5 libopenblas-dev\n!pip install wtfml\n!pip install efficientnet_pytorch","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Importing packages**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import warnings\nimport torch_xla\nimport torch_xla.debug.metrics as met\nimport torch_xla.distributed.data_parallel as dp\nimport torch_xla.distributed.parallel_loader as pl\nimport torch_xla.utils.utils as xu\nimport torch_xla.core.xla_model as xm \nimport torch_xla.distributed.xla_multiprocessing as xmp\nimport torch_xla.test.test_utils as test_utils\nimport warnings\nimport gc\n\nwarnings.filterwarnings(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torchvision.models as models\nimport torchvision\n\nimport cv2\n\nimport numpy as np \nimport pandas as pd\nimport os\n\nfrom torch.utils.data import DataLoader,TensorDataset,Dataset\nimport matplotlib.pyplot as plt\nimport albumentations\nfrom sklearn import model_selection\nfrom sklearn.metrics import roc_auc_score\nfrom efficientnet_pytorch import EfficientNet\n\nfrom wtfml.utils import EarlyStopping","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/jpeg-melanoma-384x384/train.csv')  #/kaggle/input/siim-isic-melanoma-classification/train.csv\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"making 5 fold divisions on the dataset csv","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df = train_df.sample(frac=1).reset_index(drop=True)\ndf['kfold'] = -1\ny = train_df.target.values\nkf = model_selection.StratifiedKFold(n_splits=5,shuffle=True)\nidx = kf.get_n_splits(X=df,y=y)\nprint(idx)\nfor fold,(x,y) in enumerate(kf.split(X=df,y=y)):\n    df.loc[y,'kfold'] = fold","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#df.to_csv('train_fold_tpu.csv',index=False)\ndf = pd.read_csv('/kaggle/input/tpu-csv/train_fold_tpu.csv')\n#df = pd.read_csv('/kaggle/input/melanoma-merged-external-data-512x512-jpeg/folds.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df.head(10) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **Define custom dataset**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"define helper function for image augmentation using albumentations library\n\ninp : image path,image name,valid as a boolean showing whether the image is from train or validation set\n\nout : a image vector of type torch.tensor and shape (3,256,256)\n\nAlso training images are augmented in different ways whereas validation images should keep their actual identity and hence they are only normalized.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class CustomDataset(Dataset):\n    def __init__(self,path,name,target,aug):\n        super(CustomDataset,self).__init__()\n        self.path = path\n        self.name = name\n        self.target = target\n        self.aug = aug\n        \n        \n    def __len__(self):\n        return len(self.name)\n    \n    def __getitem__(self,index):\n        \n        im_name = self.name[index]\n        y = self.target[index]\n        img_path = os.path.join(self.path,im_name + '.jpg')\n        img = cv2.resize(cv2.imread(img_path),dsize=(512,512))\n        image = self.aug(image=img)\n        l = image['image']\n        image = np.transpose(l, (2, 0, 1)).astype(np.float32)\n        \n        return torch.tensor(image,dtype=torch.float),torch.tensor(y)\n        \n        ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **Define dataloader for tpu**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class Data_Loader():\n    def __init__(self,path,name,target,aug):\n        self.path = path\n        self.name = name\n        self.target = target\n        self.aug = aug\n        self.dataset = CustomDataset(self.path,self.name,self.target,self.aug)\n        \n    def get(self,batch_size,shuffle,num_workers):\n        \n        sampler = torch.utils.data.distributed.DistributedSampler(self.dataset,\n                                                                  num_replicas = xm.xrt_world_size(),\n                                                                  rank = xm.get_ordinal(),\n                                                                  shuffle = shuffle)\n        dataloader = torch.utils.data.DataLoader(self.dataset,\n                                                 batch_size=batch_size,\n                                                 shuffle=False,\n                                                 sampler=sampler,\n                                                 num_workers=num_workers)\n        return dataloader\n        \n        ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **Set up network architecture**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class EffNet(nn.Module):\n    def __init__(self,model='b4'):\n        super(EffNet,self).__init__()\n        \n        model_name = 'efficientnet' + model\n        self.feature = EfficientNet.from_pretrained(\"efficientnet-b6\")\n        self.drop = nn.Dropout(0.3)\n        self.l0 = nn.Linear(2304,1) #b6 -- 2304  b4 - 1792 b3 - 1536 b2 - 1406\n        \n        \n    def forward(self,img):\n        batch_size = img.shape[0]\n        \n        x = self.feature.extract_features(img)\n        #print(x.shape)\n        \n        x = nn.functional.adaptive_avg_pool2d(x,1).reshape(batch_size,-1)\n        #print(x.shape)\n        \n        x = self.drop(x)\n        #print(x.shape)\n        out = self.l0(x)\n        #print(out.shape)\n        \n        return out","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **Focal Loss**\n\nChanged to focal loss which is a modified loss function based on cross entropy loss specially used in cases where there is high imbalannce in the nature of datasets.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class FocalLoss(nn.Module):\n    def __init__(self,alpha=1,gamma=2):\n        super(FocalLoss,self).__init__()\n        self.alpha = alpha\n        self.gamma = gamma\n        \n    def forward(self,preds,truth):\n        criterion = nn.BCEWithLogitsLoss()\n        logits = criterion(preds,truth.unsqueeze(-1).type_as(preds))\n        pt = torch.exp(-logits)\n        focal_loss = self.alpha*(1-pt)**self.gamma*logits\n        \n        return torch.mean(focal_loss)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **Training**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def train(model,fold):\n    \n    batch_t = 32\n    batch_v = 32\n    best_score = 0\n    device = xm.xla_device() \n    \n    mean = (0.485, 0.456, 0.406)\n    std = (0.229, 0.224, 0.225)\n    train_transpose = albumentations.Compose([\n                albumentations.Normalize(mean, std, max_pixel_value=255.0, always_apply=True),\n                albumentations.ShiftScaleRotate(shift_limit=0.0625, scale_limit=0.1, rotate_limit=15),\n                albumentations.Flip(p=0.5)\n                #albumentations.CenterCrop(150,150,always_apply=True)\n            ])\n    valid_transpose = albumentations.Compose(\n        [ albumentations.Normalize(mean, std, max_pixel_value=255.0, always_apply=True) \n        ])\n    \n    image_path = '/kaggle/input/jpeg-melanoma-512x512/train/' #'/kaggle/input/siic-isic-224x224-images/train/'\n    train_df = df[df.kfold!=fold].reset_index(drop=True)  #kfold to fold\n    valid_df = df[df.kfold==fold].reset_index(drop=True)\n    train_im = train_df.image_name.values.tolist()\n    train_y = train_df.target.values\n    valid_im = valid_df.image_name.values.tolist()\n    valid_y = valid_df.target.values\n    train_dataset = Data_Loader(image_path,train_im,train_y,\n                                train_transpose).get(batch_size=batch_t,shuffle=True,num_workers=4)\n    valid_dataset = Data_Loader(image_path,valid_im,valid_y,\n                               valid_transpose).get(batch_size=batch_v,shuffle=False,num_workers=4)\n    \n    \n    \n    \n    \n    #model = Resnext50_32x4d()\n    #model = EffNet()\n    model = model.to(device)\n    optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)\n    schedular = torch.optim.lr_scheduler.ReduceLROnPlateau(    #to update the learning rate if model auc score does not increase\n        optimizer,                                             #for 3 succesive epochs\n        patience=3,           \n        threshold=0.001,\n        mode=\"max\"\n    )\n\n    es = EarlyStopping(patience=5, mode=\"max\",tpu=True)  #early stopping function to stop training if auc score does not increase over 5 epochs\n    criterion = nn.BCEWithLogitsLoss()\n    #criterion = FocalLoss()\n    epochs = 15\n    best_score = 0\n    \n    \n    \n    for epoch in range(epochs):\n            #train mode for training the model and updating the losses\n            model.train()\n            batch = 0\n            #para_loader = pl.ParallelLoader(train_dataset,[device])\n            #train_loader = para_loader.per_device_loader(device)\n        \n            #for _,(train_data,label) in enumerate(train_loader):\n            for train_data,label in train_dataset:\n                train_data = train_data.to(device)\n                label = torch.tensor(label,dtype = torch.float32)\n                label = label.to(device)\n                \n                optimizer.zero_grad()\n                out = model(train_data)\n                loss = criterion(out,label.unsqueeze(1).type_as(out))\n                #loss = criterion(out,label)\n                batch +=1\n                del train_data,label\n                gc.collect()\n                if batch%200==0 : print(\"EPOCH {}  Loss {}  batch  {}\".format(epoch,loss.item(),batch))\n                \n                loss.backward()\n                xm.optimizer_step(optimizer,barrier=True)\n            #evaluate mode to evaluate the model on cv and update learning rate based on auc score\n            #del para_loader,train_loader\n            gc.collect()\n            model.eval()\n            preds = []\n            batch = 0\n            #para_loader = pl.ParallelLoader(valid_dataset,[device])\n            #valid_loader = para_loader.per_device_loader(device)\n            \n            #for _,(valid_data,valid_label) in enumerate(valid_dataset):\n            for valid_data,valid_label in valid_dataset:\n                valid_data = valid_data.to(device)\n                valid_label = torch.tensor(valid_label,dtype = torch.float32)\n                valid_label = valid_label.to(device)\n                batch +=1\n                \n                \n                with torch.no_grad():\n                    out = model(valid_data)\n                    #loss = criterion(out,valid_label)\n                    loss = criterion(out,valid_label.unsqueeze(1).type_as(out))\n                    preds.append(out.cpu())\n                    if batch%50==0 : xm.master_print('Valid Loss {}  batch  {}'.format(loss.item(),batch))\n                del valid_data,valid_label\n                gc.collect()\n            #del para_loader,valid_loader\n            gc.collect()\n            pred=np.vstack((preds)).ravel()\n            #print('pred',pred)\n            auc_score = roc_auc_score(valid_y.astype(np.float32),pred)\n            print(\"EPOCH {}  AUC Score {}\".format(epoch,auc_score))\n            schedular.step(auc_score)\n            es(auc_score, model, model_path=f\"model_fold_{fold}.bin\")\n            if es.early_stop:\n                print(\"Early stopping\")\n                break\n            gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = EffNet()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def _mp_fn(rank, flags):\n    torch.set_default_tensor_type('torch.FloatTensor')\n    a = train(model,0)\n\nFLAGS={}\nxmp.spawn(_mp_fn, args=(FLAGS,), nprocs=8, start_method='fork')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train(model,0)\ntrain(model,1)\ntrain(model,2)\ntrain(model,3)\ntrain(model,4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"gpu code that was used previously","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def image_aug(path,image_name,valid=False):\n    mean = (0.485, 0.456, 0.406)\n    std = (0.229, 0.224, 0.225)\n    train_transpose = albumentations.Compose([\n            albumentations.Normalize(mean, std, max_pixel_value=255.0, always_apply=True),\n            albumentations.ShiftScaleRotate(shift_limit=0.0625, scale_limit=0.1, rotate_limit=15),\n            albumentations.Flip(p=0.5)\n            #albumentations.CenterCrop(150,150,always_apply=True)\n        ])\n    valid_transpose = albumentations.Compose(\n    [ albumentations.Normalize(mean, std, max_pixel_value=255.0, always_apply=True) \n    ])\n    \n    if valid==True :\n        \n            im_path = os.path.join(path,image_name + '.jpg')\n            #img = cv2.imread(im_path)\n            img = cv2.resize(cv2.imread(im_path),dsize=(384,384))\n            aug = valid_transpose(image=img)\n            l = aug['image']\n            #print(\"Validation set image augmented\")\n               \n    else:\n        \n            im_path = os.path.join(path,image_name + '.jpg')\n            img = cv2.resize(cv2.imread(im_path),dsize=(384,384))\n            aug = train_transpose(image=img)\n            l =aug['image']\n            #print(\"Train set image augmented\")\n            \n    image = np.transpose(l, (2, 0, 1)).astype(np.float32)\n    return torch.tensor(image, dtype=torch.float)\n\n\n\nclass Data_Loader(Dataset):\n    def __init__(self,image_path,im_name,target,valid=False):\n        self.name = im_name\n        self.target = target\n        self.path = image_path\n        self.valid = valid\n        \n    def __len__(self):\n        return (len(self.name))\n    \n    def __getitem__(self,index):\n        \n        if self.valid==False:\n            im = self.name[index]\n            self.train_y = self.target[index]\n            im_tensor = image_aug(self.path,im)\n            \n            return im_tensor,self.train_y\n        \n        else:\n            im = self.name[index]\n            self.valid_y = self.target[index]\n            im_tensor = image_aug(self.path,im,valid=True)\n            \n            return im_tensor,self.valid_y\n        \n        \n        \n        \n        \ndef train(fold):\n    \n    batch_t = 16\n    batch_v = 16\n    best_score = 0\n    device = 'cuda'\n    image_path = '/kaggle/input/jpeg-melanoma-384x384/train/' #'/kaggle/input/siic-isic-224x224-images/train/'\n    train_df = df[df.kfold!=fold].reset_index(drop=True)  #kfold to fold\n    valid_df = df[df.kfold==fold].reset_index(drop=True)\n    train_im = train_df.image_name.values.tolist()\n    train_y = train_df.target.values\n    valid_im = valid_df.image_name.values.tolist()\n    valid_y = valid_df.target.values\n    train_dataset = Data_Loader(image_path,train_im,train_y)\n    train_dataset = DataLoader(train_dataset,batch_t,shuffle=False,num_workers=4)\n    valid_dataset = Data_Loader(image_path,valid_im,valid_y,valid=True)\n    valid_dataset = DataLoader(valid_dataset,batch_v,shuffle=False,num_workers=4)\n    \n    \n    \n    \n    \n    #model = Resnext50_32x4d()\n    model = EffNet()\n    model = model.cuda()\n    optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)\n    schedular = torch.optim.lr_scheduler.ReduceLROnPlateau(    #to update the learning rate if model auc score does not increase\n        optimizer,                                             #for 3 succesive epochs\n        patience=3,           \n        threshold=0.001,\n        mode=\"max\"\n    )\n\n    es = EarlyStopping(patience=5, mode=\"max\")  #early stopping function to stop training if auc score does not increase over 5 epochs\n    criterion = nn.BCEWithLogitsLoss()\n    #criterion = FocalLoss()\n    epochs = 25\n    best_score = 0\n    \n    \n    \n    for epoch in range(epochs):\n            #train mode for training the model and updating the losses\n            model.train()\n            batch = 0\n        \n            for train_data,label in train_dataset:\n                train_data = train_data.to(device)\n                label = torch.tensor(label,dtype = torch.float32)\n                label = label.to(device)\n                \n                optimizer.zero_grad()\n                out = model(train_data)\n                loss = criterion(out,label.unsqueeze(1).type_as(out))\n                #loss = criterion(out,label)\n                batch +=1\n                if batch%200==0 : print(\"EPOCH {}  Loss {}  batch  {}\".format(epoch,loss.item(),batch))\n                \n                loss.backward()\n                optimizer.step()\n            #evaluate mode to evaluate the model on cv and update learning rate based on auc score\n            model.eval()\n            preds = []\n            batch = 0\n            for valid_data,valid_label in valid_dataset:\n                valid_data = valid_data.to(device)\n                valid_label = torch.tensor(valid_label,dtype = torch.float32)\n                valid_label = valid_label.to(device)\n                batch +=1\n                \n                \n                with torch.no_grad():\n                    out = model(valid_data)\n                    #loss = criterion(out,valid_label)\n                    loss = criterion(out,valid_label.unsqueeze(1).type_as(out))\n                    preds.append(out.cpu())\n                    if batch%50==0 : print('Valid Loss {}  batch  {}'.format(loss.item(),batch))\n\n            pred=np.vstack((preds)).ravel()\n            #print('pred',pred)\n            auc_score = roc_auc_score(valid_y.astype(np.float32),pred)\n            print(\"EPOCH {}  AUC Score {}\".format(epoch,auc_score))\n            schedular.step(auc_score)\n            es(auc_score, model, model_path=f\"model_fold_{fold}.bin\")\n            if es.early_stop:\n                print(\"Early stopping\")\n                break        \n            #if auc_score>best_score:\n                        #best_score = auc_score \n                        #torch.save(model,'best_model.pth')\n                        #print(\"Validation Score Improved ======>>>>>> Saving Model\")\n            del train_data,valid_data,label,valid_label\n            gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train(0)\ntrain(1)\ntrain(2)\ntrain(3)\ntrain(4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **Prediction**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def predict(fold):\n    test_df = pd.read_csv('/kaggle/input/jpeg-melanoma-512x512/test.csv')   #/kaggle/input/siim-isic-melanoma-classification/test.csv\n    im_path = '/kaggle/input/jpeg-melanoma-512x512/test/' #'/kaggle/input/siic-isic-224x224-images/test/'\n    batch_t = 32\n    #model_path = '../working/model_fold_'+str(fold)+'.bin'\n    model_path = '/kaggle/input/effnet-b5/model_fold_'+str(fold)+'.bin' \n    \n    mean = (0.485, 0.456, 0.406)\n    std = (0.229, 0.224, 0.225)\n    valid_transpose = albumentations.Compose(\n    [ albumentations.Normalize(mean, std, max_pixel_value=255.0, always_apply=True) \n    ])\n    test_im = test_df.image_name.values.tolist()\n    test_y = np.ones(len(test_im))\n    test_dataset = Data_Loader(im_path,test_im,test_y,valid=False)\n    test_dataset = DataLoader(test_dataset,batch_t,shuffle=False,num_workers=4)\n    #test_dataset = Data_Loader(im_path,test_im,test_y,\n                               #valid_transpose).get(batch_size=batch_t,shuffle=False,num_workers=4)\n    device = 'cuda'#xm.xla_device()\n    \n    \n    model = EffNet()\n    model.load_state_dict(torch.load(model_path))\n    model.to(device)\n    model.eval()\n    preds = []\n    batch = 0\n    #for i in range(5):\n    for test_data,test_label in test_dataset:\n                test_data = test_data.to(device)\n                batch +=1\n                \n                with torch.no_grad():\n                    out = model(test_data)\n                    out = torch.sigmoid(out)\n                    preds.append(out.cpu())\n                    if batch%50==0 : print('Batch  {}'.format(batch))\n\n    pred=np.vstack((preds)).ravel()\n    return pred\n        ","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"predict_1 = predict(0)\npredict_2 = predict(1)\npredict_3 = predict(2)\npredict_4 = predict(3)\npredict_5 = predict(4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"prediction = (predict_1+predict_2+predict_3+predict_4+predict_5)/5\nsubmission = pd.read_csv(\"../input/jpeg-melanoma-512x512/sample_submission.csv\")\nsubmission.loc[:,'target'] = prediction\nsubmission.to_csv('submission.csv',index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"s1 = pd.read_csv('../working/submission.csv')  # ../input/new-submit/submit_bce.csv\ns2 = pd.read_csv('../input/new-submit/submit_fl.csv')\ns3 = pd.read_csv('../input/new-submit/submit_bce.csv')\ntarget_res = s3.target.values\ntarget_eff = s2.target.values\ntarget_eff_1 = s1.target.values\nresult = (target_res + target_eff + target_eff_1)/3\n#result_1 = (target_eff + target_eff_1)/2\nsubmission = pd.read_csv(\"../input/jpeg-melanoma-384x384/sample_submission.csv\")\nsubmission.loc[:,'target'] = result\nsubmission.to_csv('submit_2.csv',index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tabular = pd.read_csv('../input/melanoma-submissions2/submissionImage.csv')\ntabular.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"file = pd.read_csv(\"../input/jpeg-melanoma-512x512/sample_submission.csv\")\n\nexample = 0.9*sub.target.values + 0.1*tabular.target.values\nfile.loc[:,'target'] = example\nfile.to_csv('tab-image.csv',index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub = pd.read_csv('../working/submission.csv')\nsub.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**submission file**  -- single model on effnet b4 using bceloss\n\n**submit_1 file**  -- combined model trained on bceloss and focal loss","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"reduced_loss = xm.mesh_reduce(\"loss_reduce\", loss, reduce_fn)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}