{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Pseudolabelling NIH ChestX-ray8 Dataset\n\nI have put together a rough notebook showing my approach to pseudolabel the NIH CXRs.\n\n**Steps:**\n1. Download the NIH data\n2. Remove the images that are in the RANZCR dataset\n3. Run inference on the remaining images\n4. Round the results based on predetermined confidence level\n5. Add labelled images, particularly of scarce classes, to training set\n\nIf you found this notebook useful, feel free to upvote :) "},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport pandas as pd\nimport urllib.request\nimport sys\nsys.path.append('../input/timm-pytorch-image-models/pytorch-image-models-master')\nimport cv2\nfrom sklearn.metrics import accuracy_score\nimport torch\nfrom torch.utils.data import DataLoader, Dataset\nimport torch.nn as nn\nfrom tqdm import tqdm\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport timm\nfrom albumentations import *\nfrom albumentations.pytorch import ToTensorV2\ndevice = torch.device('cuda')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Download the NIH CXRs\n\nThis is the location they are stored in by NIH. Kaggle can't hold all of them in its working memory, so you'll have to find a way around that. Mine was using Google Colab."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"links = [\n    'https://nihcc.box.com/shared/static/vfk49d74nhbxq3nqjg0900w5nvkorp5c.gz',\n#   'https://nihcc.box.com/shared/static/i28rlmbvmfjbl8p2n3ril0pptcmcu9d1.gz',\n#   'https://nihcc.box.com/shared/static/f1t00wrtdk94satdfb9olcolqx20z2jp.gz',\n#   'https://nihcc.box.com/shared/static/0aowwzs5lhjrceb3qp67ahp0rd1l1etg.gz',\n#   'https://nihcc.box.com/shared/static/v5e3goj22zr6h8tzualxfsqlqaygfbsn.gz',\n#   'https://nihcc.box.com/shared/static/asi7ikud9jwnkrnkj99jnpfkjdes7l6l.gz',\n#   'https://nihcc.box.com/shared/static/jn1b4mw4n6lnh74ovmcjb8y48h8xj07n.gz',\n#   'https://nihcc.box.com/shared/static/tvpxmn7qyrgl0w8wfh9kqfjskv6nmm1j.gz',\n#   'https://nihcc.box.com/shared/static/upyy3ml7qdumlgk2rfcvlb9k6gvqq2pj.gz',\n#   'https://nihcc.box.com/shared/static/l6nilvfa9cg3s28tqv1qc1olm3gnz54p.gz',\n#   'https://nihcc.box.com/shared/static/hhq8fkdgvcari67vfhs7ppg2w6ni4jze.gz',\n#   'https://nihcc.box.com/shared/static/ioqwiy20ihqwyr8pf4c24eazhh281pbu.gz'\n]\n\nfor idx, link in enumerate(links):\n    fn = '/kaggle/working/images_%02d.tar.gz' % (idx)\n    print('downloading'+fn+'...')\n    urllib.request.urlretrieve(link, fn)\n    \nfor i in range(len(links)):\n    gzip_file_path = f'/kaggle/working/images_0{i}.tar.gz'\n    !gunzip $gzip_file_path\n    tar_file_path = f'/kaggle/working/images_0{i}.tar'\n    !tar -xf $tar_file_path","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def filenames_to(array):\n    for root, dirs, files in os.walk(\"/kaggle/working/images/\", topdown=False):\n        for name in files:\n            image_path = os.path.join(root, name)\n            array.append(image_path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"files_arr = []\nfilenames_to(files_arr)\nprint(len(files_arr))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Remove duplicates\n\n[@mohamed3abdelrazik](https://www.kaggle.com/mohamed3abdelrazik) demonstrated that you could use the imagehash library to locate duplicates in the RANZCR training set and the NIH CXRs. I've used his .csv file to locate and remove the duplicates for this notebook."},{"metadata":{"trusted":true},"cell_type":"code","source":"nih = pd.read_csv('../input/duplicates/duplicates.csv')\n\nfor i in range(len(nih)):\n    if nih.iloc[i][0] in files_arr:\n        duplicate = f'{nih.iloc[i][0]}'\n        !rm -rf $duplicate\n        print(f'Removing {nih.iloc[i][0]}')\n\nclean_files_arr = []\nfilenames_to(clean_files_arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Inference on NIH CXRs\n\nMost of this is taken from @underwearfitting's excellent notebook [here](https://www.kaggle.com/underwearfitting/resnet200d-public-benchmark-2xtta-lb0-965). \n\nYou can of course vary the number of models you want to ensemble with; the combination of these five gave LB 0.965."},{"metadata":{"trusted":true},"cell_type":"code","source":"batch_size = 1\nimage_size = 512\nmodel_path = ['../input/resnet200d-baseline-benchmark-public/resnet200d_fold0_cv953.pth',\n              '../input/resnet200d-baseline-benchmark-public/resnet200d_fold1_cv955.pth',\n              '../input/resnet200d-baseline-benchmark-public/resnet200d_fold2_cv955.pth',\n              '../input/resnet200d-baseline-benchmark-public/resnet200d_fold3_cv957.pth',\n              '../input/resnet200d-baseline-benchmark-public/resnet200d_fold4_cv954.pth'\n             ]\nenet_type = ['resnet200d'] * len(model_path)\nlabel_cols =['ETT - Abnormal', 'ETT - Borderline', 'ETT - Normal',\n             'NGT - Abnormal', 'NGT - Borderline', 'NGT - Incompletely Imaged', 'NGT - Normal', \n             'CVC - Abnormal', 'CVC - Borderline', 'CVC - Normal',\n             'Swan Ganz Catheter Present']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class RANZCRResNet200D(nn.Module):\n    def __init__(self, model_name='resnet200d', out_dim=11, pretrained=False):\n        super().__init__()\n        self.model = timm.create_model(model_name, pretrained=False)\n        n_features = self.model.fc.in_features\n        self.model.global_pool = nn.Identity()\n        self.model.fc = nn.Identity()\n        self.pooling = nn.AdaptiveAvgPool2d(1)\n        self.fc = nn.Linear(n_features, out_dim)\n\n    def forward(self, x):\n        bs = x.size(0)\n        features = self.model(x)\n        pooled_features = self.pooling(features).view(bs, -1)\n        output = self.fc(pooled_features)\n        return output","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"transforms_test = Compose([\n    Resize(image_size, image_size),\n    Normalize(\n         mean=[0.485, 0.456, 0.406],\n         std=[0.229, 0.224, 0.225],\n     ),\n    ToTensorV2()\n])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class RANZCRDataset(Dataset):\n    def __init__(self, df, mode, transform=None):\n        \n        self.df = df.reset_index(drop=True)\n        self.mode = mode\n        self.transform = transform\n        self.labels = df[label_cols].values\n        \n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, index):\n        row = self.df.loc[index]\n        img = cv2.imread(row.file_path, cv2.IMREAD_GRAYSCALE)\n        mask = img > 0\n        img = img[np.ix_(mask.any(1), mask.any(0))] # snippet to remove black borders\n        img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)\n        \n        if self.transform is not None:\n            res = self.transform(image=img)\n            img = res['image']\n        label = torch.tensor(self.labels[index]).float()\n        if self.mode == 'test':\n            return img\n        else:\n            return img, label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def inference_func(test_loader):\n    model.eval()\n    bar = tqdm(test_loader)\n    LOGITS = []\n    PREDS = []\n    \n    with torch.no_grad():\n        for batch_idx, images in enumerate(bar):\n            x = images.to(device)\n            logits = model(x)\n            LOGITS.append(logits.cpu())\n            PREDS += [logits.sigmoid().detach().cpu()]\n        PREDS = torch.cat(PREDS).cpu().numpy()\n        LOGITS = torch.cat(LOGITS).cpu().numpy()\n    return PREDS","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Create Dataset and Dataloader\n\nI've used a very janky for loop to fix up the sample_submission.csv for our purposes. Feel free to suggest something better!"},{"metadata":{"trusted":true},"cell_type":"code","source":"test = pd.read_csv('../input/ranzcr-clip-catheter-line-classification/sample_submission.csv')\n\nfor i in range(len(clean_files_arr)):\n    test.loc[i] = [0] * 12\n    test['StudyInstanceUID'][i] = clean_files_arr[i]\n\ntest['file_path'] = test.StudyInstanceUID.apply(lambda x: os.path.join('/kaggle/working/images/', f'{x}'))\n\ntest.to_csv(r'./nih_pseudolabels_test.csv', index=False)\ntest_dataset = RANZCRDataset(test, 'test', transform=transforms_test)\ntest_loader = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=False,  num_workers=24)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"test_preds = []\nfor i in range(len(enet_type)):\n    if enet_type[i] == 'resnet200d':\n        print('resnet200d loaded')\n        model = RANZCRResNet200D(enet_type[i], out_dim=len(label_cols))\n        model = model.to(device)\n    model.load_state_dict(torch.load(model_path[i], map_location='cuda:0'))\n    test_preds += [inference_func(test_loader)]\n    \nsubmission = pd.read_csv('./nih_pseudolabels_test.csv')\nsubmission[label_cols] = np.mean(test_preds, axis=0)\nsubmission.to_csv('./nih_pseudolabels_mean.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Turn the predictions into labels\n\nYou can determine for yourself how confident you would like the model to be, for it to return a positive label."},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"pseudolabel_means = pd.read_csv('./nih_pseudolabels_mean.csv')\n\nconfidence_level = 0.7\ndef roundAbove(x):\n    if x < confidence_level:\n        return 0\n    else:\n        return 1\n    \nfor i in range(len(label_cols)):\n    pseudolabel_means[label_cols[i]] = pseudolabel_means[label_cols[i]].map(roundAbove)\n\npseudolabel_means.to_csv('./nih_pseudolabels.csv', index=False)\npseudolabel_means.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Display some examples\n\nNow we could pass these images with the new pseudolabels as training data to our single model. Let's have a bit of a look first to eyeball the accuracy."},{"metadata":{"trusted":true},"cell_type":"code","source":"pseudolabels = pd.read_csv('./nih_pseudolabels.csv')\nresults_dataset = RANZCRDataset(pseudolabels, 'none', transform=transforms_test)\n\nfor i in range(100):\n    image, label = results_dataset[i]\n    np_label = label.numpy()\n    ones = np.argwhere(np_label==1)\n\n    labels = []\n\n    if len(ones):\n        for i in range(len(ones)):\n            idx = ones[i][0]\n            labels.append(label_cols[idx])\n            plt.imshow(image[0], cmap=\"gray\")\n            plt.title(f'label: {labels}')\n            plt.show() ","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}