{"cells":[{"metadata":{"id":"b5dyzqbPwa2J","colab_type":"text"},"cell_type":"markdown","source":"** BEFORE YOU FORK, PLEASE SUPPORT AND UPVOTE **"},{"metadata":{"id":"uggxhoHpwa2K","colab_type":"text"},"cell_type":"markdown","source":"# Recursion Cellular Image Classification - fastai starter\n\nWelcome to the Recursion Cellular Image Classification Kaggle competition! Here, I provide a basic fastai starter code."},{"metadata":{"id":"m67AVgPxwa2L","colab_type":"text"},"cell_type":"markdown","source":"## Load modules"},{"metadata":{"id":"92p63IbDwa2L","colab_type":"code","colab":{},"trusted":true},"cell_type":"code","source":"import os\nos.environ['CUDA_LAUNCH_BLOCKING'] = '1'\n\nimport numpy as np\nimport pandas as pd\n\nfrom fastai.vision import *","execution_count":null,"outputs":[]},{"metadata":{"id":"6mJopzu_evYF","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"def seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n\nSEED = 0\nseed_everything(SEED)","execution_count":0,"outputs":[]},{"metadata":{"id":"IAHj2Ulawa2R","colab_type":"text"},"cell_type":"markdown","source":"## Loading and formatting data\n\nHere I will load the csv into the DataFrame, and create a column in the DataFrame with the path to the corresponding image (`generate_df`)"},{"metadata":{"id":"VI6_C0cVwa2R","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":343},"outputId":"64557492-946d-4cd1-9a7a-087edc91886d","trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('../input/train.csv')\ntrain_df.head(10)","execution_count":null,"outputs":[]},{"metadata":{"id":"YeEIRS-iwa2T","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"def generate_df(train_df,sample_num=1):\n    train_df['path'] = train_df['experiment'].str.cat(train_df['plate'].astype(str).str.cat(train_df['well'],sep='/'),sep='/Plate') + '_s'+str(sample_num) + '_w'\n    train_df = train_df.drop(columns=['id_code','experiment','plate','well']).reindex(columns=['path','sirna'])\n    return train_df\nproc_train_df = generate_df(train_df)  ","execution_count":0,"outputs":[]},{"metadata":{"id":"Fq-2iKzBwa2V","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":343},"outputId":"7ca0733a-10d4-4f87-d725-4ce202967437","trusted":false},"cell_type":"code","source":"proc_train_df.head(10)","execution_count":null,"outputs":[]},{"metadata":{"id":"D8XgaghTwa2X","colab_type":"text"},"cell_type":"markdown","source":"Let's look at an example image. These images are 6-channel images, but the each of the six channels are saved as separate files. Here, I open just one channel of the image."},{"metadata":{"id":"P61K3v_Vwa2X","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":286},"outputId":"018b1df3-0b97-4881-84ed-51020711dd62","trusted":false},"cell_type":"code","source":"import cv2\nimg = cv2.imread(\"../input/train/HEPG2-01/Plate1/B03_s1_w2.png\")\nplt.imshow(img)\ngray_img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)\nplt.imshow(gray_img)\ngray_img.shape","execution_count":null,"outputs":[]},{"metadata":{"id":"a07AWSdswa2a","colab_type":"text"},"cell_type":"markdown","source":"In fastai, there is a modular data API that allows you to easily load images, add labels, split into train/valid, and add transforms. The base class for loading the images is an `ItemList`. For image classification tasks, the base class is `ImageList` which in turn subclasses the `ItemList` class. Since `ImageList` can only open 3-channel images, we will define a new `ImageList` class where we redefine the loading function:"},{"metadata":{"id":"FfDzD6Wuwa2a","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"def open_rcic_image(fn):\n    images = []\n    for i in range(6):\n        file_name = fn+str(i+1)+'.png'\n        im = cv2.imread(file_name)\n        im = cv2.cvtColor(im, cv2.COLOR_RGB2GRAY)\n        images.append(im)\n    image = np.dstack(images)\n    #print(pil2tensor(image, np.float32).shape)#.div_(255).shape)\n    return Image(pil2tensor(image, np.float32).div_(255))\n  \nclass MultiChannelImageList(ImageList):\n    def open(self, fn):\n        return open_rcic_image(fn)","execution_count":0,"outputs":[]},{"metadata":{"id":"F_WqdWsiwa2c","colab_type":"text"},"cell_type":"markdown","source":"As I subclassed the ImageList function I can load images with the `ImageList` function `.from_df`. "},{"metadata":{"id":"h4p_BcLTwa2e","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"il = MultiChannelImageList.from_df(df=proc_train_df,path='../input/train/')","execution_count":0,"outputs":[]},{"metadata":{"id":"O3mQg0A1wa2l","colab_type":"text"},"cell_type":"markdown","source":"We have to redefine the following function to be able to view the image in the notebook. I view just the first 3 channels."},{"metadata":{"id":"EVruUNW0wa2l","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"def image2np(image:Tensor)->np.ndarray:\n    \"Convert from torch style `image` to numpy/matplotlib style.\"\n    res = image.cpu().permute(1,2,0).numpy()\n    if res.shape[2]==1:\n        return res[...,0]  \n    elif res.shape[2]>3:\n        #print(res.shape)\n        #print(res[...,:3].shape)\n        return res[...,:3]\n    else:\n        return res\n\nvision.image.image2np = image2np","execution_count":0,"outputs":[]},{"metadata":{"id":"uR8wOHAXwa2r","colab_type":"text"},"cell_type":"markdown","source":"Now let's view an example image:"},{"metadata":{"id":"4904GgsJwa2r","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":529},"outputId":"f4131f6f-5d84-42da-fb61-d8ba68a63b74","trusted":false},"cell_type":"code","source":"il[0]","execution_count":null,"outputs":[]},{"metadata":{"id":"7RJelHuLwa2t","colab_type":"text"},"cell_type":"markdown","source":"With the multi-channel `ImageList` defined, we can now create a DataBunch of the train images. Let's first create a stratified split of dataset and get the indices. "},{"metadata":{"id":"L3Dlp01Bwa2u","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"from sklearn.model_selection import StratifiedKFold\n#train_idx, val_idx = next(iter(StratifiedKFold(n_splits=int(1/0.035),random_state=42).split(proc_train_df, proc_train_df.sirna)))\nfrom sklearn.model_selection import train_test_split\ntrain_df,val_df = train_test_split(proc_train_df,test_size=0.035, stratify = proc_train_df.sirna, random_state=42)\n_proc_train_df = pd.concat([train_df,val_df])","execution_count":0,"outputs":[]},{"metadata":{"id":"diwrgp9ewa2w","colab_type":"text"},"cell_type":"markdown","source":"Now we create the `DataBunch`"},{"metadata":{"id":"aXTb_YxZwa2w","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"data = (MultiChannelImageList.from_df(df=_proc_train_df,path='../input/train/')\n        .split_by_idx(list(range(len(train_df),len(_proc_train_df))))\n        .label_from_df()\n        .transform(get_transforms(),size=256)\n        .databunch(bs=128,num_workers=4)\n        .normalize()\n       )","execution_count":0,"outputs":[]},{"metadata":{"id":"Xba28Znvwa2z","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"data.show_batch()","execution_count":0,"outputs":[]},{"metadata":{"id":"_mxelLONwa21","colab_type":"text"},"cell_type":"markdown","source":"## Creating and Training a Model"},{"metadata":{"id":"RECFNRy3wa22","colab_type":"text"},"cell_type":"markdown","source":"I will use a pretrained EfficientNet. There is code for other models thatt you can try but the EfficientNet seems to do the best. I have to now adjust the CNN arch to take in 6 channels as opposed to the usual 3 channels:"},{"metadata":{"id":"jiawTWYgi9Ky","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":67},"outputId":"47872c63-45a6-4892-e328-b8ffcad88ad2","trusted":false},"cell_type":"code","source":"!pip install efficientnet_pytorch","execution_count":null,"outputs":[]},{"metadata":{"id":"0XQpndYbjVkg","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"from efficientnet_pytorch import *","execution_count":0,"outputs":[]},{"metadata":{"id":"SITNC-7_wa22","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"\"\"\"Inspired by https://github.com/wdhorton/protein-atlas-fastai/blob/master/resnet.py\"\"\"\n\nimport torchvision\nRESNET_MODELS = {\n    18: torchvision.models.resnet18,\n    34: torchvision.models.resnet34,\n    50: torchvision.models.resnet50,\n    101: torchvision.models.resnet101,\n    152: torchvision.models.resnet152,\n}\n\ndef resnet_multichannel(depth=50,pretrained=True,num_classes=1108,num_channels=6):\n        model = RESNET_MODELS[depth](pretrained=pretrained)\n        w = model.conv1.weight\n        model.conv1 = nn.Conv2d(num_channels, 64, kernel_size=7, stride=2, padding=3,\n                               bias=False)\n        model.conv1.weight = nn.Parameter(torch.stack([torch.mean(w, 1)]*num_channels, dim=1))\n        return model\n\n    \nDENSENET_MODELS = {\n    121: torchvision.models.densenet121,\n    161: torchvision.models.densenet161,\n    169: torchvision.models.densenet169,\n    201: torchvision.models.densenet201,\n}\n\ndef densenet_multichannel(depth=121,pretrained=True,num_classes=1108,num_channels=6):\n        model = DENSENET_MODELS[depth](pretrained=pretrained)\n        w = model.features.conv0.weight\n        model.features.conv0 = nn.Conv2d(num_channels, 64, kernel_size=7, stride=2, padding=3,\n                               bias=False)\n        model.features.conv0.weight = nn.Parameter(torch.stack([torch.mean(w, 1)]*num_channels, dim=1))\n        return model\n        \n        \n#EFFICIENTNET_MODELS = {\n#    'b0': '../input/efficientnet-pytorch/efficientnet-b0-08094119.pth',\n#    'b1': '../input/efficientnet-pytorch/efficientnet-b1-dbc7070a.pth',\n#    'b2': '../input/efficientnet-pytorch/efficientnet-b2-27687264.pth',\n#    'b3': '../input/efficientnet-pytorch/efficientnet-b3-c8376fa2.pth',\n#    'b4': '../input/efficientnet-pytorch/efficientnet-b4-e116e8b3.pth',\n#    'b5': '../input/efficientnet-pytorch/efficientnet-b5-586e6cc6.pth'\n#}\n\n\ndef efficientnet_multichannel(pretrained=True,name='b0',num_classes=1108,num_channels=6,image_size=256):\n    model = EfficientNet.from_pretrained('efficientnet-'+name,num_classes=num_classes)\n    #model.load_state_dict(torch.load(EFFICIENTNET_MODELS[name]))\n    w = model._conv_stem.weight\n    #s = model._conv_stem.static_padding\n    model._conv_stem = utils.Conv2dStaticSamePadding(num_channels,32,kernel_size=(3, 3), stride=(2, 2), bias=False, image_size = image_size)\n    model._conv_stem.weight = nn.Parameter(torch.stack([torch.mean(w, 1)]*num_channels, dim=1))\n    return model","execution_count":0,"outputs":[]},{"metadata":{"id":"byAmg0Zle--u","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"def resnet18(pretrained,num_channels=6):\n    return resnet_multichannel(depth=18,pretrained=pretrained,num_channels=num_channels)\n\ndef _resnet_split(m): return (m[0][6],m[1])\n\ndef densenet161(pretrained,num_channels=6):\n    return densenet_multichannel(depth=161,pretrained=pretrained,num_channels=num_channels)\n  \ndef _densenet_split(m:nn.Module): return (m[0][0][7],m[1])\n\ndef efficientnetb0(pretrained=True,num_channels=6):\n    return efficientnet_multichannel(pretrained=pretrained,name='b0',num_channels=num_channels)\n","execution_count":0,"outputs":[]},{"metadata":{"id":"duo9Fd26wa2-","colab_type":"text"},"cell_type":"markdown","source":"Let's create our Learner:"},{"metadata":{"id":"XJvIjoGlwa3A","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":84},"outputId":"5c658594-a5de-43ca-c1ff-8ac4bbc07f8a","trusted":false},"cell_type":"code","source":"from fastai.metrics import *\nlearn = Learner(data, efficientnetb0(),metrics=[accuracy]).to_fp16()\nlearn.path = Path('../')","execution_count":null,"outputs":[]},{"metadata":{"id":"WAhkRXkwwa3R","colab_type":"text"},"cell_type":"markdown","source":"We will now unfreeze and train the entire model."},{"metadata":{"id":"lHPKmKgLwa3S","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":67},"outputId":"71ecee97-0209-499b-8dd9-1bf6dd05045b","trusted":false},"cell_type":"code","source":"learn.unfreeze()\n#learn.lr_find() #<-- uncomment to determine the learning rate (commented to reduce time)\n#learn.recorder.plot(suggestion=True) ","execution_count":null,"outputs":[]},{"metadata":{"id":"OBwtrSGbwa3V","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":127},"outputId":"83e5e437-8957-499e-ba4f-ee6c75d5c731","trusted":false},"cell_type":"code","source":"learn.fit_one_cycle(18,1e-3)","execution_count":null,"outputs":[]},{"metadata":{"id":"apFsHYKXwa3X","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":178},"outputId":"cc4b5035-fea2-4a54-c022-6456e4dfaf24","trusted":false},"cell_type":"code","source":"learn.recorder.plot_losses()\nlearn.recorder.plot_metrics()","execution_count":null,"outputs":[]},{"metadata":{"id":"ITSX3vH0wa3Z","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"learn.save('stage-2')\nlearn.export()","execution_count":0,"outputs":[]},{"metadata":{"id":"GA4EcIQIwa3b","colab_type":"text"},"cell_type":"markdown","source":"## Inference and Submission Generation"},{"metadata":{"id":"C47mcIFTwa3c","colab_type":"text"},"cell_type":"markdown","source":"Let's now load our test csv and process the DataFrame like we did for the training data."},{"metadata":{"id":"ISLuc-u2wa3d","colab_type":"code","colab":{},"trusted":true},"cell_type":"code","source":"test_df = pd.read_csv('../input/test.csv')\nproc_test_df = generate_df(test_df.copy())","execution_count":null,"outputs":[]},{"metadata":{"id":"nkcB9CxXwa3g","colab_type":"text"},"cell_type":"markdown","source":"We add the data to our DataBunch:"},{"metadata":{"id":"FidBxNo3wa3h","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"data_test = MultiChannelImageList.from_df(df=proc_test_df,path='../input/test/')\nlearn.data.add_test(data_test)","execution_count":0,"outputs":[]},{"metadata":{"id":"cBHsXyj9wa3l","colab_type":"text"},"cell_type":"markdown","source":"Now we can get out predictions on the test set."},{"metadata":{"id":"ubEraC8xwa3m","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"preds, _ = learn.get_preds(DatasetType.Test)","execution_count":0,"outputs":[]},{"metadata":{"id":"OaYyqaClwa3t","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"preds_ = preds.argmax(dim=-1)","execution_count":0,"outputs":[]},{"metadata":{"id":"5UWz4Pmnwa3x","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"test_df.head(10)","execution_count":0,"outputs":[]},{"metadata":{"id":"txbsG6yDwa33","colab_type":"text"},"cell_type":"markdown","source":"Let's open the sample submission file and load it with our predictions to create a submission."},{"metadata":{"id":"Qlw2WtI5wa33","colab_type":"code","colab":{},"trusted":true},"cell_type":"code","source":"submission_df = pd.read_csv('../input/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"id":"PzV7ochrwa36","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"submission_df.sirna = preds_.numpy().astype(int)\nsubmission_df.head(10)","execution_count":0,"outputs":[]},{"metadata":{"id":"BCSGb2RRwa39","colab_type":"code","colab":{},"trusted":false},"cell_type":"code","source":"submission_df.to_csv('submission.csv',index=False)","execution_count":0,"outputs":[]},{"metadata":{"id":"DTx-gbx7wa3_","colab_type":"text"},"cell_type":"markdown","source":"That's it!"},{"metadata":{"id":"BvrnXpFkwa4A","colab_type":"text"},"cell_type":"markdown","source":"## Future work:\n\nThis is only a simple baseline. There are many different things we can change:\n* Use both sites (right now I only use site 1)\n* Model architecture\n* Train multiple classifiers for different cell types\n* **Metric learning** - This will be the key to successful submissions"}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.6"},"colab":{"name":"rcic_fastai_starter.ipynb","version":"0.3.2","provenance":[],"toc_visible":true},"accelerator":"GPU"},"nbformat":4,"nbformat_minor":1}