{"cells":[{"metadata":{},"cell_type":"markdown","source":"A concatenated EffNet approach to the Recursion Cellular Image Classification competition\n======\nIn this approach to the Recursion Cellular Image Classification competition, there are two datasets used:\n1. The original Recursion Cellular Image Classification data for this competition\n2. The Recursion Cellular Image Classification: 128 data from the first version of this kernel\n\n**Note that the options GPU and Internet have to be checked.**"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os, sys, random,copy\nimport matplotlib.pyplot as plt\nfrom keras.models import Sequential\nfrom keras.models import Model\nfrom keras.layers import Dense, Dropout, GlobalAveragePooling2D, Input, concatenate\nfrom keras.utils import np_utils, Sequence\nfrom PIL import Image\nfrom PIL import ImageFilter\nfrom sklearn.model_selection import train_test_split\n!pip install efficientnet\nimport efficientnet\nfrom keras.optimizers import Adam\nfrom keras.callbacks import ModelCheckpoint","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"After importing the necessary modules one loads the test data..."},{"metadata":{"trusted":true},"cell_type":"code","source":"test_data = pd.read_csv(\"../input/recursion-cellular-image-classification/test.csv\")\nprint(\"Shape of test_data:\", test_data.shape)\ntest_data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"... and the training data into pandas dataframes."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data = pd.read_csv(\"../input/recursion-cellular-image-classification/train.csv\")\nprint(\"Shape of train_data:\", train_data.shape)\ntrain_data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To load the data in memory efficient batches later on an image data generator is necessary. Since I started out with the original dataset that needed a custom generator, I kept and modified my self-written generator instead of using the keras ImageDataGenerator class.\n\nTo load the data from an id_code the get_input function is necessary, which uses the PIL Image interface and has two options for either loading from the training or test directory."},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_input(id_code,site,train=True):\n    if train==True:\n        base_path = '../input/recursion-cellular-image-classification-128/train/train'\n    else:\n        base_path = '../input/recursion-cellular-image-classification-128/test/test'\n    img = Image.open(base_path+\"/\"+id_code+\"_s\"+str(site)+\".jpeg\")\n    return(img)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The ImgGen inherits the keras.utils.Sequence class. It takes a pandas dataframe as input with the id_codes as index and the sirna values as the first column. The generator supports shuffling at startup and using a custom preprocessing function for image transformation purposes like augmentation. Its default mode is using training data, else it loads the test data."},{"metadata":{"trusted":true},"cell_type":"code","source":"class ImgGen (Sequence):\n    def __init__(self, label_file, batch_size = 32,preprocess=(lambda x: x),train=True,shuffle=False):\n        if shuffle==True:\n            self.label_file = label_file.sample(frac=1)\n        else:\n            self.label_file = label_file\n        self.batch_size = batch_size\n        self.preprocess = preprocess\n        self.train = train\n        self.x = list(self.label_file.index)\n        if self.train==True:\n            self.y = list(self.label_file[self.label_file.columns[0]])\n\n    def __len__(self):\n        return int(np.ceil(len(self.x) / float(self.batch_size)))\n\n    def __getitem__(self, idx):\n        batch_x = self.x[idx * self.batch_size:(idx + 1) * self.batch_size]\n\n        if self.train==True:\n            batch_y = self.y[idx * self.batch_size:(idx + 1) * self.batch_size]\n            \n            x1 = [np.array(self.preprocess(get_input(id_code,1))).reshape(128,128,3) / 255 for id_code in batch_x] \n            x2 = [np.array(self.preprocess(get_input(id_code,2))).reshape(128,128,3) / 255 for id_code in batch_x]\n            y = [sirna for sirna in batch_y]\n            return [np.array(x1),np.array(x2)], np.array(y)\n        else:\n            x1 = [np.array(self.preprocess(get_input(id_code,1,train=False))).reshape(128,128,3) / 255 for id_code in batch_x] \n            x2 = [np.array(self.preprocess(get_input(id_code,2,train=False))).reshape(128,128,3) / 255 for id_code in batch_x] \n            return [np.array(x1),np.array(x2)]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To make the model more robust during training image augmentation is used. Since I don't know which transformations result in misleading images only small modifications like blur, rotation and rank filters are used."},{"metadata":{"trusted":true},"cell_type":"code","source":"def augment(image):\n    random_transform = random.randint(-1,4)\n    if random_transform==0:\n        image = image.rotate(random.randint(-5,5))\n    if random_transform==1:\n        image = image.filter(ImageFilter.GaussianBlur(radius=1))\n    if random_transform==2:\n        image = image.filter(ImageFilter.RankFilter(size=3, rank=1))\n    if random_transform==3:\n        image = image.filter(ImageFilter.MedianFilter(size=3))\n    if random_transform==4:\n        image = image.filter(ImageFilter.MaxFilter(size=3))\n    return image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since all cell images are taken from two sites there are two input channels that are processed by an EfficientNetB1. To shorten the fitting process the pretrained model from https://github.com/qubvel/efficientnet/releases/download/v0.0.1/efficientnet-b1_imagenet_1000_notop.h5 is used which is trained on the imagenet data.\n\nAfter pooling they are concatenated and result in a dense layer to output the classification result for all 1108 sirna classes. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_model():\n    effnet = efficientnet.EfficientNetB1(weights='imagenet',include_top=False,input_shape=(128, 128, 3))\n    site1 = Input(shape=(128,128,3))\n    site2 = Input(shape=(128,128,3))\n    x = effnet(site1)\n    x = GlobalAveragePooling2D()(x)\n    x = Model(inputs=site1, outputs=x)\n    y = effnet(site2)\n    y = GlobalAveragePooling2D()(y)\n    y = Model(inputs=site2, outputs=y)\n    combined = concatenate([x.output, y.output])\n    z = Dropout(0.5)(combined)\n    z = Dense(1108, activation='softmax')(z)\n    model = Model(inputs=[x.input, y.input], outputs=z)\n    model.compile(loss='sparse_categorical_crossentropy', optimizer=Adam(0.0001), metrics=['accuracy'])\n    model.summary()\n    \n    return model\n\nmodel = create_model()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"At first the model is trained and validated via instances of ImgGen on the whole training data. Afterwards the model with the best weights is refined on each cell type and then saved in 4 different checkpoints."},{"metadata":{"trusted":true},"cell_type":"code","source":"phases = ['All','HEPG2','HUVEC','RPE','U2OS']\nbatch_size = 128\ntest_size = 0.025\n\nfor phase in phases:\n    \n    print('Start phase %s.' % (phase))\n    \n    filepath = 'ModelCheckpoint_'+phase+'.h5'\n    print(\"Set filepath to %s.\" % (filepath))\n    \n    callback = [\n        ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, save_weights_only=False, mode='auto', period=1)\n        ]\n    \n    if phase != 'All':\n        model.load_weights('ModelCheckpoint_All.h5')\n        print(\"Successfully loaded weights of ModelCheckpoint_All.h5\")\n    \n    if phase != 'All':\n        label_file_source = train_data[train_data['id_code'].str.contains(phase)]\n        print(\"Successfully created label_file_source for phase %s.\" % (phase))\n    else:\n        label_file_source = train_data\n        print(\"Successfully created label_file_source for phase All.\")\n    \n    label_file = pd.DataFrame(index=label_file_source['id_code'],data=list(label_file_source['sirna']),columns=['sirna'])\n    train, val = train_test_split(label_file, test_size=test_size)\n    train_gen = ImgGen(train,batch_size=batch_size,shuffle=True,preprocess=augment)\n    val_gen = ImgGen(val,batch_size=batch_size,shuffle=True,preprocess=augment)\n    \n    history = model.fit_generator(train_gen, \n                              steps_per_epoch=len(train)//batch_size, \n                              epochs=25, \n                              verbose=1, \n                              validation_data=val_gen,\n                              validation_steps=len(val)//batch_size,\n                              callbacks=callback\n                             )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Finally each cell type on the test data is predicted separately. This is done by loading the corresponding refined weigths and then predicting on a filtered list.\nThe 4 prediction arrays are then combined into the submission file."},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.DataFrame()\nid_codes = []\n\nfor phase in phases[1:]:\n    test_label_file = pd.DataFrame(index=(test_data[test_data['id_code'].str.contains(phase)]['id_code']))\n    \n    model.load_weights('ModelCheckpoint_'+phase+'.h5')\n    print(\"Successfully loaded weights of ModelCheckpoint_%s.h5\" % (phase))\n    test_generator = ImgGen(test_label_file,batch_size=batch_size,train=False)\n    if phase == phases[1]:\n        predictions = model.predict_generator(test_generator,verbose=1)\n    else:\n        predictions = np.append(predictions,model.predict_generator(test_generator,verbose=1), axis=0)\n    id_codes += list(test_label_file.index)\n    \nsubmission['id_code'] = id_codes\nsubmission['sirna'] = predictions.argmax(axis=-1)\nsubmission.to_csv(\"submission.csv\",index=False)\n\nprint(pd.read_csv(\"submission.csv\"))","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.6"}},"nbformat":4,"nbformat_minor":1}