{"cells":[{"metadata":{},"cell_type":"markdown","source":"As reported by Recursion [in this post](https://www.kaggle.com/c/recursion-cellular-image-classification/discussion/102905), there is a special structure in the data which simplifies predictions significantly.\n\nAssignments of sirnas to plates is not completely random in this competition. In this kernel, first I show it on the train data, and then apply the leak [on the pretrained Keras model](https://www.kaggle.com/chandyalex/recursion-cellular-keras-densenet) (kudos to [Alex](https://www.kaggle.com/chandyalex)) with LB 0.113 to get score 0.207. Same model which uses 2 sites for inference gets LB score 0.231 (the original model uses only one site but I just can't hold myself on that). "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\n\nfrom tqdm import tqdm\nimport PIL\nimport cv2\nfrom PIL import Image, ImageOps\n\nfrom keras.models import Sequential, load_model\nfrom keras.layers import (Activation, Dropout, Flatten, Dense, Input, Conv2D, GlobalAveragePooling2D)\nfrom keras.applications.densenet import DenseNet121\nimport keras\nfrom keras.models import Model\n\nSIZE = 224\nNUM_CLASSES = 1108\n\ntrain_csv = pd.read_csv(\"../input/recursion-cellular-image-classification/train.csv\")\ntest_csv = pd.read_csv(\"../input/recursion-cellular-image-classification/test.csv\")\nsub = pd.read_csv(\"../input/recursion-cellular-keras-densenet/submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train data"},{"metadata":{},"cell_type":"markdown","source":"Look at the first 10 sirnas plates assignments across the train set. One can observe that two sirnas that are on the same plate in the first experiment stay on the same plate for all experiments. Moreover, there are only 3 unique rows. "},{"metadata":{"trusted":true},"cell_type":"code","source":"np.stack([train_csv.plate.values[train_csv.sirna == i] for i in range(10)]).transpose()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The above observation can be easily verified on the whole train data, - there are 4 groups of 277 sirnas in each group which stick together. \n\nBut is there any order of groups to plates assignment? In general, there are `4*3*2=24` possible combinations of assigning 4 groups to 4 plates. But in the train data only 3 are active, each assignment appearing 22, 7 and 4 times respectively."},{"metadata":{"trusted":true},"cell_type":"code","source":"# you will see the same output here for each sirna number\ntrain_csv.loc[train_csv.sirna==0,'plate'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Later we will see that the 4th combination, missing from the training data, does in fact appear in the test data. My conlusion here is that Recursion used some kind of rotation of plates only, therefore only 4 combinations.\n\nLet's calculate which sirna belongs to which plate in every of the 4 assignments:"},{"metadata":{"trusted":true},"cell_type":"code","source":"plate_groups = np.zeros((1108,4), int)\nfor sirna in range(1108):\n    grp = train_csv.loc[train_csv.sirna==sirna,:].plate.value_counts().index.values\n    assert len(grp) == 3\n    plate_groups[sirna,0:3] = grp\n    plate_groups[sirna,3] = 10 - grp.sum()\n    \nplate_groups[:10,:]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Test data"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"Now let's take a look if we observe the same behavior in the test data. I use the output predictions from the kernel that I mentioned to calculate average probability of each assignment for every experiment."},{"metadata":{"trusted":true},"cell_type":"code","source":"all_test_exp = test_csv.experiment.unique()\n\ngroup_plate_probs = np.zeros((len(all_test_exp),4))\nfor idx in range(len(all_test_exp)):\n    preds = sub.loc[test_csv.experiment == all_test_exp[idx],'sirna'].values\n    pp_mult = np.zeros((len(preds),1108))\n    pp_mult[range(len(preds)),preds] = 1\n    \n    sub_test = test_csv.loc[test_csv.experiment == all_test_exp[idx],:]\n    assert len(pp_mult) == len(sub_test)\n    \n    for j in range(4):\n        mask = np.repeat(plate_groups[np.newaxis, :, j], len(pp_mult), axis=0) == \\\n               np.repeat(sub_test.plate.values[:, np.newaxis], 1108, axis=1)\n        \n        group_plate_probs[idx,j] = np.array(pp_mult)[mask].sum()/len(pp_mult)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we go, this is the average probabilities for each test experiment to be in every of the 4 assignments:\n\nOne can see the favorites. "},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.DataFrame(group_plate_probs, index = all_test_exp)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's select the most probable assignment for every test experiment. You may say that some of the selections here are not certain and the probabilities are too close. But we get the same assignments with our much better models, so even this relatively simple model is able to make correct assignments."},{"metadata":{"trusted":true},"cell_type":"code","source":"exp_to_group = group_plate_probs.argmax(1)\nprint(exp_to_group)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Running predictions with the existing DenseNet121 model"},{"metadata":{},"cell_type":"markdown","source":"In the code below we load the model, make predictions to get the full probabilites matrix, and set 3 out of 4 plates for every sirna to zero, according to the assignment that we previously selected."},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_model(input_shape,n_out):\n    input_tensor = Input(shape=input_shape)\n    base_model = DenseNet121(include_top=False,\n                   weights=None,\n                   input_tensor=input_tensor)\n    x = GlobalAveragePooling2D()(base_model.output)\n    x = Dense(1024, activation='relu')(x)\n \n    final_output = Dense(n_out, activation='softmax', name='final_output')(x)\n    model = Model(input_tensor, final_output)\n    \n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = create_model(input_shape=(SIZE,SIZE,3),n_out=NUM_CLASSES)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.load_weights('../input/recursion-cellular-keras-densenet/Densenet121.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predicted = []\nfor i, name in tqdm(enumerate(test_csv['id_code'])):\n    path1 = os.path.join('../input/recursion-cellular-image-classification-224-jpg/test/test/', name+'_s1.jpeg')\n    image1 = cv2.imread(path1)\n    score_predict1 = model.predict((image1[np.newaxis])/255)\n    \n    path2 = os.path.join('../input/recursion-cellular-image-classification-224-jpg/test/test/', name+'_s2.jpeg')\n    image2 = cv2.imread(path2)\n    score_predict2 = model.predict((image2[np.newaxis])/255)\n    \n    predicted.append(0.5*(score_predict1 + score_predict2))\n    #predicted.append(score_predict1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predicted = np.stack(predicted).squeeze()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"this is the function that sets 75% of the sirnas to zero according to the selected assignment:"},{"metadata":{"trusted":true},"cell_type":"code","source":"def select_plate_group(pp_mult, idx):\n    sub_test = test_csv.loc[test_csv.experiment == all_test_exp[idx],:]\n    assert len(pp_mult) == len(sub_test)\n    mask = np.repeat(plate_groups[np.newaxis, :, exp_to_group[idx]], len(pp_mult), axis=0) != \\\n           np.repeat(sub_test.plate.values[:, np.newaxis], 1108, axis=1)\n    pp_mult[mask] = 0\n    return pp_mult","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for idx in range(len(all_test_exp)):\n    #print('Experiment', idx)\n    indices = (test_csv.experiment == all_test_exp[idx])\n    \n    preds = predicted[indices,:].copy()\n    \n    preds = select_plate_group(preds, idx)\n    sub.loc[indices,'sirna'] = preds.argmax(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"(sub.sirna == pd.read_csv(\"../input/recursion-cellular-keras-densenet/submission.csv\").sirna).mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.to_csv('../working/submission.csv', index=False, columns=['id_code','sirna'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That is all! I hope this demonstration will get everyone to the same playing field on this issue. Assigning 277 sirnas to 277 wells is still a hard problem to crack."}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}