{"cells":[{"metadata":{},"cell_type":"markdown","source":"### Some notes\nThis notebook contains EDA, EfficientNet, and Creating video from training images.<br>\n<br>**Likbez on topic:**\n+ **Structure of a cell:** <br>[[en] playlist](https://www.youtube.com/playlist?list=PLSQl0a2vh4HDmOg7VVnL5kiEh7tKB-jJh) or [[rus] playlist](https://www.youtube.com/playlist?list=PLRGeEPbOb5tPQBPEP2XamxksDWR5Jm-0G)\n+ **Generation and action of siRNAs and miRNAs:**\n<br>\n[[en] 7 min video](https://www.youtube.com/watch?v=5YsTW5i0Xro&list=LL-R1-jljOQArmtGuwfFkpuw&index=2&t=1s) or [[rus] 5 min video](https://www.youtube.com/watch?v=EHWIDbsSE_Y&list=LL-R1-jljOQArmtGuwfFkpuw&index=25&t=0s)\n+ **Brief introduction from rxrx:**\n+ https://www.rxrx.ai/\n\n**Some notes about the data** (as i understood them)\n\nThe images from data are generated by carrying out biological experiments using reagents known as siRNAs.\n\nEach images instance has 6 individual channel different organelles of the cells - the nucleus, endoplasmic reticulum, actin cytoskeleton, nucleolus, mitochondria, and golgi apparatus.\n\nEach six-channel image is one of the types of cells:\n+ [HUVEC](http://www.lgcstandards-atcc.org/products/all/CRL-1730.aspx?geo_country=ua#generalinformation)\n+ [RPE](https://www.lgcstandards-atcc.org/products/all/CRL-4000.aspx)\n+ [HepG2](https://www.lgcstandards-atcc.org/products/all/HB-8065.aspx?geo_country=ua)\n+ [U2OS](https://www.lgcstandards-atcc.org/Products/All/HTB-96.aspx?geo_country=ua)<br>\n\nYou can view all RGB images from the competition's training data in [this](https://www.youtube.com/watch?v=D3sLsakGoNI&feature=youtu.be) video or short videos from each experiment in the output of this kernel."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# Loading libraries\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport matplotlib.pyplot as plt\nimport cv2\nfrom PIL import Image\nfrom matplotlib import cm\n\nimport os\nprint(os.listdir(\"../input/\"))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## EDA"},{"metadata":{"trusted":true},"cell_type":"code","source":"pixel_status = pd.read_csv('../input/recursion-cellular-image-classification/pixel_stats.csv')\ntrain_df = pd.read_csv('../input/recursion-cellular-image-classification/train.csv')\ntest_df = pd.read_csv('../input/recursion-cellular-image-classification/test.csv')\n\ntrain_controls = pd.read_csv('../input/recursion-cellular-image-classification/train_controls.csv')\ntest_controls = pd.read_csv('../input/recursion-cellular-image-classification/test_controls.csv')\n\nsub = pd.read_csv('../input/recursion-cellular-image-classification/sample_submission.csv')\n\nprint('Dimensions: \\n pixel_status: %s'\\\n     '\\n train_df: %s \\n test_df: %s' \\\n      '\\n train_controls: %s \\n test_controls: %s' \\\n      '\\n submission: %s' % (pixel_status.shape, train_df.shape, \n                            test_df.shape, train_controls.shape,\n                            test_controls.shape, sub.shape))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pixel_status.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_controls.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Image from dataset with index 1\nexp, well, plate = train_df.loc[1,['experiment', 'well', 'plate']]\n\n# List of arrays of different channels(total 6) of the same image\nimg_names = [np.array(Image.open(os.path.join('../input/recursion-cellular-image-classification/train/',\n                                              exp,\n                                              f'Plate{plate}',\n                                              f'{well}_s{1}_w{channel}.png')),\n                      dtype=np.float32) for channel in range(1,7)]\n\n# Сonversion to a six-channel image\nsample = np.stack([img_ar for img_ar in img_names],axis=0)\nsample.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_cell(sample_img):    \n    channels = ['Nuclei', 'Endoplasmic reticuli', 'Actin', 'Nucleoli', 'Mitochondria', 'Golgi apparatus']\n    cmaps = ['gist_ncar','terrain', 'gnuplot' ,'rainbow','PiYG', 'gist_earth']\n\n    fig=plt.figure(figsize=(20, 15))\n    for i in range(1,6+1):\n        fig.add_subplot(1, 6, i)\n        plt.imshow(sample_img[i-1, :, :,],cmap=cmaps[i-1]);\n        plt.axis('off');\n        plt.title(f'{channels[i-1]}')\n    fig.suptitle(\"Single image channels\", y=0.65, fontsize=15)\n    plt.show()\n    \n## Let's looking on image channels\nplot_cell(sample)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Datagenerator"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Loading libraries\nimport sys\n\npackage_path = '../input/efficientnet/efficientnet-pytorch/EfficientNet-PyTorch/'\nsys.path.append(package_path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Loading libraries\nimport sys\nfrom efficientnet_pytorch import EfficientNet\n\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset, DataLoader\nimport torch.nn.functional as F\n\nimport torchvision\nimport torchvision.transforms as transforms","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class CellDataset(Dataset):\n    def __init__(self, df, img_dir, site=1, transforms=None):\n        self.df = df\n        self.img_dir = img_dir\n        self.site = site\n        self.transforms = transforms\n        \n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, idx):\n        exp, well, plate = self.df.loc[idx,['experiment', 'well', 'plate']].values\n        img_channels = [np.array(Image.open(os.path.join(self.img_dir,\n                                             exp,\n                                             f'Plate{plate}',\n                                             f'{well}_s{self.site}_w{channel}.png')), \n                                          dtype=np.float32) for channel in range(1,7)]\n        \n        one_img = np.stack([channel for channel in img_channels],axis=2)\n        \n        if self.transforms is not None:\n            one_img = self.transforms(one_img)\n        if self.img_dir == '../input/recursion-cellular-image-classification/train/':\n            return one_img, self.df.loc[idx,['sirna']].astype('int32').values\n        else:\n            return one_img\n                                 \n            ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Augmentations for data\naug = transforms.Compose([\n      # transforms.ToPILImage(),\n        transforms.ToTensor(),\n        transforms.Normalize(mean=[0.485, 0.485, 0.456, 0.456, 0.406, 0.406],\n                                 std=[0.229, 0.229, 0.225, 0.225, 0.224, 0.224])\n])\n\n# Dataset & data loaders\ndataset = CellDataset(df=train_df, img_dir='../input/recursion-cellular-image-classification/train/', transforms=aug)\ntrain_loader = DataLoader(dataset=dataset, batch_size=15, shuffle=True)\n\ntest_dataset = CellDataset(df=test_df, img_dir='../input/recursion-cellular-image-classification/test/', transforms=aug)\ntest_loader = DataLoader(dataset=test_dataset, batch_size=15, shuffle=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data checking"},{"metadata":{"trusted":true},"cell_type":"code","source":"#train_loader checking\ndata, target = next(iter(train_loader))\nprint(data.shape, target.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#test_loader checking\ntest_data = next(iter(test_loader))\nprint(test_data.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data, target = next(iter(train_loader))\nprint('Dimension:', data.shape, \",\", target[:, 0].shape)\nprint('Datatype: ', data.type(),\",\", target.type())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_cell(data.numpy()[1,:,:,:])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Model parameters\nnum_epochs = 10\ntotal_step = len(train_loader)\nin_ch = 6\nlr = 0.001","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = EfficientNet.from_pretrained('efficientnet-b0', num_classes=1108)\n\n# Changes count input channels of our model\ntrained_kernel = model._conv_stem.weight\nnew_conv = nn.Sequential(nn.Conv2d(in_ch, 32, kernel_size=(3,3), stride=(2,2), bias=False),\n            nn.ZeroPad2d(padding=(0, 1, 0, 1)))\nwith torch.no_grad():\n    new_conv[0].weight[:,:] = torch.stack([torch.mean(trained_kernel, 1)]*6, dim=1)\nmodel._conv_stem = new_conv\nmodel = model.cuda()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Loss and Optimizer\ncriterion = nn.CrossEntropyLoss()\noptimizer = torch.optim.Adam(model.parameters(), lr=lr)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Train model\nfor epoch in range(num_epochs):\n    for batch_i, (data, target) in enumerate(train_loader):\n        data, target = data.cuda(), target[:,0].long().cuda()\n        #print(data.shape)\n        outputs = model(data)\n        loss = criterion(outputs, target)\n        \n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n        \n        if (batch_i+1) % 100 == 0:\n            print('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'\n                 .format(epoch+1, num_epochs, batch_i+1, total_step, loss.item()))\ntorch.save(model.state_dict(), 'model.pt')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions = []\nmodel.eval()\nwith torch.no_grad():\n    for data in test_loader:\n        data = data.cuda()\n        output = model(data)\n        batch_idx = output.max(dim=-1)[1].cpu().numpy()\n        for pred in batch_idx:\n            predictions.append(pred.astype(int))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub['sirna'] = predictions\nsub.to_csv('submission.csv', index=False, columns=['id_code','sirna'])\nprint('Number of unique values:',len(sub['sirna'].value_counts()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Converting six-channel image to RGB with Pillow"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(sample.shape)\n# splitting a six-channel image into two three-channel images\nrgb1 = Image.fromarray(np.uint8(sample[:3,:,:].transpose(1,2,0))).convert('RGB')\nrgb2 = Image.fromarray(np.uint8(sample[3:,:,:].transpose(1,2,0))).convert('RGB')\n# rgb1 + rgb2 (interpolation)\n#rgb3 = Image.blend(rgb1, rgb2, 0.5).convert('RGB')\nrgb3 = Image.blend(rgb1, rgb2, 0.5).convert('L')\n# after which their composition for color saturation\nimg_composit = Image.composite(rgb1, rgb2, rgb3)\nimg_composit","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Creating video from all training images"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Code from https://github.com/recursionpharma/rxrx1-utils/blob/master/rxrx/io.py\nDEFAULT_CHANNELS = (1, 2, 3, 4, 5, 6)\nFPS = float(5)\nRGB_MAP = {\n    1: {\n        'rgb': np.array([19, 0, 249]),\n        'range': [0, 51]\n    },\n    2: {\n        'rgb': np.array([42, 255, 31]),\n        'range': [0, 107]\n    },\n    3: {\n        'rgb': np.array([255, 0, 25]),\n        'range': [0, 64]\n    },\n    4: {\n        'rgb': np.array([45, 255, 252]),\n        'range': [0, 191]\n    },\n    5: {\n        'rgb': np.array([250, 0, 253]),\n        'range': [0, 89]\n    },\n    6: {\n        'rgb': np.array([254, 255, 40]),\n        'range': [0, 191]\n    }\n}\n\ndef convert_tensor_to_rgb(t, channels=DEFAULT_CHANNELS, vmax=255, rgb_map=RGB_MAP):\n    colored_channels = []\n    for i, channel in enumerate(channels):\n        x = (t[i, :, :] / vmax) / \\\n            ((rgb_map[channel]['range'][1] - rgb_map[channel]['range'][0]) / 255) + \\\n            rgb_map[channel]['range'][0] / 255\n        x = np.where(x > 1., 1., x)\n        x_rgb = np.array(\n            np.outer(x, rgb_map[channel]['rgb']).reshape(512, 512, 3),\n            dtype=int)\n        colored_channels.append(x_rgb)\n    im = np.array(np.array(colored_channels).sum(axis=0), dtype=int)\n    im = np.where(im > 255, 255, im)\n    return im","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Convert images -> tensors -> list rgb -> video\ndef tensor_rgb_video(df, img_dir, experiment, video_name):\n    \n    df = df[df['experiment']==experiment].reset_index(drop=True)\n\n    descript = np.zeros((512,512,3),dtype=np.uint8)\n    descript = cv2.putText(descript, f'Experiment: {experiment}',\n                           (10,250), cv2.FONT_ITALIC,\n                           1.3,(255,255,255),2,cv2.LINE_AA)\n\n    img_list = [descript] * 5\n    for i in range(100):#len(df)\n        well, plate = df.loc[i,['well', 'plate']]\n        img_names = [np.array(Image.open(os.path.join(img_dir,\n                                                      experiment,\n                                                      f'Plate{plate}',\n                                                      f'{well}_s{1}_w{channel}.png')),\n                              dtype=np.float32) for channel in range(1,7)]\n    \n        tensor = np.stack([img_ar for img_ar in img_names],axis=0)\n        rgb_img = convert_tensor_to_rgb(tensor)\n        img_list.append(rgb_img)\n        \n    height, width, layers = img_list[1].shape\n    video = cv2.VideoWriter(video_name, 0, FPS, (width, height))\n    for img in img_list:  \n        video.write(img.astype('uint8'))\n    video.release()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Recursive function call\ndef all_video_experiments():\n    experiments = train_df['experiment'].value_counts().to_string().split()[::2]\n    a = len(experiments)\n    while a !=0:\n        a -= 1\n        tensor_rgb_video(train_df,'../input/recursion-cellular-image-classification/train/', experiments[a], f'{experiments[a]}.avi')\n        \nall_video_experiments()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}