{"cells":[{"metadata":{"_uuid":"e53eb094e65df48e6afbf17733fac01e93f9419a"},"cell_type":"markdown","source":"**What am I predicting?**\n\nYou are predicting protein organelle localization labels for each sample. There are in total 28 different labels present in the dataset. The dataset is acquired in a highly standardized way using one imaging modality (confocal microscopy). However, the dataset comprises 27 different cell types of highly different morphology, which affect the protein patterns of the different organelles. All image samples are represented by four filters (stored as individual files), the protein of interest (green) plus three cellular landmarks: nucleus (blue), microtubules (red), endoplasmic reticulum (yellow). The green filter should hence be used to predict the label, and the other filters are used as references."},{"metadata":{"trusted":true,"_uuid":"956dd967470a77ca0bcdabf242ebe0b9fd8e9658"},"cell_type":"code","source":"import os #for OS utilities\nimport numpy as np # for linear algebra and matrices operations\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom skimage.io import imread #For reading an image\nimport matplotlib.pyplot as plt # For Plotting an images\nfrom skimage.segmentation import mark_boundaries #For drawing boundaries on the detected regions\nfrom skimage.util import montage # To plot images in a montage style (collection of small scale )\nmontage_rgb = lambda x: np.stack([montage(x[:, :, :, i]) for i in range(x.shape[3])], -1)\nbase_dir = '../input' # Base directory\ntrain_image_dir = os.path.join(base_dir, 'train') # path of the training images\ntest_image_dir = os.path.join(base_dir, 'test') # path of the test images\nimport gc # Garbage Collection for optimized memory allocation\ngc.enable() # memory is tight\nfrom itertools import chain #for chain interations of label\nfrom collections import Counter # This can be used to count the labels.\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7e75ad2d7ae3597cd79e3da5afb0a56ff64fd52f"},"cell_type":"markdown","source":"A dictionary to save the target ID and the name of the Proteun. This is very useful for humans. :)"},{"metadata":{"trusted":true,"_uuid":"b6b0faf0cc01986ab5f0af3115cd31dd0178a66f"},"cell_type":"code","source":"name_label_dictionary = {\n0:  \"Nucleoplasm\", \n1:  \"Nuclear membrane\",   \n2:  \"Nucleoli\",   \n3:  \"Nucleoli fibrillar center\" ,  \n4:  \"Nuclear speckles\"   ,\n5:  \"Nuclear bodies\"   ,\n6:  \"Endoplasmic reticulum\",   \n7:  \"Golgi apparatus\"   ,\n8:  \"Peroxisomes\"   ,\n9:  \"Endosomes\"   ,\n10:  \"Lysosomes\"   ,\n11:  \"Intermediate filaments\",   \n12:  \"Actin filaments\"   ,\n13:  \"Focal adhesion sites\",   \n14:  \"Microtubules\"   ,\n15:  \"Microtubule ends\",   \n16:  \"Cytokinetic bridge\",   \n17:  \"Mitotic spindle\"   ,\n18:  \"Microtubule organizing center\" ,  \n19:  \"Centrosome\"   ,\n20:  \"Lipid droplets\",   \n21:  \"Plasma membrane\",   \n22:  \"Cell junctions\"  , \n23:  \"Mitochondria\"   ,\n24:  \"Aggresome\"   ,\n25:  \"Cytosol\",\n26:  \"Cytoplasmic bodies\",   \n27:  \"Rods & rings\" \n}","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"gaussian_noise = 0.1 \n\n# number of validation images to use\nnumber_of_validation_images = 1000\n# maximum number of training images\nnumber_of_training_images = 15000 \n#Pre-Trained Models for Transfer Learning\n\nbase_model ='VGG16' # ['VGG16', 'RESNET52', 'InceptionV3', 'Xception', 'DenseNet169', 'DenseNet121']\nimage_size = (299, 299) # [(224, 224), (384, 384), (512, 512), (640, 640)]\nbatch_size = 64 # [1, 8, 16, 24]\ndrop_out = 0.5\ndense_count = 128\nlearning_rate = 1e-4\nepochs = 5\nrgb_flip = 1 # should rgb be flipped when rendering images\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cab729b1d63124efdd45cc8cd1ed39b280424524"},"cell_type":"code","source":"image_dataframe = pd.read_csv(os.path.join(base_dir,'train.csv')) #Create a Data Frame to store the images along with the labels\nprint(image_dataframe.shape[0], 'masks found')  #Shape of our trainig data\n\n# Lets begin by using just the green images, :)\n#image_dataframe['green_path'] = image_dataframe['Id'].map(lambda x: os.path.join(train_image_dir, '{}_green.png'.format(x)))\nimage_dataframe['target_list'] = image_dataframe['Target'].map(lambda x: [int(a) for a in x.split(' ')])\nimage_dataframe.head()\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"05166ed009eb1866b712fa93c182f2fca2575d1f"},"cell_type":"markdown","source":"A Counter is a dict subclass for counting hashable objects. It is an unordered collection where elements are stored as dictionary keys and their counts are stored as dictionary values.\n\n"},{"metadata":{"trusted":true,"_uuid":"df356ee0c841b9bd155f3d527d0ef3c71fe55301"},"cell_type":"code","source":"protein_labels = list(chain.from_iterable(image_dataframe['target_list'].values))\ncount_value = Counter(protein_labels)\nn_keys = count_value.keys()\nmax_idx = max(n_keys)\nfig, ax1 = plt.subplots(1,1, figsize = (10, 5))\nax1.bar(n_keys, [count_value[k] for k in n_keys],color ='red' )\nax1.set_xticks(range(max_idx))\nax1.set_xticklabels([name_label_dictionary[k] for k in range(max_idx)], rotation=90)\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"24a635b428bb742866ca1931f7e4c710fe71b36c","_kg_hide-output":true,"scrolled":true},"cell_type":"code","source":"count_combination_dataframe = image_dataframe['Target'].value_counts()\nprint(count_combination_dataframe)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c0071da72aed50b4922bc41810e72d31177b246e"},"cell_type":"code","source":"from PIL import Image\n\nred = np.array(Image.open(\"../input/test/00631ec8-bad9-11e8-b2b9-ac1f6b6435d0_red.png\").convert(\"L\"))\ngreen = np.array(Image.open(\"../input/test/00631ec8-bad9-11e8-b2b9-ac1f6b6435d0_green.png\").convert(\"L\"))\nblue = np.array(Image.open(\"../input/test/00631ec8-bad9-11e8-b2b9-ac1f6b6435d0_blue.png\").convert(\"L\"))\nyellow = np.array(Image.open(\"../input/test/00631ec8-bad9-11e8-b2b9-ac1f6b6435d0_yellow.png\").convert(\"L\"))\n\ndemo_rgb=Image.fromarray(np.concatenate((np.expand_dims(red,axis=2),np.expand_dims(green,axis=2),np.expand_dims(blue,axis=2)),axis=2))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"07ca42ed4b70ee885cd2e9589c5f5f8e59839f3b"},"cell_type":"code","source":"demo_rgb","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"14d9afa9a39ff5b387fa518682f3d513b887c020","trusted":true,"_kg_hide-input":false},"cell_type":"code","source":"all_red =[]\nall_green= []\nall_blue = []\nall_yellow =[]\n\nfor i in image_dataframe['Id'][0:50]:        \n    all_red.append(np.array(Image.open(\"../input/train/\"+i+\"_red.png\").convert(\"L\")))\n    all_green.append(np.array(Image.open(\"../input/train/\"+i+\"_green.png\").convert(\"L\")))\n    all_blue.append(np.array(Image.open(\"../input/train/\"+i+\"_blue.png\").convert(\"L\")))\n    all_yellow.append(np.array(Image.open(\"../input/train/\"+i+\"_yellow.png\").convert(\"L\")))\n    \nprint(len(all_red))\nprint(len(all_green))\nprint(len(all_blue))\nprint(len(all_yellow))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4f96040e77c729656b75b80018ae82100d108ced"},"cell_type":"code","source":"import pickle\nwith open('red_images.pkl','wb') as f:\n    pickle.dump(all_red,f)\nwith open('green_images.pkl','wb') as f:\n    pickle.dump(all_green,f)\nwith open('blue_images.pkl','wb') as f:\n    pickle.dump(all_blue,f)\nwith open('yellow_images.pkl','wb') as f:\n    pickle.dump(all_yellow,f)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"405f6a2e9827d2a23de80ed8d7fd56075d977833"},"cell_type":"code","source":"with open('red_images.pkl','rb') as f:\n    tmp = pickle.load(f)\nlen(tmp)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5a264d832d9190cfac5008e67ff6dcc051868565","trusted":true},"cell_type":"code","source":"Image.fromarray(tmp[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"dc1380703b0961925d4769980d88565f4e736a6d"},"cell_type":"code","source":"red = np.array(Image.open(\"../input/train/00070df0-bbc3-11e8-b2bc-ac1f6b6435d0_red.png\").convert(\"L\"))\nImage.fromarray(red)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"37e4c0c87776c636557be103d726760bb34ceb05","_kg_hide-output":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4c77413ad676b0ce3de168bacfbf1ed3bb0e5ae2"},"cell_type":"markdown","source":""}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}