{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Build a base model for the Human Protein Atlas - Single Cell Classification Competition using Tensorflow and Keras\n\n## Introduction\n\nImportant insights from the previous competition, from Ouyang et al. Nature Methods (2019) sections \"Strategies used by the top-ranking solutions\" and \"Assessing the biological relevance of the winning model with class activation maps (CAMs)\":\n\n1. Data augmentation such as random cropping, rotation, and flipping might improve model performance. \n2. Modifications of the loss function. \n3. DenseNet architecture more effective than ResNet. \n4. Medium sized networks worked better than larger ones (for example DenseNet121 performed better than DenseNet169).\n5. Using larger image sizes might improve scores.\n6. Model ensembling and stacking might improve performance. \n7. Class activation maps (CAMs) can be used for visualization of model spatial attention.\n\n\nArticles: \n\n[1] Ouyang, W., Winsnes, C.F., Hjelmare, M. et al. Analysis of the Human Protein Atlas Image Classification competition. Nat Methods 16, 1254–1261 (2019). https://doi.org/10.1038/s41592-019-0658-6\n\nNotebooks:\n\n(1) [DenseNet Trained with Old and New Data](https://www.kaggle.com/raimonds1993/aptos19-densenet-trained-with-old-and-new-data) by Federico Raimondi.\n\n(2) [Tutorial on Keras ImageDataGenerator with flow_from_dataframe](https://vijayabhaskar96.medium.com/tutorial-on-keras-imagedatagenerator-with-flow-from-dataframe-8bd5776e45c1) by Vijayabhaskar J. \n\n\nDatasets:\n\n(1) [HPA cell tiles sample balanced dataset: individual cells as RGB jpg images for rapid experimentation](https://www.kaggle.com/thedrcat/hpa-cell-tiles-sample-balanced-dataset) by Darek Kłeczek, a single-cell image version of the original dataset, below.\n\n(2) [Human Protein Atlas - Single Cell Classification Dataset](https://www.kaggle.com/c/hpa-single-cell-image-classification/data).\n\nPackage documentation:\n\n(1) [Keras DenseNet121](https://keras.io/api/applications/densenet).\n\n(2) [Tensorflow Module: tf.keras.layers.experimental.preprocessing](https://www.tensorflow.org/api_docs/python/tf/keras/layers/experimental/preprocessing/).\n\n(3) [Tensorflow Data augmentation](https://www.tensorflow.org/tutorials/images/data_augmentation).\n\n(4) [Tensorflow Image classification](https://www.tensorflow.org/tutorials/images/classification).\n\n(5) [Tensorflow Image dataset from directory](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image_dataset_from_directory).\n\n(6) [scikit-learn MultiLabelBinarizer](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MultiLabelBinarizer.html#sklearn.preprocessing.MultiLabelBinarizer).","metadata":{}},{"cell_type":"markdown","source":"\nTasks:\n\n1. Preprocessing:\n\n(1.1) Get unique single-cell image identifiers and multilabels.\n\n(1.2) Train and validation split.\n\n(1.3) Configure dataset for performance.\n\n2. Model definition.\n\n3. Training.\n\n4. Evaluation.","metadata":{}},{"cell_type":"code","source":"# libraries\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport glob\nimport warnings \nimport os,gc,cv2\nimport shutil\nimport random\nfrom tqdm.notebook import tqdm\nfrom PIL import Image, ImageDraw\nfrom sklearn.preprocessing import MultiLabelBinarizer\n\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras.preprocessing import image\nfrom tensorflow.keras.models import Model\nfrom tensorflow.keras import layers\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.applications import DenseNet121\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.optimizers import RMSprop\n\n\n%matplotlib inline\nwarnings.filterwarnings('ignore')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# directories \nCELL_IMG='../input/hpa-cell-tiles-sample-balanced-dataset/cells/'\nCELL_DF='../input/hpa-cell-tiles-sample-balanced-dataset/cell_df.csv'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1. Pre-processing","metadata":{}},{"cell_type":"markdown","source":"### (1.1) Get unique single-cell image identifiers and multilabels","metadata":{}},{"cell_type":"code","source":"# loads train dataframe\ntrain_df=pd.read_csv(CELL_DF)\ntrain_df.head(n=10)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# spliting label column\ntrain_df[\"image_labels\"] = train_df[\"image_labels\"].str.split(\"|\")\n\n# class labels\nclass_labels = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18']\n\n# binarizing each label/class\nfor label in tqdm(class_labels):\n    train_df[label] = train_df['image_labels'].map(lambda result: 1 if label in result else 0)\n\n# rename column\ntrain_df.columns = ['image_id', 'r_mean', 'g_mean', 'b_mean', 'cell_id', 'image_labels', 'size1', 'size2', 'Nucleoplasm', 'Nuclear membrane', 'Nucleoli', 'Nucleoli fibrillar center',\n                    'Nuclear speckles', 'Nuclear bodies', 'Endoplasmic reticulum', 'Golgi apparatus', 'Intermediate filaments',\n                    'Actin filaments', 'Microtubules', 'Mitotic spindle', 'Centrosome', 'Plasma membrane', 'Mitochondria',\n                    'Aggresome', 'Cytosol', 'Vesicles and punctate cytosolic patterns', 'Negative']","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# creates a new column with unique identifiers for single-cell images\n\nmultinames = ['id', 'r_mean', 'g_mean', 'b_mean', 'image_labels', 'size1', 'size2', 'Nucleoplasm', 'Nuclear membrane', 'Nucleoli', 'Nucleoli fibrillar center',\n                    'Nuclear speckles', 'Nuclear bodies', 'Endoplasmic reticulum', 'Golgi apparatus', 'Intermediate filaments',\n                    'Actin filaments', 'Microtubules', 'Mitotic spindle', 'Centrosome', 'Plasma membrane', 'Mitochondria',\n                    'Aggresome', 'Cytosol', 'Vesicles and punctate cytosolic patterns', 'Negative']\ncell_df=train_df\ncell_df[\"id\"] = cell_df['image_id'] +'_'+ cell_df['cell_id'].astype(str) \ncell_df[\"id\"] =  cell_df[\"id\"] + '.jpg'\ncell_df=cell_df.drop( columns=['image_id', 'cell_id'] )\ncell_df=cell_df.reindex( columns= multinames ) \ncell_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# change order of ids as in the cells folder\ncell_df=cell_df.sort_values('id', axis=0, ascending=True, inplace=False, \n                            kind='quicksort', na_position='last')\ncell_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# define multilabels for training\nmultilabels = ['Nucleoplasm', 'Nuclear membrane', 'Nucleoli', 'Nucleoli fibrillar center',\n                    'Nuclear speckles', 'Nuclear bodies', 'Endoplasmic reticulum', 'Golgi apparatus', 'Intermediate filaments',\n                    'Actin filaments', 'Microtubules', 'Mitotic spindle', 'Centrosome', 'Plasma membrane', 'Mitochondria',\n                    'Aggresome', 'Cytosol', 'Vesicles and punctate cytosolic patterns', 'Negative']\nprint( len(multilabels), '\\n')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### (1.2) Train and validation split","metadata":{}},{"cell_type":"markdown","source":"Use the tensorflow method 'flow_from_dataframe', as in this [notebook](https://www.kaggle.com/minniekabra/code-3may) ","metadata":{}},{"cell_type":"code","source":"# constant parameters\nIMG_SIZE = 224\nBATCH_SIZE = 32","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# image generator, rescaling is performed in a pre-processing layer below, \nimage_generator = image.ImageDataGenerator(\n    rescale=1./255,\n    data_format='channels_last',\n    preprocessing_function=None,\n    validation_split=0.2\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train set data flow from dataframe\ntrain_data = image_generator.flow_from_dataframe(\ncell_df,\ndirectory=CELL_IMG,\nx_col='id',\ny_col=multilabels,\nclass_mode='raw',    \ncolor_mode='rgb',\ntarget_size=(IMG_SIZE, IMG_SIZE),    \nbatch_size=BATCH_SIZE,\nseed=123,\nsubset='training'\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# validation set data flow from dataframe\nvalidation_data = image_generator.flow_from_dataframe(\ncell_df,\ndirectory=CELL_IMG,\nx_col='id',\ny_col=multilabels,\nclass_mode='raw',    \ncolor_mode='rgb',\ntarget_size=(IMG_SIZE, IMG_SIZE),    \nbatch_size=BATCH_SIZE,\nseed=123,\nsubset='validation'\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Model definition","metadata":{}},{"cell_type":"code","source":"# constant parameters for model definition\nNUM_CLASSES=19","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# DenseNet121 model\ndensenet = DenseNet121(\n    include_top=True,\n    weights=None,\n    input_shape=(IMG_SIZE,IMG_SIZE,3),\n    input_tensor=None,\n    pooling=None,\n    classes=NUM_CLASSES\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# model definition including a normalization layer and extra layers\nmodel_densenet = Sequential( [\nlayers.experimental.preprocessing.Rescaling( 1./255, input_shape=(IMG_SIZE, IMG_SIZE, 3) ),\nlayers.experimental.preprocessing.RandomFlip(\"horizontal\"),\nlayers.experimental.preprocessing.RandomFlip(\"vertical\"),\nlayers.experimental.preprocessing.RandomTranslation(height_factor=0.1, width_factor=0.1),\nlayers.experimental.preprocessing.RandomRotation(factor=1.0),\nlayers.experimental.preprocessing.RandomZoom(height_factor=0.25, width_factor=0.25),\ndensenet\n] )","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# shape of the output ndarray \nmodel_densenet.output","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# compile model\nlearning_rate = 1e-3\nmodel_densenet.compile(optimizer=Adam(lr=learning_rate), \n                       loss='binary_crossentropy', metrics=['categorical_accuracy'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# model summary\nmodel_densenet.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 3. Training","metadata":{}},{"cell_type":"code","source":"# constant training parameters\nEPOCHS=10","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# callbacks\nmodel_callbacks = [\n    tf.keras.callbacks.EarlyStopping(monitor='loss', patience=2, verbose=0),\n    tf.keras.callbacks.ModelCheckpoint(filepath='./densenet_model.{epoch:02d}-{val_loss:.2f}.h5'),\n    tf.keras.callbacks.TensorBoard(log_dir='./logs'),\n]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = model_densenet.fit(\ntrain_data,\nvalidation_data=validation_data,\nepochs=EPOCHS,\ncallbacks=model_callbacks     \n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot model accuracy\nplt.plot(history.history['categorical_accuracy'])\nplt.plot(history.history['val_categorical_accuracy'])\nplt.title('model accuracy')\nplt.ylabel('accuracy')\nplt.xlabel('epoch')\nplt.legend(['train', 'validation'], loc='upper left')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot model loss\nplt.plot(history.history['loss'])\nplt.plot(history.history['val_loss'])\nplt.title('model loss')\nplt.ylabel('loss')\nplt.xlabel('epoch')\nplt.legend(['train', 'validation'], loc='upper left')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Evaluation","metadata":{}}]}