{"cells":[{"metadata":{},"cell_type":"markdown","source":"## SIIM-ISIC: EDA and Model training\n---\n<!-- <font size=\"2\"> -->\nSkin cancer is the most prevalent type of cancer. Melanoma, specifically, is responsible for 75% of skin cancer deaths, despite being the least common skin cancer. The American Cancer Society estimates over 100,000 new melanoma cases will be diagnosed in 2020. It's also expected that almost 7,000 people will die from the disease. As with other cancers, early and accurate detection—potentially aided by data science—can make treatment more effective.\n\nThrough this notebook, I aim to provide you with an insight to the data, what to expect, and also help you build a model using thhe image, sex of patient, the anatomical site of the image, and approximate age of the patient as model inputs. \n\nMelanoma is a deadly disease, but if caught early, most melanomas can be cured with minor surgery. Image analysis tools that automate the diagnosis of melanoma will improve dermatologists' diagnostic accuracy. Better detection of melanoma has the opportunity to positively impact millions of people.\n<!-- </font>  -->","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n# !pip install tf-nightly\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport tensorflow as tf\nimport matplotlib.pyplot as plt\nimport cv2\nfrom sklearn.model_selection import train_test_split\nfrom kaggle_datasets import KaggleDatasets\n\nAUTOTUNE = tf.data.experimental.AUTOTUNE\nGCS_path = KaggleDatasets().get_gcs_path('melanoma-256x256')\nGCS_path","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">The block of code below handles the itsy-bitsy tasks like imputing missing values and converting paths to usable formats.</font>\n\n---","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dataframe = pd.read_csv('../input/siim-isic-melanoma-classification/train.csv')\n\ndataframe['image_path'] ='../input/siim-isic-melanoma-classification/jpeg/train/' + dataframe['image_name']+'.jpg'\ndataframe.pop('image_name')\n\ndataframe['anatom_site'] = pd.Categorical(dataframe['anatom_site_general_challenge'])\nanatom_site_categories = dataframe.anatom_site.cat.categories\ndataframe['anatom_site'] = dataframe.anatom_site.cat.codes\ndataframe['anatom_site'].fillna((dataframe['anatom_site'].mean()), inplace=True)\n\ndataframe['sex'] = pd.Categorical(dataframe['sex'])\nsex_categories = dataframe.sex.cat.categories\ndataframe['sex'] = dataframe.sex.cat.codes\ndataframe['sex'].fillna((dataframe['sex'].mean()), inplace=True)\n\ndataframe['age_approx'].fillna((dataframe['age_approx'].mean()), inplace=True)\ndataframe = dataframe.astype({\"sex\":np.int32,\"age_approx\":np.int32, 'target':np.int8})\n\ndataframe = dataframe.drop(columns=['diagnosis', 'patient_id', 'anatom_site_general_challenge'])\ndataframe.to_csv('dataframe.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since the variables 'sex' and 'anatomical site' are categorical,they have to be changed to numbers, so as to be fed to the model. The following are the respective mappings for the categories and the numbers:","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(\"Anatom_sites:\")\nfor idx, anatom_site in enumerate(anatom_site_categories):\n    print(str(idx)+\": \"+anatom_site, end='\\n')\n\nprint(\"\\nSex:\")\nfor idx, sex in enumerate(sex_categories):\n    print(str(idx)+\": \"+sex, end='\\n')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">Now let's look at some EDA.</font>","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"dataframe.sample(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dataframe.benign_malignant.value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">\nFrom the above value counts, we infer that our dataset is heavily imbalanced. There are significantly more benign images than actual malignant images. <br/> <br/>\n    \nThis is what happens mostly, medically. Although cancer is truly something very serious and dangerous, our model wouldn't be able to find a proper distinction between a benign and malignant skin cancer image, because of the hugely imbalanced data. <br/>\n\n---\n    \nWhat I have experimented over here is class weights. It tries to magnify loss of the model when it classifies a image of a person with malignant melanoma as benign.<br/> \n    \nThere is also another technique calles clas oversampling. It samples the class whose datapoints are low in number, and tries to bring homogenity to the dataset by making copies of this under-represented data.\n    ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Some images with Melanoma","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"pos_values = dataframe['target'].values != 0\npos_dataframe = dataframe.iloc[pos_values, :]\n\nplt.figure(figsize=(20,8))\n\n\nfor i in range(24):\n    img = cv2.imread(pos_dataframe.iloc[i,4])\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    ax = plt.subplot(4,6,i+1)\n    plt.imshow(img)\n    plt.axis('off')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Some images without Melanoma","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"neg_dataframe =  dataframe.iloc[~pos_values, :]\n\nplt.figure(figsize=(20,8))\n\n\nfor i in range(24):\n    img = cv2.imread(neg_dataframe.iloc[i,4])\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    ax = plt.subplot(4,6,i+1)\n    plt.imshow(img)\n    plt.axis('off')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Age is an important factor in carciongenous growth, because it helps you to understand who is more vulnerable at an early age and who is more vulnerable at later stages of their life.","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.suptitle(\"Distribution of Melanoma patient skin images with respect to age \")\nplt.hist(pos_dataframe['age_approx'].values,bins=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.suptitle(\"Distribution of Non-Melanoma patient skin images with respect to age\")\nplt.hist(neg_dataframe['age_approx'].values,bins=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It is evident from the plot below that most of the images correspond to be taken at torso. And there are relativel much examples for lower body. It leads to the hypothesis that this skin cancer can b caused by exposure to sunlight, as our torso is exposed to the sun almost always, being vulnerable to the harmful UV rays which could have caused this type of cancer.","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.bar(dataframe['anatom_site'].value_counts().keys(),dataframe['anatom_site'].value_counts().values)\nplt.suptitle('Occurence of images of various body parts, of both targets')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From the block down below, data is being prepared so as to be fed to our accelerator(GPU), for efficient and fast experimentation and training. In the consecutice cells, TF-record files are being downloaded and decoded to form a tuple of (image, sex,age_approx, anatom_site) and target, which is also batched later.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"BATCH_SIZE = 10\nEPOCHS = 10\nvalidation_split = 0.15\n\nfiles_train = tf.io.gfile.glob(GCS_path+'/train*.tfrec')\nfiles_test = tf.io.gfile.glob(GCS_path+'/test*.tfrec')\n\nsplit = int(len(files_train) * validation_split)\nvalidation_filenames = files_train[:split]\ntraining_filenames = files_train[split:]\nprint(\"Pattern matches {} data files. Splitting dataset into {} training files and {} validation files\".format(len(files_train), len(training_filenames), len(validation_filenames)))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"@tf.function\ndef read_tfrecord(example):\n    features = {\n        'image'                        : tf.io.FixedLenFeature([], tf.string),\n        'image_name'                   : tf.io.FixedLenFeature([], tf.string),\n        'patient_id'                   : tf.io.FixedLenFeature([], tf.int64),\n        'sex'                          : tf.io.FixedLenFeature([], tf.int64),\n        'age_approx'                   : tf.io.FixedLenFeature([], tf.int64),\n        'anatom_site_general_challenge': tf.io.FixedLenFeature([], tf.int64),\n        'diagnosis'                    : tf.io.FixedLenFeature([], tf.int64),\n        'target'                       : tf.io.FixedLenFeature([], tf.int64)\n    }\n    example = tf.io.parse_single_example(example, features)\n    \n    image = tf.image.decode_jpeg(example[\"image\"], channels=3)\n    image = tf.cast(image, dtype= 'float32') / 255.0\n    image = tf.image.resize(image, [256,256])\n    \n    return (image, \n            example['sex'],\n            example['age_approx'],\n            example['anatom_site_general_challenge']\n           ), example['target']\n\n@tf.function\ndef augment(data, target):\n    img = data[0]\n    img = tf.image.random_brightness(img, 0.25)\n    img = tf.image.random_contrast(img, 0.5, 0.6)\n    img = tf.image.random_flip_left_right(img)\n    img = tf.image.random_flip_up_down(img)\n    img = tf.image.random_hue(img, 0.02)\n    # img = tf.image.random_jpeg_quality(img, 85, 100)\n    img = tf.cast(img, tf.float32)\n    \n    return (img, data[1], data[2], data[3]), target\n\n\ndef load_dataset(filenames):\n    # read from TFRecords\n    option_no_order = tf.data.Options()\n    option_no_order.experimental_deterministic = False\n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTOTUNE)\n    dataset = dataset.with_options(option_no_order)\n    dataset = dataset.map(read_tfrecord, num_parallel_calls=AUTOTUNE)\n    return dataset","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_batched_dataset(filenames, training=True):\n    dataset = load_dataset(filenames)\n    if training:\n        dataset = dataset.repeat()\n    dataset = dataset.shuffle(buffer_size= 100, reshuffle_each_iteration=True)\n    dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)\n    dataset = dataset.map(augment, num_parallel_calls=AUTOTUNE)\n    dataset = dataset.prefetch(AUTOTUNE)\n    return dataset\n\ntraining_dataset = load_batched_dataset(training_filenames)\nvalidation_dataset = load_batched_dataset(validation_filenames, training=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"neg, pos = np.bincount(dataframe['target'])\ntotal = len(dataframe)\nprint(\"negative (benign): \"+str(neg))\nprint(\"potitive (malignant): \"+str(pos))\nprint(\"total examples: \"+str(total))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is our Deep Learning model. It's is inspired by Andrej Karpathy's talk at Pytorch as Tesla, in which he introduces the idea of a hydranet. In that model, it is processing images as multiple heads of the network. But in our model, only one head is a CNN, and the other heads are for the other variables. \n\nThis CNN is an ensemble of the SOTA implementations in Computer Vision like DenseNet169, InceptionResNetV2, ResNet152V2, and Xception. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"input_1 = tf.keras.Input(shape=(256,256,3), name='image')\ninput_2 = tf.keras.Input(shape=(1), name='sex')\ninput_3 = tf.keras.Input(shape=(1), name='age_approx')\ninput_4 = tf.keras.Input(shape=(1), name='anatom_site')\n\ncnn1 = tf.keras.applications.densenet.DenseNet169(input_shape=(256,256,3), include_top=False, weights=None)(input_1)\ncnn1 = tf.keras.layers.Conv2D(filters=64, kernel_size=(1,1), activation='relu', use_bias=True)(cnn1)\ncnn1 = tf.keras.layers.Flatten()(cnn1)\ncnn1 = tf.keras.layers.Dense(2048, activation='relu', use_bias=False)(cnn1)\ncnn1 = tf.keras.layers.Dropout(0.4)(cnn1)\ncnn1 = tf.keras.layers.Dense(1024, activation='relu', use_bias=False)(cnn1)\ncnn1 = tf.keras.layers.Dropout(0.3)(cnn1)\ncnn1 = tf.keras.layers.Dense(1, activation='tanh', use_bias=True)(cnn1)\n\ncnn2 = tf.keras.applications.resnet_v2.ResNet152V2(input_shape=(256,256,3), include_top=False, weights=None)(input_1)\ncnn2 = tf.keras.layers.Conv2D(filters=64, kernel_size=(1,1), activation='relu', use_bias=True)(cnn2)\ncnn2 = tf.keras.layers.Flatten()(cnn2)\ncnn2 = tf.keras.layers.Dense(2048, activation='relu', use_bias=False)(cnn2)\ncnn2 = tf.keras.layers.Dropout(0.4)(cnn2)\ncnn2 = tf.keras.layers.Dense(1024, activation='relu', use_bias=False)(cnn2)\ncnn2 = tf.keras.layers.Dropout(0.3)(cnn2)\ncnn2 = tf.keras.layers.Dense(1, activation='tanh', use_bias=True)(cnn2)\n\ncnn3 = tf.keras.applications.inception_resnet_v2.InceptionResNetV2(input_shape=(256,256,3), include_top=False, weights=None)(input_1)\ncnn3 = tf.keras.layers.Conv2D(filters=64, kernel_size=(1,1), activation='relu', use_bias=True)(cnn3)\ncnn3 = tf.keras.layers.Flatten()(cnn3)\ncnn3 = tf.keras.layers.Dense(2048, activation='relu', use_bias=False)(cnn3)\ncnn3 = tf.keras.layers.Dropout(0.4)(cnn3)\ncnn3 = tf.keras.layers.Dense(1024, activation='relu', use_bias=False)(cnn3)\ncnn3 = tf.keras.layers.Dropout(0.3)(cnn3)\ncnn3 = tf.keras.layers.Dense(1, activation='tanh', use_bias=True)(cnn3)\n\ncnn4 = tf.keras.applications.xception.Xception(input_shape=(256,256,3), include_top=False, weights=None)(input_1)\ncnn4 = tf.keras.layers.Conv2D(filters=64, kernel_size=(1,1), activation='relu', use_bias=True)(cnn4)\ncnn4 = tf.keras.layers.Flatten()(cnn4)\ncnn4 = tf.keras.layers.Dense(2048, activation='relu', use_bias=False)(cnn4)\ncnn4 = tf.keras.layers.Dropout(0.4)(cnn4)\ncnn4 = tf.keras.layers.Dense(1024, activation='relu', use_bias=False)(cnn4)\ncnn4 = tf.keras.layers.Dropout(0.3)(cnn4)\ncnn4 = tf.keras.layers.Dense(1, activation='tanh', use_bias=True)(cnn4)\n\ncnn_output = tf.keras.layers.Concatenate(axis=-1)([cnn1, cnn2, cnn3, cnn4])\ncnn_output = tf.keras.layers.Dense(1, activation='relu', use_bias=True)(cnn_output)\n\nx = tf.keras.layers.Concatenate(axis=-1)([cnn_output, input_2, input_3, input_4])\nx = tf.keras.layers.Dense(10, activation='relu', use_bias=True)(x)\nx = tf.keras.layers.Dropout(0.5)(x)\nx = tf.keras.layers.Dense(10, activation='relu', use_bias=True)(x)\nx = tf.keras.layers.Dropout(0.5)(x)\nx = tf.keras.layers.Dense(1, activation='sigmoid', use_bias=True)(x)\n\nmodel = tf.keras.models.Model(inputs=[input_1,input_2,input_3,input_4], outputs=x)\n\nmodel.summary()\nMETRICS = [\n    tf.keras.metrics.TruePositives(name='tp'),\n    tf.keras.metrics.FalsePositives(name='fp'),\n    tf.keras.metrics.TrueNegatives(name='tn'),\n    tf.keras.metrics.FalseNegatives(name='fn'), \n    tf.keras.metrics.BinaryAccuracy(name='accuracy'),\n    tf.keras.metrics.Precision(name='precision'),\n    tf.keras.metrics.Recall(name='recall'),\n    tf.keras.metrics.AUC(name='auc'),\n]\nmodel.compile(optimizer='adam',\n              loss=tf.keras.losses.BinaryCrossentropy(),\n              metrics=METRICS)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weight_for_0 = (1 / neg)*(total)/2.0 \nweight_for_1 =(1 / pos)*(total)/2.0\n\nclass_weight = {0: weight_for_0, 1: weight_for_1}\n\nprint('Weight for class 0: {:.2f}'.format(weight_for_0))\nprint('Weight for class 1: {:.2f}'.format(weight_for_1))\nSTEPS = np.ceil(total/10)\n\nmodel.fit(training_dataset, \n          epochs=1, \n          validation_steps=5,\n          validation_data = validation_dataset,\n          # class_weight=class_weight,\n          steps_per_epoch=STEPS)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The accuracy of this model might be pretty high, but technically, what the model is learning is to say that the cancer in image is benign. It is a very common situation whenthe data is so highly imbalanced. \n\nAnother problem I encountered with this type of model training is that the ram get progressivvely filled up, and after around 2.5 epochs, it just goes out of memory, even though the dataset isnt being cached. And even if it is, the tf-records I am using are only 700mb at max. \n\nIf you, the reader of this notebook, find or have an opinion on how the performance can be improved, do comment and let me know. \n\nThank you!! ","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}