{"cells":[{"metadata":{},"cell_type":"markdown","source":"![SIIM-ISIC Melanoma](https://siim.org/resource/resmgr/SIIM_logo-600x315.png)\n\n## Introduction\nAs the leading healthcare organization for informatics in medical imaging, the **Society for Imaging Informatics in Medicine (SIIM)**'s mission is to advance medical imaging informatics through education, research, and innovation in a multi-disciplinary community. SIIM is joined by the International Skin Imaging Collaboration (ISIC), an international effort to improve melanoma diagnosis. The ISIC Archive contains the largest publicly available collection of quality-controlled dermoscopic images of skin lesions.\n\n### What is Melanoma?\nMelanoma is a type of skin cancer that develops when melanocytes (the cells that give the skin its tan or brown color) start to grow out of control.\n\nCancer starts when cells in the body begin to grow out of control. Cells in nearly any part of the body can become cancer, and can then spread to other areas of the body. To learn more about cancer and how it starts and spreads, see What Is Cancer?\n\nMelanoma is much less common than some other types of skin cancers. But melanoma is more dangerous because it’s much more likely to spread to other parts of the body if not caught and treated early.\n![](https://www.cancer.org/cancer/melanoma-skin-cancer/about/what-is-melanoma/_jcr_content/par/textimage/image.img.jpg/1568208521537.jpg)\n\n### Signs and Symptoms of Melanoma\nThe most important warning sign of melanoma is a new spot on the skin or a spot that is changing in size, shape, or color.\n\nAnother important sign is a spot that looks different from all of the other spots on your skin (known as the *ugly duckling sign*).\n\nIf you have one of these warning signs, have your skin checked by a doctor.\n\nThe **ABCDE** rule is another guide to the usual signs of melanoma. Be on the lookout and tell your doctor about spots that have any of the following features:\n\n* A is for Asymmetry: One half of a mole or birthmark does not match the other.\n* B is for Border: The edges are irregular, ragged, notched, or blurred.\n* C is for Color: The color is not the same all over and may include different shades of brown or black, or sometimes with patches of pink, red, white, or blue.\n* D is for Diameter: The spot is larger than 6 millimeters across (about ¼ inch – the size of a pencil eraser), although melanomas can sometimes be smaller than this.\n* E is for Evolving: The mole is changing in size, shape, or color.\n\nsource: [cancer.org](https://www.cancer.org/cancer/melanoma-skin-cancer/)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# The Competition\n\n### Objective\nThe objective of this competition is to identify melanoma in images of skin lesions. The data given consists of not only the images, but also metadata of every patient from which the images have been acquired. Hence, it is a problem of **Binary Classification**, where the two target labels refer to **0 - Benign**, **1 - Malignant**.\n\n### Evaluation\nOur submissions will be evaluated using **area under the ROC curve**. An ROC curve (receiver operating characteristic curve) is a graph showing the performance of a classification model at all classification thresholds. This curve plots two parameters:\n![AUC-ROC](https://imgur.com/yNeAG4M.png)\nAn ROC curve plots TPR vs. FPR at different classification thresholds. Lowering the classification threshold classifies more items as positive, thus increasing both False Positives and True Positives. The following figure shows a typical ROC curve.\n<img src=\"https://developers.google.com/machine-learning/crash-course/images/ROCCurve.svg\" width=\"400\">\nAUC provides an aggregate measure of performance across all possible classification thresholds. One way of interpreting AUC is as the probability that the model ranks a random positive example more highly than a random negative example.\n\nFor more information, feel free to check out this [tutorial](https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc) by Google.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 1. Importing necessary libraries\n\nWe begin this EDA by importing the necessary libraries. If you're wondering what is `colored`, it is just a python package for printing out formatted string using Python.","execution_count":null},{"metadata":{"trusted":true,"collapsed":true,"_kg_hide-output":true},"cell_type":"code","source":"!pip install colored --upgrade","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom colored import fg, bg, attr\n\nsns.set_style(\"whitegrid\")\nsns.set_palette(\"pastel\")\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. Loading data","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"basepath = \"../input/siim-isic-melanoma-classification/\"\ntrain_df = pd.read_csv(basepath + \"train.csv\")\ntest_df = pd.read_csv(basepath + \"test.csv\")\n\nprint(\"Train df shape: %s%s%s%s\" % (attr(1), fg(156), str(train_df.shape), attr(0)))\ndisplay(train_df.head())\nprint(\"Test df shape: %s%s%s%s\" % (attr(1), fg(156), str(test_df.shape), attr(0)))\ndisplay(test_df.head())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 3. Exploration","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Column information","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"print(\"%s%s ====== train_df INFO ====== %s\" % (attr(1), fg(156), attr(0)))\nprint(train_df.info())\nprint(\"%s%s ====== test_df INFO ====== %s\" % (attr(1), fg(156), attr(0)))\nprint(test_df.info())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":false},"cell_type":"code","source":"missing_train = train_df.isnull().sum()\nmissing_test = test_df.isnull().sum()\n\nprint(\"%s%sMissing values in %straining data: %s\" % (attr(1), fg(197), fg(156), attr(0)))\nprint(missing_train[missing_train > 0].sort_values(ascending=False))\n\nprint(\"%s%sMissing values in %stest data: %s\" % (attr(1), fg(197), fg(156), attr(0)))\nprint(missing_test[missing_test > 0].sort_values(ascending=False))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We observe that there are missing values in both training data and test data. We will have to deal with this later for modelling purposes if we wish to use the metadata to make predictions.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Unique patients","execution_count":null},{"metadata":{"_kg_hide-input":false,"_kg_hide-output":false,"trusted":true},"cell_type":"code","source":"print(\"%sNo. of patients in training data: %s%s%s\" % (attr(1), fg(45), train_df.patient_id.nunique(), attr(0)))\nprint(\"%sNo. of patients in testing data: %s%s%s\" % (attr(1), fg(45), test_df.patient_id.nunique(), attr(0)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualizations","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Gender Distribution","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize=(19,6))\n\nsns.countplot(train_df.sex, ax=ax[0], palette=['dodgerblue','lightpink'], alpha=0.7)\nax[0].set_title(\"Training data\")\n\nsns.countplot(test_df.sex, ax=ax[1], palette=['dodgerblue', 'lightpink'], alpha=0.7)\nax[1].set_title(\"Test data\")\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The gender distribution seems to be more balanced in the training data. Number of males is higher is both datasets, but the test data has noticeably higher number of males.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### No. of images per patient","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_patient_count = train_df.groupby('patient_id').image_name.count()\ntest_patient_count = test_df.groupby('patient_id').image_name.count()\n\nfig, ax = plt.subplots(2,2,figsize=(19,11))\n\nsns.distplot(train_patient_count, bins=30, ax=ax[0,0], color=\"mediumspringgreen\", kde=False)\nax[0,0].set_title(\"Training Data\")\nax[0,0].set_xlabel(\"\")\nax[0,0].set_ylabel(\"Frequency\")\n\nsns.distplot(test_patient_count, bins=30, ax=ax[0,1], color='mediumorchid', kde=False)\nax[0,1].set_title(\"Test Data\")\nax[0,1].set_xlabel(\"\")\nax[0,1].set_ylabel(\"Frequency\")\n\nsns.boxplot(train_patient_count, ax=ax[1,0], color=\"mediumspringgreen\")\nax[1,0].set_xlabel(\"No. of images\")\n\nsns.boxplot(test_patient_count, ax=ax[1,1], color='mediumorchid')\nax[1,1].set_xlabel(\"No. of images\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We observe that there are multiple images for every patient. Most of them have around 0-20 images, but there are patients with upto approximately **120** images in the training data and **250** images in the test data.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Patient Age Distribution","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_patient_ages = list(train_df.groupby(\"patient_id\").age_approx.unique())\ntrain_mean_patient_ages = [np.mean(ages) for ages in train_patient_ages]\n\ntest_patient_ages = list(test_df.groupby(\"patient_id\").age_approx.unique())\ntest_mean_patient_ages = [np.mean(ages) for ages in test_patient_ages]\n\nfig, ax = plt.subplots(1,2, figsize=(19,6))\n\nsns.distplot(train_mean_patient_ages, bins=15, ax=ax[0], color='mediumspringgreen')\nax[0].set_title(\"Age Distribution in Training Data\")\nax[0].set_xlabel(\"Mean age\")\nax[0].set_ylabel(\"Frequency\")\n\nsns.distplot(test_mean_patient_ages, bins=15, ax=ax[1], color='mediumorchid')\nax[1].set_title(\"Age Distribution in Test Data\")\nax[1].set_xlabel(\"Mean age\")\nax[1].set_ylabel(\"Frequency\")\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We observe that patients might have multiple ages recorded in the metadata, probably because of varying sources of the images. Hence, I decided to take the mean of the recorded ages of each patient and plot a histogram.\n\nBoth training data and test data seems to have a similar bell-shaped distribution of age, with most of the patients being around the age of **40-60 years**. Though there is a slightly higher number of older patients in the test data. This sort of imbalance might be important to consider while modelling.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Location Distribution","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize=(19,6))\n\ntrain_locations = train_df.anatom_site_general_challenge.value_counts().sort_values(ascending=False)\ntest_locations = test_df.anatom_site_general_challenge.value_counts().sort_values(ascending=False)\n\nsns.barplot(x=train_locations.index.values, y=train_locations.values, ax=ax[0], color=\"mediumspringgreen\", alpha=0.5);\nax[0].set_xlabel(\"\");\nlabels = ax[0].get_xticklabels();\nax[0].set_xticklabels(labels, rotation=45);\nax[0].set_title(\"Image locations in Training data\");\n\nsns.barplot(x=test_locations.index.values, y=test_locations.values, ax=ax[1], color=\"mediumorchid\", alpha=0.5);\nax[1].set_xlabel(\"\");\nlabels = ax[1].get_xticklabels();\nax[1].set_xticklabels(labels, rotation=45);\nax[1].set_title(\"Image locations in Test data\");\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Both training and test data have similar distribution of images for each location, with the **Torso** region having the most number of images, followed by **Lower Extremity**.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Diagnosis Distribution","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train_diagnosis = train_df.diagnosis.value_counts().sort_values(ascending=False)\n\nfig, ax = plt.subplots(1,2, figsize=(19,6))\n\nsns.barplot(x=train_diagnosis.index.values, y=train_diagnosis.values, ax=ax[0], color=\"mediumspringgreen\", alpha=0.5);\nax[0].set_xlabel(\"\");\nlabels = ax[0].get_xticklabels();\nax[0].set_xticklabels(labels, rotation=45);\nax[0].set_title(\"Diagnoses' in Training data\");\n\nsns.countplot(train_df.benign_malignant, ax=ax[1], palette=['tomato','red'], alpha=0.7)\nax[1].set_title(\"Type of cancer in Training data\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"benign = train_df.groupby('benign_malignant').image_name.count()['benign'] / train_df.shape[0] * 100\nmalignant = train_df.groupby('benign_malignant').image_name.count()['malignant'] / train_df.shape[0] * 100\n\nprint(\"%sTarget Distribution in Training data%s\" % (attr(1), attr(0)))\nprint(\"%s%sBenign: %s%s%%%s\" % (attr(1), fg(197), fg(156), np.round(benign, 2), attr(0)))\nprint(\"%s%sMalignant: %s%s%%%s\" % (attr(1), fg(197), fg(156), np.round(malignant, 2), attr(0)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We observe that most of the diagnoses are **unknown** and there is a very high imbalance in the target variable, with around 98% of the data being labeled as **benign**. This imbalance will play a key role in how your model makes predictions, hence we have to be smart dealing with such a situation.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Age/Gender Distribution","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize=(19,6))\n\nsns.countplot(x=train_df.age_approx, hue=train_df.sex, palette='Greens', ax=ax[0], alpha=0.7)\nax[0].set_title(\"Training data\")\n\nsns.countplot(x=test_df.age_approx, hue=test_df.sex, palette='Purples', ax=ax[1], alpha=0.7)\nax[1].set_title(\"Test_data\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"These graphs are pretty interesting actually, so iin the training data, there are **more females affected** at younger ages of around **25-40 years**, whereas in the test data, most of the age groups have **more males affected** with a few exceptions.\n\nBut what's most interesting is that, in both the datasets, at the age group of **70 years**, the number of males affected is **significantly higher** than females. This sort of pattern should be considered while modelling so as our model does not overfit or so.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Correlations with Age","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize=(19,6))\n\nsns.boxplot(y=train_df.age_approx, x=train_df.diagnosis, ax=ax[1], palette='PuRd', boxprops=dict(alpha=.7))\nax[1].set_title(\"Age/Diagnosis in Training data\")\nax[1].set_xlabel(\"\");\nlabels = ax[1].get_xticklabels();\nax[1].set_xticklabels(labels, rotation=45);\n\nsns.boxplot(y=train_df.age_approx, x=train_df.anatom_site_general_challenge, ax=ax[0], palette='GnBu', boxprops=dict(alpha=.7))\nax[0].set_title(\"Age/Location in Test data\")\nax[0].set_xlabel(\"\");\nlabels = ax[0].get_xticklabels();\nax[0].set_xticklabels(labels, rotation=45);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The distribution of locations wrt to age seems to almost similar for every location, whereas the distribution of diagnoses wrt age seems to be pretty random. ***Solar lentigo and Lichenoid keratosis*** seems to occur more at older ages whereas ***Nevus*** seems to occur more at younger ages.\n\nThe unknown class seems to be more uniformly distributed.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Gender/Diagnosis Distribution","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"female_diagnosis = list(train_df.groupby(['sex','diagnosis']).image_name.count()[:7])\nz = 0\nfemale_diagnosis = [z]*2 + female_diagnosis\n\nmale_diagnosis = list(train_df.groupby(['sex', 'diagnosis']).image_name.count()[7:])\n\ndiagnosis = ['atypical melanocytic proliferation', 'cafe-au-lait macule', 'lentigo NOS', 'lichenoid keratosis', 'melanoma', 'nevus', 'seborrheic keratosis', 'solar lentigo', 'unknown']\n\nassert train_df.sex.value_counts()['male'] == np.sum(male_diagnosis)\nassert train_df.sex.value_counts()['female'] == np.sum(female_diagnosis)\n\nmale_diagnosis = (male_diagnosis / np.sum(male_diagnosis)) * 100\nfemale_diagnosis = (female_diagnosis / np.sum(female_diagnosis)) * 100\n\nprint(\"%s%sMale Diagnosis Statistics%s\" % (attr(1), fg(45), attr(0)))\nfor i in range(len(diagnosis)):\n    print(\"%s%s=> \" % (attr(1), fg(14)) + diagnosis[i] + \": %s\" % (fg(156)) + str(np.round(male_diagnosis[i],2)) + \"%%%s\" % (attr(0)))\n    \nprint(\"%s%s=============================================%s\" % (attr(1), fg(250), attr(0)))\n\nprint(\"%s%sFemale Diagnosis Statistics%s\" % (attr(1), fg(213), attr(0)))\nfor i in range(len(diagnosis)):\n    print(\"%s%s=> \" % (attr(1), fg(205)) + diagnosis[i] + \": %s\" % (fg(156)) + str(np.round(female_diagnosis[i],2)) + \"%%%s\" % (attr(0)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Instead of a graph, I decided to calculate the percentage of males and females having each of the diagnoses, giving a better understanding if any gender is more likely to be diagnosed with one of the diagnoses.\n\nMost of the data is similar for both genders, but there are no cases of **atypical melanocytic proliferation and cafe-au-lait macule** in females, atleast in the training data.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Conclusion\n\n* The more important observations from the exploration would be the **high imbalance of target variable**.\n* The fact that there are multiple images for every patient should be taken into consideration with the above point for validation of your model. Your model should generalize well enough.\n* The imbalance in gender distributions at the age of **70 years** is also a worthwhile observation.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 4. Modelling\n\nI'll be extending this notebook to demonstrate how to get started with building a **Convolutional Neural Network with PyTorch** very soon.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"> Please feel free to let me know in the **comments section** what you think about this notebook, if I've made a mistake somewhere and also how I could make my notebooks better. If you liked this notebook, please **upvote** it! I'm only getting started with Kaggle, and it would mean a lot if you guys did so. Thank you again!","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}