{"cells":[{"metadata":{},"cell_type":"markdown","source":"**Overview**  \nImagine being able to detect blindness before it happened.\n\nMillions of people suffer from diabetic retinopathy, the leading cause of blindness among working aged adults. Aravind Eye Hospital in India hopes to detect and prevent this disease among people living in rural areas where medical screening is difficult to conduct. Successful entries in this competition will improve the hospital’s ability to identify potential patients. Further, the solutions will be spread to other Ophthalmologists through the 4th Asia Pacific Tele-Ophthalmology Society (APTOS) Symposium.\n\nCurrently, Aravind technicians travel to these rural areas to capture images and then rely on highly trained doctors to review the images and provide diagnosis. Their goal is to scale their efforts through technology; to gain the ability to automatically screen images for disease and provide information on how severe the condition may be.\n\nIn this synchronous Kernels-only competition, we need to build a machine learning model to speed up disease detection. we will be working with thousands of images collected in rural areas to help identify diabetic retinopathy automatically. If successful, we will not only help to prevent lifelong blindness, but these models may be used to detect other sorts of diseases in the future, like glaucoma and macular degeneration.  \n\nCompetition Page: [Kaggle - Blindness Detection](https://www.kaggle.com/c/aptos2019-blindness-detection/overview)"},{"metadata":{},"cell_type":"markdown","source":"**About the Dataset**  \nWe will be provided with a large set of retina images taken using fundus photography under a variety of imaging conditions.\n\nA clinician has rated each image for the severity of diabetic retinopathy on a scale of 0 to 4:  \n\n* 0 - No DR  \n* 1 - Mild  \n* 2 - Moderate  \n* 3 - Severe  \n* 4 - Proliferative DR  \n\nLike any real-world data set, we will encounter noise in both the images and labels. Images may contain artifacts, be out of focus, underexposed, or overexposed. The images were gathered from multiple clinics using a variety of cameras over an extended period of time, which will introduce further variation.  "},{"metadata":{},"cell_type":"markdown","source":"**Files for Analysis & Prediction**"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nsns.set()\n\nimport os\nprint(os.listdir(\"../input\"))\n\nimport warnings\nwarnings.filterwarnings(\"ignore\", category=DeprecationWarning)\nwarnings.filterwarnings(\"ignore\", category=UserWarning)\nwarnings.filterwarnings(\"ignore\", category=FutureWarning)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Training Sample**"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_labels = pd.read_csv(\"../input/train.csv\")\nprint(train_labels.head())\n\nprint(\"There are {0} samples in the Training dataset\".format(train_labels.shape[0]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Testing Dataset**"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"test_labels = pd.read_csv(\"../input/test.csv\")\nprint(\"We need to predict {0} patients as what severity of diabetic retinopathy they have\".format(test_labels.shape[0]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from pandasql import sqldf\npysqldf = lambda q: sqldf(q, globals())\n\ndiag_q = \"\"\"\nselect diagnosis, count(distinct id_code) as cnt\nFrom train_labels\nGROUP BY diagnosis;\n\"\"\"\n\ndiag_df = pysqldf(diag_q)\n\nimport plotly.plotly as py\nimport plotly.graph_objs as go\nfrom plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot\ninit_notebook_mode(connected=True)\n\nfig = {\n  \"data\": [\n    {\n      \"values\": diag_df.cnt,\n      \"labels\": diag_df.diagnosis,\n      \"domain\": {\"x\": [0, .5]},\n      \"hoverinfo\":\"label+percent\",\n      \"hole\": .2,\n      \"type\": \"pie\"\n    },],\n \"layout\": {\n        \"title\":\"Severity Proportion of Diabetic Retinopathy\"\n    }\n}\n\niplot(fig)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Severity Proportion of Diabetic Retinopathy Summary\n* 0 - No DR - 49.3%\n* 2 - Moderate - 27.3%\n* 1 - Mild - 10.1%\n* 4 - Proliferative DR - 8.06%\n* 3 - Severe - 5.27%"},{"metadata":{},"cell_type":"markdown","source":"**Displaying sample original image without resizing**"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from IPython.display import Image\nfrom IPython.display import display\nim_0 = Image(filename ='../input/train_images/002c21358ce6.png') \nim_1 = Image(filename ='../input/train_images/0024cdab0c1e.png')\nim_2 = Image(filename ='../input/train_images/000c1434d8d7.png')\nim_3 = Image(filename ='../input/train_images/0104b032c141.png')\nim_4 = Image(filename ='../input/train_images/02685f13cefd.png')\ndisplay(im_0, im_1, im_2, im_3, im_4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Dividing the dataset based on Severity of Diabetic Retinopathy**"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"DATA_PATH = '../input/'\nTRAIN_IMG_PATH = os.path.join(DATA_PATH, 'train_images')\nTEST_IMG_PATH = os.path.join(DATA_PATH, 'test_images')\nTRAIN_LABEL_PATH = os.path.join(DATA_PATH, 'train.csv')\nTEST_LABEL_PATH = os.path.join(DATA_PATH, 'test.csv')\n\ntrain_df = pd.read_csv(TRAIN_LABEL_PATH)\ntest_df = pd.read_csv(TEST_LABEL_PATH)\n\ntrain_labels_0 = train_df[train_df.diagnosis == 0].reset_index()\ntrain_labels_1 = train_df[train_df.diagnosis == 1].reset_index()\ntrain_labels_2 = train_df[train_df.diagnosis == 2].reset_index()\ntrain_labels_3 = train_df[train_df.diagnosis == 3].reset_index()\ntrain_labels_4 = train_df[train_df.diagnosis == 4].reset_index()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"%matplotlib inline\nfrom PIL import Image\nimport matplotlib.pyplot as plt\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#0 - No DR\nfor i in range(5):\n    img_path = TRAIN_IMG_PATH+'/'+train_labels_0['id_code'][i]+'.png'\n    img = Image.open(img_path)\n    img.thumbnail((200,200))\n    ax[i].imshow(img)\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 0 - No DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from PIL import Image\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n#1 - Mild DR\nfor i in range(5):\n    img_path = TRAIN_IMG_PATH+'/'+train_labels_1['id_code'][i]+'.png'\n    img = Image.open(img_path)\n    img.thumbnail((200,200))\n    ax[i].imshow(img)\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 1 - Mild DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from PIL import Image\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#2 - Moderate DR\nfor i in range(5):\n    img_path = TRAIN_IMG_PATH+'/'+train_labels_2['id_code'][i]+'.png'\n    img = Image.open(img_path)\n    img.thumbnail((200,200))\n    ax[i].imshow(img)\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 2 - Moderate DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from PIL import Image\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#3 - Severe DR\nfor i in range(5):\n    img_path = TRAIN_IMG_PATH+'/'+train_labels_3['id_code'][i]+'.png'\n    img = Image.open(img_path)\n    img.thumbnail((200,200))\n    ax[i].imshow(img)\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 3 - Severe DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from PIL import Image\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#4 - Proliferative DR\nfor i in range(5):\n    img_path = TRAIN_IMG_PATH+'/'+train_labels_4['id_code'][i]+'.png'\n    img = Image.open(img_path)\n    img.thumbnail((200,200))\n    plt.title(train_labels_4['id_code'][i])\n    ax[i].title.set_text(train_labels_4['id_code'][i])\n    ax[i].imshow(img)\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 4 - Proliferative DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Applying 'jet' on Diabetic Retinopathy images**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#0 - No DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_0['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"jet\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 0 - No DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#1 - Mild DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_1['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"jet\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 1 - Mild DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#2 - Moderate DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_2['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"jet\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 2 - Moderate DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#3 - Severe DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_3['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"jet\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 3 - Severe DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#4 - Proliferative DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_4['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"jet\")\n    ax[i].set_axis_off()  \nprint(\"Diabetic Retinopathy of Severity 4 - Proliferative DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Applying 'PiYG' on Diabetic Retinopathy images**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#0 - No DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_0['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"PiYG\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 0 - No DR\")\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#1 - Mild DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_1['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"PiYG\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 1 - Mild DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#2 - Moderate DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_2['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"PiYG\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 2 - Moderate DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#3 - Severe DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_3['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"PiYG\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 3 - Severe DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#4 - Proliferative DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_4['id_code'][i]+'.png',0)\n    edges = cv2.Canny(img,100,200)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"PiYG\")\n    ax[i].set_axis_off()  \nprint(\"Diabetic Retinopathy of Severity 4 - Proliferative DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Applying 'gray' on Diabetic Retinopathy images**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#0 - No DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_0['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"gray\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 0 - No DR\")\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#1 - Mild DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_1['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"gray\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 1 - Mild DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#2 - Moderate DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_2['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"gray\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 2 - Moderate DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#3 - Severe DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_3['id_code'][i]+'.png',0)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"gray\")\n    ax[i].set_axis_off() \nprint(\"Diabetic Retinopathy of Severity 3 - Severe DR\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import cv2\nf,ax = plt.subplots(1,5, figsize=(15,15))\nplt.rcParams[\"axes.grid\"] = False\n\n#4 - Proliferative DR\nfor i in range(5):\n    img = cv2.imread(TRAIN_IMG_PATH+'/'+train_labels_4['id_code'][i]+'.png',0)\n    edges = cv2.Canny(img,100,200)\n    plt.imshow(img)\n    ax[i].imshow(img, cmap=\"gray\")\n    ax[i].set_axis_off()  \nprint(\"Diabetic Retinopathy of Severity 4 - Proliferative DR\")\nplt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}