{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<h1><center>APTOS 2019 Blindness Detection</center></h1>\n<h2><center>Detect diabetic retinopathy to stop blindness before it's too late</center></h2>\n<center><img src=\"https://raw.githubusercontent.com/dimitreOliveira/MachineLearning/master/Kaggle/APTOS%202019%20Blindness%20Detection/aux_img.png\"></center>\n\nIn this synchronous Kernels-only competition, you'll build a machine learning model to speed up disease detection. You’ll work with thousands of images collected in rural areas to help identify diabetic retinopathy automatically. If successful, you will not only help to prevent lifelong blindness, but these models may be used to detect other sorts of diseases in the future, like glaucoma and macular degeneration.\n\nIn this notebook, I will be using basic deep learning and transfer learning (ResNet50) to create a baseline.\n##### Image source: http://cceyemd.com/diabetes-and-eye-exams/\n\n### Dependencies","metadata":{}},{"cell_type":"code","source":"import os\nimport cv2\nimport random\nimport warnings\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import confusion_matrix, cohen_kappa_score\nfrom keras.models import Model, Sequential\nfrom keras import optimizers, applications\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.callbacks import EarlyStopping, ReduceLROnPlateau\nfrom keras.layers import Dense, Dropout, GlobalAveragePooling2D, Input, MaxPooling2D, Flatten\nfrom keras.layers.convolutional import Conv2D\n\n# Set seeds to make the experiment more reproducible.\nfrom tensorflow import set_random_seed\ndef seed_everything(seed=0):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    set_random_seed(0)\nseed_everything()\n\n%matplotlib inline\nsns.set(style=\"whitegrid\")\nwarnings.filterwarnings(\"ignore\")","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-05T13:42:16.988637Z","iopub.execute_input":"2023-05-05T13:42:16.988982Z","iopub.status.idle":"2023-05-05T13:42:19.581632Z","shell.execute_reply.started":"2023-05-05T13:42:16.988925Z","shell.execute_reply":"2023-05-05T13:42:19.580813Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load data","metadata":{"_kg_hide-output":true}},{"cell_type":"code","source":"train = pd.read_csv('../input/aptos2019-blindness-detection/train.csv')\ntest = pd.read_csv('../input/aptos2019-blindness-detection/test.csv')","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_kg_hide-input":false,"execution":{"iopub.status.busy":"2023-05-05T13:42:21.950419Z","iopub.execute_input":"2023-05-05T13:42:21.950738Z","iopub.status.idle":"2023-05-05T13:42:21.978084Z","shell.execute_reply.started":"2023-05-05T13:42:21.950682Z","shell.execute_reply":"2023-05-05T13:42:21.977265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# EDA\n\n## Data overview","metadata":{}},{"cell_type":"code","source":"print('Number of train samples: ', train.shape[0])\nprint('Number of test samples: ', test.shape[0])\ndisplay(train.head())","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-05T13:42:24.646021Z","iopub.execute_input":"2023-05-05T13:42:24.646367Z","iopub.status.idle":"2023-05-05T13:42:24.670003Z","shell.execute_reply.started":"2023-05-05T13:42:24.646305Z","shell.execute_reply":"2023-05-05T13:42:24.668840Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Label class distribution\n\nAs we can see we have an unbalanced database, we have two times more class 0 than 2, and classes 1, 2 and 4 each have less than half of the class 2 data.","metadata":{}},{"cell_type":"code","source":"f, ax = plt.subplots(figsize=(14, 8.7))\nax = sns.countplot(x=\"diagnosis\", data=train, palette=\"GnBu_d\")\nsns.despine()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-05T13:42:26.839329Z","iopub.execute_input":"2023-05-05T13:42:26.839645Z","iopub.status.idle":"2023-05-05T13:42:27.135303Z","shell.execute_reply.started":"2023-05-05T13:42:26.839590Z","shell.execute_reply":"2023-05-05T13:42:27.134165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Legend\n- 0 - No DR\n- 1 - Mild\n- 2 - Moderate\n- 3 - Severe\n- 4 - Proliferative DR ","metadata":{}},{"cell_type":"markdown","source":"### Now let's see some of the images\n\nThe images have different sizes, they may need resizing or some padding.","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"white\")\ncount = 1\nplt.figure(figsize=[20, 20])\nfor img_name in train['id_code'][:15]:\n    img = cv2.imread(\"../input/aptos2019-blindness-detection/train_images/%s.png\" % img_name)[...,[2, 1, 0]]\n    plt.subplot(5, 5, count)\n    plt.imshow(img)\n    plt.title(\"Image %s\" % count)\n    count += 1\n    \nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-05T13:42:30.794702Z","iopub.execute_input":"2023-05-05T13:42:30.795028Z","iopub.status.idle":"2023-05-05T13:42:39.977477Z","shell.execute_reply.started":"2023-05-05T13:42:30.794970Z","shell.execute_reply":"2023-05-05T13:42:39.976599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model parameters","metadata":{}},{"cell_type":"code","source":"# Model parameters\nBATCH_SIZE = 8\nEPOCHS = 15\nWARMUP_EPOCHS = 2\nLEARNING_RATE = 1e-4\nWARMUP_LEARNING_RATE = 1e-3\nHEIGHT = 128\nWIDTH = 128\nCANAL = 3\nN_CLASSES = train['diagnosis'].nunique()\nES_PATIENCE = 5\nRLROP_PATIENCE = 3\nDECAY_DROP = 0.5","metadata":{"execution":{"iopub.status.busy":"2023-05-05T13:42:44.113019Z","iopub.execute_input":"2023-05-05T13:42:44.113384Z","iopub.status.idle":"2023-05-05T13:42:44.120468Z","shell.execute_reply.started":"2023-05-05T13:42:44.113295Z","shell.execute_reply":"2023-05-05T13:42:44.119526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Preprocecss data\ntrain[\"id_code\"] = train[\"id_code\"].apply(lambda x: x + \".png\")\ntest[\"id_code\"] = test[\"id_code\"].apply(lambda x: x + \".png\")\ntrain['diagnosis'] = train['diagnosis'].astype('str')\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-05T13:42:46.358373Z","iopub.execute_input":"2023-05-05T13:42:46.358695Z","iopub.status.idle":"2023-05-05T13:42:46.378208Z","shell.execute_reply.started":"2023-05-05T13:42:46.358640Z","shell.execute_reply":"2023-05-05T13:42:46.377344Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# preprocessing","metadata":{}},{"cell_type":"code","source":"def crop_image_from_gray(img,tol=7):\n    if img.ndim ==2:\n        mask = img>tol\n        return img[np.ix_(mask.any(1),mask.any(0))]\n    elif img.ndim==3:\n        gray_img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)\n        mask = gray_img>tol\n        \n        check_shape = img[:,:,0][np.ix_(mask.any(1),mask.any(0))].shape[0]\n        if (check_shape == 0): # image is too dark so that we crop out everything,\n            return img # return original image\n        else:\n            img1=img[:,:,0][np.ix_(mask.any(1),mask.any(0))]\n            img2=img[:,:,1][np.ix_(mask.any(1),mask.any(0))]\n            img3=img[:,:,2][np.ix_(mask.any(1),mask.any(0))]\n    #         print(img1.shape,img2.shape,img3.shape)\n            img = np.stack([img1,img2,img3],axis=-1)\n    #         print(img.shape)\n        return img\n\ndef preprocess_image(image, sigmaX=10):\n#     image = cv2.imread(path)\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    image = crop_image_from_gray(image)\n    image = cv2.resize(image, (HEIGHT, WIDTH))\n    image=cv2.addWeighted ( image,4, cv2.GaussianBlur( image , (0,0) , sigmaX) ,-4 ,128)\n        \n    return image","metadata":{"execution":{"iopub.status.busy":"2023-05-05T13:42:48.821288Z","iopub.execute_input":"2023-05-05T13:42:48.821647Z","iopub.status.idle":"2023-05-05T13:42:48.833414Z","shell.execute_reply.started":"2023-05-05T13:42:48.821585Z","shell.execute_reply":"2023-05-05T13:42:48.832045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## images after preprocessing and feature selection","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 5, figsize=(15, 6))\nfor i in range(5):\n    sample = train[train['diagnosis'] == str(i)].sample(1)\n    image_name = sample['id_code'].item()\n    X = preprocess_image(cv2.imread(f\"/kaggle/input/aptos2019-blindness-detection/train_images/{ image_name }\"))\n    ax[i].set_title(f\"Image: { image_name }\\n Label = { sample['diagnosis'].item() }\", \n                    weight = 'bold', fontsize = 10)\n    ax[i].axis('off')\n    ax[i].imshow(X);","metadata":{"execution":{"iopub.status.busy":"2023-05-05T13:42:52.481620Z","iopub.execute_input":"2023-05-05T13:42:52.481957Z","iopub.status.idle":"2023-05-05T13:42:55.075115Z","shell.execute_reply.started":"2023-05-05T13:42:52.481897Z","shell.execute_reply":"2023-05-05T13:42:55.074326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data generator","metadata":{}},{"cell_type":"code","source":"train_datagen=ImageDataGenerator(rescale=1./255, \n                                 validation_split=0.2,\n                                 horizontal_flip=True)\n\ntrain_generator=train_datagen.flow_from_dataframe(\n    dataframe=train,\n    directory=\"../input/aptos2019-blindness-detection/train_images/\",\n    x_col=\"id_code\",\n    y_col=\"diagnosis\",\n    batch_size=BATCH_SIZE,\n    class_mode=\"categorical\",\n    target_size=(HEIGHT, WIDTH),\n    subset='training')\n\nvalid_generator=train_datagen.flow_from_dataframe(\n    dataframe=train,\n    directory=\"../input/aptos2019-blindness-detection/train_images/\",\n    x_col=\"id_code\",\n    y_col=\"diagnosis\",\n    batch_size=BATCH_SIZE,\n    class_mode=\"categorical\",    \n    target_size=(HEIGHT, WIDTH),\n    subset='validation')\n\ntest_datagen = ImageDataGenerator(rescale=1./255)\n\ntest_generator = test_datagen.flow_from_dataframe(  \n        dataframe=test,\n        directory = \"../input/aptos2019-blindness-detection/test_images/\",\n        x_col=\"id_code\",\n        target_size=(HEIGHT, WIDTH),\n        batch_size=1,\n        shuffle=False,\n        class_mode=None)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-05T13:42:57.930366Z","iopub.execute_input":"2023-05-05T13:42:57.930702Z","iopub.status.idle":"2023-05-05T13:43:10.964719Z","shell.execute_reply.started":"2023-05-05T13:42:57.930642Z","shell.execute_reply":"2023-05-05T13:43:10.963759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model","metadata":{}},{"cell_type":"code","source":"def create_model(input_shape, n_out):\n    input_tensor = Input(shape=input_shape)\n    base_model = Sequential()\n    \n    base_model.add(Conv2D(32, (3, 3), padding='same', activation='relu', input_shape=input_shape))\n    base_model.add(Conv2D(32, (3, 3), activation='relu'))\n    base_model.add(MaxPooling2D(pool_size=(2, 2)))\n    base_model.add(Dropout(0.25))\n    \n    base_model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))\n    base_model.add(Conv2D(64, (3, 3), activation='relu'))\n    base_model.add(MaxPooling2D(pool_size=(2, 2)))\n    base_model.add(Dropout(0.25))\n    \n    base_model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))\n    base_model.add(Conv2D(64, (3, 3), activation='relu'))\n    base_model.add(MaxPooling2D(pool_size=(2, 2)))\n    base_model.add(Dropout(0.25))\n    \n    base_model.add(Flatten())\n    base_model.add(Dense(512, activation='relu'))\n    base_model.add(Dropout(0.5))\n    base_model.add(Dense(output_dim=n_out, activation='softmax'))\n    \n    for layer in base_model.layers:\n        layer.trainable = False\n    \n    for i in range(-5, 0):\n        base_model.layers[i].trainable = True\n  \n    metric_list = [\"accuracy\"]\n    optimizer = optimizers.Adam(lr=WARMUP_LEARNING_RATE)\n    base_model.compile(optimizer=optimizer, loss=\"categorical_crossentropy\",  metrics=metric_list)\n    \n    return base_model\n\n    \n    \n#     model = Model(input_tensor, base_model)\n    \n#     \n\n#     return model\n    \n#     base_model.load_weights('../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5')\n\n#     x = GlobalAveragePooling2D()(base_model.output)\n#     x = Dropout(0.5)(x)\n#     x = Dense(2048, activation='relu')(x)\n#     x = Dropout(0.5)(x)\n#     final_output = Dense(n_out, activation='softmax', name='final_output')(x)\n#     model = Model(input_tensor, final_output)\n    \n#     return model\n\n# model = Sequential()\n#     # first set of CONV => RELU => MAX POOL layers    ","metadata":{"execution":{"iopub.status.busy":"2023-05-05T13:44:05.048022Z","iopub.execute_input":"2023-05-05T13:44:05.048372Z","iopub.status.idle":"2023-05-05T13:44:05.060905Z","shell.execute_reply.started":"2023-05-05T13:44:05.048312Z","shell.execute_reply":"2023-05-05T13:44:05.059666Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = create_model(input_shape=(HEIGHT, WIDTH, CANAL), n_out=N_CLASSES)\n\nmodel.summary()\n\n# for layer in model.layers:\n#     layer.trainable = False\n\n# for i in range(-5, 0):\n#     model.layers[i].trainable = True\n\n# metric_list = [\"accuracy\"]\n# optimizer = optimizers.Adam(lr=WARMUP_LEARNING_RATE)\n# model.compile(optimizer=optimizer, loss=\"categorical_crossentropy\",  metrics=metric_list)\n# model.summary()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-05T13:44:07.910228Z","iopub.execute_input":"2023-05-05T13:44:07.910547Z","iopub.status.idle":"2023-05-05T13:44:08.134364Z","shell.execute_reply.started":"2023-05-05T13:44:07.910491Z","shell.execute_reply":"2023-05-05T13:44:08.133615Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train top layers","metadata":{}},{"cell_type":"code","source":"STEP_SIZE_TRAIN = train_generator.n//train_generator.batch_size\nSTEP_SIZE_VALID = valid_generator.n//valid_generator.batch_size\n\nhistory_warmup = model.fit_generator(generator=train_generator,\n                                     steps_per_epoch=STEP_SIZE_TRAIN,\n                                     validation_data=valid_generator,\n                                     validation_steps=STEP_SIZE_VALID,\n                                     epochs=WARMUP_EPOCHS,\n                                     verbose=1).history","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-05T13:44:11.159043Z","iopub.execute_input":"2023-05-05T13:44:11.159780Z","iopub.status.idle":"2023-05-05T13:59:03.878024Z","shell.execute_reply.started":"2023-05-05T13:44:11.159438Z","shell.execute_reply":"2023-05-05T13:59:03.877056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Fine-tune the complete model","metadata":{}},{"cell_type":"code","source":"for layer in model.layers:\n    layer.trainable = True\n\nes = EarlyStopping(monitor='val_loss', mode='min', patience=ES_PATIENCE, restore_best_weights=True, verbose=1)\nrlrop = ReduceLROnPlateau(monitor='val_loss', mode='min', patience=RLROP_PATIENCE, factor=DECAY_DROP, min_lr=1e-6, verbose=1)\n\nmetric_list = [\"accuracy\"]\n\ncallback_list = [es, rlrop]\noptimizer = optimizers.Adam(lr=LEARNING_RATE)\nmodel.compile(optimizer=optimizer, loss=\"binary_crossentropy\",  metrics=metric_list)\nmodel.summary()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-05T14:06:14.519877Z","iopub.execute_input":"2023-05-05T14:06:14.520210Z","iopub.status.idle":"2023-05-05T14:06:14.582326Z","shell.execute_reply.started":"2023-05-05T14:06:14.520142Z","shell.execute_reply":"2023-05-05T14:06:14.581560Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history_finetunning = model.fit_generator(generator=train_generator,\n                                          steps_per_epoch=STEP_SIZE_TRAIN,\n                                          validation_data=valid_generator,\n                                          validation_steps=STEP_SIZE_VALID,\n                                          epochs=EPOCHS,\n                                          callbacks=callback_list,\n                                          verbose=1).history","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-05T14:06:18.914598Z","iopub.execute_input":"2023-05-05T14:06:18.914921Z","iopub.status.idle":"2023-05-05T15:40:44.689524Z","shell.execute_reply.started":"2023-05-05T14:06:18.914866Z","shell.execute_reply":"2023-05-05T15:40:44.688670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model loss graph ","metadata":{}},{"cell_type":"code","source":"history = {'loss': history_warmup['loss'] + history_finetunning['loss'], \n           'val_loss': history_warmup['val_loss'] + history_finetunning['val_loss'], \n           'acc': history_warmup['acc'] + history_finetunning['acc'], \n           'val_acc': history_warmup['val_acc'] + history_finetunning['val_acc']}\n\nsns.set_style(\"whitegrid\")\nfig, (ax1, ax2) = plt.subplots(2, 1, sharex='col', figsize=(20, 14))\n\nax1.plot(history['loss'], label='Train loss')\nax1.plot(history['val_loss'], label='Validation loss')\nax1.legend(loc='best')\nax1.set_title('Loss')\n\nax2.plot(history['acc'], label='Train Accuracy')\nax2.plot(history['val_acc'], label='Validation accuracy')\nax2.legend(loc='best')\nax2.set_title('Accuracy')\n\nplt.xlabel('Epochs')\nsns.despine()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-05T15:48:10.986681Z","iopub.execute_input":"2023-05-05T15:48:10.987046Z","iopub.status.idle":"2023-05-05T15:48:11.566697Z","shell.execute_reply.started":"2023-05-05T15:48:10.986987Z","shell.execute_reply":"2023-05-05T15:48:11.565920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Evaluation","metadata":{}},{"cell_type":"code","source":"complete_datagen = ImageDataGenerator(rescale=1./255)\ncomplete_generator = complete_datagen.flow_from_dataframe(  \n        dataframe=train,\n        directory = \"../input/aptos2019-blindness-detection/train_images/\",\n        x_col=\"id_code\",\n        target_size=(HEIGHT, WIDTH),\n        batch_size=1,\n        shuffle=False,\n        class_mode=None)\n\nSTEP_SIZE_COMPLETE = complete_generator.n//complete_generator.batch_size\ntrain_preds = model.predict_generator(complete_generator, steps=STEP_SIZE_COMPLETE)\ntrain_preds = [np.argmax(pred) for pred in train_preds]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-05T15:48:16.570250Z","iopub.execute_input":"2023-05-05T15:48:16.570577Z","iopub.status.idle":"2023-05-05T15:54:45.855296Z","shell.execute_reply.started":"2023-05-05T15:48:16.570523Z","shell.execute_reply":"2023-05-05T15:54:45.854123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Confusion Matrix","metadata":{}},{"cell_type":"code","source":"labels = ['0 - No DR', '1 - Mild', '2 - Moderate', '3 - Severe', '4 - Proliferative DR']\ncnf_matrix = confusion_matrix(train['diagnosis'].astype('int'), train_preds)\ncnf_matrix_norm = cnf_matrix.astype('float') / cnf_matrix.sum(axis=1)[:, np.newaxis]\ndf_cm = pd.DataFrame(cnf_matrix_norm, index=labels, columns=labels)\nplt.figure(figsize=(16, 7))\nsns.heatmap(df_cm, annot=True, fmt='.2f', cmap=\"Blues\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-05T16:19:54.461519Z","iopub.execute_input":"2023-05-05T16:19:54.461869Z","iopub.status.idle":"2023-05-05T16:19:54.914895Z","shell.execute_reply.started":"2023-05-05T16:19:54.461811Z","shell.execute_reply":"2023-05-05T16:19:54.913655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Classification Report","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import classification_report\n\ncr = classification_report(train_preds, train['diagnosis'].astype('int'))\nprint(cr)","metadata":{"execution":{"iopub.status.busy":"2023-05-05T16:19:58.523937Z","iopub.execute_input":"2023-05-05T16:19:58.524275Z","iopub.status.idle":"2023-05-05T16:19:58.541315Z","shell.execute_reply.started":"2023-05-05T16:19:58.524214Z","shell.execute_reply":"2023-05-05T16:19:58.540299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Quadratic Weighted Kappa","metadata":{}},{"cell_type":"code","source":"print(\"Train Cohen Kappa score: %.3f\" % cohen_kappa_score(train_preds, train['diagnosis'].astype('int'), weights='quadratic'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-05T16:20:05.684457Z","iopub.execute_input":"2023-05-05T16:20:05.684786Z","iopub.status.idle":"2023-05-05T16:20:05.699623Z","shell.execute_reply.started":"2023-05-05T16:20:05.684728Z","shell.execute_reply":"2023-05-05T16:20:05.698909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Apply model to test set and output predictions","metadata":{}},{"cell_type":"code","source":"test_generator.reset()\nSTEP_SIZE_TEST = test_generator.n//test_generator.batch_size\npreds = model.predict_generator(test_generator, steps=STEP_SIZE_TEST)\npredictions = [np.argmax(pred) for pred in preds]","metadata":{"execution":{"iopub.status.busy":"2023-05-05T16:20:10.069092Z","iopub.execute_input":"2023-05-05T16:20:10.069464Z","iopub.status.idle":"2023-05-05T16:21:53.360290Z","shell.execute_reply.started":"2023-05-05T16:20:10.069407Z","shell.execute_reply":"2023-05-05T16:21:53.359359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"filenames = test_generator.filenames\nresults = pd.DataFrame({'id_code':filenames, 'diagnosis':predictions})\nresults['id_code'] = results['id_code'].map(lambda x: str(x)[:-4])\nresults.to_csv('submission.csv',index=False)\nresults.head(10)","metadata":{"execution":{"iopub.status.busy":"2023-05-04T10:18:59.881253Z","iopub.execute_input":"2023-05-04T10:18:59.881751Z","iopub.status.idle":"2023-05-04T10:19:00.443452Z","shell.execute_reply.started":"2023-05-04T10:18:59.881531Z","shell.execute_reply":"2023-05-04T10:19:00.442511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Predictions class distribution","metadata":{}},{"cell_type":"code","source":"f, ax = plt.subplots(figsize=(14, 8.7))\nax = sns.countplot(x=\"diagnosis\", data=results, palette=\"GnBu_d\")\nsns.despine()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-04T10:19:00.445Z","iopub.execute_input":"2023-05-04T10:19:00.445547Z","iopub.status.idle":"2023-05-04T10:19:00.743188Z","shell.execute_reply.started":"2023-05-04T10:19:00.445469Z","shell.execute_reply":"2023-05-04T10:19:00.741993Z"},"trusted":true},"execution_count":null,"outputs":[]}]}