{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

APTOS 2019 Blindness Detection

\n

Detect diabetic retinopathy to stop blindness before it's too late

\n
\n\nIn this synchronous Kernels-only competition, you'll build a machine learning model to speed up disease detection. You’ll work with thousands of images collected in rural areas to help identify diabetic retinopathy automatically. If successful, you will not only help to prevent lifelong blindness, but these models may be used to detect other sorts of diseases in the future, like glaucoma and macular degeneration.\n\nIn this notebook, I will be using basic deep learning and transfer learning (ResNet50) to create a baseline.\n##### Image source: http://cceyemd.com/diabetes-and-eye-exams/\n\n### Dependencies","metadata":{}},{"cell_type":"code","source":"import os\nimport cv2\nimport random\nimport warnings\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import confusion_matrix, cohen_kappa_score,classification_report\nfrom keras.models import Model,load_model\nfrom keras import optimizers, applications\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.callbacks import EarlyStopping, ReduceLROnPlateau\nfrom keras.layers import Dense, Dropout, GlobalAveragePooling2D, Input\n\n# Set seeds to make the experiment more reproducible.\nimport tensorflow as tf\ndef seed_everything(seed=0):\n random.seed(seed)\n os.environ['PYTHONHASHSEED'] = str(seed)\n np.random.seed(seed)\n tf.random.set_seed(0)\nseed_everything()\n\n%matplotlib inline\nsns.set(style=\"whitegrid\")\nwarnings.filterwarnings(\"ignore\")","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.status.busy":"2022-02-10T16:21:25.977701Z","iopub.execute_input":"2022-02-10T16:21:25.978046Z","iopub.status.idle":"2022-02-10T16:21:26.080595Z","shell.execute_reply.started":"2022-02-10T16:21:25.978018Z","shell.execute_reply":"2022-02-10T16:21:26.079426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load data","metadata":{"_kg_hide-output":true}},{"cell_type":"code","source":"train = pd.read_csv('../input/aptos2019-blindness-detection/train.csv')\ntest = pd.read_csv('../input/aptos2019-blindness-detection/test.csv')","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_kg_hide-input":false,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","execution":{"iopub.status.busy":"2022-02-10T16:21:27.475261Z","iopub.execute_input":"2022-02-10T16:21:27.475746Z","iopub.status.idle":"2022-02-10T16:21:27.511686Z","shell.execute_reply.started":"2022-02-10T16:21:27.475716Z","shell.execute_reply":"2022-02-10T16:21:27.510679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# EDA\n\n## Data overview","metadata":{}},{"cell_type":"code","source":"print('Number of train samples: ', train.shape[0])\nprint('Number of test samples: ', test.shape[0])\ndisplay(train.head())","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T16:21:30.877889Z","iopub.execute_input":"2022-02-10T16:21:30.878295Z","iopub.status.idle":"2022-02-10T16:21:30.895065Z","shell.execute_reply.started":"2022-02-10T16:21:30.878262Z","shell.execute_reply":"2022-02-10T16:21:30.893831Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Label class distribution\n\nAs we can see we have an unbalanced database, we have two times more class 0 than 2, and classes 1, 2 and 4 each have less than half of the class 2 data.","metadata":{}},{"cell_type":"code","source":"f, ax = plt.subplots(figsize=(14, 8.7))\nax = sns.countplot(x=\"diagnosis\", data=train, palette=\"GnBu_d\")\nsns.despine()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T16:21:32.892290Z","iopub.execute_input":"2022-02-10T16:21:32.892700Z","iopub.status.idle":"2022-02-10T16:21:33.099834Z","shell.execute_reply.started":"2022-02-10T16:21:32.892669Z","shell.execute_reply":"2022-02-10T16:21:33.098792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##### Legend\n- 0 - No DR\n- 1 - Mild\n- 2 - Moderate\n- 3 - Severe\n- 4 - Proliferative DR ","metadata":{}},{"cell_type":"markdown","source":"### Now let's see some of the images\n\nThe images have different sizes, they may need resizing or some padding.","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"white\")\ncount = 1\nplt.figure(figsize=[20, 20])\nfor img_name in train['id_code'][:15]:\n img = cv2.imread(\"../input/aptos2019-blindness-detection/train_images/%s.png\" % img_name)[...,[2, 1, 0]]\n plt.subplot(5, 5, count)\n plt.imshow(img)\n plt.title(\"Image %s\" % count)\n count += 1\n \nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T16:21:37.275258Z","iopub.execute_input":"2022-02-10T16:21:37.275832Z","iopub.status.idle":"2022-02-10T16:21:48.758531Z","shell.execute_reply.started":"2022-02-10T16:21:37.275798Z","shell.execute_reply":"2022-02-10T16:21:48.757338Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model parameters","metadata":{}},{"cell_type":"code","source":"# Model parameters\nBATCH_SIZE = 8\nEPOCHS = 20\nWARMUP_EPOCHS = 2\nLEARNING_RATE = 1e-4\nWARMUP_LEARNING_RATE = 1e-3\nHEIGHT = 728\nWIDTH = 728\nCANAL = 3\nN_CLASSES = train['diagnosis'].nunique()\nES_PATIENCE = 5\nRLROP_PATIENCE = 3\nDECAY_DROP = 0.5","metadata":{"execution":{"iopub.status.busy":"2022-02-10T16:21:50.755523Z","iopub.execute_input":"2022-02-10T16:21:50.755947Z","iopub.status.idle":"2022-02-10T16:21:50.763319Z","shell.execute_reply.started":"2022-02-10T16:21:50.755916Z","shell.execute_reply":"2022-02-10T16:21:50.761892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Preprocecss data\ntrain[\"id_code\"] = train[\"id_code\"].apply(lambda x: x + \".png\")\ntest[\"id_code\"] = test[\"id_code\"].apply(lambda x: x + \".png\")\ntrain['diagnosis'] = train['diagnosis'].astype('str')\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-02-10T16:21:52.834130Z","iopub.execute_input":"2022-02-10T16:21:52.834507Z","iopub.status.idle":"2022-02-10T16:21:52.860781Z","shell.execute_reply.started":"2022-02-10T16:21:52.834467Z","shell.execute_reply":"2022-02-10T16:21:52.859515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data generator","metadata":{}},{"cell_type":"code","source":"train_datagen=ImageDataGenerator(rescale=1./255, \n validation_split=0.2,\n horizontal_flip=True)\n\ntrain_generator=train_datagen.flow_from_dataframe(\n dataframe=train,\n directory=\"../input/aptos2019-blindness-detection/train_images/\",\n x_col=\"id_code\",\n y_col=\"diagnosis\",\n batch_size=BATCH_SIZE,\n class_mode=\"categorical\",\n target_size=(HEIGHT, WIDTH),\n subset='training')\n\nvalid_generator=train_datagen.flow_from_dataframe(\n dataframe=train,\n directory=\"../input/aptos2019-blindness-detection/train_images/\",\n x_col=\"id_code\",\n y_col=\"diagnosis\",\n batch_size=BATCH_SIZE,\n class_mode=\"categorical\", \n target_size=(HEIGHT, WIDTH),\n subset='validation')\n\ntest_datagen = ImageDataGenerator(rescale=1./255)\n\ntest_generator = test_datagen.flow_from_dataframe( \n dataframe=test,\n directory = \"../input/aptos2019-blindness-detection/test_images/\",\n x_col=\"id_code\",\n target_size=(HEIGHT, WIDTH),\n batch_size=1,\n shuffle=False,\n class_mode=None)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T16:23:49.324000Z","iopub.execute_input":"2022-02-10T16:23:49.324465Z","iopub.status.idle":"2022-02-10T16:23:52.625579Z","shell.execute_reply.started":"2022-02-10T16:23:49.324421Z","shell.execute_reply":"2022-02-10T16:23:52.624279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model","metadata":{}},{"cell_type":"code","source":"def create_model(input_shape, n_out):\n input_tensor = Input(shape=input_shape)\n base_model = applications.ResNet50(weights='imagenet', \n include_top=False,\n input_tensor=input_tensor)\n #base_model.load_weights('../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5')\n\n x = GlobalAveragePooling2D()(base_model.output)\n x = Dropout(0.5)(x)\n x = Dense(2048, activation='relu')(x)\n x = Dropout(0.5)(x)\n final_output = Dense(n_out, activation='softmax', name='final_output')(x)\n model = Model(input_tensor, final_output)\n \n return model","metadata":{"execution":{"iopub.status.busy":"2022-02-10T16:23:58.156186Z","iopub.execute_input":"2022-02-10T16:23:58.156582Z","iopub.status.idle":"2022-02-10T16:23:58.164170Z","shell.execute_reply.started":"2022-02-10T16:23:58.156551Z","shell.execute_reply":"2022-02-10T16:23:58.162967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = create_model(input_shape=(HEIGHT, WIDTH, CANAL), n_out=N_CLASSES)\n\nfor layer in model.layers:\n layer.trainable = False\n\nfor i in range(-5, 0):\n model.layers[i].trainable = True\n\nmetric_list = [\"accuracy\"]\noptimizer = optimizers.Adam(lr=WARMUP_LEARNING_RATE)\nmodel.compile(optimizer=optimizer, loss=\"categorical_crossentropy\", metrics=metric_list)\nmodel.summary()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-10T16:23:58.482439Z","iopub.execute_input":"2022-02-10T16:23:58.482778Z","iopub.status.idle":"2022-02-10T16:24:00.374226Z","shell.execute_reply.started":"2022-02-10T16:23:58.482749Z","shell.execute_reply":"2022-02-10T16:24:00.372996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train top layers","metadata":{}},{"cell_type":"code","source":"STEP_SIZE_TRAIN = train_generator.n//train_generator.batch_size\nSTEP_SIZE_VALID = valid_generator.n//valid_generator.batch_size\n\nhistory_warmup = model.fit_generator(generator=train_generator,\n steps_per_epoch=50,\n validation_data=valid_generator,\n validation_steps=30,\n epochs=10,\n verbose=1).history","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-10T16:24:05.592816Z","iopub.execute_input":"2022-02-10T16:24:05.593180Z","iopub.status.idle":"2022-02-10T16:38:30.639923Z","shell.execute_reply.started":"2022-02-10T16:24:05.593149Z","shell.execute_reply":"2022-02-10T16:38:30.638842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Fine-tune the complete model","metadata":{}},{"cell_type":"code","source":"for layer in model.layers:\n layer.trainable = True\n\nes = EarlyStopping(monitor='val_loss', mode='min', patience=ES_PATIENCE, restore_best_weights=True, verbose=1)\nrlrop = ReduceLROnPlateau(monitor='val_loss', mode='min', patience=RLROP_PATIENCE, factor=DECAY_DROP, min_lr=1e-6, verbose=1)\n\ncallback_list = [es, rlrop]\noptimizer = optimizers.Adam(lr=LEARNING_RATE)\nmodel.compile(optimizer=optimizer, loss=\"categorical_crossentropy\", metrics=metric_list)\nmodel.summary()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-10T14:52:44.685724Z","iopub.execute_input":"2022-02-10T14:52:44.686097Z","iopub.status.idle":"2022-02-10T14:52:44.850588Z","shell.execute_reply.started":"2022-02-10T14:52:44.686068Z","shell.execute_reply":"2022-02-10T14:52:44.849245Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history_finetunning = model.fit_generator(generator=train_generator,\n steps_per_epoch=50,\n validation_data=valid_generator,\n validation_steps=30,\n epochs=10,\n callbacks=callback_list,\n verbose=1).history","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-02-10T14:52:53.318602Z","iopub.execute_input":"2022-02-10T14:52:53.319019Z","iopub.status.idle":"2022-02-10T15:06:12.410586Z","shell.execute_reply.started":"2022-02-10T14:52:53.318989Z","shell.execute_reply":"2022-02-10T15:06:12.409438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.applications.vgg16 import VGG16 \nfrom keras.models import Sequential\nfrom tifffile.tifffile import sequence\nfrom keras.layers import Dense, Dropout, Flatten\nVGG_MODEL = VGG16(weights = 'imagenet', include_top = False ,input_shape =(728,728,3))\n\nfor layer in VGG_MODEL.layers[:-4]:\n layer.trainable = False\n\nmodel = Sequential()\nmodel.add(VGG_MODEL)\nmodel.add(Flatten())\nmodel.add(Dense(256, activation = 'relu'))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(126, activation = 'relu'))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(5, activation = 'softmax'))","metadata":{"execution":{"iopub.status.busy":"2022-02-10T16:54:56.561664Z","iopub.execute_input":"2022-02-10T16:54:56.562095Z","iopub.status.idle":"2022-02-10T16:54:56.981422Z","shell.execute_reply.started":"2022-02-10T16:54:56.562064Z","shell.execute_reply":"2022-02-10T16:54:56.980336Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n\nes = EarlyStopping(monitor='val_loss', mode='min', patience=ES_PATIENCE, restore_best_weights=True, verbose=1)\nrlrop = ReduceLROnPlateau(monitor='val_loss', mode='min', patience=RLROP_PATIENCE, factor=DECAY_DROP, min_lr=1e-6, verbose=1)\n\ncallback_list = [es, rlrop]\noptimizer = optimizers.Adam(lr=LEARNING_RATE)\nmodel.compile(optimizer=optimizer, loss=\"categorical_crossentropy\", metrics=metric_list)\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-02-10T16:54:58.631496Z","iopub.execute_input":"2022-02-10T16:54:58.631874Z","iopub.status.idle":"2022-02-10T16:54:58.660631Z","shell.execute_reply.started":"2022-02-10T16:54:58.631827Z","shell.execute_reply":"2022-02-10T16:54:58.659524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history_finetunning = model.fit_generator(generator=train_generator,\n steps_per_epoch=50,\n validation_data=valid_generator,\n validation_steps=30,\n epochs=10,\n callbacks=callback_list,\n verbose=1).history","metadata":{"execution":{"iopub.status.busy":"2022-02-10T16:54:59.633195Z","iopub.execute_input":"2022-02-10T16:54:59.633554Z","iopub.status.idle":"2022-02-10T17:09:42.890209Z","shell.execute_reply.started":"2022-02-10T16:54:59.633521Z","shell.execute_reply":"2022-02-10T17:09:42.889122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.applications.xception import Xception\nxception = Xception(include_top=False, input_shape = (728,728,3))\nfor layer in xception.layers[:-4]:\n layer.trainable = False\n\nmodel = Sequential()\nmodel.add(VGG_MODEL)\nmodel.add(Flatten())\nmodel.add(Dense(256, activation = 'relu'))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(126, activation = 'relu'))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(5, activation = 'softmax'))","metadata":{"execution":{"iopub.status.busy":"2022-02-10T17:11:19.485034Z","iopub.execute_input":"2022-02-10T17:11:19.485528Z","iopub.status.idle":"2022-02-10T17:11:23.347427Z","shell.execute_reply.started":"2022-02-10T17:11:19.485491Z","shell.execute_reply":"2022-02-10T17:11:23.346289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n\nes = EarlyStopping(monitor='val_loss', mode='min', patience=ES_PATIENCE, restore_best_weights=True, verbose=1)\nrlrop = ReduceLROnPlateau(monitor='val_loss', mode='min', patience=RLROP_PATIENCE, factor=DECAY_DROP, min_lr=1e-6, verbose=1)\n\ncallback_list = [es, rlrop]\noptimizer = optimizers.Adam(lr=LEARNING_RATE)\nmodel.compile(optimizer=optimizer, loss=\"categorical_crossentropy\", metrics=metric_list)\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-02-10T17:11:26.421215Z","iopub.execute_input":"2022-02-10T17:11:26.421657Z","iopub.status.idle":"2022-02-10T17:11:26.451999Z","shell.execute_reply.started":"2022-02-10T17:11:26.421621Z","shell.execute_reply":"2022-02-10T17:11:26.450915Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history_finetunning = model.fit_generator(generator=train_generator,\n steps_per_epoch=50,\n validation_data=valid_generator,\n validation_steps=30,\n epochs=10,\n callbacks=callback_list,\n verbose=1).history","metadata":{"execution":{"iopub.status.busy":"2022-02-10T17:11:30.069034Z","iopub.execute_input":"2022-02-10T17:11:30.069429Z","iopub.status.idle":"2022-02-10T17:25:58.060642Z","shell.execute_reply.started":"2022-02-10T17:11:30.069398Z","shell.execute_reply":"2022-02-10T17:25:58.059544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model loss graph ","metadata":{}},{"cell_type":"code","source":"history = {'loss': history_warmup['loss'] + history_finetunning['loss'], \n 'val_loss': history_warmup['val_loss'] + history_finetunning['val_loss'], \n 'accuracy': history_warmup['accuracy'] + history_finetunning['accuracy'], \n 'val_accuracy': history_warmup['val_accuracy'] + history_finetunning['val_accuracy']}\n\nsns.set_style(\"whitegrid\")\nfig, (ax1, ax2) = plt.subplots(2, 1, sharex='col', figsize=(20, 14))\n\nax1.plot(history['loss'], label='Train loss')\nax1.plot(history['val_loss'], label='Validation loss')\nax1.legend(loc='best')\nax1.set_title('Loss')\n\nax2.plot(history['accuracy'], label='Train Accuracy')\nax2.plot(history['val_accuracy'], label='Validation accuracy')\nax2.legend(loc='best')\nax2.set_title('Accuracy')\n\nplt.xlabel('Epochs')\nsns.despine()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T17:26:09.803363Z","iopub.execute_input":"2022-02-10T17:26:09.803770Z","iopub.status.idle":"2022-02-10T17:26:10.273381Z","shell.execute_reply.started":"2022-02-10T17:26:09.803739Z","shell.execute_reply":"2022-02-10T17:26:10.272273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save(\"./my_model.h5\")","metadata":{"execution":{"iopub.status.busy":"2022-02-10T15:38:06.428441Z","iopub.execute_input":"2022-02-10T15:38:06.428769Z","iopub.status.idle":"2022-02-10T15:38:07.767404Z","shell.execute_reply.started":"2022-02-10T15:38:06.428741Z","shell.execute_reply":"2022-02-10T15:38:07.766312Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Evaluation","metadata":{}},{"cell_type":"code","source":"complete_datagen = ImageDataGenerator(rescale=1./255)\ncomplete_generator = complete_datagen.flow_from_dataframe( \n dataframe=train,\n directory = \"../input/aptos2019-blindness-detection/train_images/\",\n x_col=\"id_code\",\n target_size=(HEIGHT, WIDTH),\n batch_size=1,\n shuffle=False,\n class_mode=None)\n\nSTEP_SIZE_COMPLETE = complete_generator.n//complete_generator.batch_size\ntrain_preds = model.predict_generator(complete_generator, steps=STEP_SIZE_COMPLETE)\ntrain_preds = [np.argmax(pred) for pred in train_preds]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T17:26:20.709121Z","iopub.execute_input":"2022-02-10T17:26:20.709510Z","iopub.status.idle":"2022-02-10T17:33:15.031537Z","shell.execute_reply.started":"2022-02-10T17:26:20.709477Z","shell.execute_reply":"2022-02-10T17:33:15.030409Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Confusion Matrix","metadata":{}},{"cell_type":"code","source":"labels = ['0 - No DR', '1 - Mild', '2 - Moderate', '3 - Severe', '4 - Proliferative DR']\ncnf_matrix = confusion_matrix(train['diagnosis'].astype('int'), train_preds)\ncnf_matrix_norm = cnf_matrix.astype('float') / cnf_matrix.sum(axis=1)[:, np.newaxis]\ndf_cm = pd.DataFrame(cnf_matrix_norm, index=labels, columns=labels)\nplt.figure(figsize=(16, 7))\nsns.heatmap(df_cm, annot=True, fmt='.2f', cmap=\"Blues\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T17:33:15.033424Z","iopub.execute_input":"2022-02-10T17:33:15.033902Z","iopub.status.idle":"2022-02-10T17:33:15.442949Z","shell.execute_reply.started":"2022-02-10T17:33:15.033861Z","shell.execute_reply":"2022-02-10T17:33:15.441396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(classification_report(train['diagnosis'].astype('int'), train_preds))","metadata":{"execution":{"iopub.status.busy":"2022-02-10T17:33:15.445309Z","iopub.execute_input":"2022-02-10T17:33:15.445655Z","iopub.status.idle":"2022-02-10T17:33:15.468241Z","shell.execute_reply.started":"2022-02-10T17:33:15.445625Z","shell.execute_reply":"2022-02-10T17:33:15.467132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Quadratic Weighted Kappa","metadata":{}},{"cell_type":"code","source":"print(\"Train Cohen Kappa score: %.3f\" % cohen_kappa_score(train_preds, train['diagnosis'].astype('int'), weights='quadratic'))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T17:33:15.470587Z","iopub.execute_input":"2022-02-10T17:33:15.471027Z","iopub.status.idle":"2022-02-10T17:33:15.490022Z","shell.execute_reply.started":"2022-02-10T17:33:15.470985Z","shell.execute_reply":"2022-02-10T17:33:15.488541Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Apply model to test set and output predictions","metadata":{}},{"cell_type":"code","source":"test_generator.reset()\nSTEP_SIZE_TEST = test_generator.n//test_generator.batch_size\npreds = model.predict_generator(test_generator, steps=STEP_SIZE_TEST,verbose =1)\npredictions = [np.argmax(pred) for pred in preds]","metadata":{"execution":{"iopub.status.busy":"2022-02-10T17:33:15.491483Z","iopub.execute_input":"2022-02-10T17:33:15.491835Z","iopub.status.idle":"2022-02-10T17:35:31.230644Z","shell.execute_reply.started":"2022-02-10T17:33:15.491805Z","shell.execute_reply":"2022-02-10T17:35:31.229521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"filenames = test_generator.filenames\nresults = pd.DataFrame({'id_code':filenames, 'diagnosis':predictions})\nresults['id_code'] = results['id_code'].map(lambda x: str(x)[:-4])\nresults.to_csv('submission.csv',index=False)\nresults.head(10)","metadata":{"execution":{"iopub.status.busy":"2022-02-10T17:35:31.232340Z","iopub.execute_input":"2022-02-10T17:35:31.232757Z","iopub.status.idle":"2022-02-10T17:35:31.262885Z","shell.execute_reply.started":"2022-02-10T17:35:31.232726Z","shell.execute_reply":"2022-02-10T17:35:31.261754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Predictions class distribution","metadata":{}},{"cell_type":"code","source":"f, ax = plt.subplots(figsize=(14, 8.7))\nax = sns.countplot(x=\"diagnosis\", data=results, palette=\"GnBu_d\")\nsns.despine()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-10T17:35:31.264275Z","iopub.execute_input":"2022-02-10T17:35:31.264901Z","iopub.status.idle":"2022-02-10T17:35:31.477199Z","shell.execute_reply.started":"2022-02-10T17:35:31.264856Z","shell.execute_reply":"2022-02-10T17:35:31.475736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"my_model = load_model(\"./my_model.h5\")\nmy_model.summary()","metadata":{"execution":{"iopub.status.busy":"2022-02-10T16:12:58.009646Z","iopub.execute_input":"2022-02-10T16:12:58.010002Z","iopub.status.idle":"2022-02-10T16:12:59.960372Z","shell.execute_reply.started":"2022-02-10T16:12:58.009972Z","shell.execute_reply":"2022-02-10T16:12:59.959213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}