{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n

Herbarium 2021 - Half-Earth Challenge - FGVC8

\n\n\n#### Identify plant species of the Americas, Oceania and the Pacific from herbarium specimens\n\n\n\n\n","metadata":{}},{"cell_type":"markdown","source":"## Data Overview\nThe training and test set contain images of herbarium specimens from nearly 65,000 species of vascular plants. Each image contains exactly one specimen. The text labels on the specimen images have been blurred to remove category information in the image.\n\nThe data has been approximately split 80%/20% for training/test. Each category has at least 1 instance in both the training and test datasets. Note that the test set distribution is slightly different from the training set distribution. The training set contains species with hundreds of examples, but the test set has the number of examples per species capped at a maximum of 10.\n\n\n## Dataset Details\nEach image has different image dimensions, with a maximum of 1000 pixels in the larger dimension. These have been resized from the original image resolution. All images are in JPEG format.\n\n## Dataset Format\nThis dataset uses the COCO dataset format with additional annotation fields. In addition to the species category labels, we also provide region and supercategory information.\n\nThe training set metadata (train/metadata.json) and test set metadata (test/metadata.json) are JSON files in the format below. Naturally, the test set metadata file omits the \"annotations\", \"categories,\" and \"regions\" elements.\n\n#### Dataset Link \n\n##### [Here](https://www.kaggle.com/c/herbarium-2021-fgvc8/overview)\n","metadata":{}},{"cell_type":"markdown","source":"## What is VGG16 model?\nVGG16 (also called OxfordNet) is a convolutional neural network architecture named after the Visual Geometry Group from Oxford, who developed it. ... By only keeping the convolutional modules, our model can be adapted to arbitrary input sizes. The model loads a set of weights pre-trained on ImageNet.\n\n\n","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport cv2\nimport matplotlib.pyplot as plt\nimport json\nimport PIL\nimport skimage.io\nimport seaborn as sn\nfrom collections import Counter\nimport tensorflow as tf\nfrom sklearn.model_selection import train_test_split\nimport keras.backend as K\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom keras.utils import to_categorical, Sequence\nfrom keras.models import Sequential,Model\nfrom keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, Activation,BatchNormalization\nfrom keras.optimizers import RMSprop,Adam\nfrom keras.applications import ResNet50\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, ModelCheckpoint, EarlyStopping\nfrom tensorflow.keras.optimizers import Adam\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:16.939771Z","iopub.execute_input":"2021-05-30T09:23:16.940327Z","iopub.status.idle":"2021-05-30T09:23:24.130315Z","shell.execute_reply.started":"2021-05-30T09:23:16.940221Z","shell.execute_reply":"2021-05-30T09:23:24.129386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"path = '/kaggle/input/herbarium-2021-fgvc8/'\nos.listdir(path)","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:24.133821Z","iopub.execute_input":"2021-05-30T09:23:24.134153Z","iopub.status.idle":"2021-05-30T09:23:24.143324Z","shell.execute_reply.started":"2021-05-30T09:23:24.134122Z","shell.execute_reply":"2021-05-30T09:23:24.14228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"samp_subm = pd.read_csv(path+'sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:24.145419Z","iopub.execute_input":"2021-05-30T09:23:24.145712Z","iopub.status.idle":"2021-05-30T09:23:24.271972Z","shell.execute_reply.started":"2021-05-30T09:23:24.145685Z","shell.execute_reply":"2021-05-30T09:23:24.270998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(path+'train/'+'metadata.json') as f:\n train_data = json.load(f)\nwith open(path+'test/'+'metadata.json') as f:\n test_data = json.load(f)","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:24.273201Z","iopub.execute_input":"2021-05-30T09:23:24.273491Z","iopub.status.idle":"2021-05-30T09:23:39.207676Z","shell.execute_reply.started":"2021-05-30T09:23:24.273463Z","shell.execute_reply":"2021-05-30T09:23:39.206647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_examples():\n fig, axs = plt.subplots(4, 4, figsize=(20, 20))\n fig.subplots_adjust(hspace = .1, wspace=.1)\n \n axs = axs.ravel()\n for i in range(16):\n img = cv2.imread(path+'train/'+train_data['images'][i]['file_name'])\n axs[i].imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))\n axs[i].set_title(train_data['categories'][i]['family'])\n axs[i].set_xticklabels([])\n axs[i].set_yticklabels([])\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:39.213219Z","iopub.execute_input":"2021-05-30T09:23:39.213601Z","iopub.status.idle":"2021-05-30T09:23:39.220372Z","shell.execute_reply.started":"2021-05-30T09:23:39.213561Z","shell.execute_reply":"2021-05-30T09:23:39.219635Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Number of train images:', len(train_data['images']))\nprint('Number of test images:', len(test_data['images']))","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:39.222929Z","iopub.execute_input":"2021-05-30T09:23:39.223218Z","iopub.status.idle":"2021-05-30T09:23:39.237791Z","shell.execute_reply.started":"2021-05-30T09:23:39.22319Z","shell.execute_reply":"2021-05-30T09:23:39.236568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data['annotations'][0]","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:39.239516Z","iopub.execute_input":"2021-05-30T09:23:39.239991Z","iopub.status.idle":"2021-05-30T09:23:39.252463Z","shell.execute_reply.started":"2021-05-30T09:23:39.239944Z","shell.execute_reply":"2021-05-30T09:23:39.251511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data['categories'][0]","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:39.253973Z","iopub.execute_input":"2021-05-30T09:23:39.254292Z","iopub.status.idle":"2021-05-30T09:23:39.267021Z","shell.execute_reply.started":"2021-05-30T09:23:39.254261Z","shell.execute_reply":"2021-05-30T09:23:39.265951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data['images'][0]","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:39.268662Z","iopub.execute_input":"2021-05-30T09:23:39.269124Z","iopub.status.idle":"2021-05-30T09:23:39.281861Z","shell.execute_reply.started":"2021-05-30T09:23:39.269081Z","shell.execute_reply":"2021-05-30T09:23:39.280753Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_examples()","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:39.283232Z","iopub.execute_input":"2021-05-30T09:23:39.283625Z","iopub.status.idle":"2021-05-30T09:23:42.736101Z","shell.execute_reply.started":"2021-05-30T09:23:39.283595Z","shell.execute_reply":"2021-05-30T09:23:42.735292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_image = pd.json_normalize(train_data['images'])\ndf_annot = pd.json_normalize(train_data['annotations'])\ndf_train_data = pd.DataFrame()\ndf_train_data['file_name'] = df_image['file_name']\ndf_train_data['category_id'] = df_annot['category_id']","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:23:42.737124Z","iopub.execute_input":"2021-05-30T09:23:42.737486Z","iopub.status.idle":"2021-05-30T09:24:55.507385Z","shell.execute_reply.started":"2021-05-30T09:23:42.737459Z","shell.execute_reply":"2021-05-30T09:24:55.50646Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_data, df_val_data = train_test_split(df_train_data, test_size=0.3)\ndf_train_data.index = range(len(df_train_data.index))\ndf_val_data.index = range(len(df_val_data.index))","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:24:55.508511Z","iopub.execute_input":"2021-05-30T09:24:55.50899Z","iopub.status.idle":"2021-05-30T09:24:56.46901Z","shell.execute_reply.started":"2021-05-30T09:24:55.508954Z","shell.execute_reply":"2021-05-30T09:24:56.468029Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_image = pd.json_normalize(test_data['images'])\ndf_test_data = pd.DataFrame()\ndf_test_data['file_name'] = df_image['file_name']\ndf_test_data['category_id'] = 0","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:24:56.470297Z","iopub.execute_input":"2021-05-30T09:24:56.47061Z","iopub.status.idle":"2021-05-30T09:25:00.792114Z","shell.execute_reply.started":"2021-05-30T09:24:56.47058Z","shell.execute_reply":"2021-05-30T09:25:00.790969Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Number of train samples:', len(df_train_data))\nprint('Number of val samples:', len(df_val_data))\nprint('Number of test samples:', len(df_test_data))","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.793737Z","iopub.execute_input":"2021-05-30T09:25:00.79418Z","iopub.status.idle":"2021-05-30T09:25:00.800576Z","shell.execute_reply.started":"2021-05-30T09:25:00.794092Z","shell.execute_reply":"2021-05-30T09:25:00.799287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Number of categories:', len(df_train_data['category_id'].unique()))","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.802359Z","iopub.execute_input":"2021-05-30T09:25:00.802789Z","iopub.status.idle":"2021-05-30T09:25:00.831942Z","shell.execute_reply.started":"2021-05-30T09:25:00.802746Z","shell.execute_reply":"2021-05-30T09:25:00.830801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_data['category_id'].value_counts()[0:10]","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.833636Z","iopub.execute_input":"2021-05-30T09:25:00.834064Z","iopub.status.idle":"2021-05-30T09:25:00.868788Z","shell.execute_reply.started":"2021-05-30T09:25:00.83402Z","shell.execute_reply":"2021-05-30T09:25:00.867405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"q_size = 64\nimg_channel = 3\nnum_classes = 64500\nbatch_size = 32\nepochs = 5","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.870202Z","iopub.execute_input":"2021-05-30T09:25:00.870498Z","iopub.status.idle":"2021-05-30T09:25:00.875746Z","shell.execute_reply.started":"2021-05-30T09:25:00.87047Z","shell.execute_reply":"2021-05-30T09:25:00.874619Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class DataGenerator(Sequence):\n def __init__(self, path, list_IDs, labels, batch_size, img_size, img_channel, num_classes):\n self.path = path\n self.list_IDs = list_IDs\n self.labels = labels\n self.batch_size = batch_size\n self.img_size = img_size\n self.img_channel = img_channel\n self.num_classes = num_classes\n self.indexes = np.arange(len(self.list_IDs))\n\n \n def __len__(self):\n len_ = int(len(self.list_IDs)/self.batch_size)\n if len_*self.batch_size < len(self.list_IDs):\n len_ += 1\n return len_\n \n \n def __getitem__(self, index):\n indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n list_IDs_temp = [self.list_IDs[k] for k in indexes]\n X, y = self.__data_generation(list_IDs_temp)\n return X, y\n \n \n def __data_generation(self, list_IDs_temp):\n X = np.zeros((self.batch_size, self.img_size, self.img_size, self.img_channel))\n y = np.zeros((self.batch_size, self.num_classes), dtype=int)\n for i, ID in enumerate(list_IDs_temp):\n img = cv2.imread(self.path+ID)\n img = cv2.resize(img, (self.img_size, self.img_size))\n X[i, ] = img/255\n y[i, ] = to_categorical(self.labels[i], num_classes=self.num_classes)\n return X, y","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.877177Z","iopub.execute_input":"2021-05-30T09:25:00.877573Z","iopub.status.idle":"2021-05-30T09:25:00.891405Z","shell.execute_reply.started":"2021-05-30T09:25:00.877537Z","shell.execute_reply":"2021-05-30T09:25:00.890555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"number_samples = 10000\ndf_train_data = df_train_data[0:number_samples]\ndf_val_data = df_val_data[0:number_samples]\ndf_test_data = df_test_data[0:number_samples]","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.89318Z","iopub.execute_input":"2021-05-30T09:25:00.893871Z","iopub.status.idle":"2021-05-30T09:25:00.912884Z","shell.execute_reply.started":"2021-05-30T09:25:00.893826Z","shell.execute_reply":"2021-05-30T09:25:00.911801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train_data","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.915088Z","iopub.execute_input":"2021-05-30T09:25:00.915679Z","iopub.status.idle":"2021-05-30T09:25:00.940412Z","shell.execute_reply.started":"2021-05-30T09:25:00.915634Z","shell.execute_reply":"2021-05-30T09:25:00.939187Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_generator = DataGenerator(path+'train/', df_train_data['file_name'], df_train_data['category_id'],\n batch_size, q_size, img_channel, num_classes)\nval_generator = DataGenerator(path+'train/',df_val_data['file_name'], df_val_data['category_id'],\n batch_size, q_size, img_channel, num_classes)\ntest_generator = DataGenerator(path+'test/',df_test_data['file_name'], df_test_data['category_id'],\n batch_size, q_size, img_channel, num_classes)","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.942389Z","iopub.execute_input":"2021-05-30T09:25:00.942842Z","iopub.status.idle":"2021-05-30T09:25:00.950082Z","shell.execute_reply.started":"2021-05-30T09:25:00.942798Z","shell.execute_reply":"2021-05-30T09:25:00.949059Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"base_model = tf.keras.applications.VGG16(input_shape=(q_size, q_size, img_channel),include_top=False,weights=\"imagenet\")","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:00.951284Z","iopub.execute_input":"2021-05-30T09:25:00.951723Z","iopub.status.idle":"2021-05-30T09:25:02.286942Z","shell.execute_reply.started":"2021-05-30T09:25:00.951693Z","shell.execute_reply":"2021-05-30T09:25:02.285953Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Freezing Layers\n\nfor layer in base_model.layers[:-20]:\n layer.trainable=False","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:02.288459Z","iopub.execute_input":"2021-05-30T09:25:02.288783Z","iopub.status.idle":"2021-05-30T09:25:02.293563Z","shell.execute_reply.started":"2021-05-30T09:25:02.288753Z","shell.execute_reply":"2021-05-30T09:25:02.292729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Building Model\nmodel=Sequential()\nmodel.add(base_model)\nmodel.add(Dropout(0.5))\n# Add new layers\nmodel.add(Flatten())\nmodel.add(Dense(4096 , activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(4096 , activation='relu'))\nmodel.add(Dropout(0.3))\nmodel.add(Dense(4096, activation='relu'))\nmodel.add(Dropout(0.4))\nmodel.add(Dense(num_classes, activation='softmax'))\n\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:02.294755Z","iopub.execute_input":"2021-05-30T09:25:02.295031Z","iopub.status.idle":"2021-05-30T09:25:04.628354Z","shell.execute_reply.started":"2021-05-30T09:25:02.295005Z","shell.execute_reply":"2021-05-30T09:25:04.627309Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.utils import plot_model\nfrom IPython.display import Image\nplot_model(model, to_file='convnet.png', show_shapes=True,show_layer_names=True)\nImage(filename='convnet.png') ","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:04.629414Z","iopub.execute_input":"2021-05-30T09:25:04.629755Z","iopub.status.idle":"2021-05-30T09:25:05.281904Z","shell.execute_reply.started":"2021-05-30T09:25:04.629695Z","shell.execute_reply":"2021-05-30T09:25:05.280889Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def f1_score(y_true, y_pred): #taken from old keras source code\n true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))\n possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))\n predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))\n precision = true_positives / (predicted_positives + K.epsilon())\n recall = true_positives / (possible_positives + K.epsilon())\n f1_val = 2*(precision*recall)/(precision+recall+K.epsilon())\n return f1_val","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:05.283349Z","iopub.execute_input":"2021-05-30T09:25:05.283678Z","iopub.status.idle":"2021-05-30T09:25:05.290942Z","shell.execute_reply.started":"2021-05-30T09:25:05.28365Z","shell.execute_reply":"2021-05-30T09:25:05.290034Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"METRICS = [\n tf.keras.metrics.BinaryAccuracy(name='accuracy'),\n tf.keras.metrics.Precision(name='precision'),\n tf.keras.metrics.Recall(name='recall'), \n tf.keras.metrics.AUC(name='auc'),\n f1_score,\n]","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:05.292255Z","iopub.execute_input":"2021-05-30T09:25:05.29255Z","iopub.status.idle":"2021-05-30T09:25:05.334238Z","shell.execute_reply.started":"2021-05-30T09:25:05.292507Z","shell.execute_reply":"2021-05-30T09:25:05.333311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"METRICS","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:05.335475Z","iopub.execute_input":"2021-05-30T09:25:05.335766Z","iopub.status.idle":"2021-05-30T09:25:05.341512Z","shell.execute_reply.started":"2021-05-30T09:25:05.335738Z","shell.execute_reply":"2021-05-30T09:25:05.340742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lrd = ReduceLROnPlateau(monitor = 'val_loss',patience = 2,verbose = 1,factor = 0.50, min_lr = 1e-4)\n\nmcp = ModelCheckpoint('model.h5')\n\nes = EarlyStopping(verbose=1, patience=2)","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:05.343252Z","iopub.execute_input":"2021-05-30T09:25:05.343556Z","iopub.status.idle":"2021-05-30T09:25:05.354251Z","shell.execute_reply.started":"2021-05-30T09:25:05.343512Z","shell.execute_reply":"2021-05-30T09:25:05.353184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.compile(optimizer=Adam(), loss='categorical_crossentropy',metrics=METRICS)","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:05.355673Z","iopub.execute_input":"2021-05-30T09:25:05.35595Z","iopub.status.idle":"2021-05-30T09:25:05.380159Z","shell.execute_reply.started":"2021-05-30T09:25:05.355924Z","shell.execute_reply":"2021-05-30T09:25:05.378932Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%time\nhistory = model.fit_generator(generator=train_generator,validation_data=val_generator,epochs = epochs,verbose = 1)","metadata":{"execution":{"iopub.status.busy":"2021-05-30T09:25:05.381556Z","iopub.execute_input":"2021-05-30T09:25:05.381874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#%% PLOTTING RESULTS (Train vs Validation FOLDER 1)\n\ndef Train_Val_Plot(acc,val_acc,loss,val_loss,auc,val_auc,precision,val_precision,f1,val_f1):\n \n fig, (ax1, ax2,ax3,ax4,ax5) = plt.subplots(1,5, figsize= (20,5))\n fig.suptitle(\" MODEL'S METRICS VISUALIZATION \")\n\n ax1.plot(range(1, len(acc) + 1), acc)\n ax1.plot(range(1, len(val_acc) + 1), val_acc)\n ax1.set_title('History of Accuracy')\n ax1.set_xlabel('Epochs')\n ax1.set_ylabel('Accuracy')\n ax1.legend(['training', 'validation'])\n\n\n ax2.plot(range(1, len(loss) + 1), loss)\n ax2.plot(range(1, len(val_loss) + 1), val_loss)\n ax2.set_title('History of Loss')\n ax2.set_xlabel('Epochs')\n ax2.set_ylabel('Loss')\n ax2.legend(['training', 'validation'])\n \n ax3.plot(range(1, len(auc) + 1), auc)\n ax3.plot(range(1, len(val_auc) + 1), val_auc)\n ax3.set_title('History of AUC')\n ax3.set_xlabel('Epochs')\n ax3.set_ylabel('AUC')\n ax3.legend(['training', 'validation'])\n \n ax4.plot(range(1, len(precision) + 1), precision)\n ax4.plot(range(1, len(val_precision) + 1), val_precision)\n ax4.set_title('History of Precision')\n ax4.set_xlabel('Epochs')\n ax4.set_ylabel('Precision')\n ax4.legend(['training', 'validation'])\n \n ax5.plot(range(1, len(f1) + 1), f1)\n ax5.plot(range(1, len(val_f1) + 1), val_f1)\n ax5.set_title('History of F1-score')\n ax5.set_xlabel('Epochs')\n ax5.set_ylabel('F1 score')\n ax5.legend(['training', 'validation'])\n\n\n plt.show()\n \n\nTrain_Val_Plot(history.history['accuracy'],history.history['val_accuracy'],\n history.history['loss'],history.history['val_loss'],\n history.history['auc'],history.history['val_auc'],\n history.history['precision'],history.history['val_precision'],\n history.history['f1_score'],history.history['val_f1_score']\n )","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predict = model.predict_generator(test_generator, verbose=1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predict.argmax(axis=1)\nsamp_subm.loc[0:len(df_test_data.index)-1, 'Predicted'] = predict.argmax(axis=1)[0:len(df_test_data.index)]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Sub = samp_subm.copy()\nSub.to_csv('submission.csv', index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}