{"cells":[{"metadata":{},"cell_type":"markdown","source":"\"Flowers\"","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"In this notebook i will experiment 6 most popular pre trained convolutional neural networks for identify the type of flowers in a dataset of images (for simplicity, we’re sticking to just over 100 types).\n\nI will made an assemblage using this 8 architectures by creating fair voting system between them, at the end every CNN architecture will return type of flower and i will considertate the choice of the majority.\n\nFair play =).\n\nand after that i will belend prediction probabilites for each CNN and see whitch assembling aproach works better.\n\nI will take advantage of the powerful Tensor Processing Units (TPUs) provided by Kaggle in cloud, it allows to greatly increase the learning speed and also it pushes the limits of RAM as what happens for gpu which allows to consider high resolution images.\n\ni will also do data augmentation on pictures by zoom, rotate, inverse, shift and share them. it decrease overfitting.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\n\nIn this notebook i will use transfert learning and evaluate the following pretrained CNN.\n\n* Xception\n* VGG16\n* DenseNet201\n* InceptionV3 \n* InceptionResNetV2\n\nI will use only these 5 models but there are others like: \n\n* ResNet152V2\n* InceptionResNetV2\n* VGG19\n* EfficientNetB1 to B7\n* DenseNet169 and 121\n* MobileNet\n\nAnd many others.\n\nI cannot use more than 6 for reasons of insufficient hardware resources.\n\nHowever, it is possible to swap the modes by changing their name in the dictionary that lists them : models\n\n
\n\nKeras reference","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\"Flowers\"","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Referances :**\n\nRotation Augmentation GPU/TPU - [0.96+]\n\nFlowers@TPU EfficientNet B7+B6+B5+B4\n\nA Simple Petals TF 2.2 notebook","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Solution Architecture","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\"Flowers\"","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Contents\n\n* [1. Libraries](#1)\n* [2. Configuration and Data Access](#2)\n* [3. Data Augmentation](#3)\n* [4. Pretrained Models Creation](#4)\n* [5. Transfer Learning and Prediction](#5)\n* [6. Models Performance](#6)\n * [Xception](#6.1)\n * [VGG16](#6.2)\n * [DenseNet201](#6.3)\n * [InceptionV3](#6.4)\n * [InceptionResNetV2](#6.6)\n* [7. Assembling and Submission](#7)\n* [8. Conclusion](#7)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 1. Libraries ","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import random, re, math\nimport numpy as np, pandas as pd\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import f1_score, precision_score, recall_score, confusion_matrix\nimport tensorflow as tf, tensorflow.keras.backend as K\nfrom kaggle_datasets import KaggleDatasets\nfrom IPython.display import Image\nfrom tensorflow.keras.utils import plot_model\nprint('Tensorflow version ' + tf.__version__)\nfrom sklearn.model_selection import KFold\nimport gc\nfrom scipy import stats\nimport gc\nfrom collections import Counter\nimport time\nimport seaborn as sns\nfrom sklearn.metrics import confusion_matrix\nimport shap\nimport matplotlib.image as mpimg\n\n!pip install -q efficientnet\n\nfrom tensorflow.keras.applications import Xception\nfrom tensorflow.keras.applications import VGG16\nfrom tensorflow.keras.applications import DenseNet201\nfrom tensorflow.keras.applications import InceptionV3\nfrom efficientnet.tfkeras import EfficientNetB7\nfrom tensorflow.keras.applications import ResNet152V2\nfrom tensorflow.keras.applications import MobileNetV2\nfrom tensorflow.keras.applications import InceptionResNetV2","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. Configuration and Data Access ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Detect hardware, return appropriate distribution strategy\ntry:\n tpu = tf.distribute.cluster_resolver.TPUClusterResolver() # TPU detection. No parameters necessary if TPU_NAME environment variable is set. On Kaggle this is always the case.\n print('Running on TPU ', tpu.master())\nexcept ValueError:\n tpu = None\n\nif tpu:\n tf.config.experimental_connect_to_cluster(tpu)\n tf.tpu.experimental.initialize_tpu_system(tpu)\n strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n strategy = tf.distribute.get_strategy() # default distribution strategy in Tensorflow. Works on CPU and single GPU.\n\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"AUTO = tf.data.experimental.AUTOTUNE\n\n# Configuration\nIMAGE_SIZE = [224, 224]\nEPOCHS = 30\nFOLDS = 3\nSEED = 777\nBATCH_SIZE = 16 * strategy.num_replicas_in_sync","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"MIXED_PRECISION = False\nXLA_ACCELERATE = False\n\nif MIXED_PRECISION:\n from tensorflow.keras.mixed_precision import experimental as mixed_precision\n if tpu: policy = tf.keras.mixed_precision.experimental.Policy('mixed_bfloat16')\n else: policy = tf.keras.mixed_precision.experimental.Policy('mixed_float16')\n mixed_precision.set_policy(policy)\n print('Mixed precision enabled')\n\nif XLA_ACCELERATE:\n tf.config.optimizer.set_jit(True)\n print('Accelerated Linear Algebra enabled')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Data access\nGCS_DS_PATH = KaggleDatasets().get_gcs_path('tpu-getting-started')\n\nGCS_PATH_SELECT = { # available image sizes\n 192: GCS_DS_PATH + '/tfrecords-jpeg-192x192',\n 224: GCS_DS_PATH + '/tfrecords-jpeg-224x224',\n 331: GCS_DS_PATH + '/tfrecords-jpeg-331x331',\n 512: GCS_DS_PATH + '/tfrecords-jpeg-512x512'\n}\n\nGCS_PATH = GCS_PATH_SELECT[IMAGE_SIZE[0]]\n\nTRAINING_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec') + tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\n\nTRAINING_FILENAMES_ONLY = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec')\nVAL_FILENAMES_ONLY = tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\n\nTEST_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/test/*.tfrec') # predictions on this dataset should be submitted for the competition\n\nCLASSES = ['pink primrose', 'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea', 'wild geranium', 'tiger lily', 'moon orchid', 'bird of paradise', 'monkshood', 'globe thistle', # 00 - 09\n 'snapdragon', \"colt's foot\", 'king protea', 'spear thistle', 'yellow iris', 'globe-flower', 'purple coneflower', 'peruvian lily', 'balloon flower', 'giant white arum lily', # 10 - 19\n 'fire lily', 'pincushion flower', 'fritillary', 'red ginger', 'grape hyacinth', 'corn poppy', 'prince of wales feathers', 'stemless gentian', 'artichoke', 'sweet william', # 20 - 29\n 'carnation', 'garden phlox', 'love in the mist', 'cosmos', 'alpine sea holly', 'ruby-lipped cattleya', 'cape flower', 'great masterwort', 'siam tulip', 'lenten rose', # 30 - 39\n 'barberton daisy', 'daffodil', 'sword lily', 'poinsettia', 'bolero deep blue', 'wallflower', 'marigold', 'buttercup', 'daisy', 'common dandelion', # 40 - 49\n 'petunia', 'wild pansy', 'primula', 'sunflower', 'lilac hibiscus', 'bishop of llandaff', 'gaura', 'geranium', 'orange dahlia', 'pink-yellow dahlia', # 50 - 59\n 'cautleya spicata', 'japanese anemone', 'black-eyed susan', 'silverbush', 'californian poppy', 'osteospermum', 'spring crocus', 'iris', 'windflower', 'tree poppy', # 60 - 69\n 'gazania', 'azalea', 'water lily', 'rose', 'thorn apple', 'morning glory', 'passion flower', 'lotus', 'toad lily', 'anthurium', # 70 - 79\n 'frangipani', 'clematis', 'hibiscus', 'columbine', 'desert-rose', 'tree mallow', 'magnolia', 'cyclamen ', 'watercress', 'canna lily', # 80 - 89\n 'hippeastrum ', 'bee balm', 'pink quill', 'foxglove', 'bougainvillea', 'camellia', 'mallow', 'mexican petunia', 'bromelia', 'blanket flower', # 90 - 99\n 'trumpet creeper', 'blackberry lily', 'common tulip', 'wild rose'] \n\n\n# Learning rate schedule for TPU, GPU and CPU.\n# Using an LR ramp up because fine-tuning a pre-trained model.\n# Starting with a high LR would break the pre-trained weights.\n\nLR_START = 0.00001\nLR_MAX = 0.00005 * strategy.num_replicas_in_sync\nLR_MIN = 0.00001\nLR_RAMPUP_EPOCHS = 5\nLR_SUSTAIN_EPOCHS = 0\nLR_EXP_DECAY = .8\n\n \ndef scheduler(epoch):\n if epoch < 4:\n return 0.0005\n elif epoch < 8:\n return 0.0002\n elif epoch < 12:\n return 0.0001\n elif epoch < 16:\n return 0.00005\n elif epoch < 20:\n return 0.00002\n else:\n return 0.00001\n \nlr_callback = tf.keras.callbacks.LearningRateScheduler(scheduler, verbose = True)\n\nrng = [i for i in range(25 if EPOCHS<25 else EPOCHS)]\ny = [scheduler(x) for x in rng]\nplt.plot(rng, y)\nprint(\"Learning rate schedule: {:.3g} to {:.3g} to {:.3g}\".format(y[0], max(y), y[-1]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def decode_image(image_data):\n image = tf.image.decode_jpeg(image_data, channels=3)\n image = tf.cast(image, tf.float32) / 255.0 # convert image to floats in [0, 1] range\n image = tf.reshape(image, [*IMAGE_SIZE, 3]) # explicit size needed for TPU\n return image\n\ndef read_labeled_tfrecord(example):\n LABELED_TFREC_FORMAT = {\n \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n \"class\": tf.io.FixedLenFeature([], tf.int64), # shape [] means single element\n }\n example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n image = decode_image(example['image'])\n label = tf.cast(example['class'], tf.int32)\n return image, label # returns a dataset of (image, label) pairs\n\ndef read_unlabeled_tfrecord(example):\n UNLABELED_TFREC_FORMAT = {\n \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n \"id\": tf.io.FixedLenFeature([], tf.string), # shape [] means single element\n # class is missing, this competitions's challenge is to predict flower classes for the test dataset\n }\n example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n image = decode_image(example['image'])\n idnum = example['id']\n return image, idnum # returns a dataset of image(s)\n\ndef load_dataset(filenames, labeled = True, ordered = False):\n # Read from TFRecords. For optimal performance, reading from multiple files at once and\n # Diregarding data order. Order does not matter since we will be shuffling the data anyway\n \n ignore_order = tf.data.Options()\n if not ordered:\n ignore_order.experimental_deterministic = False # disable order, increase speed\n \n dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads = AUTO) # automatically interleaves reads from multiple files\n dataset = dataset.with_options(ignore_order) # use data as soon as it streams in, rather than in its original order\n dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord, num_parallel_calls = AUTO) # returns a dataset of (image, label) pairs if labeled = True or (image, id) pair if labeld = False\n return dataset\n\ndef data_augment(image, label):\n # data augmentation. Thanks to the dataset.prefetch(AUTO) statement in the next function (below),\n # this happens essentially for free on TPU. Data pipeline code is executed on the \"CPU\" part\n # of the TPU while the TPU itself is computing gradients.\n image = tf.image.random_flip_left_right(image)\n return image, label \n\ndef get_training_dataset(dataset,do_aug=True):\n dataset = dataset.map(data_augment, num_parallel_calls=AUTO)\n if do_aug: dataset = dataset.map(transform, num_parallel_calls=AUTO)\n dataset = dataset.repeat() # the training dataset must repeat for several epochs\n dataset = dataset.shuffle(2048)\n dataset = dataset.batch(BATCH_SIZE)\n dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n return dataset\n\ndef get_validation_dataset(dataset):\n dataset = dataset.batch(BATCH_SIZE)\n dataset = dataset.cache()\n dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n return dataset\n\ndef get_test_dataset(ordered=False):\n dataset = load_dataset(TEST_FILENAMES, labeled=False, ordered=ordered)\n dataset = dataset.batch(BATCH_SIZE)\n dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n return dataset\n\ndef count_data_items(filenames):\n # the number of data items is written in the name of the .tfrec files, i.e. flowers00-230.tfrec = 230 data items\n n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) for filename in filenames]\n return np.sum(n)\n\nNUM_TRAINING_IMAGES = int( count_data_items(TRAINING_FILENAMES) * (FOLDS-1.)/FOLDS )\nNUM_VALIDATION_IMAGES = int( count_data_items(TRAINING_FILENAMES) * (1./FOLDS) )\nNUM_TEST_IMAGES = count_data_items(TEST_FILENAMES)\nSTEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\n\nprint('Dataset: {} training images, {} validation images, {} unlabeled test images'.format(NUM_TRAINING_IMAGES, NUM_VALIDATION_IMAGES, NUM_TEST_IMAGES))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 3. Data Augmentation \n\nBig Thanks to Chris Deotte for this\n\nRotation Augmentation GPU/TPU - [0.96+]","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def get_mat(rotation, shear, height_zoom, width_zoom, height_shift, width_shift):\n # returns 3x3 transformmatrix which transforms indicies\n \n # CONVERT DEGREES TO RADIANS\n rotation = math.pi * rotation / 180.\n shear = math.pi * shear / 180.\n \n # ROTATION MATRIX\n c1 = tf.math.cos(rotation)\n s1 = tf.math.sin(rotation)\n one = tf.constant([1],dtype='float32')\n zero = tf.constant([0],dtype='float32')\n rotation_matrix = tf.reshape( tf.concat([c1,s1,zero, -s1,c1,zero, zero,zero,one],axis=0),[3,3] )\n \n # SHEAR MATRIX\n c2 = tf.math.cos(shear)\n s2 = tf.math.sin(shear)\n shear_matrix = tf.reshape( tf.concat([one,s2,zero, zero,c2,zero, zero,zero,one],axis=0),[3,3] ) \n \n # ZOOM MATRIX\n zoom_matrix = tf.reshape( tf.concat([one/height_zoom,zero,zero, zero,one/width_zoom,zero, zero,zero,one],axis=0),[3,3] )\n \n # SHIFT MATRIX\n shift_matrix = tf.reshape( tf.concat([one,zero,height_shift, zero,one,width_shift, zero,zero,one],axis=0),[3,3] )\n \n return K.dot(K.dot(rotation_matrix, shear_matrix), K.dot(zoom_matrix, shift_matrix))\n\ndef transform(image,label):\n # input image - is one image of size [dim,dim,3] not a batch of [b,dim,dim,3]\n # output - image randomly rotated, sheared, zoomed, and shifted\n DIM = IMAGE_SIZE[0]\n XDIM = DIM%2 #fix for size 331\n \n rot = 15. * tf.random.normal([1],dtype='float32')\n shr = 5. * tf.random.normal([1],dtype='float32') \n h_zoom = 1.0 + tf.random.normal([1],dtype='float32')/10.\n w_zoom = 1.0 + tf.random.normal([1],dtype='float32')/10.\n h_shift = 16. * tf.random.normal([1],dtype='float32') \n w_shift = 16. * tf.random.normal([1],dtype='float32') \n \n # GET TRANSFORMATION MATRIX\n m = get_mat(rot,shr,h_zoom,w_zoom,h_shift,w_shift) \n\n # LIST DESTINATION PIXEL INDICES\n x = tf.repeat( tf.range(DIM//2,-DIM//2,-1), DIM )\n y = tf.tile( tf.range(-DIM//2,DIM//2),[DIM] )\n z = tf.ones([DIM*DIM],dtype='int32')\n idx = tf.stack( [x,y,z] )\n \n # ROTATE DESTINATION PIXELS ONTO ORIGIN PIXELS\n idx2 = K.dot(m,tf.cast(idx,dtype='float32'))\n idx2 = K.cast(idx2,dtype='int32')\n idx2 = K.clip(idx2,-DIM//2+XDIM+1,DIM//2)\n \n # FIND ORIGIN PIXEL VALUES \n idx3 = tf.stack( [DIM//2-idx2[0,], DIM//2-1+idx2[1,]] )\n d = tf.gather_nd(image,tf.transpose(idx3))\n \n return tf.reshape(d,[DIM,DIM,3]),label\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":false},"cell_type":"code","source":"row = 3; col = 4;\nall_elements = get_training_dataset(load_dataset(TRAINING_FILENAMES),do_aug=False).unbatch()\none_element = tf.data.Dataset.from_tensors( next(iter(all_elements)) )\naugmented_element = one_element.repeat().map(transform).batch(row*col)\n\nfor (img,label) in augmented_element:\n plt.figure(figsize=(15,int(15*row/col)))\n for j in range(row*col):\n plt.subplot(row,col,j+1)\n plt.axis('off')\n plt.imshow(img[j,])\n plt.show()\n break","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"row = 3; col = 4;\nall_elements = get_training_dataset(load_dataset(TRAINING_FILENAMES),do_aug=False).unbatch()\none_element = tf.data.Dataset.from_tensors( next(iter(all_elements)) )\naugmented_element = one_element.repeat().map(transform).batch(row*col)\n\nfor (img,label) in augmented_element:\n plt.figure(figsize=(15,int(15*row/col)))\n for j in range(row*col):\n plt.subplot(row,col,j+1)\n plt.axis('off')\n plt.imshow(img[j,])\n plt.show()\n break","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"row = 3; col = 4;\nall_elements = get_training_dataset(load_dataset(TRAINING_FILENAMES),do_aug=False).unbatch()\none_element = tf.data.Dataset.from_tensors( next(iter(all_elements)) )\naugmented_element = one_element.repeat().map(transform).batch(row*col)\n\nfor (img,label) in augmented_element:\n plt.figure(figsize=(15,int(15*row/col)))\n for j in range(row*col):\n plt.subplot(row,col,j+1)\n plt.axis('off')\n plt.imshow(img[j,])\n plt.show()\n break","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 4. Pretrained Models Creation ","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Create Test, TRain and validation Data\n\ntrain_dataset_all = load_dataset(list(pd.DataFrame({'TRAINING_FILENAMES': TRAINING_FILENAMES})['TRAINING_FILENAMES']), labeled = True)\ntest_dataset_all = load_dataset(list(pd.DataFrame({'TEST_FILENAMES': TEST_FILENAMES})['TEST_FILENAMES']), labeled = True, ordered=True)\n\ntrain_dataset = load_dataset(list(pd.DataFrame({'TRAINING_FILENAMES_ONLY': TRAINING_FILENAMES_ONLY})['TRAINING_FILENAMES_ONLY']), labeled = True)\nval_dataset = load_dataset(list(pd.DataFrame({'VAL_FILENAMES_ONLY': VAL_FILENAMES_ONLY})['VAL_FILENAMES_ONLY']), labeled=True, ordered=True)\n\n\ntrain_all = get_training_dataset(train_dataset_all) #Both validation and train concatenated for final fitting\ntest = get_test_dataset(test_dataset_all)\ntest_images_ds = test.map(lambda image, idnum: image)\n\ntrain = get_training_dataset(train_dataset)\nval = get_validation_dataset(val_dataset)\n\ndef Xception_model():\n with strategy.scope():\n rnet = Xception(\n input_shape=(IMAGE_SIZE[0], IMAGE_SIZE[1], 3),\n weights='imagenet',\n include_top=False\n )\n # trainable rnet\n rnet.trainable = True\n model = tf.keras.Sequential([\n rnet,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax',dtype='float32')\n ])\n model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n return model\n\ndef VGG16_model():\n with strategy.scope():\n rnet = VGG16(\n input_shape=(IMAGE_SIZE[0], IMAGE_SIZE[1], 3),\n weights='imagenet',\n include_top=False\n )\n # trainable rnet\n rnet.trainable = True\n model = tf.keras.Sequential([\n rnet,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax',dtype='float32')\n ])\n model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n return model\n\ndef DenseNet201_model():\n with strategy.scope():\n rnet = DenseNet201(\n input_shape=(IMAGE_SIZE[0], IMAGE_SIZE[1], 3),\n weights='imagenet',\n include_top=False\n )\n # trainable rnet\n rnet.trainable = True\n model = tf.keras.Sequential([\n rnet,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax',dtype='float32')\n ])\n model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n return model\n\ndef InceptionV3_model():\n with strategy.scope():\n rnet = InceptionV3(\n input_shape=(IMAGE_SIZE[0], IMAGE_SIZE[1], 3),\n weights='imagenet',\n include_top=False\n )\n # trainable rnet\n rnet.trainable = True\n model = tf.keras.Sequential([\n rnet,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dropout(0.2),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax',dtype='float32')\n ])\n model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n return model\n\ndef EfficientNetB7_model():\n with strategy.scope():\n rnet = EfficientNetB7(\n input_shape=(IMAGE_SIZE[0], IMAGE_SIZE[1], 3),\n weights='imagenet',\n include_top=False\n )\n # trainable rnet\n rnet.trainable = True\n model = tf.keras.Sequential([\n rnet,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dropout(0.2),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax',dtype='float32')\n ])\n model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n return model\n\ndef ResNet152V2_model():\n with strategy.scope():\n rnet = ResNet152V2(\n input_shape=(IMAGE_SIZE[0], IMAGE_SIZE[1], 3),\n weights='imagenet',\n include_top=False\n )\n # trainable rnet\n rnet.trainable = True\n model = tf.keras.Sequential([\n rnet,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dropout(0.2),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax',dtype='float32')\n ])\n model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n return model\n\ndef MobileNetV2_model():\n with strategy.scope():\n rnet = MobileNetV2(\n input_shape=(IMAGE_SIZE[0], IMAGE_SIZE[1], 3),\n weights='imagenet',\n include_top=False\n )\n # trainable rnet\n rnet.trainable = True\n model = tf.keras.Sequential([\n rnet,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dropout(0.2),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax',dtype='float32')\n ])\n model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n return model\n\ndef InceptionResNetV2_model():\n with strategy.scope():\n rnet = InceptionResNetV2(\n input_shape=(IMAGE_SIZE[0], IMAGE_SIZE[1], 3),\n weights='imagenet',\n include_top=False\n )\n # trainable rnet\n rnet.trainable = True\n model = tf.keras.Sequential([\n rnet,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dropout(0.2),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax',dtype='float32')\n ])\n model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n return model\n\nmodels = {'Xception' : Xception_model,\n 'VGG16' : VGG16_model,\n 'DenseNet201' : DenseNet201_model,\n 'InceptionV3' : InceptionV3_model,\n# 'EfficientNetB7' : EfficientNetB7_model, \n# 'ResNet152V2' : ResNet152V2_model, \n# 'MobileNetV2' : MobileNetV2_model, \n 'InceptionResNetV2' : InceptionResNetV2_model\n }\nhistorys = {}\npredictions = {}\npredictions_val = {}\npredictions_prob = {}\ntimes = {}\n\nMODELS_NUMBER = 5 #By RAM constraints i took only 5 models\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 5. Transfer Learning ","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"for name, model_ in models.items() :\n print('Running ' + name)\n model = model_()\n plot_model(model, to_file= name+'.png', show_shapes=True)\n \n history = model.fit(\n train, \n steps_per_epoch = STEPS_PER_EPOCH,\n epochs = EPOCHS,\n callbacks = [lr_callback],#, early_stopping],\n validation_data = val,\n verbose = 3\n )\n\n historys[name] = history # Save historys\n predictions_val[name] = np.argmax(model.predict(val), axis=-1)\n# Train on Train and validation Data for prediction\n del model\n gc.collect()\n \n model = model_()\n start_time = time.time()\n history = model.fit(\n train_all, \n steps_per_epoch = STEPS_PER_EPOCH,\n epochs = EPOCHS,\n callbacks = [lr_callback],#, early_stopping],\n verbose = 2\n )\n times[name] = \"--- %s seconds ---\" % (time.time() - start_time) \n predictions_prob[name] = model.predict_proba(test_images_ds)\n predictions[name] = np.argmax(model.predict(test_images_ds), axis=-1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 6. Evaluation and Performance Check ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 6.1 Xception ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\n\ninterpretation of Inception modules in convolutional neural networks as being an intermediate step in-between regular convolution and the depthwise separable convolution operation (a depthwise convolution followed by a pointwise convolution). In this light, a depthwise separable convolution can be understood as an Inception module with a maximally large number of towers. This observation leads us to propose a novel deep convolutional neural network architecture inspired by Inception, where Inception modules have been replaced with depthwise separable convolutions. We show that this architecture, dubbed Xception, slightly outperforms Inception V3 on the ImageNet dataset (which Inception V3 was designed for), and significantly outperforms Inception V3 on a larger image classification dataset comprising 350 million images and 17,000 classes. Since the Xception architecture has the same number of parameters as Inception V3, the performance gains are not due to increased capacity but rather to a more efficient use of model parameters.\n\nXception: Deep Learning with Depthwise Separable Convolutions\n\n
","execution_count":null},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def plot_perf(name) :\n \n print('{} Training took {}'.format(name, times[name]))\n \n history = historys[name]\n\n fig = plt.figure(figsize = [30,5])\n\n plt.subplot(1, 3, 1)\n\n img=mpimg.imread(name+'.png')\n plt.title(name + ' Architecture')\n plt.imshow(img)\n\n plt.subplot(1, 3, 2)\n\n plt.plot(history.history['sparse_categorical_accuracy'])\n plt.plot(history.history['val_sparse_categorical_accuracy'])\n plt.title(name + ' accuracy')\n plt.ylabel('accuracy')\n plt.xlabel('epoch')\n plt.legend(['train', 'test'], loc='upper left')\n\n plt.subplot(1, 3, 3)\n\n plt.plot(history.history['loss'])\n plt.plot(history.history['val_loss'])\n plt.title(name + ' loss')\n plt.ylabel('loss')\n plt.xlabel('epoch')\n plt.legend(['train', 'test'], loc='upper left')\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_perf('Xception')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# tf.data.Dataset.from_tensor_slices(list(val))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def display_confusion_matrix(cmat, score, precision, recall, name):\n plt.figure(figsize=(45,45))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = name + \" with noisy-student weights\"\n if score is not None:\n titlestring += '\\n f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\n precision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\n recall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 30, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n\ncmdataset = val\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy() \n# cm_probabilities = model1.predict(images_ds)\ncm_predictions = predictions_val['Xception']\ncmat = confusion_matrix(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)))\nscore = f1_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nprecision = precision_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nrecall = recall_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\ncmat = (cmat.T / cmat.sum(axis=1)).T \ndisplay_confusion_matrix(cmat, score, precision, recall, 'Xception')\nprint('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Xception took about 4 minutes for 30 epochs.\n* Relatively large difference in performance between train data and validation. is it mini a overfitting ?\n* Good performances f1 score: 0.921, precision: 0.929, recall: 0.919\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 6.2 VGG16 ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"VGG16 is a convolutional neural network model proposed by K. Simonyan and A. Zisserman from the University of Oxford in the paper “Very Deep Convolutional Networks for Large-Scale Image Recognition”. The model achieves 92.7% top-5 test accuracy in ImageNet, which is a dataset of over 14 million images belonging to 1000 classes. It was one of the famous model submitted to ILSVRC-2014. It makes the improvement over AlexNet by replacing large kernel-sized filters (11 and 5 in the first and second convolutional layer, respectively) with multiple 3×3 kernel-sized filters one after another. VGG16 was trained for weeks and was using NVIDIA Titan Black GPU’s.\n\nVery Deep Convolutional Networks for Large-Scale Image Recognition\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_perf('VGG16')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def display_confusion_matrix(cmat, score, precision, recall, name):\n plt.figure(figsize=(45,45))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = name + \" with noisy-student weights\"\n if score is not None:\n titlestring += '\\n f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\n precision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\n recall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 30, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n\ncmdataset = val\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy() \n# cm_probabilities = model1.predict(images_ds)\ncm_predictions = predictions_val['VGG16']\ncmat = confusion_matrix(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)))\nscore = f1_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nprecision = precision_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nrecall = recall_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\ncmat = (cmat.T / cmat.sum(axis=1)).T \ndisplay_confusion_matrix(cmat, score, precision, recall, 'VGG16')\nprint('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* VGG16 took about 3 minute.\n* Slow gradient descent maybe biger LR would be better.\n* relativly bad performances : f1 score: 0.563, precision: 0.614, recall: 0.556","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 6.3 DenseNet201 ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Recent work has shown that convolutional networks can be substantially\ndeeper, more accurate, and efficient to train if they contain shorter\nconnections between layers close to the input and those close to the output. In\nthis paper, we embrace this observation and introduce the Dense Convolutional\nNetwork (DenseNet), which connects each layer to every other layer in a\nfeed-forward fashion. Whereas traditional convolutional networks with L layers\nhave L connections - one between each layer and its subsequent layer - our\nnetwork has L(L+1)/2 direct connections. For each layer, the feature-maps of\nall preceding layers are used as inputs, and its own feature-maps are used as\ninputs into all subsequent layers. DenseNets have several compelling\nadvantages: they alleviate the vanishing-gradient problem, strengthen feature\npropagation, encourage feature reuse, and substantially reduce the number of\nparameters. We evaluate our proposed architecture on four highly competitive\nobject recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet).\nDenseNets obtain significant improvements over the state-of-the-art on most of\nthem, whilst requiring less computation to achieve high performance. Code and\npre-trained models are available at this https URL .\n\nDensely Connected Convolutional Networks\n\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_perf('DenseNet201')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def display_confusion_matrix(cmat, score, precision, recall, name):\n plt.figure(figsize=(45,45))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = name + \" with noisy-student weights\"\n if score is not None:\n titlestring += '\\n f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\n precision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\n recall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 30, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n\ncmdataset = val\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy() \n# cm_probabilities = model1.predict(images_ds)\ncm_predictions = predictions_val['DenseNet201']\ncmat = confusion_matrix(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)))\nscore = f1_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nprecision = precision_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nrecall = recall_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\ncmat = (cmat.T / cmat.sum(axis=1)).T \ndisplay_confusion_matrix(cmat, score, precision, recall, 'DenseNet201')\nprint('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* DenseNet201 took about 11 minutes which is relatively long.\n* Seems to be a disturbance at first and it is stabilized but there is a gap between the test and is the same as Xception.\n* Good performances : f1 score: 0.936, precision: 0.940, recall: 0.936","execution_count":null},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## 6.4 InceptionV3 ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Convolutional networks are at the core of most state-of-the-art computer vision solutions for a wide variety of tasks. Since 2014 very deep convolutional networks started to become mainstream, yielding substantial gains in various benchmarks. Although increased model size and computational cost tend to translate to immediate quality gains for most tasks (as long as enough labeled data is provided for training), computational efficiency and low parameter count are still enabling factors for various use cases such as mobile vision and big-data scenarios. Here we explore ways to scale up networks in ways that aim at utilizing the added computation as efficiently as possible by suitably factorized convolutions and aggressive regularization. We benchmark our methods on the ILSVRC 2012 classification challenge validation set demonstrate substantial gains over the state of the art: 21.2% top-1 and 5.6% top-5 error for single frame evaluation using a network with a computational cost of 5 billion multiply-adds per inference and with using less than 25 million parameters. With an ensemble of 4 models and multi-crop evaluation, we report 3.5% top-5 error on the validation set (3.6% error on the test set) and 17.3% top-1 error \non the validation set.\n\nRethinking the Inception Architecture for Computer Vision\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_perf('InceptionV3')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def display_confusion_matrix(cmat, score, precision, recall, name):\n plt.figure(figsize=(45,45))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = name + \" with noisy-student weights\"\n if score is not None:\n titlestring += '\\n f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\n precision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\n recall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 30, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n\ncmdataset = val\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy() \n# cm_probabilities = model1.predict(images_ds)\ncm_predictions = predictions_val['InceptionV3']\ncmat = confusion_matrix(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)))\nscore = f1_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nprecision = precision_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nrecall = recall_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\ncmat = (cmat.T / cmat.sum(axis=1)).T \ndisplay_confusion_matrix(cmat, score, precision, recall, 'InceptionV3')\nprint('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* InceptionV3 took about 5.38.\n* Good progression of the accuracy and loss but there is a gap between the test and val.\n* good performances : f1 score: 0.903, precision: 0.912, recall: 0.903","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## 6.5 InceptionResNetV2 ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Very deep convolutional networks have been central to the largest advances in image recognition performance in recent years. One example is the Inception architecture that has been shown to achieve very good performance at relatively low computational cost. Recently, the introduction of residual connections in conjunction with a more traditional architecture has yielded state-of-the-art performance in the 2015 ILSVRC challenge; its performance was similar to the latest generation Inception-v3 network. This raises the question of whether there are any benefit in combining the Inception architecture with residual connections. Here we give clear empirical evidence that training with residual connections accelerates the training of Inception networks significantly. There is also some evidence of residual Inception networks outperforming similarly expensive Inception networks without residual connections by a thin margin. We also present several new streamlined architectures for both residual and non-residual Inception networks. These variations improve the single-frame recognition performance on the ILSVRC 2012 classification task significantly. We further demonstrate how proper activation scaling stabilizes the training of very wide residual Inception networks. With an ensemble of three residual and one Inception-v4, we achieve 3.08 percent top-5 error on the test set of the ImageNet classification (CLS) challenge.\n\nInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_perf('InceptionResNetV2')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def display_confusion_matrix(cmat, score, precision, recall, name):\n plt.figure(figsize=(45,45))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 18})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = name + \" with noisy-student weights\"\n if score is not None:\n titlestring += '\\n f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\n precision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\n recall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 30, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n\ncmdataset = val\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy() \n# cm_probabilities = model1.predict(images_ds)\ncm_predictions = predictions_val['InceptionResNetV2']\ncmat = confusion_matrix(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)))\nscore = f1_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nprecision = precision_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\nrecall = recall_score(cm_correct_labels, cm_predictions, labels=range(len(CLASSES)), average='macro')\ncmat = (cmat.T / cmat.sum(axis=1)).T \ndisplay_confusion_matrix(cmat, score, precision, recall, 'InceptionResNetV2')\nprint('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* ResNet152V2 took about 10 minutes.\n* Good progression of accuracy and loss, maybe the best.\n* Best performances : f1 score: 0.927, precision: 0.934, recall: 0.925","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# 7. Assembling and Submission ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df = pd.DataFrame(predictions)\npred = []\nfor i in range(0, 7382) :\n if df.loc[i,:].unique().shape[0] < MODELS_NUMBER :\n pred.append(stats.mode(df.loc[i,:].values)[0][0])\n else :\n pred.append(df.loc[i,'InceptionResNetV2'])\n \ndf = pd.DataFrame(predictions_val)\npred_val = []\nfor i in range(0, 3712) :\n if df.loc[i,:].unique().shape[0] < MODELS_NUMBER :\n pred_val.append(stats.mode(df.loc[i,:].values)[0][0])\n else :\n pred_val.append(df.loc[i,'InceptionResNetV2'])\n\navg_prob = predictions_prob['Xception'] + predictions_prob['VGG16'] + predictions_prob['DenseNet201'] + predictions_prob['InceptionV3'] + predictions_prob['InceptionResNetV2']\npred_avg = pd.DataFrame(np.argmax(avg_prob, axis=-1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_ids_ds = test.map(lambda image, idnum: idnum).unbatch()\ntest_ids = next(iter(test_ids_ds.batch(NUM_TEST_IMAGES))).numpy().astype('U') # all in one batch\nnp.savetxt('submission_vote.csv', np.rec.fromarrays([test_ids, pred]), fmt=['%s', '%d'], delimiter=',', header='id,label', comments='')\nnp.savetxt('submission.csv', np.rec.fromarrays([test_ids, np.argmax(avg_prob, axis=-1)]), fmt=['%s', '%d'], delimiter=',', header='id,label', comments='')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 8. Conclusion ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We can not extrapolate on what has just happened because it is related to the problematic, each model has its proliferation medium they can perform well on a set of images as it can not perform on another.\n\nAnd also I want to clarify that I used the same hyperparameters for each model, if we change them that is likely to change.\n\nHowever, this gives us an overall idea of the performance of each and personally I chose InceptionResNetV2 it is good.\n\nUpvote if u like it =) Bye.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\"Flowers\"","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}