\nThe other 3 models uses:
\nS2 - EfficientNetB7
\nS3 - DenseNet201
\nS4 - DenseNet201
\n
\nFor training apart from the default competition training set, this model uses additional data from the [5 image source provided by Kirill Blinov](https://www.kaggle.com/kirillblinov/tf-flower-photo-tfrec). For validation of the model, the default competition validation set is used.
\nThe image size used is [224, 224]
\nLR_MAX used for this kernel is 0.00005 * strategy.num_replicas_in_sync
\nLR_EXP_DECAY used for this kernel is 0.80
\nEarly stopping is used while monitoring for val_loss.
\nNumber of Epochs is set to a maximum of 50.
\nResults obtained from each of these models are saved and reused during ensembling. The values stored include validation probabilities and test probabilities derived from the model predictions.
"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -q efficientnet","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"import math, re, gc\nimport numpy as np # linear algebra\nimport pickle\nfrom datetime import datetime, timedelta\nimport tensorflow as tf\nimport efficientnet.tfkeras as efficientnet\nfrom matplotlib import pyplot as plt\nfrom kaggle_datasets import KaggleDatasets\nfrom sklearn.metrics import f1_score, precision_score, recall_score, confusion_matrix\nprint('TensorFlow version', tf.__version__)\nAUTO = tf.data.experimental.AUTOTUNE","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Initial Setup"},{"metadata":{"trusted":true},"cell_type":"code","source":"try:\n tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n print('Running on TPU', tpu.master())\nexcept ValueError:\n tpu = None\n\nif tpu:\n tf.config.experimental_connect_to_cluster(tpu)\n tf.tpu.experimental.initialize_tpu_system(tpu)\n strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n strategy = tf.distribute.get_strategy()\n\nprint('Replicas:', strategy.num_replicas_in_sync)\n\nGCS_DS_PATH = KaggleDatasets().get_gcs_path('flower-classification-with-tpus')\nMORE_IMAGES_GCS_DS_PATH = This is no longer permitted and this line is intended to fail compilation.\nprint(GCS_DS_PATH, '\\n', MORE_IMAGES_GCS_DS_PATH)\n#!ls -l /kaggle/input/tf-flower-photo-tfrec/*/tfrecords-jpeg-224x224/*.tfrec\n#!ls -l /kaggle/input/tf-flower-photo-tfrec/imagenet/tfrecords-jpeg-224x224/*.tfrec\n#!ls -l /kaggle/input/tf-flower-photo-tfrec/inaturalist/tfrecords-jpeg-224x224/*.tfrec\n#!ls -l /kaggle/input/tf-flower-photo-tfrec/openimage/tfrecords-jpeg-224x224/*.tfrec\n#!ls -l /kaggle/input/tf-flower-photo-tfrec/oxford_102/tfrecords-jpeg-224x224/*.tfrec\n#!ls -l /kaggle/input/tf-flower-photo-tfrec/tf_flowers/tfrecords-jpeg-224x224/*.tfrec","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"start_time = datetime.now()\nprint('Time now is', start_time)\nend_training_by_tdelta = timedelta(seconds=8400)\nthis_run_file_prefix = start_time.strftime('%Y%m%d_%H%M_')\nprint(this_run_file_prefix)\n\nIMAGE_SIZE = [224, 224] # [512, 512]\n\nEPOCHS = 12\nBATCH_SIZE = 16 * strategy.num_replicas_in_sync\n\nGCS_PATH_SELECT = {\n 192: GCS_DS_PATH + '/tfrecords-jpeg-192x192',\n 224: GCS_DS_PATH + '/tfrecords-jpeg-224x224',\n 331: GCS_DS_PATH + '/tfrecords-jpeg-331x331',\n 512: GCS_DS_PATH + '/tfrecords-jpeg-512x512'\n}\nGCS_PATH = GCS_PATH_SELECT[IMAGE_SIZE[0]]\n\nTRAINING_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec')\nVALIDATION_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\nTEST_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/test/*.tfrec')\n\nMOREIMAGES_PATH_SELECT = {\n 192: '/tfrecords-jpeg-192x192',\n 224: '/tfrecords-jpeg-224x224',\n 331: '/tfrecords-jpeg-331x331',\n 512: '/tfrecords-jpeg-512x512'\n}\nMOREIMAGES_PATH = MOREIMAGES_PATH_SELECT[IMAGE_SIZE[0]]\n\nIMAGENET_FILES = tf.io.gfile.glob(MORE_IMAGES_GCS_DS_PATH + '/imagenet' + MOREIMAGES_PATH + '/*.tfrec')\nINATURELIST_FILES = tf.io.gfile.glob(MORE_IMAGES_GCS_DS_PATH + '/inaturalist' + MOREIMAGES_PATH + '/*.tfrec')\nOPENIMAGE_FILES = tf.io.gfile.glob(MORE_IMAGES_GCS_DS_PATH + '/openimage' + MOREIMAGES_PATH + '/*.tfrec')\nOXFORD_FILES = tf.io.gfile.glob(MORE_IMAGES_GCS_DS_PATH + '/oxford_102' + MOREIMAGES_PATH + '/*.tfrec')\nTENSORFLOW_FILES = tf.io.gfile.glob(MORE_IMAGES_GCS_DS_PATH + '/tf_flowers' + MOREIMAGES_PATH + '/*.tfrec')\nADDITIONAL_TRAINING_FILENAMES = IMAGENET_FILES + INATURELIST_FILES + OPENIMAGE_FILES + OXFORD_FILES + TENSORFLOW_FILES\n#print(VALIDATION_FILENAMES)\nprint('----')\nTRAINING_FILENAMES = TRAINING_FILENAMES + ADDITIONAL_TRAINING_FILENAMES\n#print(TRAINING_FILENAMES)\n\n# This is so awkward. Everyone is doing this for an extra few points.\n# TRAINING_FILENAMES = TRAINING_FILENAMES + VALIDATION_FILENAMES\n# VALIDATION_FILENAMES = TRAINING_FILENAMES\n\nCLASSES = ['pink primrose', 'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea', 'wild geranium', 'tiger lily', 'moon orchid', 'bird of paradise', 'monkshood', 'globe thistle', # 00 - 09\n 'snapdragon', \"colt's foot\", 'king protea', 'spear thistle', 'yellow iris', 'globe-flower', 'purple coneflower', 'peruvian lily', 'balloon flower', 'giant white arum lily', # 10 - 19\n 'fire lily', 'pincushion flower', 'fritillary', 'red ginger', 'grape hyacinth', 'corn poppy', 'prince of wales feathers', 'stemless gentian', 'artichoke', 'sweet william', # 20 - 29\n 'carnation', 'garden phlox', 'love in the mist', 'cosmos', 'alpine sea holly', 'ruby-lipped cattleya', 'cape flower', 'great masterwort', 'siam tulip', 'lenten rose', # 30 - 39\n 'barberton daisy', 'daffodil', 'sword lily', 'poinsettia', 'bolero deep blue', 'wallflower', 'marigold', 'buttercup', 'daisy', 'common dandelion', # 40 - 49\n 'petunia', 'wild pansy', 'primula', 'sunflower', 'lilac hibiscus', 'bishop of llandaff', 'gaura', 'geranium', 'orange dahlia', 'pink-yellow dahlia', # 50 - 59\n 'cautleya spicata', 'japanese anemone', 'black-eyed susan', 'silverbush', 'californian poppy', 'osteospermum', 'spring crocus', 'iris', 'windflower', 'tree poppy', # 60 - 69\n 'gazania', 'azalea', 'water lily', 'rose', 'thorn apple', 'morning glory', 'passion flower', 'lotus', 'toad lily', 'anthurium', # 70 - 79\n 'frangipani', 'clematis', 'hibiscus', 'columbine', 'desert-rose', 'tree mallow', 'magnolia', 'cyclamen ', 'watercress', 'canna lily', # 80 - 89\n 'hippeastrum ', 'bee balm', 'pink quill', 'foxglove', 'bougainvillea', 'camellia', 'mallow', 'mexican petunia', 'bromelia', 'blanket flower', # 90 - 99\n 'trumpet creeper', 'blackberry lily', 'common tulip', 'wild rose'] # 100 - 102","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Out of 4 models used for ensembling, two of the models use LR_EXP_DECAY of 0.8 and two of the models use 0.75."},{"metadata":{"trusted":true},"cell_type":"code","source":"LR_START = 0.00001\nLR_MAX = 0.00005 * strategy.num_replicas_in_sync\nLR_MIN = LR_START\nLR_RAMPUP_EPOCHS = 5\nLR_SUSTAIN_EPOCHS = 0 # 3\nLR_EXP_DECAY = 0.80\n\ndef lrfn(epoch):\n if epoch < LR_RAMPUP_EPOCHS:\n lr = LR_START + (epoch * (LR_MAX - LR_START) / LR_RAMPUP_EPOCHS)\n elif epoch < (LR_RAMPUP_EPOCHS + LR_SUSTAIN_EPOCHS):\n lr = LR_MAX\n else:\n lr = LR_MIN + (LR_MAX - LR_MIN) * LR_EXP_DECAY ** (epoch - LR_RAMPUP_EPOCHS - LR_SUSTAIN_EPOCHS)\n# print('For epoch', epoch, 'setting lr to', lr)\n return lr\n\nlr_callback = tf.keras.callbacks.LearningRateScheduler(lrfn, verbose = True)\n\nrng = [i for i in range(20)]\ny = [lrfn(x) for x in rng]\nprint(y)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Helper Functions"},{"metadata":{"trusted":true},"cell_type":"code","source":"# numpy and matplotlib defaults\nnp.set_printoptions(threshold=15, linewidth=80)\n\ndef batch_to_numpy_images_and_labels(data):\n images, labels = data\n numpy_images = images.numpy()\n numpy_labels = labels.numpy()\n if numpy_labels.dtype == object: # binary string in this case, these are image ID strings\n numpy_labels = [None for _ in enumerate(numpy_images)]\n # If no labels, only image IDs, return None for labels (this is the case for test data)\n return numpy_images, numpy_labels\n\ndef title_from_label_and_target(label, correct_label):\n if correct_label is None:\n return CLASSES[label], True\n correct = (label == correct_label)\n return \"{} [{}{}{}]\".format(CLASSES[label], 'OK' if correct else 'NO', u\"\\u2192\" if not correct else '',\n CLASSES[correct_label] if not correct else ''), correct\n\ndef display_one_flower(image, title, subplot, red=False, titlesize=16):\n plt.subplot(*subplot)\n plt.axis('off')\n plt.imshow(image)\n if len(title) > 0:\n plt.title(title, fontsize=int(titlesize) if not red else int(titlesize/1.2), color='red' if red else 'black', fontdict={'verticalalignment':'center'}, pad=int(titlesize/1.5))\n return (subplot[0], subplot[1], subplot[2]+1)\n \ndef display_batch_of_images(databatch, predictions=None):\n \"\"\"This will work with:\n display_batch_of_images(images)\n display_batch_of_images(images, predictions)\n display_batch_of_images((images, labels))\n display_batch_of_images((images, labels), predictions)\n \"\"\"\n # data\n images, labels = batch_to_numpy_images_and_labels(databatch)\n if labels is None:\n labels = [None for _ in enumerate(images)]\n \n # auto-squaring: this will drop data that does not fit into square or square-ish rectangle\n rows = int(math.sqrt(len(images)))\n cols = len(images)//rows\n \n # size and spacing\n FIGSIZE = 13.0\n SPACING = 0.1\n subplot=(rows,cols,1)\n if rows < cols:\n plt.figure(figsize=(FIGSIZE,FIGSIZE/cols*rows))\n else:\n plt.figure(figsize=(FIGSIZE/rows*cols,FIGSIZE))\n \n # display\n for i, (image, label) in enumerate(zip(images[:rows*cols], labels[:rows*cols])):\n title = '' if label is None else CLASSES[label]\n correct = True\n if predictions is not None:\n title, correct = title_from_label_and_target(predictions[i], label)\n dynamic_titlesize = FIGSIZE*SPACING/max(rows,cols)*40+3 # magic formula tested to work from 1x1 to 10x10 images\n subplot = display_one_flower(image, title, subplot, not correct, titlesize=dynamic_titlesize)\n \n #layout\n plt.tight_layout()\n if label is None and predictions is None:\n plt.subplots_adjust(wspace=0, hspace=0)\n else:\n plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n plt.show()\n\ndef display_confusion_matrix(cmat, score, precision, recall):\n plt.figure(figsize=(15,15))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 7})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 7})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = \"\"\n if score is not None:\n titlestring += 'f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\nprecision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\nrecall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 18, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n \ndef display_training_curves(training, validation, title, subplot):\n if subplot%10==1: # set up the subplots on the first call\n plt.subplots(figsize=(10,10), facecolor='#F0F0F0')\n plt.tight_layout()\n ax = plt.subplot(subplot)\n ax.set_facecolor('#F8F8F8')\n ax.plot(training)\n ax.plot(validation)\n ax.set_title('model '+ title)\n ax.set_ylabel(title)\n #ax.set_ylim(0.28,1.05)\n ax.set_xlabel('epoch')\n ax.legend(['train', 'valid.'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def decode_image(image_data):\n image = tf.image.decode_jpeg(image_data, channels = 3)\n image = tf.cast(image, tf.float32) / 255.0\n image = tf.reshape(image, [*IMAGE_SIZE, 3])\n return image\n#\n\ndef read_labeled_tfrecord(example):\n LABELED_TFREC_FORMAT = {\n 'image': tf.io.FixedLenFeature([], tf.string),\n 'class': tf.io.FixedLenFeature([], tf.int64),\n }\n example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n image = decode_image(example['image'])\n label = tf.cast(example['class'], tf.int32)\n return image, label\n#\n\ndef read_unlabeled_tfrecord(example):\n UNLABELED_TFREC_FORMAT = {\n 'image': tf.io.FixedLenFeature([], tf.string),\n 'id': tf.io.FixedLenFeature([], tf.string),\n }\n example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n image = decode_image(example['image'])\n idnum = example['id']\n return image, idnum\n#\n\ndef load_dataset(filenames, labeled = True, ordered = False):\n ignore_order = tf.data.Options()\n if not ordered:\n ignore_order.experimental_deterministic = False\n dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads = AUTO)\n dataset = dataset.with_options(ignore_order)\n dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord, num_parallel_calls = AUTO)\n return dataset\n#\n\ndef data_augment(image, label):\n image = tf.image.random_flip_left_right(image)\n image = tf.image.random_saturation(image, 0, 2)\n return image, label\n#\n\ndef get_training_dataset():\n dataset = load_dataset(TRAINING_FILENAMES, labeled = True)\n dataset = dataset.map(data_augment, num_parallel_calls = AUTO)\n dataset = dataset.repeat()\n dataset = dataset.shuffle(2048)\n dataset = dataset.batch(BATCH_SIZE)\n dataset = dataset.prefetch(AUTO)\n return dataset\n#\n\ndef get_validation_dataset(ordered = False):\n dataset = load_dataset(VALIDATION_FILENAMES, labeled = True, ordered=ordered)\n dataset = dataset.batch(BATCH_SIZE)\n dataset = dataset.cache()\n dataset = dataset.prefetch(AUTO)\n return dataset\n#\n\ndef get_test_dataset(ordered = False):\n dataset = load_dataset(TEST_FILENAMES, labeled = False, ordered = ordered)\n dataset = dataset.batch(BATCH_SIZE)\n dataset = dataset.prefetch(AUTO)\n return dataset\n#\n\ndef count_data_items(filenames):\n n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) for filename in filenames]\n return np.sum(n)\n#\n\nNUM_TRAINING_IMAGES = count_data_items(TRAINING_FILENAMES)\nNUM_VALIDATION_IMAGES = count_data_items(VALIDATION_FILENAMES)\nNUM_TEST_IMAGES = count_data_items(TEST_FILENAMES)\nSTEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\nprint('Dataset: {} training images, {} validation images, {} unlabeled test images'.format(NUM_TRAINING_IMAGES, NUM_VALIDATION_IMAGES, NUM_TEST_IMAGES))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Training data shapes')\nfor image, label in get_training_dataset().take(3):\n print(image.numpy().shape, label.numpy().shape)\nprint('Training data label examples:', label.numpy())\n#\n\nprint('Validation data shapes')\nfor image, label in get_validation_dataset().take(3):\n print(image.numpy().shape, label.numpy().shape)\nprint('Validation data label examples:', label.numpy())\n#\n\nprint('Test data shapes')\nfor image, idnum in get_test_dataset().take(3):\n print(image.numpy().shape, idnum.numpy().shape)\nprint('Test data IDs:', idnum.numpy().astype('U'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"training_dataset = get_training_dataset()\ntraining_dataset = training_dataset.unbatch().batch(20)\ntrain_batch = iter(training_dataset)\n#\n#display_batch_of_images(next(train_batch))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_dataset = get_test_dataset()\ntest_dataset = test_dataset.unbatch().batch(20)\ntest_batch = iter(test_dataset)\n#\n#display_batch_of_images(next(test_batch))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Model Setup and Training\nA lot of code below are not used in this particular notebook. But code previously used in other notebooks were not deleted intentionally. It should give an indication on the different kinds of experiments one could run for this competition."},{"metadata":{"trusted":true},"cell_type":"code","source":"early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=20, restore_best_weights = True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_VGG16_model():\n pretrained_model = tf.keras.applications.VGG16(weights = 'imagenet', include_top = False, input_shape = [*IMAGE_SIZE, 3])\n pretrained_model.trainable = True # False\n\n model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation = 'softmax')\n ])\n\n model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy'])\n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_Xception_model():\n pretrained_model = tf.keras.applications.Xception(include_top = False, input_shape = [*IMAGE_SIZE, 3])\n pretrained_model.trainable = True\n\n model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation = 'softmax')\n ])\n\n model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy'])\n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_DenseNet_model():\n pretrained_model = tf.keras.applications.DenseNet201(weights = 'imagenet', include_top = False, input_shape = [*IMAGE_SIZE, 3])\n pretrained_model.trainable = True\n\n model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation = 'softmax')\n ])\n\n model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy'])\n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_EfficientNet_model():\n pretrained_model = efficientnet.EfficientNetB7(weights = 'noisy-student', include_top = False, input_shape = [*IMAGE_SIZE, 3])\n pretrained_model.trainable = True\n\n model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation = 'softmax')\n ])\n\n model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy'])\n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_InceptionV3_model():\n pretrained_model = tf.keras.applications.InceptionV3(weights = 'imagenet', include_top = False, input_shape = [*IMAGE_SIZE, 3])\n pretrained_model.trainable = True\n\n model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation = 'softmax')\n ])\n\n model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy'])\n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_ResNet152_model():\n pretrained_model = tf.keras.applications.ResNet152V2(weights = 'imagenet', include_top = False, input_shape = [*IMAGE_SIZE, 3])\n pretrained_model.trainable = True\n\n model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation = 'softmax')\n ])\n\n model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy'])\n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_MobileNetV2_model():\n pretrained_model = tf.keras.applications.MobileNetV2(weights = 'imagenet', include_top = False, input_shape = [*IMAGE_SIZE, 3])\n pretrained_model.trainable = True\n\n model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation = 'softmax')\n ])\n\n model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy'])\n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_InceptionResNetV2_model():\n pretrained_model = tf.keras.applications.InceptionResNetV2(weights = 'imagenet', include_top = False, input_shape = [*IMAGE_SIZE, 3])\n pretrained_model.trainable = True\n\n model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation = 'softmax')\n ])\n\n model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['sparse_categorical_accuracy'])\n return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"no_of_models = 1\nmodels = [0] * no_of_models\nstart_model = 0\nend_model = 1\nmodel_indx_0 = start_model\n#model_indx_1 = start_model + 1\n#model_indx_2 = start_model + 2\n\nval_probabilities = [0] * no_of_models\ntest_probabilities = [0] * no_of_models\nall_probabilities = [0] * no_of_models","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Out of the four models used in ensembling, two of the models use EfficientNetB7 and two others use DenseNet201."},{"metadata":{"trusted":true},"cell_type":"code","source":"#with strategy.scope():\n# models[0] = create_DenseNet_model()\n# models[1] = create_EfficientNet_model()\n#print(models[0].summary())\n#print(models[1].summary())\n#\nwith strategy.scope():\n for j in range(no_of_models):\n# models[j] = create_VGG16_model()\n# models[j] = create_Xception_model()\n# models[j] = create_DenseNet_model()\n models[j] = create_EfficientNet_model()\n# models[j] = create_InceptionV3_model()\n# models[j] = create_ResNet152_model()\n# models[j] = create_MobileNetV2_model()\n# models[j] = create_InceptionResNetV2_model()\n#\nmodels[0].summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def write_history(j):\n history_dict = [0] * no_of_models\n for i in range(j + 1):\n if (historys[i] != 0):\n history_dict[i] = historys[i].history\n#\n filename = './' + this_run_file_prefix + 'model_history_' + str(j) + '.pkl'\n pklfile = open(filename, 'ab')\n pickle.dump(history_dict, pklfile)\n pklfile.close()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"EPOCHS = 50 # 30 # 50 # 35 # 2 # 20\nhistorys = [0] * no_of_models\n#lr_exp_decay_values = [0.5,0.6,0.5,0.7] # [0.6,0.7,0.8,0.9,0.6,0.7,0.8,0.9,0.6,0.7,0.8,0.9,0.6,0.7,0.8,0.9,0.5,0.5,0.5,0.5]\n#lr_max_values = [0.00005,0.00003,0.00004,0.00003] # [0.00003,0.00003,0.00003,0.00003,0.00004,0.00004,0.00004,0.00004,0.00005,0.00005,0.00005,0.00005,0.00006,0.00006,0.00006,0.00006,0.00003,0.00004,0.00005,0.00006]\nfinished_models = 0\n\nfor j in range(start_model, end_model):\n start_training = datetime.now()\n print(start_training)\n time_from_start_program_tdelta = start_training - start_time\n if time_from_start_program_tdelta > end_training_by_tdelta:\n print(j, 'time limit for doing training over, get out')\n break\n# with strategy.scope():\n# models[j] = create_DenseNet_model()\n# if j == 0:\n# models[0].summary()\n# print('----------------------------------------------------')\n# LR_EXP_DECAY = lr_exp_decay_values[j]\n# LR_MAX = lr_max_values[j] * strategy.num_replicas_in_sync\n print('LR_EXP_DECAY:', LR_EXP_DECAY, '. LR_MAX:', LR_MAX)\n historys[j] = models[j].fit(get_training_dataset(), steps_per_epoch = STEPS_PER_EPOCH, epochs = EPOCHS, validation_data = get_validation_dataset(), callbacks = [lr_callback, early_stop])\n write_history(j)\n filename = this_run_file_prefix + 'models_' + str(j) + '.h5'\n models[j].save(filename)\n# model_to_delete = models[j]\n# models[j] = 0\n# del model_to_delete\n gc.collect()\n finished_models = j + 1\n\nprint(datetime.now())\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cmdataset = get_validation_dataset(ordered = True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_ds = get_test_dataset(ordered = True)\n\n#print('Computing predictions...')\ntest_images_ds = test_ds.map(lambda image, idnum: image)\ntest_ids_ds = test_ds.map(lambda image, idnum: idnum).unbatch()\ntest_ids = next(iter(test_ids_ds.batch(NUM_TEST_IMAGES))).numpy().astype('U')\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dataset = get_validation_dataset()\ndataset = dataset.unbatch().batch(20)\nbatch = iter(dataset)\n\nimages, labels = next(batch)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(datetime.now())\n#\nfor j in range(start_model, end_model):\n val_probabilities[j] = models[j].predict(images_ds)\n test_probabilities[j] = models[j].predict(test_images_ds)\n all_probabilities[j] = models[j].predict(images)\n\nprint(datetime.now())\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for j in range(start_model, finished_models):\n display_training_curves(historys[j].history['loss'], historys[j].history['val_loss'], 'loss', 211)\n display_training_curves(historys[j].history['sparse_categorical_accuracy'], historys[j].history['val_sparse_categorical_accuracy'], 'accuracy', 212)\n#\nfor j in range(start_model, finished_models):\n print('model number:', j, ', Train Accuracy:', max(historys[j].history['sparse_categorical_accuracy']), ', Validation Accuracy:', max(historys[j].history['val_sparse_categorical_accuracy']))\nfor j in range(start_model, finished_models):\n print('model number:', j, ', Train Loss:', min(historys[j].history['loss']), ', Validation Loss:', min(historys[j].history['val_loss']))\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cm_probabilities = np.zeros((val_probabilities[0].shape)) # = val_probabilities[0] + val_probabilities[1] + val_probabilities[2]\nfor j in range(no_of_models):\n cm_probabilities = cm_probabilities + val_probabilities[j]\n\ncm_predictions = np.argmax(cm_probabilities, axis = -1)\nprint('Correct labels: ', cm_correct_labels.shape, cm_correct_labels)\nprint('Predicted labels: ', cm_predictions.shape, cm_predictions)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def getFitPrecisionRecall(correct_labels, predictions):\n score = f1_score(correct_labels, predictions, labels = range(len(CLASSES)), average = 'macro')\n precision = precision_score(correct_labels, predictions, labels = range(len(CLASSES)), average = 'macro')\n recall = recall_score(correct_labels, predictions, labels = range(len(CLASSES)), average = 'macro')\n return score, precision, recall\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cmat = confusion_matrix(cm_correct_labels, cm_predictions, labels = range(len(CLASSES)))\nscore, precision, recall = getFitPrecisionRecall(cm_correct_labels, cm_predictions)\ncmat = (cmat.T / cmat.sum(axis = -1)).T\ndisplay_confusion_matrix(cmat, score, precision, recall)\nprint('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_submission_file(filename, probabilities):\n predictions = np.argmax(probabilities, axis = -1)\n print('Generating submission file...', filename)\n test_ids_ds = test_ds.map(lambda image, idnum: idnum).unbatch()\n test_ids = next(iter(test_ids_ds.batch(NUM_TEST_IMAGES))).numpy().astype('U')\n\n np.savetxt(filename, np.rec.fromarrays([test_ids, predictions]), fmt = ['%s', '%d'], delimiter = ',', header = 'id,label', comments = '')\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"probabilities = np.zeros((test_probabilities[0].shape)) # = test_probabilities[0] + test_probabilities[1] + test_probabilities[2]\nfor j in range(no_of_models):\n probabilities = probabilities + test_probabilities[j]\n\nfilename = this_run_file_prefix + 'submission.csv'\ncreate_submission_file(filename, probabilities)\ncreate_submission_file('submission.csv', probabilities)\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def combine_two(correct_labels, probability_0, probability_1):\n print('Start. ', datetime.now())\n alphas0_to_try = np.linspace(0, 1, 101)\n best_score = -1\n best_alpha0 = -1\n best_alpha1 = -1\n best_precision = -1\n best_recall = -1\n best_val_predictions = None\n\n for alpha0 in alphas0_to_try:\n alpha1 = 1.0 - alpha0\n probabilities = alpha0 * probability_0 + alpha1 * probability_1 #\n predictions = np.argmax(probabilities, axis = -1)\n\n score, precision, recall = getFitPrecisionRecall(correct_labels, predictions)\n if score > best_score:\n best_alpha0 = alpha0\n best_alpha1 = alpha1\n best_score = score\n best_precision = precision\n best_recall = recall\n best_val_predictions = predictions\n #\n return best_alpha0, best_alpha1, best_val_predictions, best_score, best_precision, best_recall","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def combine_three(correct_labels, probability_0, probability_1, probability_2):\n print('Start. ', datetime.now())\n alphas0_to_try = np.linspace(0, 1, 101)\n alphas1_to_try = np.linspace(0, 1, 101)\n best_score = -1\n best_alpha0 = -1\n best_alpha1 = -1\n best_alpha2 = -1\n best_precision = -1\n best_recall = -1\n best_val_predictions = None\n\n for alpha0 in alphas0_to_try:\n for alpha1 in alphas1_to_try:\n if (alpha0 + alpha1) > 1.0:\n break\n\n alpha2 = 1.0 - alpha0 - alpha1\n probabilities = alpha0 * probability_0 + alpha1 * probability_1 + alpha2 * probability_2\n predictions = np.argmax(probabilities, axis = -1)\n\n score, precision, recall = getFitPrecisionRecall(correct_labels, predictions)\n if score > best_score:\n best_alpha0 = alpha0\n best_alpha1 = alpha1\n best_alpha2 = alpha2\n best_score = score\n best_precision = precision\n best_recall = recall\n best_val_predictions = predictions\n #\n return best_alpha0, best_alpha1, best_alpha2, best_val_predictions, best_score, best_precision, best_recall","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_best_combination(no_models, cm_correct_labels, val_probabilities, test_probabilities):\n best_fit_score = -10000.0\n best_predictions = 0\n choose_filename = ''\n\n curr_predictions = np.argmax(val_probabilities[0], axis = -1)\n score, precision, recall = getFitPrecisionRecall(cm_correct_labels, curr_predictions)\n print('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))\n filename = this_run_file_prefix + 'submission_0.csv'\n if best_fit_score < score:\n best_fit_score = score\n best_predictions = curr_predictions\n choose_filename = filename\n create_submission_file('./submission.csv', test_probabilities[0])\n create_submission_file(filename, test_probabilities[0])\n\n if no_models > 1:\n curr_predictions = np.argmax(val_probabilities[1], axis = -1)\n score, precision, recall = getFitPrecisionRecall(cm_correct_labels, curr_predictions)\n print('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))\n filename = this_run_file_prefix + 'submission_1.csv'\n if best_fit_score < score:\n best_fit_score = score\n best_predictions = curr_predictions\n choose_filename = filename\n create_submission_file('./submission.csv', test_probabilities[1])\n create_submission_file(filename, test_probabilities[1])\n\n if no_models > 2:\n curr_predictions = np.argmax(val_probabilities[2], axis = -1)\n score, precision, recall = getFitPrecisionRecall(cm_correct_labels, curr_predictions)\n print('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(score, precision, recall))\n filename = this_run_file_prefix + 'submission_2.csv'\n if best_fit_score < score:\n best_fit_score = score\n best_predictions = curr_predictions\n choose_filename = filename\n create_submission_file('./submission.csv', test_probabilities[2])\n create_submission_file(filename, test_probabilities[2])\n\n if no_models > 1:\n best_alpha0, best_alpha1, best_val_predictions, best_score, best_precision, best_recall = combine_two(cm_correct_labels, val_probabilities[0], val_probabilities[1])\n print('For indx', [0, 1], 'best_alpha0:', best_alpha0, 'best_alpha1:', best_alpha1, '. ', datetime.now())\n print('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(best_score, best_precision, best_recall))\n combined_probabilities = best_alpha0 * test_probabilities[0] + best_alpha1 * test_probabilities[1]\n filename = this_run_file_prefix + 'submission_01.csv'\n if best_fit_score < best_score:\n best_fit_score = best_score\n best_predictions = best_val_predictions\n choose_filename = filename\n create_submission_file('./submission.csv', combined_probabilities)\n create_submission_file(filename, combined_probabilities)\n\n if no_models > 2:\n best_alpha0, best_alpha1, best_val_predictions, best_score, best_precision, best_recall = combine_two(cm_correct_labels, val_probabilities[0], val_probabilities[2])\n print('For indx', [0, 2], 'best_alpha0:', best_alpha0, 'best_alpha1:', best_alpha1, '. ', datetime.now())\n print('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(best_score, best_precision, best_recall))\n combined_probabilities = best_alpha0 * test_probabilities[0] + best_alpha1 * test_probabilities[2]\n filename = this_run_file_prefix + 'submission_02.csv'\n if best_fit_score < best_score:\n best_fit_score = best_score\n best_predictions = best_val_predictions\n choose_filename = filename\n create_submission_file('./submission.csv', combined_probabilities)\n create_submission_file(filename, combined_probabilities)\n\n best_alpha0, best_alpha1, best_val_predictions, best_score, best_precision, best_recall = combine_two(cm_correct_labels, val_probabilities[1], val_probabilities[2])\n print('For indx', [1, 2], 'best_alpha0:', best_alpha0, 'best_alpha1:', best_alpha1, '. ', datetime.now())\n print('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(best_score, best_precision, best_recall))\n combined_probabilities = best_alpha0 * test_probabilities[1] + best_alpha1 * test_probabilities[2]\n filename = this_run_file_prefix + 'submission_12.csv'\n if best_fit_score < best_score:\n best_fit_score = best_score\n best_predictions = best_val_predictions\n choose_filename = filename\n create_submission_file('./submission.csv', combined_probabilities)\n create_submission_file(filename, combined_probabilities)\n\n best_alpha0, best_alpha1, best_alpha2, best_val_predictions, best_score, best_precision, best_recall = combine_three(cm_correct_labels, val_probabilities[0], val_probabilities[1], val_probabilities[2])\n print('For indx', [0, 1, 2], 'best_alpha0:', best_alpha0, 'best_alpha1:', best_alpha1, 'best_alpha2:', best_alpha2, '. ', datetime.now())\n print('f1 score: {:.3f}, precision: {:.3f}, recall: {:.3f}'.format(best_score, best_precision, best_recall))\n combined_probabilities = best_alpha0 * test_probabilities[0] + best_alpha1 * test_probabilities[1] + best_alpha2 * test_probabilities[2]\n filename = this_run_file_prefix + 'submission_012.csv'\n if best_fit_score < best_score:\n best_fit_score = best_score\n best_predictions = best_val_predictions\n choose_filename = filename\n create_submission_file('./submission.csv', combined_probabilities)\n create_submission_file(filename, combined_probabilities)\n#\n cmat = confusion_matrix(cm_correct_labels, best_predictions, labels = range(len(CLASSES)))\n cmat = (cmat.T / cmat.sum(axis = -1)).T\n display_confusion_matrix(cmat, best_fit_score, precision, recall)\n#\n print('Best score from all combination was', best_fit_score, '. For submission file used is', choose_filename)\n return best_predictions\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"best_predictions = cm_predictions\nif no_of_models > 1:\n# bp = get_best_combination(no_of_models, cm_correct_labels_results[0], val_probabilities, test_probabilities)\n bp = get_best_combination(no_of_models, cm_correct_labels, val_probabilities, test_probabilities)\n best_predictions = bp\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"probabilities = np.zeros((all_probabilities[0].shape)) # = all_probabilities[0] + all_probabilities[1] + all_probabilities[2]\nfor j in range(no_of_models):\n probabilities = probabilities + all_probabilities[j]\n\npredictions = np.argmax(probabilities, axis =-1)\ndisplay_batch_of_images((images, labels), predictions)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Storing the Output for Ensembling\nThe initial reason for storing outputs of these models were to save time during ensembling. Loading the model saved and re-predicting for the validation and test set would take unusally longer compared to the time it took in the original kernel. Many a times giving the feeling that the kernel ensembling had hanged."},{"metadata":{"trusted":true},"cell_type":"code","source":"#val_probs = [cm_correct_labels, cm_predictions, test_ids, val_probabilities[0], val_probabilities[1], val_probabilities[2], test_probabilities[0], test_probabilities[1], test_probabilities[2]]\n#val_probs = [cm_correct_labels, cm_predictions, test_ids, val_probabilities[0], val_probabilities[1], test_probabilities[0], test_probabilities[1]]\nval_probs = [cm_correct_labels, cm_predictions, test_ids, val_probabilities[0], test_probabilities[model_indx_0]]\nfilename = this_run_file_prefix + 'tests_vals_0.pkl'\npklfile = open(filename, 'ab')\npickle.dump(val_probs, pklfile)\npklfile.close()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here the intention was to display the validation images that were incorrectly predicted. But some how the commented code which had worked on the local PC did not work through the kernel. And so abandoned the idea of displaying them."},{"metadata":{"trusted":true},"cell_type":"code","source":"#images_ds_unbatched = images_ds.unbatch()\n#cm_images_ds_numpy = next(iter(images_ds_unbatched.batch(NUM_VALIDATION_IMAGES))).numpy()\nuse_correct_labels = cm_correct_labels\nuse_val_predictions = best_predictions","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('type of labels_ds is {}'.format(type(labels_ds)))\nprint('type of use_val_predictions is {}. shape of use_val_predictions is {}'.format(type(use_val_predictions), use_val_predictions.shape))\n#print('type of use_correct_labels is {}, cm_images_ds_numpy is {}'.format(type(use_correct_labels), type(cm_images_ds_numpy)))\n#print('shape of use_correct_labels is {}, cm_images_ds_numpy is {}'.format(use_correct_labels.shape, cm_images_ds_numpy.shape))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"correct_labels_cnt = 0\nincorrect_labels_cnt = 0\ncorrect_labels = []\nincorrect_labels = []\nvals_actual_true = {}\nvals_tp = {}\nvals_fn = {}\nvals_fp = {}\nfor i in range(len(CLASSES)):\n vals_actual_true[i] = 0\n vals_tp[i] = 0\n vals_fn[i] = 0\n vals_fp[i] = 0\n\nfor i in range(len(use_correct_labels)):\n correct_label = use_correct_labels[i]\n predict_label = use_val_predictions[i]\n vals_actual_true[correct_label] = vals_actual_true[correct_label] + 1\n if use_val_predictions[i] != use_correct_labels[i]:\n incorrect_labels_cnt = incorrect_labels_cnt + 1\n incorrect_labels.append(i)\n vals_fn[correct_label] = vals_fn[correct_label] + 1\n vals_fp[predict_label] = vals_fp[predict_label] + 1\n else:\n correct_labels_cnt = correct_labels_cnt + 1\n correct_labels.append(i)\n vals_tp[correct_label] = vals_tp[correct_label] + 1\n# print(i)\n#\nprint('Number of correct_labels is {}, incorrect_labels is {}'.format(correct_labels_cnt, incorrect_labels_cnt))\n#print('Correct labels', correct_labels)\nprint('Incorrect labels', incorrect_labels)\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def display_my_batch_of_images(databatch, rows = 0, cols = 0, predictions=None):\n \"\"\"This will work with:\n display_batch_of_images(images)\n display_batch_of_images(images, predictions)\n display_batch_of_images((images, labels))\n display_batch_of_images((images, labels), predictions)\n \"\"\"\n # data\n images, labels = databatch\n if labels is None:\n labels = [None for _ in enumerate(images)]\n\n # auto-squaring: this will drop data that does not fit into square or square-ish rectangle\n if rows == 0 or cols == 0:\n rows = int(math.sqrt(len(images)))\n cols = (len(images) + rows - 1)//rows\n print('Total number of images is {}, rows is {}, cols is {}'.format(len(images), rows, cols))\n\n # size and spacing\n FIGSIZE = 20.0\n SPACING = 0.1\n subplot=(rows,cols,1)\n if rows < cols:\n plt.figure(figsize=(FIGSIZE,FIGSIZE/cols*rows))\n else:\n plt.figure(figsize=(FIGSIZE/rows*cols,FIGSIZE))\n\n # display\n for i, (image, label) in enumerate(zip(images[:rows*cols], labels[:rows*cols])):\n title = '' if label is None else CLASSES[label]\n correct = True\n if predictions is not None:\n title, correct = title_from_label_and_target(predictions[i], label)\n dynamic_titlesize = FIGSIZE*SPACING/max(rows,cols)*40+3 # magic formula tested to work from 1x1 to 10x10 images\n subplot = display_one_flower(image, title, subplot, not correct, titlesize=dynamic_titlesize)\n\n #layout\n plt.tight_layout()\n if label is None and predictions is None:\n plt.subplots_adjust(wspace=0, hspace=0)\n else:\n plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n plt.show()\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#disp_images = []\ndisp_labels = []\ndisp_predictions = []\nfor i in range(54):\n if i >= incorrect_labels_cnt:\n break\n id = incorrect_labels[i]\n disp_labels.append(use_correct_labels[id])\n disp_predictions.append(use_val_predictions[id])\n# disp_images.append(cm_images_ds_numpy[id])\n#\nprint(disp_labels)\nprint(disp_predictions)\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#print('type of disp_images is {}, disp_labels is {}'.format(type(disp_images), type(disp_labels)))\n#print('length of disp_images is {}, disp_labels is {}'.format(len(disp_images), len(disp_labels)))\n#print('type of disp_images[0] is {}, disp_labels[0] is {}'.format(type(disp_images[0]), type(disp_labels[0])))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#display_my_batch_of_images((disp_images, disp_labels), rows = 9, cols = 6, predictions = disp_predictions)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The goal here was to gather information on which class of flowers had failed badly and to be able to use data augmentation for only those class of flowers. Finally never got to pursue it."},{"metadata":{"trusted":true},"cell_type":"code","source":"val_ids = list(range(len(use_correct_labels)))\nfilename = this_run_file_prefix + 'validation_results.csv'\nnp.savetxt(filename, np.rec.fromarrays([val_ids, use_correct_labels, use_val_predictions]), fmt = ['%d', '%d', '%d'], delimiter = ',', header = 'id,correct_label,predicted_label', comments = '')\n#\ncls_ids = list(range(len(CLASSES)))\n#print(len(cls_ids), len(vals_actual_true), len(vals_tp), len(vals_fn), len(vals_fp))\nfilename = this_run_file_prefix + 'validation_statistics.csv'\nnp.savetxt(filename, np.rec.fromarrays([cls_ids, list(vals_actual_true.values()), list(vals_tp.values()), list(vals_fn.values()), list(vals_fp.values())]), fmt = ['%d', '%d', '%d', '%d', '%d'], delimiter = ',', header = 'cls_id,actual_true,true_positive,false_negative,false_positive', comments = '')\n#","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.6"}},"nbformat":4,"nbformat_minor":4}