{"cells":[{"metadata":{},"cell_type":"markdown","source":"The TPUs provided on Kaggle makes training extremely fast, allowing us to explore more hyperparameters to get the best result. [Keras-Tuner](https://github.com/keras-team/keras-tuner) is a convenient solution for hyperparameter tuning.\n\nTo start with, this notebook shows using the built-in tuning algorithms and hypermodels (models including tunable hyperparameter) to tackle this classification task with simple code. In particular, I will use a hypermodel based on EfficientNet (shipped in [keras.applications](https://keras.io/api/applications/) since TF2.3) with random search tuning algorithm. Quite good result can be achieved with default options. \n\nIn general, hyperparameter searching requires large amount of resources. In the example you can choose `TESTING_LEVEL = 0` for a quick debug, `TESTING_LEVEL = 1` for a relatively short test to see some reasonable result (~0.9), or `TESTING_LEVEL = 2` and set your own search epochs limit for an extended run.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Configuration\nTESTING_LEVEL = 2\n\nif TESTING_LEVEL == 0:\n    # For debugging\n    IMAGE_SIZE = [192, 192]\n    EPOCHS_SEARCH = 5\n    MAX_TRIALS = 3\n    EPOCHS_FINAL = 5\n    BATCH_SIZE_PER_REPLICA = 16\nelif TESTING_LEVEL == 1:\n    # For relatively short test to see some reasonable result\n    IMAGE_SIZE = [331, 331]\n    EPOCHS_SEARCH = 15\n    MAX_TRIALS = 5\n    EPOCHS_FINAL = 10\n    BATCH_SIZE_PER_REPLICA = 16\nelse:\n    # For an extended run.\n    # Can set even larger MAX_TRIALS and EPOCHS_SEARCH for even better result.\n    IMAGE_SIZE = [331, 331]\n    EPOCHS_SEARCH = 60\n    MAX_TRIALS = 10\n    EPOCHS_FINAL = 10\n    BATCH_SIZE_PER_REPLICA = 16","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -q tensorflow==2.3.0 # Use 2.3.0 for built-in EfficientNet\n!pip install -q git+https://github.com/keras-team/keras-tuner@master # Use github head for newly added TPU support\n!pip install -q cloud-tpu-client # Needed for sync TPU version","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"import random, re, math\nimport numpy as np, pandas as pd\nimport matplotlib.pyplot as plt\nimport tensorflow as tf, tensorflow.keras.backend as K\nfrom kaggle_datasets import KaggleDatasets\nprint('Tensorflow version ' + tf.__version__)\nimport kerastuner as kt","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Configurations\n\nConfigure for TPU if TPU is available for use. In order to use TF2.3 on TPU, you need to manually configure TPU version using cloud-tpu-client. This is not yet officially supported, and may conflict with `user_secrets.set_tensorflow_credential(user_credential)` for now (as of 8/12). ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Detect hardware, return appropriate distribution strategy\ntry:\n    # Sync TPU version\n    from cloud_tpu_client import Client\n    c = Client()\n    c.configure_tpu_version(tf.__version__, restart_type='ifNeeded')\n    \n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()  # TPU detection. No parameters necessary if TPU_NAME environment variable is set. On Kaggle this is always the case.\n    print('Running on TPU ', tpu.master())\nexcept ValueError:\n    tpu = None\n    \n\nif tpu:\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.TPUStrategy(tpu)\nelse:\n    strategy = tf.distribute.get_strategy() # default distribution strategy in Tensorflow. Works on CPU and single GPU.\n\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)\nBATCH_SIZE = BATCH_SIZE_PER_REPLICA * strategy.num_replicas_in_sync","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we set a few over-all hyperparameters. These may also be searched through Keras-Tuner with customized tuner classes and model bulding function / HyperModels. ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Data preparation\nThe code for data preparation are mostly adapted from this [notebook](https://www.kaggle.com/cdeotte/rotation-augmentation-gpu-tpu-0-96) with some minor modification. Note that I am converting labels to one-hot encoding.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Data access\nGCS_DS_PATH = KaggleDatasets().get_gcs_path('tpu-getting-started')\n\nGCS_PATH_SELECT = { # available image sizes\n    192: GCS_DS_PATH + '/tfrecords-jpeg-192x192',\n    224: GCS_DS_PATH + '/tfrecords-jpeg-224x224',\n    331: GCS_DS_PATH + '/tfrecords-jpeg-331x331',\n    512: GCS_DS_PATH + '/tfrecords-jpeg-512x512'\n}\n\nGCS_PATH = GCS_PATH_SELECT[IMAGE_SIZE[0]]\n\nTRAINING_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec')\nVALIDATION_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\nTEST_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/test/*.tfrec') # predictions on this dataset should be submitted for the competition","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"CLASSES = ['pink primrose',    'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea',     'wild geranium',     'tiger lily',           'moon orchid',              'bird of paradise', 'monkshood',        'globe thistle',         # 00 - 09\n           'snapdragon',       \"colt's foot\",               'king protea',      'spear thistle', 'yellow iris',       'globe-flower',         'purple coneflower',        'peruvian lily',    'balloon flower',   'giant white arum lily', # 10 - 19\n           'fire lily',        'pincushion flower',         'fritillary',       'red ginger',    'grape hyacinth',    'corn poppy',           'prince of wales feathers', 'stemless gentian', 'artichoke',        'sweet william',         # 20 - 29\n           'carnation',        'garden phlox',              'love in the mist', 'cosmos',        'alpine sea holly',  'ruby-lipped cattleya', 'cape flower',              'great masterwort', 'siam tulip',       'lenten rose',           # 30 - 39\n           'barberton daisy',  'daffodil',                  'sword lily',       'poinsettia',    'bolero deep blue',  'wallflower',           'marigold',                 'buttercup',        'daisy',            'common dandelion',      # 40 - 49\n           'petunia',          'wild pansy',                'primula',          'sunflower',     'lilac hibiscus',    'bishop of llandaff',   'gaura',                    'geranium',         'orange dahlia',    'pink-yellow dahlia',    # 50 - 59\n           'cautleya spicata', 'japanese anemone',          'black-eyed susan', 'silverbush',    'californian poppy', 'osteospermum',         'spring crocus',            'iris',             'windflower',       'tree poppy',            # 60 - 69\n           'gazania',          'azalea',                    'water lily',       'rose',          'thorn apple',       'morning glory',        'passion flower',           'lotus',            'toad lily',        'anthurium',             # 70 - 79\n           'frangipani',       'clematis',                  'hibiscus',         'columbine',     'desert-rose',       'tree mallow',          'magnolia',                 'cyclamen ',        'watercress',       'canna lily',            # 80 - 89\n           'hippeastrum ',     'bee balm',                  'pink quill',       'foxglove',      'bougainvillea',     'camellia',             'mallow',                   'mexican petunia',  'bromelia',         'blanket flower',        # 90 - 99\n           'trumpet creeper',  'blackberry lily',           'common tulip',     'wild rose']                                                                                                                                               # 100 - 102","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Functions to create dataset\nAdapted from starter [kernel][1]\n\n[1]: https://www.kaggle.com/mgornergoogle/getting-started-with-100-flowers-on-tpu","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from tensorflow.data.experimental import AUTOTUNE\n\ndef decode_image(image_data):\n    image = tf.image.decode_jpeg(image_data, channels=3)\n    image = tf.cast(image, tf.float32) \n    # For keras.application implementation of EfficientNet, input should be [0, 255]\n    image = tf.ensure_shape(image, [*IMAGE_SIZE, 3]) # explicit size needed for TPU\n    return image\n\ndef read_labeled_tfrecord(example):\n    LABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n        \"class\": tf.io.FixedLenFeature([], tf.int64),  # shape [] means single element\n    }\n    example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    label = tf.cast(example['class'], tf.int32)\n    return image, label # returns a dataset of (image, label) pairs\n\ndef read_unlabeled_tfrecord(example):\n    UNLABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n        \"id\": tf.io.FixedLenFeature([], tf.string),  # shape [] means single element\n        # class is missing, this competitions's challenge is to predict flower classes for the test dataset\n    }\n    example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    idnum = example['id']\n    return image, idnum # returns a dataset of image(s)\n\ndef load_dataset(filenames, labeled = True, ordered = False):\n    # Read from TFRecords. For optimal performance, reading from multiple files at once and\n    # Diregarding data order. Order does not matter since we will be shuffling the data anyway\n    \n    ignore_order = tf.data.Options()\n    if not ordered:\n        ignore_order.experimental_deterministic = False # disable order, increase speed\n\n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads = AUTOTUNE) # automatically interleaves reads from multiple files\n    dataset = dataset.with_options(ignore_order) # use data as soon as it streams in, rather than in its original order\n    dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord,\n                          num_parallel_calls = AUTOTUNE) # returns a dataset of (image, label) pairs if labeled = True or (image, id) pair if labeld = False\n    return dataset\n\n\ndef one_hot_encoding(image, label, num_classes=len(CLASSES)):\n    return image, tf.one_hot(label, num_classes)\n\ndef get_training_dataset(dataset,do_aug=True):\n    dataset = dataset.map(one_hot_encoding, num_parallel_calls=AUTOTUNE)\n    if do_aug: dataset = dataset.map(transform, num_parallel_calls=AUTOTUNE)\n    dataset = dataset.repeat() # the training dataset must repeat for several epochs\n    dataset = dataset.shuffle(2048)\n    dataset = dataset.batch(BATCH_SIZE, drop_remainder=True) # Drop remainder to ensure same batch size for all.\n    dataset = dataset.prefetch(AUTOTUNE) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset\n\ndef get_validation_dataset(dataset):\n    dataset = dataset.map(one_hot_encoding, num_parallel_calls=AUTOTUNE)\n    dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTOTUNE) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset\n\ndef get_test_dataset(ordered=False):\n    dataset = load_dataset(TEST_FILENAMES, labeled=False, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE, drop_remainder=False)\n    dataset = dataset.prefetch(AUTOTUNE) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset\n\ndef count_data_items(filenames):\n    # the number of data items is written in the name of the .tfrec files, i.e. flowers00-230.tfrec = 230 data items\n    n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) for filename in filenames]\n    return np.sum(n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data Augmentation\nCurrently Keras Preprocessing Layer (KPL) is under experimental stage and is not fully compatible with TPU. Hence augmentation functions adapted from [this notebook](https://www.kaggle.com/cdeotte/rotation-augmentation-gpu-tpu-0-96) is used. \n\nWhen KPL layer become fully available, you will be able to use HyperModels for augmentation based on KPL layers that is shipped with Keras-Tuner. See [this notebook](https://www.kaggle.com/fuyixing/flower-classification-with-keras-tuner-and-kpl) for a GPU/CPU version of this notebook using KPL based tunable augmentation. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_mat(rotation, shear, height_zoom, width_zoom, height_shift, width_shift):\n    # returns 3x3 transformmatrix which transforms indicies\n        \n    # CONVERT DEGREES TO RADIANS\n    rotation = math.pi * rotation / 180.\n    shear = math.pi * shear / 180.\n    \n    # ROTATION MATRIX\n    c1 = tf.math.cos(rotation)\n    s1 = tf.math.sin(rotation)\n    one = tf.constant([1],dtype='float32')\n    zero = tf.constant([0],dtype='float32')\n    rotation_matrix = tf.reshape( tf.concat([c1,s1,zero, -s1,c1,zero, zero,zero,one],axis=0),[3,3] )\n        \n    # SHEAR MATRIX\n    c2 = tf.math.cos(shear)\n    s2 = tf.math.sin(shear)\n    shear_matrix = tf.reshape( tf.concat([one,s2,zero, zero,c2,zero, zero,zero,one],axis=0),[3,3] )    \n    \n    # ZOOM MATRIX\n    zoom_matrix = tf.reshape( tf.concat([one/height_zoom,zero,zero, zero,one/width_zoom,zero, zero,zero,one],axis=0),[3,3] )\n    \n    # SHIFT MATRIX\n    shift_matrix = tf.reshape( tf.concat([one,zero,height_shift, zero,one,width_shift, zero,zero,one],axis=0),[3,3] )\n    \n    return K.dot(K.dot(rotation_matrix, shear_matrix), K.dot(zoom_matrix, shift_matrix))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def transform(image,label):\n    image = tf.image.random_flip_left_right(image)\n    \n    # input image - is one image of size [dim,dim,3] not a batch of [b,dim,dim,3]\n    # output - image randomly rotated, sheared, zoomed, and shifted\n    DIM = IMAGE_SIZE[0]\n    XDIM = DIM%2 #fix for size 331\n    \n    rot = 20. * tf.random.normal([1],dtype='float32')\n    shr = 5. * tf.random.normal([1],dtype='float32') \n    h_zoom = 1.0 + tf.random.normal([1],dtype='float32')/10.\n    w_zoom = 1.0 + tf.random.normal([1],dtype='float32')/10.\n    h_shift = 16. * tf.random.normal([1],dtype='float32') \n    w_shift = 16. * tf.random.normal([1],dtype='float32') \n  \n    # GET TRANSFORMATION MATRIX\n    m = get_mat(rot,shr,h_zoom,w_zoom,h_shift,w_shift) \n\n    # LIST DESTINATION PIXEL INDICES\n    x = tf.repeat( tf.range(DIM//2,-DIM//2,-1), DIM )\n    y = tf.tile( tf.range(-DIM//2,DIM//2),[DIM] )\n    z = tf.ones([DIM*DIM],dtype='int32')\n    idx = tf.stack( [x,y,z] )\n    \n    # ROTATE DESTINATION PIXELS ONTO ORIGIN PIXELS\n    idx2 = K.dot(m,tf.cast(idx,dtype='float32'))\n    idx2 = K.cast(idx2,dtype='int32')\n    idx2 = K.clip(idx2,-DIM//2+XDIM+1,DIM//2)\n    \n    # FIND ORIGIN PIXEL VALUES           \n    idx3 = tf.stack( [DIM//2-idx2[0,], DIM//2-1+idx2[1,]] )\n    d = tf.gather_nd(image,tf.transpose(idx3))\n    \n    return tf.reshape(d,[DIM,DIM,3]),label","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For TPU, we need explicit batch number for both training and validation.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"num_val_samples = count_data_items(VALIDATION_FILENAMES)\nnum_train_samples = count_data_items(TRAINING_FILENAMES)\n\ntrain_ds = get_training_dataset(load_dataset(TRAINING_FILENAMES))\nvalidation_ds = get_validation_dataset(load_dataset(VALIDATION_FILENAMES))\nnum_train_batches = num_train_samples // BATCH_SIZE\nnum_val_batches = num_val_samples // BATCH_SIZE","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualizing examples","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"row = 3; col = 4;\nall_elements = get_training_dataset(load_dataset(TRAINING_FILENAMES),do_aug=False).unbatch()\none_element = tf.data.Dataset.from_tensors( next(iter(all_elements)) )\naugmented_element = one_element.repeat().map(transform).batch(row*col)\n\nfor (img,label) in augmented_element:\n    plt.figure(figsize=(15,int(15*row/col)))\n    for j in range(row*col):\n        plt.subplot(row,col,j+1)\n        plt.axis('off')\n        plt.imshow(img[j,] / 255.)\n    plt.show()\n    break","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Search Hyper-parameters with Keras-Tuner\n\nNow we search hyperparameters with Keras-Tuner.\n\nA HyperModel in Keras-Tuner is class with a `build` method that creates a *compiled* Keras model using a set of hyperparameters for each trial. A tuner takes a [HyperModel](https://keras-team.github.io/keras-tuner/documentation/hypermodels/) or simply a model builder function, and tries the combinations of the hyperparameters for times depending on different tuning algorithms (defined by [Oracle](https://keras-team.github.io/keras-tuner/documentation/oracles/)). Each of the built-in [Tuner](https://keras-team.github.io/keras-tuner/documentation/tuners/) have corresponding oracle.\n\nIn this example I only use pre-built HyperModel. It is also possible to create any HyperModel or model building function, and to create custom tuning algorithms by subclassing Oracles. Besides, we may [subclassing Tuner](https://keras-team.github.io/keras-tuner/tutorials/subclass-tuner/) to customize what happens in each trial. Here we subclass Tuner to create a `FineTuner` class that first fit the model with feature extracotr part frozen, and then finetune the entire model.\n\nA side note: TF2.3 provides `experimental_steps_per_execution` keyword for `model.compile`. This greatly improves TPU efficiency.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import copy\n\n# Helper function: re-compile with the same loss/metric/optimizer\ndef recompile(model):\n    metrics = model.compiled_metrics.metrics\n    metrics = [x.name for x in metrics]\n    model.compile(loss=model.loss,\n                  metrics=metrics,\n                  optimizer=model.optimizer)\n\nclass FineTuner(kt.engine.tuner.Tuner):\n    def run_trial(self, trial, *fit_args, **fit_kwargs):       \n        copied_fit_kwargs = copy.copy(fit_kwargs)\n\n        model = self.hypermodel.build(trial.hyperparameters)\n        #dry run to build metrics\n        model.evaluate(*fit_args, steps=1, batch_size=1)\n        \n        # freeze pretrained feature extractor\n        for l in model.layers:\n            # For efficientnet implementation we use, layers in the\n            # Feature extraction part of model all have 'block', \n            # 'stem' or 'top_conv' in name.\n            if any(x in l.name for x in ['block', 'stem', 'top_conv']):\n                l.trainable = False\n        recompile(model)\n        model.fit(*fit_args, **copied_fit_kwargs)\n        \n        for l in model.layers:\n            if not isinstance(l, tf.keras.layers.BatchNormalization):\n                l.trainable = True\n        recompile(model)\n        \n        \n        callbacks = fit_kwargs.pop('callbacks', [])\n        callbacks = self._deepcopy_callbacks(callbacks)\n        # TunerCallback reports results to the `Oracle` and save the trained Model.\n        callbacks.append(kt.engine.tuner_utils.TunerCallback(self, trial))\n        copied_fit_kwargs['callbacks'] = callbacks\n        model.fit(*fit_args, **copied_fit_kwargs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When `Tuner` is customized, we need to initiate `Oracle` and pass the `Oracle` object to initiate the `Tuner` object. If using built-in tuners, we can directly initiate tuners such as `RandomSearch`. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define HyperModel using built-in application\nfrom kerastuner.applications.efficientnet import HyperEfficientNet\nhm = HyperEfficientNet(input_shape=[IMAGE_SIZE[0], IMAGE_SIZE[1], 3] , classes=len(CLASSES))\n\n# Define Oracle\noracle = kt.tuners.bayesian.BayesianOptimizationOracle(\n    objective='val_accuracy',\n    max_trials=MAX_TRIALS,\n)\n\n# Initiate Tuner\ntuner = FineTuner(\n    hypermodel=hm,\n    oracle=oracle,\n    directory='flower_classification_kt_tpu',\n    project_name='randomsearch_efficientnet',\n    # Distribution strategy is passed in here.\n    distribution_strategy=strategy,\n    overwrite=True\n)\n\ntuner.search_space_summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`tuner.search()` is called just like the way you would call `model.fit()`.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tuner.search(train_ds,\n             epochs=EPOCHS_SEARCH,\n             validation_data=validation_ds,\n             steps_per_epoch=num_train_batches,\n             validation_steps=num_val_batches,\n             # We can add callbacks here just like in model.fit()\n             callbacks=[tf.keras.callbacks.ReduceLROnPlateau(),\n                        tf.keras.callbacks.EarlyStopping(patience=5)],\n             verbose=2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As long as some trials are complete, we may move on to get the best result up to now even if search fail to finish. Also, as long as the project directory is not deleted, you may run the same code and it will continue search from where it stopped.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tuner.results_summary()\nmodel = tuner.get_best_models()[0]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It is usually good to fit the best model with all data including validation data after hyperparameter search is done.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"ds_all =  get_training_dataset(load_dataset(TRAINING_FILENAMES + VALIDATION_FILENAMES))\n\n# Train the best model with all data\nmodel.fit(ds_all,\n          epochs=EPOCHS_FINAL,\n          steps_per_epoch=num_train_batches + num_val_batches,\n          callbacks=[tf.keras.callbacks.ReduceLROnPlateau()],\n          verbose=2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Prediction and create submission file","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"ds_test = get_test_dataset(ordered=True)\n\nprint('Computing predictions...')\npredictions = []\n\nfor i, (test_img, test_id) in enumerate(ds_test):\n    print('Processing batch ', i)\n    probabilities = model(test_img)\n    prediction = np.argmax(probabilities, axis=-1)\n    predictions.append(prediction)\n\npredictions = np.concatenate(predictions)\nprint('Number of test examples predicted: ', predictions.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Get image ids from test set and convert to unicode\nds_test_ids = ds_test.map(lambda image, idnum: idnum).unbatch()\ntest_ids = next(iter(ds_test_ids.batch(np.iinfo(np.int64).max))).numpy().astype('U')\n\n# Write the submission file\nnp.savetxt(\n    'submission.csv',\n    np.rec.fromarrays([test_ids, predictions]),\n    fmt=['%s', '%d'],\n    delimiter=',',\n    header='id,label',\n    comments='',\n)\n\n# Look at the first few predictions\n!head submission.csv","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}