{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Petals to the Metal: Flower Classification using TPU\n\nThe aim is to classify all the flower images in the tpu-getting-started into 104 classes. All the files are in .tfrec format which tensorflow records format. It is the tensorflow's own binary storage format.\n\n### Why .tfrec format ?\nIf you are working with large datasets, using a binary file format for storage of your data can have a significant impact on the performance of your import pipeline and as a consequence on the training time of your model. Binary data takes up less space on disk, takes less time to copy and can be read much more efficiently from disk. This is especially true if your data is stored on spinning disks, due to the much lower read/write performance in comparison with SSDs.\n\nFor more information, read https://medium.com/mostly-ai/tensorflow-records-what-they-are-and-how-to-use-them-c46bc4bbb564"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport torch\nfrom torch.utils.data import Dataset, DataLoader\nimport os, cv2, re\nimport tensorflow as tf\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom kaggle_datasets import KaggleDatasets\nfrom keras.layers import Input, Dense, GlobalAveragePooling2D, Dropout, BatchNormalization\nfrom keras.applications.xception import Xception\nfrom keras.models import Model, load_model\nfrom keras.callbacks import ModelCheckpoint","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Turn on TPU"},{"metadata":{"trusted":true},"cell_type":"code","source":"try:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver() \n    print('Running on TPU ', tpu.master())\nexcept ValueError:\n    tpu = None\n\nif tpu:\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n    strategy = tf.distribute.get_strategy() \n    \nstrategy.num_replicas_in_sync","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"GPUs and TPUs can radically reduce the time required to execute a single training step. Achieving peak performance requires an efficient input pipeline that delivers data for the next step before the current step has finished. The tf.data API helps to build flexible and efficient input pipelines. \n\nThe tf.data API provides the tf.data.Dataset.prefetch transformation. It can be used to decouple the time when data is produced from the time when data is consumed. In particular, the transformation uses a background thread and an internal buffer to prefetch elements from the input dataset ahead of the time they are requested. The number of elements to prefetch should be equal to (or possibly greater than) the number of batches consumed by a single training step.\n\n     tf.data.experimental.AUTOTUNE \nwhich will prompt the tf.data runtime to tune the value dynamically at runtime."},{"metadata":{"trusted":true},"cell_type":"code","source":"Image_size=[512,512]\nbatch_size=16 * strategy.num_replicas_in_sync\n\nAUTO = tf.data.experimental.AUTOTUNE","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Use KaggleDatasets().get_gcs_path() to retrieve public GCS paths from a public Kaggle dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"GCS_DS_PATH = KaggleDatasets().get_gcs_path('tpu-getting-started')\n\nGCS_PATH_SELECT = { \n    192: GCS_DS_PATH + '/tfrecords-jpeg-192x192',\n    224: GCS_DS_PATH + '/tfrecords-jpeg-224x224',\n    331: GCS_DS_PATH + '/tfrecords-jpeg-331x331',\n    512: GCS_DS_PATH + '/tfrecords-jpeg-512x512'\n}\n\nGCS_PATH = GCS_PATH_SELECT[Image_size[0]]\n\ntrain_files = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec') \nval_files = tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\ntest_files = tf.io.gfile.glob(GCS_PATH + '/test/*.tfrec')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Helper functions"},{"metadata":{"trusted":true},"cell_type":"code","source":"def data_augment(image, label):\n    # data augmentation. Thanks to the dataset.prefetch(AUTO) statement in the next function (below),\n    # this happens essentially for free on TPU. Data pipeline code is executed on the \"CPU\" part\n    # of the TPU while the TPU itself is computing gradients.\n    image = tf.image.random_flip_left_right(image)\n    return image, label \n\ndef decode_image(image_data):\n    image = tf.image.decode_jpeg(image_data, channels=3)\n    image = tf.cast(image, tf.float32) / 255.0  # convert image to floats in [0, 1] range\n    image = tf.reshape(image, [*Image_size, 3]) # explicit size needed for TPU\n    return image\n\ndef read_labeled_tfrecord(example):\n    LABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n        \"class\": tf.io.FixedLenFeature([], tf.int64),  # shape [] means single element\n    }\n    example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    label = tf.cast(example['class'], tf.int32)\n    return image, label\n\ndef read_unlabeled_tfrecord(example):\n    UNLABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n        \"id\": tf.io.FixedLenFeature([], tf.string),  # shape [] means single element\n        # class is missing, this competitions's challenge is to predict flower classes for the test dataset\n    }\n    example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    idnum = example['id']\n    return image, idnum\n\ndef load_dataset(filenames, labeled=True, ordered=False):\n    # Read from TFRecords. For optimal performance, reading from multiple files at once and\n    # disregarding data order. Order does not matter since we will be shuffling the data anyway.\n\n    ignore_order = tf.data.Options()\n    if not ordered:\n        ignore_order.experimental_deterministic = False # disable order, increase speed\n\n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO) # automatically interleaves reads from multiple files\n    dataset = dataset.with_options(ignore_order) # uses data as soon as it streams in, rather than in its original order\n    dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord, num_parallel_calls=AUTO)\n    # returns a dataset of (image, label) pairs if labeled=True or (image, id) pairs if labeled=False\n    return dataset\n\ndef get_training_dataset(train_files):\n    dataset = load_dataset(train_files, labeled=True)\n    dataset = dataset.map(data_augment, num_parallel_calls=AUTO)\n    dataset = dataset.repeat() # the training dataset must repeat for several epochs\n    dataset = dataset.shuffle(2048)\n    dataset = dataset.batch(batch_size)\n    dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset\n\ndef get_validation_dataset(val_files, ordered=False):\n    dataset = load_dataset(val_files, labeled=True, ordered=False)\n    dataset = dataset.batch(batch_size)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO) \n    return dataset\n    \ndef get_test_dataset(test_files, ordered=False):\n    dataset = load_dataset(test_files , labeled=False, ordered=ordered)\n    dataset = dataset.batch(batch_size)\n    dataset = dataset.cache() \n    dataset = dataset.prefetch(AUTO)\n    return dataset\n\ndef count_data_items(filenames):\n    # the number of data items is written in the name of the .tfrec files, i.e. flowers00-230.tfrec = 230 data items\n    n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) for filename in filenames]\n    return np.sum(n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Make training, validation and test data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_dataset = get_training_dataset(train_files)\nvalidation_dataset = get_validation_dataset(val_files)\ntest_dataset = get_test_dataset(test_files)\n\nprint(validation_dataset)\nprint(test_dataset)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"NUM_TRAINING_IMAGES = count_data_items(train_files)\nNUM_VALIDATION_IMAGES = count_data_items(val_files)\nNUM_TEST_IMAGES = count_data_items(test_files)\n\nprint('Training_size=',NUM_TRAINING_IMAGES  ,'Validation size=',NUM_VALIDATION_IMAGES , 'Test size=',NUM_TEST_IMAGES )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    image_input = Input(shape=(*Image_size,3))\n    base_model = Xception(include_top=False, input_tensor=image_input, weights='imagenet')\n    x = base_model.layers[-1].output\n    x = BatchNormalization()(x)\n    x = GlobalAveragePooling2D()(x)\n#     x = Dropout(0.25)(x)\n    output = Dense(104, activation='softmax')(x)\n\n    model = Model(inputs = image_input, outputs = output )\n    model.compile(loss= 'sparse_categorical_crossentropy', optimizer='adam', metrics =['accuracy'])\n\n# model.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Callbacks for the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# define checkpoint callback\nfilepath = './model-ep{epoch:02d}-val_acc{val_accuracy:.3f}.h5'\n\ncallbacks = [ ModelCheckpoint(filepath= filepath, save_best_only=True, monitor='val_accuracy', mode='max') ]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Train the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# epochs = 30\n# STEPS_FOR_EPOCH = NUM_TRAINING_IMAGES// batch_size\n\n# history = model.fit(train_dataset, \n#                     steps_per_epoch=STEPS_FOR_EPOCH, \n#                     epochs = epochs, \n#                     batch_size = batch_size,\n#                     callbacks = callbacks,\n#                     validation_data=validation_dataset)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Plot the performance of the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# plt.figure(figsize=(20,6))\n# titles = ['LOSS', 'ACCURACY']\n# ylabel = ['LOSSES','ACCURACY']\n# par = ['loss', 'accuracy']\n# color = ['Red','Blue','Orange','Green']\n# j =0\n# for i in range(2):\n#     plt.subplot(1,2, i+1)\n#     plt.plot(history.history[par[i]], c= color[j])\n#     plt.plot(history.history['val_'+ par[i] ], c= color[j+1])\n#     plt.title('MODEL '+ titles[i])\n#     plt.xlabel('EPOCHS')\n#     plt.ylabel(ylabel[i])\n#     j = j+2\n#     plt.legend(['training', 'validation'])\n# plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Loading the model trained in the previous version with an accuracy of 90 %"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = load_model('../input/flower-classification-model/model-ep30-val_acc0.907.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Make predictions"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_ds = get_test_dataset(test_files, ordered=True) \n\ntest_images_ds = test_ds.map(lambda image, idnum: image)\nprobabilities = model.predict(test_images_ds)\npredictions = np.argmax(probabilities, axis=-1)\nprint(predictions)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Save the predictions in the csv format"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_ids_ds = test_ds.map(lambda image, idnum: idnum).unbatch()\ntest_ids = next(iter(test_ids_ds.batch(NUM_TEST_IMAGES))).numpy().astype('U') # all in one batch\nnp.savetxt('submission.csv', np.rec.fromarrays([test_ids, np.array(predictions)]), fmt=['%s', '%d'], delimiter=',', header='id,label', comments='')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.read_csv('./submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}