{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Petals to the Metal - Flower Classification on TPU\n### Getting Started with TPUs on Kaggle!","metadata":{"execution":{"iopub.status.busy":"2023-07-07T09:38:43.450208Z","iopub.execute_input":"2023-07-07T09:38:43.450631Z","iopub.status.idle":"2023-07-07T09:38:43.456260Z","shell.execute_reply.started":"2023-07-07T09:38:43.450598Z","shell.execute_reply":"2023-07-07T09:38:43.455135Z"}}},{"cell_type":"markdown","source":"## The Challenge\nIt’s difficult to fathom just how vast and diverse our natural world is.\n\nThere are over 5,000 species of mammals, 10,000 species of birds, 30,000 species of fish – and astonishingly, over 400,000 different types of flowers.\n\nIn this competition, you’re challenged to build a machine learning model that identifies the type of flowers in a dataset of images (for simplicity, we’re sticking to just over 100 types).","metadata":{}},{"cell_type":"markdown","source":"## Dataset Description\nIn this Getting Started competition (what is a Getting Started competition?), we're classifying 104 types of flowers based on their images drawn from five different public datasets. Some classes are very narrow, containing only a particular sub-type of flower (e.g. pink primroses) while other classes contain many sub-types (e.g. wild roses).\n\nThe dataset contains imperfections - images of flowers in odd places, or as a backdrop to modern machinery - but that's part of the challenge! Build a classifier than can see past all that, to the flowers at the heart of the images.\n\n## Files\nThis competition provides its files in TFRecord format. The TFRecord format is a container format frequently used in Tensorflow to group and shard data data files for optimal training performace.\nEach file contains the id, label (the class of the sample, for training data) and img (the actual pixels in array form) information for many images.\nPlease see our Getting Started notebook or our Learn exercise for notes on how to load and use them! Additional information is available in the TPU documentation.\n\n* train/*.tfrec - training samples, including labels.\n\n* val/*.tfrec - pre-split training samples w/ labels intended to help with checking your model's performance on TPU. The split was stratified across labels.\n\n* test/*.tfrec - samples without labels - you'll be predicting what classes of flowers these fall into.\n\n* sample_submission.csv - a sample submission file in the correct format\n\n* id - a unique ID for each sample.\n\n* label - (in training data) the class of flower represented by the sample","metadata":{}},{"cell_type":"markdown","source":"This command will install the EfficientNet library, which provides efficient and accurate convolutional neural network architectures. The EfficientNet models have achieved state-of-the-art performance on various image classification tasks.","metadata":{}},{"cell_type":"code","source":"!pip install efficientnet","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import math\nimport numpy as np\nimport tensorflow as tf\nimport tensorflow.keras as keras\nfrom tensorflow.keras import mixed_precision\nfrom efficientnet.keras import EfficientNetB7\nfrom kaggle_datasets import KaggleDatasets\nfrom sklearn.metrics import f1_score, precision_score, recall_score, confusion_matrix\nimport re,os\nfrom matplotlib import pyplot as plt","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Step | Explanation |\n|------|-------------|\n| 1    | Attempt to create a `TPUClusterResolver` object to detect the presence of a TPU. |\n| 2    | If a TPU is found, print a message indicating that the code is running on a TPU and connect to the TPU cluster using `tf.config.experimental_connect_to_cluster`. |\n| 3    | Initialize the TPU system using `tf.tpu.experimental.initialize_tpu_system` to configure the TPU and its associated devices. |\n| 4    | Return a `TPUStrategy` object created with the `TPUClusterResolver`, enabling distributed training on the TPU. |\n| 5    | If no TPU is found or an exception occurs, fall back to using the default distribution strategy (`tf.distribute.get_strategy()`). |\n\nBy following these steps, the `get_strategy` function allows you to detect and utilize TPUs for faster and more efficient training when available, while gracefully falling back to other devices if TPUs are not present.","metadata":{}},{"cell_type":"code","source":"# Detect TPU, return appropriate distribution strategy\ndef get_strategy():\n    try:\n        tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n        print('Running on TPU ', tpu.master())\n        tf.config.experimental_connect_to_cluster(tpu)\n        tf.tpu.experimental.initialize_tpu_system(tpu)\n        return tf.distribute.experimental.TPUStrategy(tpu)\n    except ValueError:\n        return tf.distribute.get_strategy()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"strategy = get_strategy()\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"GCS_DS_PATH = KaggleDatasets().get_gcs_path('tpu-getting-started')\nprint(GCS_DS_PATH)  # What do GCS paths look like?","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IMAGE_SIZE = [512, 512]\nGCS_PATH = GCS_DS_PATH + '/tfrecords-jpeg-512x512'\nAUTO = tf.data.experimental.AUTOTUNE","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TRAINING_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec')\nVALIDATION_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\nTEST_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/test/*.tfrec')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"CLASSES = ['pink primrose', 'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea', 'wild geranium', 'tiger lily',\n           'moon orchid', 'bird of paradise', 'monkshood', 'globe thistle', 'snapdragon', \"colt's foot\", 'king protea',\n           'spear thistle', 'yellow iris', 'globe-flower', 'purple coneflower', 'peruvian lily', 'balloon flower',\n           'giant white arum lily', 'fire lily', 'pincushion flower', 'fritillary', 'red ginger', 'grape hyacinth',\n           'corn poppy', 'prince of wales feathers', 'stemless gentian', 'artichoke', 'sweet william',\n           'carnation', 'garden phlox', 'love in the mist', 'cosmos', 'alpine sea holly', 'ruby-lipped cattleya',\n           'cape flower', 'great masterwort', 'siam tulip', 'lenten rose', 'barberton daisy', 'daffodil', 'sword lily',\n           'poinsettia', 'bolero deep blue', 'wallflower', 'marigold', 'buttercup', 'daisy', 'common dandelion',\n           'petunia', 'wild pansy', 'primula', 'sunflower', 'lilac hibiscus', 'bishop of llandaff', 'gaura', 'geranium',\n           'orange dahlia', 'pink-yellow dahlia', 'cautleya spicata', 'japanese anemone', 'black-eyed susan',\n           'silverbush', 'californian poppy', 'osteospermum', 'spring crocus', 'iris', 'windflower', 'tree poppy',\n           'gazania', 'azalea', 'water lily', 'rose', 'thorn apple', 'morning glory', 'passion flower', 'lotus',\n           'toad lily', 'anthurium', 'frangipani', 'clematis', 'hibiscus', 'columbine', 'desert-rose', 'tree mallow',\n           'magnolia', 'cyclamen ', 'watercress', 'canna lily', 'hippeastrum ', 'bee balm', 'pink quill', 'foxglove',\n           'bougainvillea', 'camellia', 'mallow', 'mexican petunia', 'bromelia', 'blanket flower', 'trumpet creeper',\n           'blackberry lily', 'common tulip', 'wild rose']","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Line                                              | Explanation                                                                                           |\n|---------------------------------------------------|-------------------------------------------------------------------------------------------------------|\n| `image = tf.image.decode_jpeg(image_data, channels=3)`      | Decode the JPEG image data using TensorFlow's `decode_jpeg` function with 3 color channels (RGB). |\n| `image = tf.cast(image, tf.float32) / 255.0`                | Convert the image to floating-point values in the range [0, 1] by dividing by 255.0.              |\n| `image = tf.reshape(image, [*IMAGE_SIZE, 3])`               | Reshape the image tensor to the desired `IMAGE_SIZE` dimensions, where `IMAGE_SIZE` is specified.  |\n| Return `image`                                         | Return the decoded and processed image tensor.                                                      |","metadata":{}},{"cell_type":"code","source":"def decode_image(image_data):\n    image = tf.image.decode_jpeg(image_data, channels=3)\n    image = tf.cast(image, tf.float32) / 255.0  # convert image to floats in [0, 1] range\n    image = tf.reshape(image, [*IMAGE_SIZE, 3])  # explicit size needed for TPU\n    return image","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Line                                                       | Explanation                                                                                                    |\n|------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------|\n| `LABELED_TFREC_FORMAT = {...}`                              | Define the format of a labeled TFRecord example, specifying the features and their data types.                 |\n| `example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)` | Parse a single example from the TFRecord using the defined format, extracting the features as tensors.     |\n| `image = decode_image(example['image'])`                           | Decode the image feature using the `decode_image` function defined earlier to obtain the image tensor.     |\n| `label = tf.cast(example['class'], tf.int32)`                      | Extract the label feature from the example and cast it to the `int32` data type to obtain the label tensor. |\n| Return `image, label`                                      | Return the image and label tensors as a tuple.                                                                |","metadata":{}},{"cell_type":"code","source":"def read_labeled_tfrecord(example):\n    LABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),  # tf.string means bytestring\n        \"class\": tf.io.FixedLenFeature([], tf.int64),  # shape [] means single element\n    }\n    example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    label = tf.cast(example['class'], tf.int32)\n    return image, label","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Line                                                       | Explanation                                                                                                    |\n|------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------|\n| `UNLABELED_TFREC_FORMAT = {...}`                            | Define the format of an unlabeled TFRecord example, specifying the features and their data types.               |\n| `example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)` | Parse a single example from the TFRecord using the defined format, extracting the features as tensors.     |\n| `image = decode_image(example['image'])`                           | Decode the image feature using the `decode_image` function defined earlier to obtain the image tensor.     |\n| `idnum = example['id']`                                            | Extract the ID feature from the example to obtain the ID tensor.                                             |\n| Return `image, idnum`                                      | Return the image tensor and ID tensor as a tuple.                                                               |","metadata":{}},{"cell_type":"code","source":"def read_unlabeled_tfrecord(example):\n    UNLABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),  # tf.string means bytestring\n        \"id\": tf.io.FixedLenFeature([], tf.string),  # shape [] means single element\n    }\n    example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    idnum = example['id']\n    return image, idnum","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Line                                                | Explanation                                                                                                                                                      |\n|-----------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `ignore_order = tf.data.Options()`                   | Create an instance of `tf.data.Options` to configure the dataset options.                                                                                       |\n| `if not ordered:`                                    | Check if the dataset should be ordered.                                                                                                                         |\n| `ignore_order.experimental_deterministic = False`    | Set the `experimental_deterministic` attribute of `ignore_order` to `False` if the dataset does not need to be ordered.                                         |\n| `dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO)` | Create a `TFRecordDataset` object with the given filenames to read the TFRecord files.                                                                      |\n| `dataset = dataset.with_options(ignore_order)`       | Apply the dataset options specified by `ignore_order` to the dataset.                                                                                            |\n| `dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord, num_parallel_calls=AUTO)` | Apply the appropriate parsing function (`read_labeled_tfrecord` or `read_unlabeled_tfrecord`) to each example in the dataset using parallel calls. |\n| Return `dataset`                                      | Return the processed dataset.                                                                                                                                    |","metadata":{}},{"cell_type":"code","source":"def load_dataset(filenames, labeled=True, ordered=False):\n    ignore_order = tf.data.Options()\n    if not ordered:\n        ignore_order.experimental_deterministic = False\n\n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO)\n    dataset = dataset.with_options(ignore_order)\n    dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord, num_parallel_calls=AUTO)\n    return dataset","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Line                               | Explanation                                                                                                                                              |\n|------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `image = tf.image.random_flip_left_right(image)` | Apply a random horizontal flip to the image. This randomly flips the image horizontally with a 50% probability.                                           |\n| Return `(image, label)`             | Return the augmented image and the original label.                                                                                                      |","metadata":{}},{"cell_type":"code","source":"def data_augment(image, label):\n    image = tf.image.random_flip_left_right(image)\n    return image, label","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Line                        | Explanation                                                                                                                                                                                                                                     |\n|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `dataset = load_dataset(TRAINING_FILENAMES, labeled=True)` | Load the training dataset using the `load_dataset` function with `labeled=True`. This will create a dataset of (image, label) pairs.                                                                                                          |\n| `dataset = dataset.map(data_augment, num_parallel_calls=AUTO)` | Apply data augmentation to the dataset using the `data_augment` function. This function performs random horizontal flips on the images. The `num_parallel_calls` argument specifies the level of parallelism to use during the mapping operation. |\n| `dataset = dataset.repeat()`          | Repeat the dataset indefinitely. This is done to ensure that the dataset repeats for multiple epochs during training.                                                                                                                          |\n| `dataset = dataset.shuffle(2048)`       | Shuffle the dataset with a buffer size of 786. This randomly shuffles the examples in the dataset to introduce randomness and prevent the model from overfitting to the order of the training data.                                             |\n| `dataset = dataset.batch(BATCH_SIZE)` | Create batches of the dataset with a batch size of `BATCH_SIZE`. This groups multiple examples together into a single batch.                                                                                                                   |\n| `dataset = dataset.prefetch(AUTO)`     | Prefetch the dataset to optimize data loading. This allows the dataset to fetch batches in the background while the model is training on the current batch, reducing the training time.                                                        |\n| Return `dataset`             | Return the processed training dataset.                                                                                                                                                                                                          |","metadata":{}},{"cell_type":"code","source":"def get_training_dataset():\n    dataset = load_dataset(TRAINING_FILENAMES, labeled=True)\n    dataset = dataset.map(data_augment, num_parallel_calls=AUTO)\n    dataset = dataset.repeat()\n    dataset = dataset.shuffle(786)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO)\n    return dataset","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Line                        | Explanation                                                                                                                                                                                                                          |\n|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `dataset = load_dataset(VALIDATION_FILENAMES, labeled=True, ordered=ordered)` | Load the validation dataset using the `load_dataset` function with `labeled=True` and `ordered=ordered`. This will create a dataset of (image, label) pairs from the validation filenames. The `ordered` argument specifies whether the dataset should maintain the order of the examples. |\n| `dataset = dataset.batch(BATCH_SIZE)`    | Create batches of the dataset with a batch size of `BATCH_SIZE`. This groups multiple examples together into a single batch.                                                                                                         |\n| `dataset = dataset.cache()`                | Cache the dataset in memory. This improves the data loading speed as the dataset does not need to be read from disk repeatedly.                                                                                                     |\n| `dataset = dataset.prefetch(AUTO)`    | Prefetch the dataset to optimize data loading. This allows the dataset to fetch batches in the background while the model is training on the current batch, reducing the validation time.                                              |\n| Return `dataset`             | Return the processed validation dataset.                                                                                                                                                                                             |","metadata":{}},{"cell_type":"code","source":"def get_validation_dataset(ordered=False):\n    dataset = load_dataset(VALIDATION_FILENAMES, labeled=True, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO)\n    return dataset","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"| Line                        | Explanation                                                                                                                                                                                                                          |\n|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `dataset = load_dataset(TEST_FILENAMES, labeled=False, ordered=ordered)` | Load the test dataset using the `load_dataset` function with `labeled=False` and `ordered=ordered`. This will create a dataset of (image, id) pairs from the test filenames. The `ordered` argument specifies whether the dataset should maintain the order of the examples. |\n| `dataset = dataset.batch(BATCH_SIZE)`    | Create batches of the dataset with a batch size of `BATCH_SIZE`. This groups multiple examples together into a single batch.                                                                                                         |\n| `dataset = dataset.prefetch(AUTO)`    | Prefetch the dataset to optimize data loading. This allows the dataset to fetch batches in the background while the model is predicting on the current batch, reducing the inference time.                                              |\n| Return `dataset`             | Return the processed test dataset.                                                                                                                                                                                                   |","metadata":{}},{"cell_type":"code","source":"def get_test_dataset(ordered=False):\n    dataset = load_dataset(TEST_FILENAMES, labeled=False, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO)\n    return dataset","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The `count_data_items` function takes a list of filenames as input and counts the number of data items based on the filename patterns. Here's an explanation of each line:\n\n| Line           | Explanation                                                                                                                                                                                         |\n|----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `pattern = r\"-([0-9]*)\\.\"` | Define a regular expression pattern that matches the numerical part in the filename. The pattern captures digits between a hyphen (`-`) and a dot (`.`).                                                   |\n| `count = 0`    | Initialize a variable `count` to keep track of the total count of data items.                                                                                                                       |\n| `for filename in filenames:` | Iterate over each filename in the given list of filenames.                                                                                                                                          |\n| `match = re.search(pattern, filename)` | Use the `re.search()` function to search for a match of the pattern in the current filename.                                                                                                       |\n| `if match:`    | Check if a match is found.                                                                                                                                             |\n| `count += int(match.group(1))` | If a match is found, extract the numerical part using `match.group(1)` and convert it to an integer. Increment the count by the extracted value.                                              |\n| `return count` | Return the final count of data items.                                                                                                                                                               |\n\nThe `count_data_items` function calculates the total count of data items by extracting the numerical part from the filenames and summing them up.","metadata":{}},{"cell_type":"code","source":"def count_data_items(filenames):\n    pattern = r\"-([0-9]*)\\.\"\n    count = 0\n    for filename in filenames:\n        match = re.search(pattern, filename)\n        if match:\n            count += int(match.group(1))\n    return count","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"NUM_TRAINING_IMAGES = count_data_items(TRAINING_FILENAMES)\nNUM_VALIDATION_IMAGES = count_data_items(VALIDATION_FILENAMES)\nNUM_TEST_IMAGES = count_data_items(TEST_FILENAMES)\nprint('Dataset: {} training images, {} validation images, {} unlabeled test images'.format(NUM_TRAINING_IMAGES,\n                                                                                           NUM_VALIDATION_IMAGES,\n                                                                                           NUM_TEST_IMAGES))\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BATCH_SIZE = 8 * strategy.num_replicas_in_sync","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_train = get_training_dataset()\nds_valid = get_validation_dataset()\nds_test = get_test_dataset()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Training:\", ds_train)\nprint(\"Validation:\", ds_valid)\nprint(\"Test:\", ds_test)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.set_printoptions(threshold=15, linewidth=80)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The `batch_to_numpy_images_and_labels` function takes a batch of data as input and converts the images and labels to NumPy arrays. Here's an explanation of each line:\n\n| Line           | Explanation                                                                                                                                                                                         |\n|----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `images, labels = data` | Unpack the input `data` into `images` and `labels` variables.                                                                                                                                        |\n| `numpy_images = images.numpy()` | Convert the `images` tensor to a NumPy array using the `numpy()` method.                                                                                                                            |\n| `numpy_labels = labels.numpy()` | Convert the `labels` tensor to a NumPy array using the `numpy()` method.                                                                                                                            |\n| `if numpy_labels.dtype == object:` | Check if the data type of `numpy_labels` is `object`, indicating that it contains string labels (e.g., image IDs).                                                                                 |\n| `numpy_labels = [None for _ in enumerate(numpy_images)]` | If `numpy_labels` contains string labels, replace each label with `None`. This is done to handle the case when the labels are not available (e.g., for test data).                            |\n| `return numpy_images, numpy_labels` | Return the NumPy arrays `numpy_images` and `numpy_labels`.                                                                                                                                           |\n\nThe `batch_to_numpy_images_and_labels` function converts the images and labels in a batch of data to NumPy arrays. If the labels are of type `object`, indicating string labels, they are replaced with `None`.","metadata":{}},{"cell_type":"code","source":"def batch_to_numpy_images_and_labels(data):\n    images, labels = data\n    numpy_images = images.numpy()\n    numpy_labels = labels.numpy()\n    if numpy_labels.dtype == object:\n        numpy_labels = [None for _ in enumerate(numpy_images)]\n    return numpy_images, numpy_labels","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The `title_from_label_and_target` function generates a title based on the predicted label (`label`) and the correct label (`correct_label`). It also determines whether the prediction is correct or not. Here's an explanation of each line:\n\n| Line           | Explanation                                                                                                                                                                                         |\n|----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `if correct_label is None:` | Check if the `correct_label` is `None`, indicating that the correct label is not available (e.g., for test data).                                                                                   |\n| &emsp; `return CLASSES[label], True` | If the correct label is not available, return the class name corresponding to the predicted label (`label`) and `True` to indicate that the prediction is correct.                               |\n| `correct = (label == correct_label)` | Determine whether the predicted label matches the correct label.                                                                                                                                     |\n| `return \"{} [{}{}{}]\".format(CLASSES[label], 'OK' if correct else 'NO', u\"\\u2192\" if not correct else '', CLASSES[correct_label] if not correct else ''), correct` | Generate the title based on the predicted label, the correctness indicator ('OK' or 'NO'), the arrow symbol ('→'), and the correct label. Return the title and the correctness indicator. |\n\nThe `title_from_label_and_target` function provides a formatted title based on the predicted and correct labels, along with a correctness indicator.","metadata":{}},{"cell_type":"code","source":"def title_from_label_and_target(label, correct_label):\n    if correct_label is None:\n        return CLASSES[label], True\n    correct = (label == correct_label)\n    return \"{} [{}{}{}]\".format(CLASSES[label], 'OK' if correct else 'NO', u\"\\u2192\" if not correct else '',\n                                CLASSES[correct_label] if not correct else ''), correct\n\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The `display_one_flower` function is used to display a single flower image along with its title in a subplot. Here's an explanation of each line:\n\n| Line           | Explanation                                                                                                                                                                                         |\n|----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `plt.subplot(*subplot)` | Create a subplot using the dimensions specified in the `subplot` parameter.                                                                                                                         |\n| `plt.axis('off')` | Turn off the axis for the subplot.                                                                                                                                                                  |\n| `plt.imshow(image)` | Display the image in the subplot.                                                                                                                                                                   |\n| `if len(title) > 0:` | Check if a title is provided (i.e., the length of the title is greater than 0).                                                                                                                     |\n| &emsp; `plt.title(title, fontsize=int(titlesize) if not red else int(titlesize / 1.2), color='red' if red else 'black', fontdict={'verticalalignment': 'center'}, pad=int(titlesize / 1.5))` | Set the title for the subplot with the specified properties, such as font size, color, alignment, and padding. The title may be displayed in red if `red` is `True`. |\n| `return (subplot[0], subplot[1], subplot[2] + 1)` | Return the updated subplot dimensions `(nrows, ncols, index)` for the next subplot. The `index` is incremented by 1 to move to the next position.                                              |\n\nThe `display_one_flower` function allows you to display a single flower image with a customizable title in a subplot. The function is useful for visualizing images and their corresponding titles.","metadata":{}},{"cell_type":"code","source":"def display_one_flower(image, title, subplot, red=False, titlesize=16):\n    plt.subplot(*subplot)\n    plt.axis('off')\n    plt.imshow(image)\n    if len(title) > 0:\n        plt.title(title, fontsize=int(titlesize) if not red else int(titlesize / 1.2),\n                  color='red' if red else 'black',\n                  fontdict={'verticalalignment': 'center'}, pad=int(titlesize / 1.5))\n    return (subplot[0], subplot[1], subplot[2] + 1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_batch_of_images(databatch, predictions=None):\n    rows = int(math.sqrt(len(images)))\n    cols = len(images) // rows\n\n    FIGSIZE = 13.0\n    SPACING = 0.1\n    subplot = (rows, cols, 1)\n    if rows < cols:\n        plt.figure(figsize=(FIGSIZE, FIGSIZE / cols * rows))\n    else:\n        plt.figure(figsize=(FIGSIZE / rows * cols, FIGSIZE))\n\n    for i, (image, label) in enumerate(zip(images[:rows * cols], labels[:rows * cols])):\n        title = '' if label is None else CLASSES[label]\n        correct = True\n        if predictions is not None:\n            title, correct = title_from_label_and_target(predictions[i], label)\n        dynamic_titlesize = FIGSIZE * SPACING / max(rows, cols) * 40 + 3\n        subplot = display_one_flower(image, title, subplot, not correct, titlesize=dynamic_titlesize)\n\n    plt.tight_layout()\n    if label is None and predictions is None:\n        plt.subplots_adjust(wspace=0, hspace=0)\n    else:\n        plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n    plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_training_curves(training, validation, title, subplot):\n    if subplot % 10 == 1:\n        plt.subplots(figsize=(10, 10), facecolor='#F0F0F0')\n        plt.tight_layout()\n    ax = plt.subplot(subplot)\n    ax.set_facecolor('#F8F8F8')\n    ax.plot(training)\n    ax.plot(validation)\n    ax.set_title('model ' + title)\n    ax.set_ylabel(title)\n    ax.set_xlabel('epoch')\n    ax.legend(['train', 'valid.'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_confusion_matrix(cmat, score, precision, recall):\n    plt.figure(figsize=(15, 15))\n    ax = plt.gca()\n    ax.matshow(cmat, cmap='Reds')\n    ax.set_xticks(range(len(CLASSES)))\n    ax.set_xticklabels(CLASSES, fontdict={'fontsize': 7})\n    plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n    ax.set_yticks(range(len(CLASSES)))\n    ax.set_yticklabels(CLASSES, fontdict={'fontsize': 7})\n    plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n    titlestring = \"\"\n    if score is not None:\n        titlestring += 'f1 = {:.3f} '.format(score)\n    if precision is not None:\n        titlestring += '\\nprecision = {:.3f} '.format(precision)\n    if recall is not None:\n        titlestring += '\\nrecall = {:.3f} '.format(recall)\n    if len(titlestring) > 0:\n        ax.text(101, 1, titlestring, fontdict={'fontsize': 18, 'horizontalalignment': 'right',\n                                                'verticalalignment': 'top', 'color': '#804040'})\n    plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_model():\n    with strategy.scope():\n        pretrained_model = EfficientNetB7(\n            weights='imagenet',\n            include_top=False,\n            input_shape=[*IMAGE_SIZE, 3]\n        )\n        pretrained_model.trainable = False\n\n        model = keras.Sequential([\n            pretrained_model,\n            tf.keras.layers.GlobalAveragePooling2D(),\n            tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n        ])\n\n        model.compile(\n            optimizer='adam',\n            loss='sparse_categorical_crossentropy',\n            metrics=['sparse_categorical_accuracy'],\n        )\n    return model","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def exponential_lr(epoch, start_lr=0.00001, min_lr=0.00001, max_lr=0.00005, rampup_epochs=15, sustain_epochs=0,\n                   exp_decay=0.9):\n    def lr(epoch, start_lr, min_lr, max_lr, rampup_epochs, sustain_epochs, exp_decay):\n        if epoch < rampup_epochs:\n            lr = ((max_lr - start_lr) / rampup_epochs * epoch + start_lr)\n        elif epoch < rampup_epochs + sustain_epochs:\n            lr = max_lr\n        else:\n            lr = ((max_lr - min_lr) *\n                  exp_decay ** (epoch - rampup_epochs - sustain_epochs) + min_lr)\n        return lr\n\n    return lr(epoch, start_lr, min_lr, max_lr, rampup_epochs, sustain_epochs, exp_decay)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def create_lr_callback():\n    lr_callback = tf.keras.callbacks.LearningRateScheduler(exponential_lr, verbose=True)\n    return lr_callback","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def train_model(model, ds_train, ds_valid):\n    EPOCHS = 100\n    STEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\n\n    history = model.fit(\n        ds_train,\n        validation_data=ds_valid,\n        epochs=EPOCHS,\n        steps_per_epoch=STEPS_PER_EPOCH,\n        callbacks=[create_lr_callback()],\n    )\n    return history","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def compute_metrics(model, cm_dataset):\n    images_ds = cm_dataset.map(lambda image, label: image)\n    labels_ds = cm_dataset.map(lambda image, label: label).unbatch()\n\n    cm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\n    cm_probabilities = model.predict(images_ds)\n    cm_predictions = np.argmax(cm_probabilities, axis=-1)\n\n    labels = range(len(CLASSES))\n    cmat = confusion_matrix(\n        cm_correct_labels,\n        cm_predictions,\n        labels=labels,\n    )\n    cmat = (cmat.T / cmat.sum(axis=1)).T\n\n    score = f1_score(\n        cm_correct_labels,\n        cm_predictions,\n        labels=labels,\n        average='macro',\n    )\n    precision = precision_score(\n        cm_correct_labels,\n        cm_predictions,\n        labels=labels,\n        average='macro',\n    )\n    recall = recall_score(\n        cm_correct_labels,\n        cm_predictions,\n        labels=labels,\n        average='macro',\n    )\n    return cmat, score, precision, recall","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def generate_submission_file(model):\n    test_ds = get_test_dataset(ordered=True)\n    test_images_ds = test_ds.map(lambda image, idnum: image)\n    probabilities = model.predict(test_images_ds)\n    predictions = np.argmax(probabilities, axis=-1)\n\n    test_ids_ds = test_ds.map(lambda image, idnum: idnum).unbatch()\n    test_ids = next(iter(test_ids_ds.batch(NUM_TEST_IMAGES))).numpy().astype('U')\n\n    np.savetxt(\n        'submission.csv',\n        np.rec.fromarrays([test_ids, predictions]),\n        fmt=['%s', '%d'],\n        delimiter=',',\n        header='id,label',\n        comments='',\n    )\n\n    !head submission.csv","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = get_model()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = train_model(model, ds_train, ds_valid)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_training_curves(\n    history.history['loss'],\n    history.history['val_loss'],\n    'loss',\n    211,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_training_curves(\n    history.history['sparse_categorical_accuracy'],\n    history.history['val_sparse_categorical_accuracy'],\n    'accuracy',\n    212,\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cm_dataset = get_validation_dataset(ordered=True)\ncmat, score, precision, recall = compute_metrics(model, cm_dataset)\ndisplay_confusion_matrix(cmat, score, precision, recall)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataset = get_validation_dataset()\ndataset = dataset.unbatch().batch(20)\nbatch = iter(dataset)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"images, labels = next(batch)\nprobabilities = model.predict(images)\npredictions = np.argmax(probabilities, axis=-1)\ndisplay_batch_of_images((images, labels), predictions)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"generate_submission_file(model)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]}]}