{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceType":"competition","sourceId":21154,"databundleVersionId":1243559}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction","metadata":{}},{"cell_type":"markdown","source":"This notebook is inspired by both the [Create Your First Submission](https://www.kaggle.com/code/ryanholbrook/create-your-first-submission) from [Ryan Holbrook](https://www.kaggle.com/ryanholbrook) as well as the great [Road to the top series](https://www.kaggle.com/code/jhoward/first-steps-road-to-the-top-part-1) from [Jeremy Howard](https://www.kaggle.com/jhoward).\n\nI try to update the approach from Ryan Holbrook and get my notebook running on a current Kaggle environment. Now in order to achieve this quickly I follow the philosophie of Jeremy and just strive an MVP approach stripping everything intended to improve performance in order to have a running setup as quick and easy as possible. Just sticking with **TensorFlow** and **Keras** as opposed to **fastai** this time.","metadata":{}},{"cell_type":"markdown","source":"# Step 1: Setting the stage\n","metadata":{}},{"cell_type":"code","source":"import os, math, re\nimport numpy as np\nimport tensorflow as tf\nimport kaggle_datasets\n\n\nCOMPETITION = 'tpu-getting-started'\n\n# Copied from https://www.kaggle.com/code/ryanholbrook/create-your-first-submission\nCLASSES = ['pink primrose',    'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea',     'wild geranium',     'tiger lily',           'moon orchid',              'bird of paradise', 'monkshood',        'globe thistle',         # 00 - 09\n           'snapdragon',       \"colt's foot\",               'king protea',      'spear thistle', 'yellow iris',       'globe-flower',         'purple coneflower',        'peruvian lily',    'balloon flower',   'giant white arum lily', # 10 - 19\n           'fire lily',        'pincushion flower',         'fritillary',       'red ginger',    'grape hyacinth',    'corn poppy',           'prince of wales feathers', 'stemless gentian', 'artichoke',        'sweet william',         # 20 - 29\n           'carnation',        'garden phlox',              'love in the mist', 'cosmos',        'alpine sea holly',  'ruby-lipped cattleya', 'cape flower',              'great masterwort', 'siam tulip',       'lenten rose',           # 30 - 39\n           'barberton daisy',  'daffodil',                  'sword lily',       'poinsettia',    'bolero deep blue',  'wallflower',           'marigold',                 'buttercup',        'daisy',            'common dandelion',      # 40 - 49\n           'petunia',          'wild pansy',                'primula',          'sunflower',     'lilac hibiscus',    'bishop of llandaff',   'gaura',                    'geranium',         'orange dahlia',    'pink-yellow dahlia',    # 50 - 59\n           'cautleya spicata', 'japanese anemone',          'black-eyed susan', 'silverbush',    'californian poppy', 'osteospermum',         'spring crocus',            'iris',             'windflower',       'tree poppy',            # 60 - 69\n           'gazania',          'azalea',                    'water lily',       'rose',          'thorn apple',       'morning glory',        'passion flower',           'lotus',            'toad lily',        'anthurium',             # 70 - 79\n           'frangipani',       'clematis',                  'hibiscus',         'columbine',     'desert-rose',       'tree mallow',          'magnolia',                 'cyclamen ',        'watercress',       'canna lily',            # 80 - 89\n           'hippeastrum ',     'bee balm',                  'pink quill',       'foxglove',      'bougainvillea',     'camellia',             'mallow',                   'mexican petunia',  'bromelia',         'blanket flower',        # 90 - 99\n           'trumpet creeper',  'blackberry lily',           'common tulip',     'wild rose'] ","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Step 2: Get the \"Cloud Streaming\" Path","metadata":{}},{"cell_type":"markdown","source":"Since I am training on a TPU, I won't download any data to a local drive. Instead, I am setting up my data pipeline to \"stream\" data directly from a Google Cloud Storage (GCS) bucket. I like to think of this as building a high-speed digital pipeline that connects my TPU hardware directly to a remote storage vault.\n\n## How to Retrieve the Path\n\nI'm using the KaggleDatasets library to automatically locate the specific URL for this competition’s data. This ensures that no matter where the data is moved behind the scenes, I’ll always have the right \"address.\"\n","metadata":{}},{"cell_type":"code","source":"GCS_DS_PATH = kaggle_datasets.KaggleDatasets().get_gcs_path(f'competitions/{COMPETITION}')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Step 3: Building the data loader","metadata":{}},{"cell_type":"markdown","source":"I did extract the hard coded image size out of the GCS_PATH to make it easier to run the notebook using differing image resolutions. Now switching resolution is just one edit of IMAGE_SIZE.","metadata":{}},{"cell_type":"code","source":"IMAGE_SIZE = [192, 192]\nNUM_RGB_CHANNELS = 3\nAUTO = tf.data.AUTOTUNE\nGCS_PATH = f\"{GCS_DS_PATH}/tfrecords-jpeg-{IMAGE_SIZE[0]}x{IMAGE_SIZE[1]}\"\n\nTRAINING_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec')\nVALIDATION_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\nTEST_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/test/*.tfrec')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(f\"Training files found: {len(TRAINING_FILENAMES)}\")\nprint(f\"Validation files found: {len(VALIDATION_FILENAMES)}\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"I collected the functions for creating the datasets into the following cell. The code was fully functional in the current environment. The only intentional refactoring change I introduced was replacing the magic number `3` with `NUM_RGB_CHANNELS = 3` to make explicit that TensorFlow represents RGB images as 3-dimensional tensors with the shape `(height, width, channels)`.\n\nAdditionally, I simplified the dataset loading configuration by replacing the deprecated-style `experimental_deterministic` handling with the more direct `options.deterministic = ordered`.\n\nI also deliberately removed the data augmentation step from `get_training_dataset()`. At this stage my focus was simply getting a working training pipeline before starting optimization experiments. Consequently, the now-unused `data_augment()` function was removed as well.","metadata":{}},{"cell_type":"code","source":"def decode_image(image_data):\n    image = tf.image.decode_jpeg(image_data, channels=3)\n    image = tf.cast(image, tf.float32) / 255.0  # convert image to floats in [0, 1] range\n    image = tf.reshape(image, [*IMAGE_SIZE, NUM_RGB_CHANNELS]) # Size and layout (RGB layers!) needed for TPU\n    return image\n\ndef read_labeled_tfrecord(example):\n    LABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"class\": tf.io.FixedLenFeature([], tf.int64),\n    }\n    example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    label = tf.cast(example['class'], tf.int32)\n    return image, label # returns a dataset of (image, label) pairs\n\ndef read_unlabeled_tfrecord(example):\n    UNLABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"id\": tf.io.FixedLenFeature([], tf.string),\n    }\n    example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    idnum = example['id']\n    return image, idnum # returns a dataset of (image, idnum) pairs\n\ndef load_dataset(filenames, labeled=True, ordered=False):\n    # Read from TFRecords. For optimal performance defaults to ignoring data order \n    # Returns a dataset of (image, label) pairs if labeled=True or (image, id) pairs if labeled=False\n    \n    options = tf.data.Options()\n    options.deterministic = ordered\n\n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO) # automatically interleaves reads from multiple files\n    dataset = dataset.with_options(options) # uses data as soon as it streams in, rather than in its original order\n    dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord, num_parallel_calls=AUTO)\n    return dataset\n\ndef get_training_dataset():\n    dataset = load_dataset(TRAINING_FILENAMES, labeled=True)\n    dataset = dataset.repeat() # the training dataset must repeat for several epochs\n    dataset = dataset.shuffle(2048)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset\n\ndef get_validation_dataset(ordered=False):\n    dataset = load_dataset(VALIDATION_FILENAMES, labeled=True, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO)\n    return dataset\n\ndef get_test_dataset(ordered=False):\n    dataset = load_dataset(TEST_FILENAMES, labeled=False, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO)\n    return dataset","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def count_data_items(filenames):\n    # the number of data items is written in the name of the .tfrec\n    # files, i.e. flowers00-230.tfrec = 230 data items\n    n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) for filename in filenames]\n    return np.sum(n)\n\nNUM_TRAINING_IMAGES = count_data_items(TRAINING_FILENAMES)\nNUM_VALIDATION_IMAGES = count_data_items(VALIDATION_FILENAMES)\nNUM_TEST_IMAGES = count_data_items(TEST_FILENAMES)\n\nprint('Dataset: {} training images, {} validation images, {} unlabeled test images'.format(NUM_TRAINING_IMAGES, NUM_VALIDATION_IMAGES, NUM_TEST_IMAGES))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Step 4: Distribution Strategy","metadata":{}},{"cell_type":"markdown","source":"I refactored the hardware strategy setup into a dedicated `select_strategy()` function to make the TPU/GPU/CPU initialization logic easier to read and reuse.\n\nThe original notebook only attempted TPU detection and otherwise silently fell back to TensorFlow’s default strategy. In the refactored version I added explicit hardware detection for both GPUs and TPUs:\n\n```python\nprint(\"GPUs Available: \", tf.config.list_physical_devices('GPU'))\nprint(\"TPUs Available: \", tf.config.list_physical_devices('TPU'))\n```\n\nI also modernized the TPU initialization by replacing the older multi-step setup:\n\n```python\ntf.config.experimental_connect_to_cluster(tpu)\ntf.tpu.experimental.initialize_tpu_system(tpu)\nstrategy = tf.distribute.experimental.TPUStrategy(tpu)\n```\n\nwith the newer convenience method:\n\n```python\ntpu = tf.distribute.cluster_resolver.TPUClusterResolver.connect()\nstrategy = tf.distribute.TPUStrategy(tpu)\n```\n\nAdditionally, the refactored version now explicitly distinguishes between TPU, GPU and CPU execution:\n\n* TPU → `tf.distribute.TPUStrategy`\n* GPU → `tf.distribute.MirroredStrategy`\n* CPU → default TensorFlow strategy\n\nThis makes the runtime behavior more transparent and improves portability across different Kaggle environments.","metadata":{}},{"cell_type":"code","source":"print(\"GPUs Available: \", tf.config.list_physical_devices('GPU'))\nprint(\"TPUs Available: \", tf.config.list_physical_devices('TPU'))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def select_strategy():\n    try:\n        tpu = tf.distribute.cluster_resolver.TPUClusterResolver.connect() \n        strategy = tf.distribute.TPUStrategy(tpu)\n        print(f'Running on TPU: {tpu.master()}')\n    except (ValueError, tf.errors.NotFoundError):\n        # 2. Fallback to GPU/CPU\n        if tf.config.list_physical_devices('GPU'):\n            strategy = tf.distribute.MirroredStrategy()\n            print('Running on GPU')\n        else:\n            strategy = tf.distribute.get_strategy()\n            print('Running on CPU')\n\n    print(f\"REPLICAS: {strategy.num_replicas_in_sync}\")\n    return strategy\n\nstrategy = select_strategy()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Step 5: Define datasets","metadata":{"execution":{"iopub.status.busy":"2026-05-12T11:35:41.067134Z","iopub.execute_input":"2026-05-12T11:35:41.067458Z","iopub.status.idle":"2026-05-12T11:35:41.079870Z","shell.execute_reply.started":"2026-05-12T11:35:41.067408Z","shell.execute_reply":"2026-05-12T11:35:41.079136Z"}}},{"cell_type":"markdown","source":"I replaced the previously hardcoded batch size scaling logic with separate base hyperparameters for both the batch size and learning rate:\n\n```python\nBASE_BATCH_SIZE = 32\nBASE_LEARNING_RATE = 0.0001\n```\n\nThe original notebook only scaled the batch size according to the number of synchronized replicas:\n\n```python\nBATCH_SIZE = 16 * strategy.num_replicas_in_sync\n```\n\nIn the refactored version both values are now derived dynamically from the active distribution strategy:\n\n```python\nnum_replicas = strategy.num_replicas_in_sync\nBATCH_SIZE = BASE_BATCH_SIZE * num_replicas\nLEARNING_RATE = BASE_LEARNING_RATE * num_replicas\n```\n\nThis makes the configuration more explicit and easier to tune, while also following the common distributed training practice of scaling the learning rate together with the effective batch size.\n\nSeparating the base hyperparameters from their hardware-scaled values also improves readability and simplifies experimentation across CPU, GPU and TPU environments.","metadata":{}},{"cell_type":"code","source":"BASE_BATCH_SIZE = 16\nBASE_LEARNING_RATE = 0.00005\n\nnum_replicas = strategy.num_replicas_in_sync\nBATCH_SIZE = BASE_BATCH_SIZE * num_replicas\nLEARNING_RATE = BASE_LEARNING_RATE * num_replicas\n\nds_train = get_training_dataset()\nds_valid = get_validation_dataset()\nds_test = get_test_dataset()\n\nprint(\"Training:\", ds_train)\nprint (\"Validation:\", ds_valid)\nprint(\"Test:\", ds_test)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"np.set_printoptions(threshold=15, linewidth=80)\n\nprint(\"Training data shapes:\")\nfor image, label in ds_train.take(3):\n    print(image.numpy().shape, label.numpy().shape)\nprint(\"Training data label examples:\", label.numpy())","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"Test data shapes:\")\nfor image, idnum in ds_test.take(3):\n    print(image.numpy().shape, idnum.numpy().shape)\nprint(\"Test data IDs:\", idnum.numpy().astype('U')) # U=unicode string","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Step 6: Some Exploration of the Data","metadata":{}},{"cell_type":"markdown","source":"I kept most of the original visualization helper functions unchanged, as they were already well-structured and useful for quickly inspecting image batches during training.\n\nThe primary refactoring change in this section was simplifying the notebook flow by moving the `display_training_curves()` helper function into the model fit evaluation section. At this stage the focus was still on validating the data pipeline and training setup rather than evaluating model performance visually.\n\nI also shortened the surrounding explanatory text and streamlined the example usage to keep the notebook more compact and focused on the essential dataset inspection workflow.","metadata":{}},{"cell_type":"code","source":"from matplotlib import pyplot as plt\n\ndef batch_to_numpy_images_and_labels(data):\n    images, labels = data\n    numpy_images = images.numpy()\n    numpy_labels = labels.numpy()\n    if numpy_labels.dtype == object: # binary string in this case,\n                                     # these are image ID strings\n        numpy_labels = [None for _ in enumerate(numpy_images)]\n    # If no labels, only image IDs, return None for labels (this is\n    # the case for test data)\n    return numpy_images, numpy_labels\n\ndef title_from_label_and_target(label, correct_label):\n    if correct_label is None:\n        return CLASSES[label], True\n    correct = (label == correct_label)\n    return \"{} [{}{}{}]\".format(CLASSES[label], 'OK' if correct else 'NO', u\"\\u2192\" if not correct else '',\n                                CLASSES[correct_label] if not correct else ''), correct\n\ndef display_one_flower(image, title, subplot, red=False, titlesize=16):\n    plt.subplot(*subplot)\n    plt.axis('off')\n    plt.imshow(image)\n    if len(title) > 0:\n        plt.title(title, fontsize=int(titlesize) if not red else int(titlesize/1.2), color='red' if red else 'black', fontdict={'verticalalignment':'center'}, pad=int(titlesize/1.5))\n    return (subplot[0], subplot[1], subplot[2]+1)\n    \ndef display_batch_of_images(databatch, predictions=None):\n    \"\"\"This will work with:\n    display_batch_of_images(images)\n    display_batch_of_images(images, predictions)\n    display_batch_of_images((images, labels))\n    display_batch_of_images((images, labels), predictions)\n    \"\"\"\n    # data\n    images, labels = batch_to_numpy_images_and_labels(databatch)\n    if labels is None:\n        labels = [None for _ in enumerate(images)]\n        \n    # auto-squaring: this will drop data that does not fit into square\n    # or square-ish rectangle\n    rows = int(math.sqrt(len(images)))\n    cols = len(images)//rows\n        \n    # size and spacing\n    FIGSIZE = 13.0\n    SPACING = 0.1\n    subplot=(rows,cols,1)\n    if rows < cols:\n        plt.figure(figsize=(FIGSIZE,FIGSIZE/cols*rows))\n    else:\n        plt.figure(figsize=(FIGSIZE/rows*cols,FIGSIZE))\n    \n    # display\n    for i, (image, label) in enumerate(zip(images[:rows*cols], labels[:rows*cols])):\n        title = '' if label is None else CLASSES[label]\n        correct = True\n        if predictions is not None:\n            title, correct = title_from_label_and_target(predictions[i], label)\n        dynamic_titlesize = FIGSIZE*SPACING/max(rows,cols)*40+3 # magic formula tested to work from 1x1 to 10x10 images\n        subplot = display_one_flower(image, title, subplot, not correct, titlesize=dynamic_titlesize)\n    \n    #layout\n    plt.tight_layout()\n    if label is None and predictions is None:\n        plt.subplots_adjust(wspace=0, hspace=0)\n    else:\n        plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n    plt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ds_iter = iter(ds_train.unbatch().batch(20))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"one_batch = next(ds_iter)\ndisplay_batch_of_images(one_batch)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Step 7: Define the Model","metadata":{}},{"cell_type":"markdown","source":"The model definition section caused some confusion in the original notebook, as reflected in several discussion comments. The main issue was that the creation of the pretrained model and the compilation of the final model were split across separate notebook cells.\n\nIn the original version, only the model construction was executed inside the `strategy.scope()` context, while the subsequent `model.compile()` call happened outside of it. This can lead to problems when using distributed training strategies such as TPUs or multi-GPU setups, because TensorFlow expects both model creation and compilation to occur within the same distribution scope.\n\nTo avoid these issues, I merged both steps into a single cell and ensured that the entire model setup is consistently executed inside the same `strategy.scope()` block.\n\nAdditionally, I replaced the shorthand optimizer definition:\n\n```python\noptimizer='adam'\n```\n\nwith an explicit optimizer configuration:\n\n```python\noptimizer=tf.keras.optimizers.Adam(learning_rate=LEARNING_RATE)\n```\n\nThis makes the learning rate scaling introduced earlier in the notebook actually take effect and improves the transparency of the training configuration.\n\nFor faster experimentation during development, I also temporarily reduced the number of training epochs from `12` to `6`.","metadata":{}},{"cell_type":"code","source":"EPOCHS = 6\n\nwith strategy.scope():\n    pretrained_model = tf.keras.applications.VGG16(\n        weights='imagenet',\n        include_top=False ,\n        input_shape=[*IMAGE_SIZE, 3]\n    )\n    pretrained_model.trainable = False\n    \n    model = tf.keras.Sequential([\n        # To a base pretrained on ImageNet to extract features from images...\n        pretrained_model,\n        # ... attach a new head to act as a classifier.\n        tf.keras.layers.GlobalAveragePooling2D(),\n        tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n    ])\n\n    model.compile(\n        optimizer=tf.keras.optimizers.Adam(learning_rate=LEARNING_RATE),\n        loss = 'sparse_categorical_crossentropy',\n        metrics=['sparse_categorical_accuracy'],\n    )","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.summary()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Step 8: Training the Model","metadata":{}},{"cell_type":"markdown","source":"For the sake of simplicity, I decided to remove the dynamic learning rate schedule used in the original notebook.\n\nWhile the custom LearningRateScheduler implementation was elegant and well-suited for fine-tuning experiments, it also added a considerable amount of additional code and complexity to the training setup. Since the primary goal at this stage was establishing a stable and understandable baseline pipeline, I opted for a fixed learning rate instead.\n\nAs part of this simplification, I removed:\n\n* the `custom exponential_lr()` scheduling function,\n* the `LearningRateScheduler` callback,\n* and the associated visualization of the learning rate progression across epochs.\n\nThe learning rate itself is now defined directly during model compilation through the optimizer configuration introduced earlier:\n\n```python\noptimizer=tf.keras.optimizers.Adam(learning_rate=LEARNING_RATE)\n```\n\nThis keeps the training configuration more transparent and reduces the number of moving parts while debugging and validating the overall workflow. The training loop was therefore reduced to a more minimal and easier-to-follow configuration:","metadata":{}},{"cell_type":"code","source":"STEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\n\nhistory = model.fit(\n    ds_train,\n    validation_data=ds_valid,\n    epochs=EPOCHS,\n    steps_per_epoch=STEPS_PER_EPOCH,\n)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"I kept the original training curve visualization mostly unchanged, since the helper function already provided a clean way to compare training and validation metrics.\n\nThe main refactoring change was replacing:\n\n```python\nplt.subplots(...)\n```\n\nwith:\n\n```python\nplt.figure(...)\n```\n\nduring figure initialization. Since the code manages subplots manually via `plt.subplot()`, using `plt.figure()` is more appropriate and avoids creating unused subplot objects.\n\nI also removed an outdated commented line to slightly simplify the helper function.","metadata":{}},{"cell_type":"code","source":"def display_training_curves(training, validation, title, subplot):\n    if subplot%10==1:\n        plt.figure(figsize=(10,10), facecolor='#F0F0F0')  # ← plt.figure, not plt.subplots\n        plt.tight_layout()\n    ax = plt.subplot(subplot)\n    ax.set_facecolor('#F8F8F8')\n    ax.plot(training)\n    ax.plot(validation)\n    ax.set_title('model '+ title)\n    ax.set_ylabel(title)\n    ax.set_xlabel('epoch')\n    ax.legend(['train', 'valid.'])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"display_training_curves(\n    history.history['loss'],\n    history.history['val_loss'],\n    'loss',\n    211,\n)\ndisplay_training_curves(\n    history.history['sparse_categorical_accuracy'],\n    history.history['val_sparse_categorical_accuracy'],\n    'accuracy',\n    212,\n)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Step 9: Submit to competition","metadata":{}},{"cell_type":"markdown","source":"I removed the entire evaluation section for the sake of simplicity and iteration speed.\n\nThe original notebook invested considerable effort into validation tooling, including:\n\n* confusion matrix generation,\n* F1, precision and recall metrics,\n* visual validation of predictions,\n* and additional helper functions for evaluation plots.\n\nWhile these are useful for deeper model analysis, they also added substantial notebook complexity and execution time.\n\nInspired by Jeremy Howard’s “move fast” philosophy, I decided to skip intermediate evaluation entirely and rely instead on Kaggle’s submission feedback as the primary validation signal. At this stage the focus was less on carefully diagnosing model behavior and more on establishing a fast end-to-end experimentation workflow.","metadata":{}},{"cell_type":"code","source":"test_ds = get_test_dataset(ordered=True)\n\ntest_images_ds = test_ds.map(lambda image, idnum: image)\nprobabilities = model.predict(test_images_ds)\npredictions = np.argmax(probabilities, axis=-1)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_ids_ds = test_ds.map(lambda image, idnum: idnum).unbatch()\ntest_ids = next(iter(test_ids_ds.batch(NUM_TEST_IMAGES))).numpy().astype('U')\n\nnp.savetxt(\n    'submission.csv',\n    np.rec.fromarrays([test_ids, predictions]),\n    fmt=['%s', '%d'],\n    delimiter=',',\n    header='id,label',\n    comments='',\n)\n\n!head submission.csv","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This first submission scored `0.05070`, which is a good starting point and leaves plenty of room for improvement in future iterations.\n\nFeedback and upvotes are appreciated to help improve visibility for others.","metadata":{}}]}