{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":21154,"databundleVersionId":1243559,"sourceType":"competition"},{"sourceId":6948717,"sourceType":"datasetVersion","datasetId":3990766},{"sourceId":6969744,"sourceType":"datasetVersion","datasetId":4004448}],"dockerImageVersionId":30579,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **Introduction**","metadata":{}},{"cell_type":"markdown","source":"In leveraging the power of TensorFlow and Google's Tensor Processing Units (TPUs), this project will use an image classification model, meticulously designed with a Convolutional Neural Network (CNN), exemplifies cutting-edge capabilities for large-scale image analysis. The model, adept at discerning intricate patterns and features in images, is tailored for deployment on TPUs, accelerating computations and optimizing performance. TPUs, renowned for their parallel processing prowess, enhance the model's training efficiency, enabling seamless scalability for image classification tasks. This strategic integration of advanced CNN architecture with TPU acceleration propels the model's capabilities, promising swift and accurate image classification on a scale that meets the demands of modern computational challenges.","metadata":{}},{"cell_type":"markdown","source":"**Required Library Imports**","metadata":{}},{"cell_type":"code","source":"import math \nimport re\nimport os\nimport pandas as pd\nimport numpy as np\nimport tensorflow as tf\nfrom tensorflow.keras import layers, models\nfrom matplotlib import pyplot as plt\nfrom sklearn.metrics import f1_score, precision_score, recall_score, confusion_matrix\nfrom sklearn.metrics import classification_report","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:41.201931Z","iopub.execute_input":"2023-11-16T14:47:41.202306Z","iopub.status.idle":"2023-11-16T14:47:51.892608Z","shell.execute_reply.started":"2023-11-16T14:47:41.202274Z","shell.execute_reply":"2023-11-16T14:47:51.891572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Initialize the Running Strategy**","metadata":{}},{"cell_type":"code","source":"# Detect hardware, return appropriate distribution strategy\ntry:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()  \n    print('Running on TPU ', tpu.master())\nexcept ValueError:\n    tpu = None\n\nif tpu:\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n    strategy = tf.distribute.get_strategy() \n\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:51.894843Z","iopub.execute_input":"2023-11-16T14:47:51.895596Z","iopub.status.idle":"2023-11-16T14:47:51.908307Z","shell.execute_reply.started":"2023-11-16T14:47:51.895559Z","shell.execute_reply":"2023-11-16T14:47:51.907019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above section initializes a distribution strategy based on the hardware available, with a focus on Tensor Processing Units (TPUs). The process begins by attempting to detect a TPU using tf.distribute.cluster_resolver.TPUClusterResolver(). If a TPU is found, it connects to the TPU cluster, initializes the TPU system, and creates a TPUStrategy object using tf.distribute.experimental.TPUStrategy(tpu). This strategy is crucial for distributing the training across multiple devices, such as TPUs, enabling parallel processing and significantly accelerating model training. If no TPU is detected, it defaults to using the standard tf.distribute.get_strategy(). The final print statement provides information about the number of replicas in the strategy, reflecting the distributed training setup.","metadata":{}},{"cell_type":"code","source":"!mkdir -p /root/.kaggle","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:51.909729Z","iopub.execute_input":"2023-11-16T14:47:51.910424Z","iopub.status.idle":"2023-11-16T14:47:52.920805Z","shell.execute_reply.started":"2023-11-16T14:47:51.910395Z","shell.execute_reply":"2023-11-16T14:47:52.919539Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cp /kaggle/input/json-file/kaggle.json /root/.kaggle/","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:52.923296Z","iopub.execute_input":"2023-11-16T14:47:52.923602Z","iopub.status.idle":"2023-11-16T14:47:53.881108Z","shell.execute_reply.started":"2023-11-16T14:47:52.923573Z","shell.execute_reply":"2023-11-16T14:47:53.879746Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Load the Flower Dataset**","metadata":{}},{"cell_type":"code","source":"#pip install kaggle","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:53.882785Z","iopub.execute_input":"2023-11-16T14:47:53.883116Z","iopub.status.idle":"2023-11-16T14:47:53.888053Z","shell.execute_reply.started":"2023-11-16T14:47:53.883086Z","shell.execute_reply":"2023-11-16T14:47:53.887085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#from kaggle.api.kaggle_api_extended import KaggleApi\n\nGCS_DS_PATH = '/kaggle/input/tpu-getting-started'\nprint(GCS_DS_PATH)","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:53.889500Z","iopub.execute_input":"2023-11-16T14:47:53.889921Z","iopub.status.idle":"2023-11-16T14:47:53.898499Z","shell.execute_reply.started":"2023-11-16T14:47:53.889893Z","shell.execute_reply":"2023-11-16T14:47:53.897392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Establish Flower Classification Classes**","metadata":{}},{"cell_type":"code","source":"IMAGE_SIZE = [512, 512]\nGCS_PATH = '/kaggle/input/tpu-getting-started/tfrecords-jpeg-512x512'\nAUTO = tf.data.experimental.AUTOTUNE\n\nTRAINING_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec')\nVALIDATION_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\nTEST_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/test/*.tfrec') \n\nCLASSES = ['pink primrose',    'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea',     'wild geranium',     'tiger lily',           'moon orchid',              'bird of paradise', 'monkshood',        'globe thistle',         # 00 - 09\n           'snapdragon',       \"colt's foot\",               'king protea',      'spear thistle', 'yellow iris',       'globe-flower',         'purple coneflower',        'peruvian lily',    'balloon flower',   'giant white arum lily', # 10 - 19\n           'fire lily',        'pincushion flower',         'fritillary',       'red ginger',    'grape hyacinth',    'corn poppy',           'prince of wales feathers', 'stemless gentian', 'artichoke',        'sweet william',         # 20 - 29\n           'carnation',        'garden phlox',              'love in the mist', 'cosmos',        'alpine sea holly',  'ruby-lipped cattleya', 'cape flower',              'great masterwort', 'siam tulip',       'lenten rose',           # 30 - 39\n           'barberton daisy',  'daffodil',                  'sword lily',       'poinsettia',    'bolero deep blue',  'wallflower',           'marigold',                 'buttercup',        'daisy',            'common dandelion',      # 40 - 49\n           'petunia',          'wild pansy',                'primula',          'sunflower',     'lilac hibiscus',    'bishop of llandaff',   'gaura',                    'geranium',         'orange dahlia',    'pink-yellow dahlia',    # 50 - 59\n           'cautleya spicata', 'japanese anemone',          'black-eyed susan', 'silverbush',    'californian poppy', 'osteospermum',         'spring crocus',            'iris',             'windflower',       'tree poppy',            # 60 - 69\n           'gazania',          'azalea',                    'water lily',       'rose',          'thorn apple',       'morning glory',        'passion flower',           'lotus',            'toad lily',        'anthurium',             # 70 - 79\n           'frangipani',       'clematis',                  'hibiscus',         'columbine',     'desert-rose',       'tree mallow',          'magnolia',                 'cyclamen ',        'watercress',       'canna lily',            # 80 - 89\n           'hippeastrum ',     'bee balm',                  'pink quill',       'foxglove',      'bougainvillea',     'camellia',             'mallow',                   'mexican petunia',  'bromelia',         'blanket flower',        # 90 - 99\n           'trumpet creeper',  'blackberry lily',           'common tulip',     'wild rose']                                                                                                                                               # 100 - 102\n","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:53.899958Z","iopub.execute_input":"2023-11-16T14:47:53.900507Z","iopub.status.idle":"2023-11-16T14:47:53.925943Z","shell.execute_reply.started":"2023-11-16T14:47:53.900471Z","shell.execute_reply":"2023-11-16T14:47:53.925054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def decode_image(image_data):\n    image = tf.image.decode_jpeg(image_data, channels=3)\n    image = tf.cast(image, tf.float32) / 255.0  # convert image to floats in [0, 1] range\n    image = tf.reshape(image, [*IMAGE_SIZE, 3]) # explicit size needed for TPU\n    return image\n\ndef read_labeled_tfrecord(example):\n    LABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n        \"class\": tf.io.FixedLenFeature([], tf.int64),  # shape [] means single element\n    }\n    example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    label = tf.cast(example['class'], tf.int32)\n    return image, label # returns a dataset of (image, label) pairs\n\ndef read_unlabeled_tfrecord(example):\n    UNLABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n        \"id\": tf.io.FixedLenFeature([], tf.string),  # shape [] means single element\n        # class is missing, this competitions's challenge is to predict flower classes for the test dataset\n    }\n    example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    idnum = example['id']\n    return image, idnum # returns a dataset of image(s)\n\ndef load_dataset(filenames, labeled=True, ordered=False):\n    # Read from TFRecords. For optimal performance, reading from multiple files at once and\n    # disregarding data order. Order does not matter since we will be shuffling the data anyway.\n\n    ignore_order = tf.data.Options()\n    if not ordered:\n        ignore_order.experimental_deterministic = False # disable order, increase speed\n\n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO) # automatically interleaves reads from multiple files\n    dataset = dataset.with_options(ignore_order) # uses data as soon as it streams in, rather than in its original order\n    dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord, num_parallel_calls=AUTO)\n    # returns a dataset of (image, label) pairs if labeled=True or (image, id) pairs if labeled=False\n    return dataset","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:53.928494Z","iopub.execute_input":"2023-11-16T14:47:53.928797Z","iopub.status.idle":"2023-11-16T14:47:53.939330Z","shell.execute_reply.started":"2023-11-16T14:47:53.928771Z","shell.execute_reply":"2023-11-16T14:47:53.938302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def data_augment(image, label):\n    image = tf.image.random_flip_left_right(image)\n    image = tf.image.random_saturation(image, 0, 2)\n    image = tf.image.adjust_contrast(image,1)\n    image = tf.image.per_image_standardization(image)\n    return image, label   \n\ndef get_training_dataset():\n    dataset = load_dataset(TRAINING_FILENAMES, labeled=True)\n    dataset = dataset.map(data_augment, num_parallel_calls=AUTO)\n    dataset = dataset.repeat() # the training dataset must repeat for several epochs\n    dataset = dataset.shuffle(2048)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO) # prefetch next batch while training (autotune prefetch buffer size)\n    return dataset\n\ndef get_validation_dataset(ordered=False):\n    dataset = load_dataset(VALIDATION_FILENAMES, labeled=True, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO)\n    return dataset\n\ndef get_test_dataset(ordered=False):\n    dataset = load_dataset(TEST_FILENAMES, labeled=False, ordered=ordered)\n    dataset = dataset.batch(BATCH_SIZE)\n    dataset = dataset.prefetch(AUTO)\n    return dataset\n\ndef count_data_items(filenames):\n    # the number of data items is written in the name of the .tfrec\n    # files, i.e. flowers00-230.tfrec = 230 data items\n    n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) for filename in filenames]\n    return np.sum(n)\n\nNUM_TRAINING_IMAGES = count_data_items(TRAINING_FILENAMES)\nNUM_VALIDATION_IMAGES = count_data_items(VALIDATION_FILENAMES)\nNUM_TEST_IMAGES = count_data_items(TEST_FILENAMES)\nprint('Dataset: {} training images, {} validation images, {} unlabeled test images'.format(NUM_TRAINING_IMAGES, NUM_VALIDATION_IMAGES, NUM_TEST_IMAGES))","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:53.940996Z","iopub.execute_input":"2023-11-16T14:47:53.941572Z","iopub.status.idle":"2023-11-16T14:47:53.955529Z","shell.execute_reply.started":"2023-11-16T14:47:53.941539Z","shell.execute_reply":"2023-11-16T14:47:53.954680Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"These dataset sizes and splits are crucial components in the machine learning workflow, contributing to the training, evaluation, and testing phases of model development.\n\n* **Training Dataset: Contains 12,753 images.** This dataset is typically used to train a machine learning model. During training, the model learns patterns and features from these labeled images to make predictions on new, unseen data.\n* **Validation Dataset: Consists of 3,712 images.** The validation set is used to evaluate the performance of the model during training. It helps to assess how well the model generalizes to new, unseen data and to fine-tune hyperparameters to improve overall performance.\n* **Unlabeled Test Dataset: Comprises 7,382 images.** This set lacks labels, meaning the ground truth (correct answer) is not provided. It is reserved for assessing the model's performance on completely new and unlabeled data, simulating real-world scenarios where the model needs to make predictions without prior knowledge of correct outcomes.","metadata":{}},{"cell_type":"code","source":"# Define the batch size to be used by the strategy\nBATCH_SIZE = 48 * strategy.num_replicas_in_sync\n\n#df represents the dataframes that will hold the prior defined functions\ndf_train = get_training_dataset()\ndf_valid = get_validation_dataset()\ndf_test = get_test_dataset()\n\nprint(\"Training:\", df_train)\nprint (\"Validation:\", df_valid)\nprint(\"Test:\", df_test)","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:53.958796Z","iopub.execute_input":"2023-11-16T14:47:53.959504Z","iopub.status.idle":"2023-11-16T14:47:58.604394Z","shell.execute_reply.started":"2023-11-16T14:47:53.959479Z","shell.execute_reply":"2023-11-16T14:47:58.603466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From the results above, datasets are set up for training, validating, and testing a machine learning model, with the output providing insight into the dimensions and data types of the tensors within each dataset.\n\n* **Training Dataset:** The output indicates that the training dataset is a PrefetchDataset, a type of dataset in TensorFlow that efficiently loads and pre-processes data for training. The element_spec of this dataset consists of tuples, where the first element is a tensor with shape (None, 512, 512, 3) representing images (presumably in the format of height 512, width 512, and 3 color channels), and the second element is a tensor with shape (None,) of data type tf.int32, representing the corresponding labels.\n \n* **Validation Dataset:** Similar to the training dataset, the validation dataset is also a PrefetchDataset. It has the same element_spec structure, consisting of image tensors and label tensors. This dataset is typically used to assess the model's performance during training and prevent overfitting.\n\n* **Test Dataset:** The test dataset is also a PrefetchDataset. However, the element_spec here differs slightly. The image tensors remain the same with shape (None, 512, 512, 3), but the label tensors are of data type tf.string. This suggests that the test dataset is composed of images without corresponding numeric labels. Instead, it might have string labels or other identifiers that are relevant for evaluation but are not numeric.","metadata":{}},{"cell_type":"markdown","source":"# **Visualization of Flower Data**","metadata":{}},{"cell_type":"code","source":"def batch_to_numpy_images_and_labels(data):\n    images, labels = data\n    numpy_images = images.numpy()\n    numpy_labels = labels.numpy()\n    if numpy_labels.dtype == object: # binary string in this case,\n                                     # these are image ID strings\n        numpy_labels = [None for _ in enumerate(numpy_images)]\n    # If no labels, only image IDs, return None for labels (this is\n    # the case for test data)\n    return numpy_images, numpy_labels\n\ndef title_from_label_and_target(label, correct_label):\n    if correct_label is None:\n        return CLASSES[label], True\n    correct = (label == correct_label)\n    return \"{} [{}{}{}]\".format(CLASSES[label], 'OK' if correct else 'NO', u\"\\u2192\" if not correct else '',\n                                CLASSES[correct_label] if not correct else ''), correct\n\ndef display_one_flower(image, title, subplot, red=False, titlesize=16):\n    plt.subplot(*subplot)\n    plt.axis('off')\n    plt.imshow(image)\n    if len(title) > 0:\n        plt.title(title, fontsize=int(titlesize) if not red else int(titlesize/1.2), color='red' if red else 'black', fontdict={'verticalalignment':'center'}, pad=int(titlesize/1.5))\n    return (subplot[0], subplot[1], subplot[2]+1)\n    \ndef display_batch_of_images(databatch, predictions=None):\n    # data\n    images, labels = batch_to_numpy_images_and_labels(databatch)\n    if labels is None:\n        labels = [None for _ in enumerate(images)]\n        \n    # auto-squaring: this will drop data that does not fit into square\n    # or square-ish rectangle\n    rows = int(math.sqrt(len(images)))\n    cols = len(images)//rows\n        \n    # size and spacing\n    FIGSIZE = 13.0\n    SPACING = 0.1\n    subplot=(rows,cols,1)\n    if rows < cols:\n        plt.figure(figsize=(FIGSIZE,FIGSIZE/cols*rows))\n    else:\n        plt.figure(figsize=(FIGSIZE/rows*cols,FIGSIZE))\n    \n    # display\n    for i, (image, label) in enumerate(zip(images[:rows*cols], labels[:rows*cols])):\n        title = '' if label is None else CLASSES[label]\n        correct = True\n        if predictions is not None:\n            title, correct = title_from_label_and_target(predictions[i], label)\n        dynamic_titlesize = FIGSIZE*SPACING/max(rows,cols)*40+3 # magic formula tested to work from 1x1 to 10x10 images\n        subplot = display_one_flower(image, title, subplot, not correct, titlesize=dynamic_titlesize)\n    \n    #layout\n    plt.tight_layout()\n    if label is None and predictions is None:\n        plt.subplots_adjust(wspace=0, hspace=0)\n    else:\n        plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n    plt.show()\n\n\ndef display_training_curves(training, validation, title, subplot):\n    if subplot%10==1: # set up the subplots on the first call\n        plt.subplots(figsize=(10,10), facecolor='#F0F0F0')\n        plt.tight_layout()\n    ax = plt.subplot(subplot)\n    ax.set_facecolor('#F8F8F8')\n    ax.plot(training)\n    ax.plot(validation)\n    ax.set_title('model '+ title)\n    ax.set_ylabel(title)\n    #ax.set_ylim(0.28,1.05)\n    ax.set_xlabel('epoch')\n    ax.legend(['train', 'valid.'])","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:58.606104Z","iopub.execute_input":"2023-11-16T14:47:58.606484Z","iopub.status.idle":"2023-11-16T14:47:58.624515Z","shell.execute_reply.started":"2023-11-16T14:47:58.606447Z","shell.execute_reply":"2023-11-16T14:47:58.623514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_iter = iter(df_train.unbatch().batch(10))\none_batch = next(ds_iter)\ndisplay_batch_of_images(one_batch)","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:47:58.625731Z","iopub.execute_input":"2023-11-16T14:47:58.626044Z","iopub.status.idle":"2023-11-16T14:48:10.296853Z","shell.execute_reply.started":"2023-11-16T14:47:58.626019Z","shell.execute_reply":"2023-11-16T14:48:10.295813Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Exploration of Data","metadata":{}},{"cell_type":"code","source":"np.set_printoptions(threshold=15, linewidth=80)\n\nprint(\"Training data shapes:\")\nfor image, label in df_train.take(3):\n    print(image.numpy().shape, label.numpy().shape)\nprint(\"Training data label examples:\", label.numpy())","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:48:10.298138Z","iopub.execute_input":"2023-11-16T14:48:10.298443Z","iopub.status.idle":"2023-11-16T14:48:20.597419Z","shell.execute_reply.started":"2023-11-16T14:48:10.298417Z","shell.execute_reply":"2023-11-16T14:48:20.596056Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.set_printoptions(threshold=15, linewidth=80)\n\nprint(\"Training data shapes:\")\nfor image, label in df_test.take(3):\n    print(image.numpy().shape, label.numpy().shape)\nprint(\"Training data label examples:\", label.numpy())","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:48:20.598616Z","iopub.execute_input":"2023-11-16T14:48:20.598926Z","iopub.status.idle":"2023-11-16T14:48:21.572915Z","shell.execute_reply.started":"2023-11-16T14:48:20.598901Z","shell.execute_reply":"2023-11-16T14:48:21.571692Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Modeling","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow.keras import layers, models\n\ndef create_cnn_model(input_shape, num_classes):\n    model = models.Sequential()\n\n    # Convolutional layers\n    model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=input_shape))\n    model.add(layers.MaxPooling2D((2, 2)))\n    model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n    model.add(layers.MaxPooling2D((2, 2)))\n    model.add(layers.Conv2D(128, (3, 3), activation='relu'))\n    model.add(layers.MaxPooling2D((2, 2)))\n\n    # Flatten the output and add Dense layers\n    model.add(layers.Flatten())\n    model.add(layers.Dense(128, activation='relu'))\n    model.add(layers.Dropout(0.5))  # Adding dropout for regularization\n    model.add(layers.Dense(num_classes, activation='softmax'))\n\n    # Compile the model\n    model.compile(optimizer='adam',\n                  loss='sparse_categorical_crossentropy',\n                  metrics=['accuracy'])\n\n    return model\n\n# Replace with your actual image dimensions\ninput_shape = (512, 512, 3)\nnum_classes = len(CLASSES)  # Assuming CLASSES is a list of class labels\n\n# Create the model\ncnn_model = create_cnn_model(input_shape, num_classes)\n\n# Display model summary\ncnn_model.summary()","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:48:21.574698Z","iopub.execute_input":"2023-11-16T14:48:21.575375Z","iopub.status.idle":"2023-11-16T14:48:21.756283Z","shell.execute_reply.started":"2023-11-16T14:48:21.575337Z","shell.execute_reply":"2023-11-16T14:48:21.755232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau\nEPOCHS = 32\n# Callback for early stopping\nearly_stopping_callback = EarlyStopping(\n    monitor='val_loss',\n    patience=10,\n    verbose=1,\n    restore_best_weights=True\n)\n\n# Learning Rate Schedule for Fine Tuning #\ndef exponential_lr(epoch,\n                   start_lr = 0.00001, min_lr = 0.00001, max_lr = 0.00005,\n                   rampup_epochs = 5, sustain_epochs = 0,\n                   exp_decay = 0.8):\n\n    def lr(epoch, start_lr, min_lr, max_lr, rampup_epochs, sustain_epochs, exp_decay):\n        # linear increase from start to rampup_epochs\n        if epoch < rampup_epochs:\n            lr = ((max_lr - start_lr) /\n                  rampup_epochs * epoch + start_lr)\n        # constant max_lr during sustain_epochs\n        elif epoch < rampup_epochs + sustain_epochs:\n            lr = max_lr\n        # exponential decay towards min_lr\n        else:\n            lr = ((max_lr - min_lr) *\n                  exp_decay**(epoch - rampup_epochs - sustain_epochs) +\n                  min_lr)\n        return lr\n    return lr(epoch,\n              start_lr,\n              min_lr,\n              max_lr,\n              rampup_epochs,\n              sustain_epochs,\n              exp_decay)\n\nlr_callback = tf.keras.callbacks.LearningRateScheduler(exponential_lr, verbose=True)\n\nrng = [i for i in range(EPOCHS)]\ny = [exponential_lr(x) for x in rng]\nplt.plot(rng, y)\nprint(\"Learning rate schedule: {:.3g} to {:.3g} to {:.3g}\".format(y[0], max(y), y[-1]))","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:48:21.757594Z","iopub.execute_input":"2023-11-16T14:48:21.757980Z","iopub.status.idle":"2023-11-16T14:48:22.061957Z","shell.execute_reply.started":"2023-11-16T14:48:21.757944Z","shell.execute_reply":"2023-11-16T14:48:22.060838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_ds = get_test_dataset(ordered=True) \n\ntest_images_ds = test_ds.map(lambda image, idnum: image)\nprobabilities = cnn_model.predict(test_images_ds)\npredictions = np.argmax(probabilities, axis=-1)\nprint(predictions)\n\n\ntest_ids_ds = test_ds.map(lambda image, idnum: idnum).unbatch()\ntest_ids = next(iter(test_ids_ds.batch(NUM_TEST_IMAGES))).numpy().astype('U') # все в одной партии\nnp.savetxt('submission.csv', np.rec.fromarrays([test_ids, predictions]), fmt=['%s', '%d'], delimiter=',', header='id,label', comments='')","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:48:22.063275Z","iopub.execute_input":"2023-11-16T14:48:22.065766Z","iopub.status.idle":"2023-11-16T14:48:56.815598Z","shell.execute_reply.started":"2023-11-16T14:48:22.065730Z","shell.execute_reply":"2023-11-16T14:48:56.814721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load the CSV file into a Pandas DataFrame and print the head\nsubmission_df = pd.read_csv('submission.csv')\nprint(submission_df.head(100))","metadata":{"execution":{"iopub.status.busy":"2023-11-16T14:48:56.816878Z","iopub.execute_input":"2023-11-16T14:48:56.817263Z","iopub.status.idle":"2023-11-16T14:48:56.844777Z","shell.execute_reply.started":"2023-11-16T14:48:56.817233Z","shell.execute_reply":"2023-11-16T14:48:56.843824Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}