{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":13836,"databundleVersionId":1718836,"sourceType":"competition"},{"sourceId":1698093,"sourceType":"datasetVersion","datasetId":1006299},{"sourceId":1710684,"sourceType":"datasetVersion","datasetId":1014348},{"sourceId":1715857,"sourceType":"datasetVersion","datasetId":999219},{"sourceId":1735390,"sourceType":"datasetVersion","datasetId":1029968},{"sourceId":1735480,"sourceType":"datasetVersion","datasetId":1029991},{"sourceId":1750107,"sourceType":"datasetVersion","datasetId":1001841},{"sourceId":1750109,"sourceType":"datasetVersion","datasetId":1006301}],"dockerImageVersionId":30299,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"###### <center><img src=\"https://raw.githubusercontent.com/dimitreOliveira/MachineLearning/master/Kaggle/Cassava%20Leaf%20Disease%20Classification/banner.png\" width=\"1000\"></center>\n<br>\n<center><h1>Cassava Leaf - Supervised Contrastive Learning</h1></center>\n<br>\n\n\n- Dataset source `center cropped` [512x512](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-50-tfrecords-center-512x512) - `divided by classes` [512x512](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-50-tfrecords-classes-512x512)\n- Dataset source `external data` `center cropped` [512x512](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-50-tfrecords-external-512x512) - `divided by classes` [512x512](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-ext-50-tfrec-classes-512x512)\n- Dataset source [discussion thread](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198744)\n- Dataset [creation source](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-stratified-tfrecords-256x256)","metadata":{"papermill":{"duration":0.034254,"end_time":"2020-10-10T21:25:09.791326","exception":false,"start_time":"2020-10-10T21:25:09.757072","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Supervised Contrastive Learning\n\n### How it works?\n\n> Clusters of points belonging to the same class\nare pulled together in embedding space, while simultaneously pushing apart clusters of samples from different classes.\n\n\n### Self Supervised Contrastive Learning vs Supervised Contrastive Learning\n\n<center><img src=\"https://raw.githubusercontent.com/dimitreOliveira/MachineLearning/master/Kaggle/Cassava%20Leaf%20Disease%20Classification/sscl_vs_scl.png\" width=\"1000\"></center>\n\n<br>\n\n> Supervised vs. self-supervised contrastive losses: The self-supervised contrastive loss (left)\ncontrasts a single positive for each anchor (i.e., an augmented version of the same image) against a set of\nnegatives consisting of the entire remainder of the batch. The supervised contrastive loss (right) considered\nin this paper, however, contrasts the set of all samples from the same class as positives against the\nnegatives from the remainder of the batch. As demonstrated by the photo of the black and white puppy, taking\nclass label information into account results in an embedding space where elements of the same class are more\nclosely aligned than in the self-supervised case.\n\n<br>\n\n<center><img src=\"https://raw.githubusercontent.com/dimitreOliveira/MachineLearning/master/Kaggle/Cassava%20Leaf%20Disease%20Classification/scl_training_setup.png\" width=\"1000\"></center>\n\n<br>\n\n> Cross entropy, self-supervised contrastive loss and supervised contrastive loss: The cross entropy\nloss (left) uses labels and a softmax loss to train a classifier; the self-supervised contrastive loss (middle) uses\na contrastive loss and data augmentations to learn representations. The supervised contrastive loss (right) also\nlearns representations using a contrastive loss, but uses label information to sample positives in addition to\naugmentations of the same image. Both contrastive methods can have an optional second stage which trains a\nmodel on top of the learned representations.\n\nReferences:\n- [Supervised Contrastive Learning (Paper)](https://arxiv.org/pdf/2004.11362.pdf)\n- [Supervised Contrastive Learning (Keras tutorial)](https://keras.io/examples/vision/supervised-contrastive-learning/)\n- [Great videos reviewing the paper by Yannic Kilcher](https://www.youtube.com/watch?v=MpdbFLXOOIw)","metadata":{}},{"cell_type":"markdown","source":"### Supervised contrastive learning experiments summary\n\n- What improved performance\n - Larger `batch size`\n - Average `encoder size`\n - `Data augmentation`\n - Lager `image resolution`\n - Using `pre-trained` weights, I tried training from scratch but the results were very poor.\n - `Oversampling` the minority classes from the dataset, gave a little help, but I expected more.\n- What made no difference\n - `Temperature` parameter, according to the paper, lower `temperature` can benefit from longer training, since the data here is limited, tweaking this parameter maybe not worth it.\n - Different `projection heads`, I experimented a little with the number of neurons but got no relevant improvements.\n - Different `classifier heads`, I experimented a little with the number of neurons but got no relevant improvements.\n- Next steps\n - Different `pooling` heads (AVG+MAX, Multi-sample, etc).\n - Different `optimizers`.\n - More elaborated `data augmentation` schedules (MixUp, CutMix, etc).\n - Use external data.\n - It may be interesting to try doing `2nd stage` training but fine-tuning the complete network.","metadata":{}},{"cell_type":"markdown","source":"## Dependencies","metadata":{"papermill":{"duration":0.033986,"end_time":"2020-10-10T21:25:09.858267","exception":false,"start_time":"2020-10-10T21:25:09.824281","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!pip install --quiet efficientnet","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-05-29T04:43:59.679515Z","iopub.execute_input":"2024-05-29T04:43:59.679841Z","iopub.status.idle":"2024-05-29T04:44:12.736748Z","shell.execute_reply.started":"2024-05-29T04:43:59.679765Z","shell.execute_reply":"2024-05-29T04:44:12.735513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import math, os, re, warnings, random, time\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nfrom matplotlib import pyplot as plt\nfrom kaggle_datasets import KaggleDatasets\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import classification_report, confusion_matrix\nimport tensorflow as tf\nimport tensorflow.keras.layers as L\nimport tensorflow.keras.backend as K\nfrom tensorflow.keras import optimizers, losses, metrics, Model\nimport efficientnet.tfkeras as efn\nimport tensorflow_addons as tfa\nfrom sklearn.manifold import TSNE\n\n\ndef seed_everything(seed=0):\n    random.seed(seed)\n    np.random.seed(seed)\n    tf.random.set_seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    os.environ['TF_DETERMINISTIC_OPS'] = '1'\n\nseed = 0\nseed_everything(seed)\nwarnings.filterwarnings('ignore')","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":16.665386,"end_time":"2020-10-10T21:25:26.556557","exception":false,"start_time":"2020-10-10T21:25:09.891171","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:44:12.738967Z","iopub.execute_input":"2024-05-29T04:44:12.739326Z","iopub.status.idle":"2024-05-29T04:44:19.113339Z","shell.execute_reply.started":"2024-05-29T04:44:12.739291Z","shell.execute_reply":"2024-05-29T04:44:19.112330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Hardware configuration","metadata":{"papermill":{"duration":0.03302,"end_time":"2020-10-10T21:25:26.623975","exception":false,"start_time":"2020-10-10T21:25:26.590955","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# TPU or GPU detection\n# Detect hardware, return appropriate distribution strategy\ntry:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n    print(f'Running on TPU {tpu.master()}')\nexcept ValueError:\n    tpu = None\n\nif tpu:\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n    strategy = tf.distribute.get_strategy()\n\nAUTO = tf.data.experimental.AUTOTUNE\nREPLICAS = strategy.num_replicas_in_sync\nprint(f'REPLICAS: {REPLICAS}')\n\n\n# Mixed precision\nfrom tensorflow.keras.mixed_precision import experimental as mixed_precision\npolicy = mixed_precision.Policy('mixed_bfloat16')\nmixed_precision.set_policy(policy)\n\n# XLA\ntf.config.optimizer.set_jit(True)","metadata":{"_kg_hide-input":true,"papermill":{"duration":5.225184,"end_time":"2020-10-10T21:25:31.882571","exception":false,"start_time":"2020-10-10T21:25:26.657387","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:44:19.114773Z","iopub.execute_input":"2024-05-29T04:44:19.115490Z","iopub.status.idle":"2024-05-29T04:44:19.132084Z","shell.execute_reply.started":"2024-05-29T04:44:19.115449Z","shell.execute_reply":"2024-05-29T04:44:19.131061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model parameters","metadata":{"papermill":{"duration":0.034123,"end_time":"2020-10-10T21:25:31.952284","exception":false,"start_time":"2020-10-10T21:25:31.918161","status":"completed"},"tags":[]}},{"cell_type":"code","source":"BATCH_SIZE = 64 * REPLICAS\nLEARNING_RATE = 3e-5 * REPLICAS\nEPOCHS_SCL = 15\nEPOCHS = 10\nHEIGHT = 512\nWIDTH = 512\nHEIGHT_RS = 512\nWIDTH_RS = 512\nCHANNELS = 3\nN_CLASSES = 5\nN_FOLDS = 5\nFOLDS_USED = 1","metadata":{"papermill":{"duration":0.044623,"end_time":"2020-10-10T21:25:32.031164","exception":false,"start_time":"2020-10-10T21:25:31.986541","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:44:19.135341Z","iopub.execute_input":"2024-05-29T04:44:19.136363Z","iopub.status.idle":"2024-05-29T04:44:19.147967Z","shell.execute_reply.started":"2024-05-29T04:44:19.136318Z","shell.execute_reply":"2024-05-29T04:44:19.147131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load data","metadata":{}},{"cell_type":"code","source":"def count_data_items(filenames):\n    n = [int(re.compile(r'-([0-9]*)\\.').search(filename).group(1)) for filename in filenames]\n    return np.sum(n)\n\n\ndatabase_base_path = '/kaggle/input/cassava-leaf-disease-classification/'\ntrain = pd.read_csv(f'{database_base_path}train.csv')\nprint(f'Train samples: {len(train)}')\n\n# Dataset paths\nGCS_PATH = KaggleDatasets().get_gcs_path(f'cassava-leaf-disease-tfrecords-center-{HEIGHT}x{WIDTH}') # Center croped and resized (15 TFRecord)\nGCS_PATH_EXT = KaggleDatasets().get_gcs_path(f'cassava-leaf-disease-tfrecords-external-{HEIGHT}x{WIDTH}') # Center croped and resized (15 TFRecord) (External)\nGCS_PATH_CLASSES = KaggleDatasets().get_gcs_path(f'cassava-leaf-disease-tfrecords-classes-{HEIGHT}x{WIDTH}') # Center croped and resized (15 TFRecord) by classes\nGCS_PATH_EXT_CLASSES = KaggleDatasets().get_gcs_path(f'cassava-leaf-disease-tfrecords-classes-ext-{HEIGHT}x{WIDTH}') # Center croped and resized (15 TFRecord) (External) by classes\n\n# Dataset TFRecords\nFILENAMES_COMP = tf.io.gfile.glob(GCS_PATH + '/*.tfrec')\nFILENAMES_2019 = tf.io.gfile.glob(GCS_PATH_EXT + '/*.tfrec')\n\nFILENAMES_COMP_CBB = tf.io.gfile.glob(GCS_PATH_CLASSES + '/CBB*.tfrec')\nFILENAMES_COMP_CBSD = tf.io.gfile.glob(GCS_PATH_CLASSES + '/CBSD*.tfrec')\nFILENAMES_COMP_CGM = tf.io.gfile.glob(GCS_PATH_CLASSES + '/CGM*.tfrec')\nFILENAMES_COMP_CMD = tf.io.gfile.glob(GCS_PATH_CLASSES + '/CMD*.tfrec')\nFILENAMES_COMP_Healthy = tf.io.gfile.glob(GCS_PATH_CLASSES + '/Healthy*.tfrec')\n\nFILENAMES_2019_CBB = tf.io.gfile.glob(GCS_PATH_EXT_CLASSES + '/CBB*.tfrec')\nFILENAMES_2019_CBSD = tf.io.gfile.glob(GCS_PATH_EXT_CLASSES + '/CBSD*.tfrec')\nFILENAMES_2019_CGM = tf.io.gfile.glob(GCS_PATH_EXT_CLASSES + '/CGM*.tfrec')\nFILENAMES_2019_CMD = tf.io.gfile.glob(GCS_PATH_EXT_CLASSES + '/CMD*.tfrec')\nFILENAMES_2019_Healthy = tf.io.gfile.glob(GCS_PATH_EXT_CLASSES + '/Healthy*.tfrec')\n\nTRAINING_FILENAMES = (FILENAMES_COMP + \n                      (10 * FILENAMES_COMP_CBB) + \n                      (5 * FILENAMES_COMP_CBSD) + \n                      (4 * FILENAMES_COMP_CGM) + \n                      (4 * FILENAMES_COMP_Healthy))\nNUM_TRAINING_IMAGES = count_data_items(TRAINING_FILENAMES)\n\nprint(f'GCS: train images: {NUM_TRAINING_IMAGES}')\ndisplay(train.head())\n\nCLASSES = ['Cassava Bacterial Blight', \n           'Cassava Brown Streak Disease', \n           'Cassava Green Mottle', \n           'Cassava Mosaic Disease', \n           'Healthy']","metadata":{"papermill":{"duration":0.61129,"end_time":"2020-10-10T21:25:32.677805","exception":false,"start_time":"2020-10-10T21:25:32.066515","status":"completed"},"tags":[],"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:44:19.149165Z","iopub.execute_input":"2024-05-29T04:44:19.149425Z","iopub.status.idle":"2024-05-29T04:44:22.206018Z","shell.execute_reply.started":"2024-05-29T04:44:19.149400Z","shell.execute_reply":"2024-05-29T04:44:22.204780Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Augmentation","metadata":{}},{"cell_type":"code","source":"def data_augment(image, label):\n    p_rotation = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_spatial = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_rotate = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_pixel_1 = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_pixel_2 = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_pixel_3 = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_shear = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_crop = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_cutout = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    \n    # Shear\n    if p_shear > .2:\n        if p_shear > .6:\n            image = transform_shear(image, HEIGHT, shear=20.)\n        else:\n            image = transform_shear(image, HEIGHT, shear=-20.)\n            \n    # Rotation\n    if p_rotation > .2:\n        if p_rotation > .6:\n            image = transform_rotation(image, HEIGHT, rotation=45.)\n        else:\n            image = transform_rotation(image, HEIGHT, rotation=-45.)\n            \n    # Flips\n    image = tf.image.random_flip_left_right(image)\n    image = tf.image.random_flip_up_down(image)\n    if p_spatial > .75:\n        image = tf.image.transpose(image)\n        \n    # Rotates\n    if p_rotate > .75:\n        image = tf.image.rot90(image, k=3) # rotate 270º\n    elif p_rotate > .5:\n        image = tf.image.rot90(image, k=2) # rotate 180º\n    elif p_rotate > .25:\n        image = tf.image.rot90(image, k=1) # rotate 90º\n        \n    # Pixel-level transforms\n    if p_pixel_1 >= .4:\n        image = tf.image.random_saturation(image, lower=.7, upper=1.3)\n    if p_pixel_2 >= .4:\n        image = tf.image.random_contrast(image, lower=.8, upper=1.2)\n    if p_pixel_3 >= .4:\n        image = tf.image.random_brightness(image, max_delta=.1)\n        \n    # Crops\n    if p_crop > .6:\n        if p_crop > .9:\n            image = tf.image.central_crop(image, central_fraction=.5)\n        elif p_crop > .8:\n            image = tf.image.central_crop(image, central_fraction=.6)\n        elif p_crop > .7:\n            image = tf.image.central_crop(image, central_fraction=.7)\n        else:\n            image = tf.image.central_crop(image, central_fraction=.8)\n    elif p_crop > .3:\n        crop_size = tf.random.uniform([], int(HEIGHT*.6), HEIGHT, dtype=tf.int32)\n        image = tf.image.random_crop(image, size=[crop_size, crop_size, CHANNELS])\n            \n    image = tf.image.resize(image, size=[HEIGHT, WIDTH])\n\n    if p_cutout > .5:\n        image = data_augment_cutout(image)\n        \n    return image, label","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:44:22.207230Z","iopub.execute_input":"2024-05-29T04:44:22.207571Z","iopub.status.idle":"2024-05-29T04:44:22.234305Z","shell.execute_reply.started":"2024-05-29T04:44:22.207539Z","shell.execute_reply":"2024-05-29T04:44:22.232891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Auxiliary functions","metadata":{}},{"cell_type":"code","source":"# data augmentation @cdeotte kernel: https://www.kaggle.com/cdeotte/rotation-augmentation-gpu-tpu-0-96\ndef transform_rotation(image, height, rotation):\n    # input image - is one image of size [dim,dim,3] not a batch of [b,dim,dim,3]\n    # output - image randomly rotated\n    DIM = height\n    XDIM = DIM%2 #fix for size 331\n    \n    rotation = rotation * tf.random.uniform([1],dtype='float32')\n    # CONVERT DEGREES TO RADIANS\n    rotation = math.pi * rotation / 180.\n    \n    # ROTATION MATRIX\n    c1 = tf.math.cos(rotation)\n    s1 = tf.math.sin(rotation)\n    one = tf.constant([1],dtype='float32')\n    zero = tf.constant([0],dtype='float32')\n    rotation_matrix = tf.reshape(tf.concat([c1,s1,zero, -s1,c1,zero, zero,zero,one],axis=0),[3,3])\n\n    # LIST DESTINATION PIXEL INDICES\n    x = tf.repeat( tf.range(DIM//2,-DIM//2,-1), DIM )\n    y = tf.tile( tf.range(-DIM//2,DIM//2),[DIM] )\n    z = tf.ones([DIM*DIM],dtype='int32')\n    idx = tf.stack( [x,y,z] )\n    \n    # ROTATE DESTINATION PIXELS ONTO ORIGIN PIXELS\n    idx2 = K.dot(rotation_matrix,tf.cast(idx,dtype='float32'))\n    idx2 = K.cast(idx2,dtype='int32')\n    idx2 = K.clip(idx2,-DIM//2+XDIM+1,DIM//2)\n    \n    # FIND ORIGIN PIXEL VALUES \n    idx3 = tf.stack( [DIM//2-idx2[0,], DIM//2-1+idx2[1,]] )\n    d = tf.gather_nd(image, tf.transpose(idx3))\n        \n    return tf.reshape(d,[DIM,DIM,3])\n\ndef transform_shear(image, height, shear):\n    # input image - is one image of size [dim,dim,3] not a batch of [b,dim,dim,3]\n    # output - image randomly sheared\n    DIM = height\n    XDIM = DIM%2 #fix for size 331\n    \n    shear = shear * tf.random.uniform([1],dtype='float32')\n    shear = math.pi * shear / 180.\n        \n    # SHEAR MATRIX\n    one = tf.constant([1],dtype='float32')\n    zero = tf.constant([0],dtype='float32')\n    c2 = tf.math.cos(shear)\n    s2 = tf.math.sin(shear)\n    shear_matrix = tf.reshape(tf.concat([one,s2,zero, zero,c2,zero, zero,zero,one],axis=0),[3,3])    \n\n    # LIST DESTINATION PIXEL INDICES\n    x = tf.repeat( tf.range(DIM//2,-DIM//2,-1), DIM )\n    y = tf.tile( tf.range(-DIM//2,DIM//2),[DIM] )\n    z = tf.ones([DIM*DIM],dtype='int32')\n    idx = tf.stack( [x,y,z] )\n    \n    # ROTATE DESTINATION PIXELS ONTO ORIGIN PIXELS\n    idx2 = K.dot(shear_matrix,tf.cast(idx,dtype='float32'))\n    idx2 = K.cast(idx2,dtype='int32')\n    idx2 = K.clip(idx2,-DIM//2+XDIM+1,DIM//2)\n    \n    # FIND ORIGIN PIXEL VALUES \n    idx3 = tf.stack( [DIM//2-idx2[0,], DIM//2-1+idx2[1,]] )\n    d = tf.gather_nd(image, tf.transpose(idx3))\n        \n    return tf.reshape(d,[DIM,DIM,3])\n\n# CutOut\ndef data_augment_cutout(image, min_mask_size=(int(HEIGHT * .1), int(HEIGHT * .1)), \n                        max_mask_size=(int(HEIGHT * .125), int(HEIGHT * .125))):\n    p_cutout = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    \n    if p_cutout > .85: # 10~15 cut outs\n        n_cutout = tf.random.uniform([], 10, 15, dtype=tf.int32)\n        image = random_cutout(image, HEIGHT, WIDTH, \n                              min_mask_size=min_mask_size, max_mask_size=max_mask_size, k=n_cutout)\n    elif p_cutout > .6: # 5~10 cut outs\n        n_cutout = tf.random.uniform([], 5, 10, dtype=tf.int32)\n        image = random_cutout(image, HEIGHT, WIDTH, \n                              min_mask_size=min_mask_size, max_mask_size=max_mask_size, k=n_cutout)\n    elif p_cutout > .25: # 2~5 cut outs\n        n_cutout = tf.random.uniform([], 2, 5, dtype=tf.int32)\n        image = random_cutout(image, HEIGHT, WIDTH, \n                              min_mask_size=min_mask_size, max_mask_size=max_mask_size, k=n_cutout)\n    else: # 1 cut out\n        image = random_cutout(image, HEIGHT, WIDTH, \n                              min_mask_size=min_mask_size, max_mask_size=max_mask_size, k=1)\n\n    return image\n\ndef random_cutout(image, height, width, channels=3, min_mask_size=(10, 10), max_mask_size=(80, 80), k=1):\n    assert height > min_mask_size[0]\n    assert width > min_mask_size[1]\n    assert height > max_mask_size[0]\n    assert width > max_mask_size[1]\n\n    for i in range(k):\n      mask_height = tf.random.uniform(shape=[], minval=min_mask_size[0], maxval=max_mask_size[0], dtype=tf.int32)\n      mask_width = tf.random.uniform(shape=[], minval=min_mask_size[1], maxval=max_mask_size[1], dtype=tf.int32)\n\n      pad_h = height - mask_height\n      pad_top = tf.random.uniform(shape=[], minval=0, maxval=pad_h, dtype=tf.int32)\n      pad_bottom = pad_h - pad_top\n\n      pad_w = width - mask_width\n      pad_left = tf.random.uniform(shape=[], minval=0, maxval=pad_w, dtype=tf.int32)\n      pad_right = pad_w - pad_left\n\n      cutout_area = tf.zeros(shape=[mask_height, mask_width, channels], dtype=tf.uint8)\n\n      cutout_mask = tf.pad([cutout_area], [[0,0],[pad_top, pad_bottom], [pad_left, pad_right], [0,0]], constant_values=1)\n      cutout_mask = tf.squeeze(cutout_mask, axis=0)\n      image = tf.multiply(tf.cast(image, tf.float32), tf.cast(cutout_mask, tf.float32))\n\n    return image","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:44:22.236156Z","iopub.execute_input":"2024-05-29T04:44:22.236826Z","iopub.status.idle":"2024-05-29T04:44:22.280696Z","shell.execute_reply.started":"2024-05-29T04:44:22.236783Z","shell.execute_reply":"2024-05-29T04:44:22.279767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Datasets utility functions\ndef decode_image(image_data):\n    \"\"\"\n        Decode a JPEG-encoded image to a uint8 tensor.\n    \"\"\"\n    image = tf.image.decode_jpeg(image_data, channels=3)\n    return image\n\ndef scale_image(image, label):\n    \"\"\"\n        Cast tensor to float and normalizes (range between 0 and 1).\n    \"\"\"\n    image = tf.cast(image, tf.float32)\n    image /= 255.0\n    return image, label\n\ndef prepare_image(image, label):\n    \"\"\"\n        Resize and reshape images to the expected size.\n    \"\"\"\n    image = tf.image.resize(image, [HEIGHT_RS, WIDTH_RS])\n    image = tf.reshape(image, [HEIGHT_RS, WIDTH_RS, 3])\n    return image, label\n\ndef read_tfrecord(example, labeled=True):\n    \"\"\"\n        1. Parse data based on the 'TFREC_FORMAT' map.\n        2. Decode image.\n        3. If 'labeled' returns (image, label) if not (image, name).\n    \"\"\"\n    if labeled:\n        TFREC_FORMAT = {\n            'image': tf.io.FixedLenFeature([], tf.string), \n            'target': tf.io.FixedLenFeature([], tf.int64), \n        }\n    else:\n        TFREC_FORMAT = {\n            'image': tf.io.FixedLenFeature([], tf.string), \n            'image_name': tf.io.FixedLenFeature([], tf.string), \n        }\n    example = tf.io.parse_single_example(example, TFREC_FORMAT)\n    image = decode_image(example['image'])\n    if labeled:\n        label_or_name = tf.cast(example['target'], tf.int32)\n        # One-Hot Encoding needed to use \"categorical_crossentropy\" loss\n#         label_or_name = tf.one_hot(tf.cast(label_or_name, tf.int32), N_CLASSES)\n    else:\n        label_or_name = example['image_name']\n    return image, label_or_name\n\ndef get_dataset(FILENAMES, labeled=True, ordered=False, repeated=False, \n                cached=False, augment=False):\n    \"\"\"\n        Return a Tensorflow dataset ready for training or inference.\n    \"\"\"\n    ignore_order = tf.data.Options()\n    if not ordered:\n        ignore_order.experimental_deterministic = False\n        dataset = tf.data.Dataset.list_files(FILENAMES)\n        dataset = dataset.interleave(tf.data.TFRecordDataset, num_parallel_calls=AUTO)\n    else:\n        dataset = tf.data.TFRecordDataset(FILENAMES, num_parallel_reads=AUTO)\n        \n    dataset = dataset.with_options(ignore_order)\n    \n    dataset = dataset.map(lambda x: read_tfrecord(x, labeled=labeled), num_parallel_calls=AUTO)\n    \n    if augment:\n        dataset = dataset.map(data_augment, num_parallel_calls=AUTO)\n        \n    dataset = dataset.map(scale_image, num_parallel_calls=AUTO)\n    dataset = dataset.map(prepare_image, num_parallel_calls=AUTO)\n    \n    if not ordered:\n        dataset = dataset.shuffle(2048)\n    if repeated:\n        dataset = dataset.repeat()\n        \n    dataset = dataset.batch(BATCH_SIZE)\n    \n    if cached:\n        dataset = dataset.cache()\n    dataset = dataset.prefetch(AUTO)\n    return dataset","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_kg_hide-input":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","papermill":{"duration":0.072304,"end_time":"2020-10-10T21:25:32.785102","exception":false,"start_time":"2020-10-10T21:25:32.712798","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:44:22.282760Z","iopub.execute_input":"2024-05-29T04:44:22.283178Z","iopub.status.idle":"2024-05-29T04:44:22.304425Z","shell.execute_reply.started":"2024-05-29T04:44:22.283136Z","shell.execute_reply":"2024-05-29T04:44:22.303432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualization utility functions\nnp.set_printoptions(threshold=15, linewidth=80)\n\ndef batch_to_numpy_images_and_labels(data):\n    images, labels = data\n    numpy_images = images.numpy()\n    numpy_labels = labels.numpy()\n    if numpy_labels.dtype == object: # binary string in this case, these are image ID strings\n        numpy_labels = [None for _ in enumerate(numpy_images)]\n    # If no labels, only image IDs, return None for labels (this is the case for test data)\n    return numpy_images, numpy_labels\n\ndef title_from_label_and_target(label, correct_label):\n    if correct_label is None:\n        return CLASSES[label], True\n    correct = (label == correct_label)\n    return \"{} [{}{}{}]\".format(CLASSES[label], 'OK' if correct else 'NO', u\"\\u2192\" if not correct else '',\n                                CLASSES[correct_label] if not correct else ''), correct\n\ndef display_one_flower(image, title, subplot, red=False, titlesize=16):\n    plt.subplot(*subplot)\n    plt.axis('off')\n    plt.imshow(image)\n    if len(title) > 0:\n        plt.title(title, fontsize=int(titlesize) if not red else int(titlesize/1.2), color='red' if red else 'black', \n                  fontdict={'verticalalignment':'center'}, pad=int(titlesize/1.5))\n    return (subplot[0], subplot[1], subplot[2]+1)\n\ndef display_batch_of_images(databatch, predictions=None):\n    \"\"\"This will work with:\n    display_batch_of_images(images)\n    display_batch_of_images(images, predictions)\n    display_batch_of_images((images, labels))\n    display_batch_of_images((images, labels), predictions)\n    \"\"\"\n    # data\n    images, labels = batch_to_numpy_images_and_labels(databatch)\n    if labels is None:\n        labels = [None for _ in enumerate(images)]\n        \n    # auto-squaring: this will drop data that does not fit into square or square-ish rectangle\n    rows = int(math.sqrt(len(images)))\n    cols = len(images)//rows\n        \n    # size and spacing\n    FIGSIZE = 13.0\n    SPACING = 0.1\n    subplot=(rows,cols,1)\n    if rows < cols:\n        plt.figure(figsize=(FIGSIZE,FIGSIZE/cols*rows))\n    else:\n        plt.figure(figsize=(FIGSIZE/rows*cols,FIGSIZE))\n    \n    # display\n    for i, (image, label) in enumerate(zip(images[:rows*cols], labels[:rows*cols])):\n        title = '' if label is None else CLASSES[label]\n        correct = True\n        if predictions is not None:\n            title, correct = title_from_label_and_target(predictions[i], label)\n        dynamic_titlesize = FIGSIZE*SPACING/max(rows,cols)*40+3 # magic formula tested to work from 1x1 to 10x10 images\n        subplot = display_one_flower(image, title, subplot, not correct, titlesize=dynamic_titlesize)\n    \n    #layout\n    plt.tight_layout()\n    if label is None and predictions is None:\n        plt.subplots_adjust(wspace=0, hspace=0)\n    else:\n        plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n    plt.show()\n    \n# Visualize model predictions\ndef dataset_to_numpy_util(dataset, N):\n    dataset = dataset.unbatch().batch(N)\n    for images, labels in dataset:\n        numpy_images = images.numpy()\n        numpy_labels = labels.numpy()\n        break;  \n    return numpy_images, numpy_labels\n\ndef title_from_label_and_target(label, correct_label):\n    label = np.argmax(label, axis=-1)\n    correct = (label == correct_label)\n    return \"{} [{}{}{}]\".format(label, str(correct), ', shoud be ' if not correct else '',\n                                correct_label if not correct else ''), correct\n\ndef display_one_flower_eval(image, title, subplot, red=False):\n    plt.subplot(subplot)\n    plt.axis('off')\n    plt.imshow(image)\n    plt.title(title, fontsize=14, color='red' if red else 'black')\n    return subplot+1\n\ndef display_9_images_with_predictions(images, predictions, labels):\n    subplot=331\n    plt.figure(figsize=(13,13))\n    for i, image in enumerate(images):\n        title, correct = title_from_label_and_target(predictions[i], labels[i])\n        subplot = display_one_flower_eval(image, title, subplot, not correct)\n        if i >= 8:\n            break;\n              \n    plt.tight_layout()\n    plt.subplots_adjust(wspace=0.1, hspace=0.1)\n    plt.show()\n\n\n# Model evaluation\ndef plot_metrics(history):\n    fig, axes = plt.subplots(2, 1, sharex='col', figsize=(20, 8))\n    axes = axes.flatten()\n    \n    axes[0].plot(history['loss'], label='Train loss')\n    axes[0].plot(history['val_loss'], label='Validation loss')\n    axes[0].legend(loc='best', fontsize=16)\n    axes[0].set_title('Loss')\n    axes[0].axvline(np.argmin(history['loss']), linestyle='dashed')\n    axes[0].axvline(np.argmin(history['val_loss']), linestyle='dashed', color='orange')\n    \n    axes[1].plot(history['sparse_categorical_accuracy'], label='Train accuracy')\n    axes[1].plot(history['val_sparse_categorical_accuracy'], label='Validation accuracy')\n    axes[1].legend(loc='best', fontsize=16)\n    axes[1].set_title('Accuracy')\n    axes[1].axvline(np.argmax(history['sparse_categorical_accuracy']), linestyle='dashed')\n    axes[1].axvline(np.argmax(history['val_sparse_categorical_accuracy']), linestyle='dashed', color='orange')\n\n    plt.xlabel('Epochs', fontsize=16)\n    sns.despine()\n    plt.show()\n    \n    \ndef visualize_embeddings(embeddings, labels, figsize=(16, 16)):\n    # Extract TSNE values from embeddings\n    embed2D = TSNE(n_components=2, n_jobs=-1, random_state=seed).fit_transform(embeddings)\n    embed2D_x = embed2D[:,0]\n    embed2D_y = embed2D[:,1]\n\n    # Create dataframe with labels and TSNE values\n    df_embed = pd.DataFrame({'labels': labels})\n    df_embed = df_embed.assign(x=embed2D_x, y=embed2D_y)\n\n    # Create classes dataframes\n    df_embed_cbb = df_embed[df_embed['labels'] == 0]\n    df_embed_cbsd = df_embed[df_embed['labels'] == 1]\n    df_embed_cgm = df_embed[df_embed['labels'] == 2]\n    df_embed_cmd = df_embed[df_embed['labels'] == 3]\n    df_embed_healthy = df_embed[df_embed['labels'] == 4]\n    \n    # Plot embeddings\n    plt.figure(figsize=figsize)\n    plt.scatter(df_embed_cbb['x'], df_embed_cbb['y'],color='yellow',s=10,label='CBB')\n    plt.scatter(df_embed_cbsd['x'], df_embed_cbsd['y'],color='blue',s=10,label='CBSD')\n    plt.scatter(df_embed_cgm['x'], df_embed_cgm['y'],color='red',s=10,label='CGM')\n    plt.scatter(df_embed_cmd['x'], df_embed_cmd['y'],color='orange',s=10,label='CMD')\n    plt.scatter(df_embed_healthy['x'], df_embed_healthy['y'],color='green',s=10,label='Healthy')\n\n    plt.legend()\n    plt.show()","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.08061,"end_time":"2020-10-10T21:25:32.900668","exception":false,"start_time":"2020-10-10T21:25:32.820058","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:44:22.306261Z","iopub.execute_input":"2024-05-29T04:44:22.306620Z","iopub.status.idle":"2024-05-29T04:44:22.347402Z","shell.execute_reply.started":"2024-05-29T04:44:22.306586Z","shell.execute_reply":"2024-05-29T04:44:22.346428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training data samples (with augmentation)","metadata":{}},{"cell_type":"code","source":"train_dataset = get_dataset(FILENAMES_COMP, ordered=True, augment=True)\ntrain_iter = iter(train_dataset.unbatch().batch(20))\n\ndisplay_batch_of_images(next(train_iter))\ndisplay_batch_of_images(next(train_iter))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:44:22.352123Z","iopub.execute_input":"2024-05-29T04:44:22.352431Z","iopub.status.idle":"2024-05-29T04:44:38.801310Z","shell.execute_reply.started":"2024-05-29T04:44:22.352402Z","shell.execute_reply":"2024-05-29T04:44:38.800267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Dataset distribution (oversampled)","metadata":{}},{"cell_type":"code","source":"ds_dist = get_dataset(TRAINING_FILENAMES)\nlabels_comp = [target.numpy() for img, target in iter(ds_dist.unbatch())]\n\nfig, ax = plt.subplots(1, 1, figsize=(18, 8))\nax = sns.countplot(y=labels_comp, palette='viridis')\nax.tick_params(labelsize=16)\n\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:44:38.802746Z","iopub.execute_input":"2024-05-29T04:44:38.803126Z","iopub.status.idle":"2024-05-29T04:52:04.481162Z","shell.execute_reply.started":"2024-05-29T04:44:38.803089Z","shell.execute_reply":"2024-05-29T04:52:04.480243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Learning rate schedule\n\nWe are going to use a `cosine learning rate schedule with a warm-up phase`, this may be a good idea since we are using a pre-trained model, the warm-up phase will be useful to avoid the pre-trained weights degradation resulting in catastrophic forgetting, during the schedule the learning rate will slowly decrease to very low values, this helps the model to land on more stable weights.","metadata":{"papermill":{"duration":0.151388,"end_time":"2020-10-10T21:30:00.486807","exception":false,"start_time":"2020-10-10T21:30:00.335419","status":"completed"},"tags":[]}},{"cell_type":"code","source":"lr_start = 1e-8\nlr_min = 1e-8\nlr_max = LEARNING_RATE\nnum_cycles = 1.\nwarmup_epochs = 1\nhold_max_epochs = 0\ntotal_epochs = EPOCHS\nwarmup_steps = warmup_epochs * (NUM_TRAINING_IMAGES//BATCH_SIZE)\nhold_max_steps = hold_max_epochs * (NUM_TRAINING_IMAGES//BATCH_SIZE)\ntotal_steps = total_epochs * (NUM_TRAINING_IMAGES//BATCH_SIZE)\n\n@tf.function\ndef cosine_schedule_with_warmup(step, total_steps, warmup_steps=0, hold_max_steps=0, \n                                lr_start=1e-4, lr_max=1e-3, lr_min=None, num_cycles=0.5):\n    if step < warmup_steps:\n        lr = (lr_max - lr_start) / warmup_steps * step + lr_start\n    else:\n        progress = (step - warmup_steps) / (total_steps - warmup_steps)\n        lr = lr_max * (0.5 * (1.0 + tf.math.cos(np.pi * ((num_cycles * progress) % 1.0))))\n        if lr_min is not None:\n            lr = tf.math.maximum(lr_min, float(lr))\n\n    return lr\n\n\nrng = [i for i in range(total_steps)]\ny = [cosine_schedule_with_warmup(tf.cast(x, tf.float32), tf.cast(total_steps, tf.float32), \n                                 tf.cast(warmup_steps, tf.float32), hold_max_steps, \n                                 lr_start, lr_max, lr_min, num_cycles) for x in rng]\n\nsns.set(style='whitegrid')\nfig, ax = plt.subplots(figsize=(20, 6))\nplt.plot(rng, y)\n\nprint(f'{total_steps} total steps and {NUM_TRAINING_IMAGES//BATCH_SIZE} steps per epoch')\nprint(f'Learning rate schedule: {y[0]:.3g} to {max(y):.3g} to {y[-1]:.3g}')","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.532334,"end_time":"2020-10-10T21:30:01.170556","exception":false,"start_time":"2020-10-10T21:30:00.638222","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:52:04.482298Z","iopub.execute_input":"2024-05-29T04:52:04.482583Z","iopub.status.idle":"2024-05-29T04:52:21.926925Z","shell.execute_reply.started":"2024-05-29T04:52:04.482557Z","shell.execute_reply":"2024-05-29T04:52:21.925923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model","metadata":{"papermill":{"duration":0.149079,"end_time":"2020-10-10T21:26:06.196269","exception":false,"start_time":"2020-10-10T21:26:06.04719","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def encoder_fn(input_shape):\n    inputs = L.Input(shape=input_shape, name='inputs')\n    base_model = efn.EfficientNetB3(input_tensor=inputs, \n                                    include_top=False, \n                                    weights='noisy-student', \n                                    pooling='avg')\n    \n    model = Model(inputs=inputs, outputs=base_model.outputs)\n\n    return model\n\n\ndef classifier_fn(input_shape, N_CLASSES, encoder, trainable=True):\n    for layer in encoder.layers:\n        layer.trainable = trainable\n        \n    inputs = L.Input(shape=input_shape, name='inputs')\n    \n    features = encoder(inputs)\n    features = L.Dropout(.5)(features)\n    features = L.Dense(1000, activation='relu')(features)\n    features = L.Dropout(.5)(features)\n    outputs = L.Dense(N_CLASSES, activation='softmax', name='outputs', dtype='float32')(features)\n\n    model = Model(inputs=inputs, outputs=outputs)\n\n    return model","metadata":{"papermill":{"duration":0.159056,"end_time":"2020-10-10T21:26:06.50331","exception":false,"start_time":"2020-10-10T21:26:06.344254","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:52:21.928350Z","iopub.execute_input":"2024-05-29T04:52:21.928678Z","iopub.status.idle":"2024-05-29T04:52:21.938244Z","shell.execute_reply.started":"2024-05-29T04:52:21.928647Z","shell.execute_reply":"2024-05-29T04:52:21.937168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Supervised Contrastive learning parameters","metadata":{}},{"cell_type":"code","source":"temperature = 0.1\n\nclass SupervisedContrastiveLoss(losses.Loss):\n    def __init__(self, temperature=1, name=None):\n        super(SupervisedContrastiveLoss, self).__init__(name=name)\n        self.temperature = temperature\n\n    def __call__(self, labels, feature_vectors, sample_weight=None):\n        # Normalize feature vectors\n        feature_vectors_normalized = tf.math.l2_normalize(feature_vectors, axis=1)\n        # Compute logits\n        logits = tf.divide(\n            tf.matmul(\n                feature_vectors_normalized, tf.transpose(feature_vectors_normalized)\n            ),\n            temperature,\n        )\n        return tfa.losses.npairs_loss(tf.squeeze(labels), logits)\n\n\ndef add_projection_head(input_shape, encoder):\n    inputs = L.Input(shape=input_shape, name='inputs')\n    features = encoder(inputs)\n    outputs = L.Dense(128, activation='relu', name='projection_head', dtype='float32')(features)\n    \n    model = Model(inputs=inputs, outputs=outputs)\n    \n    return model","metadata":{"execution":{"iopub.status.busy":"2024-05-29T04:52:21.939593Z","iopub.execute_input":"2024-05-29T04:52:21.939900Z","iopub.status.idle":"2024-05-29T04:52:21.956971Z","shell.execute_reply.started":"2024-05-29T04:52:21.939872Z","shell.execute_reply":"2024-05-29T04:52:21.956047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training (regular training)","metadata":{"papermill":{"duration":0.152125,"end_time":"2020-10-10T21:30:01.481735","exception":false,"start_time":"2020-10-10T21:30:01.32961","status":"completed"},"tags":[]}},{"cell_type":"code","source":"#####源原来的代码","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#####源原来的代码\nskf = KFold(n_splits=N_FOLDS, shuffle=True, random_state=seed)\noof_reg_pred = []; oof_reg_labels = []; oof_reg_embed = []\n\nfor fold,(idxT, idxV) in enumerate(skf.split(np.arange(15))):\n    if fold >= FOLDS_USED:\n        break\n    print(f'\\nFOLD: {fold+1}')\n    print(f'TRAIN: {idxT} VALID: {idxV}')\n\n    # Create train and validation sets\n    FILENAMES_COMP = tf.io.gfile.glob([GCS_PATH + '/Id_train%.2i*.tfrec' % x for x in idxT])\n#     FILENAMES_2019 = tf.io.gfile.glob([GCS_PATH_EXT + '/Id_train%.2i*.tfrec' % x for x in idxT])\n\n    FILENAMES_COMP_CBB = tf.io.gfile.glob([GCS_PATH_CLASSES + '/CBB%.2i*.tfrec' % x for x in idxT])\n    FILENAMES_COMP_CBSD = tf.io.gfile.glob([GCS_PATH_CLASSES + '/CBSD%.2i*.tfrec' % x for x in idxT])\n    FILENAMES_COMP_CGM = tf.io.gfile.glob([GCS_PATH_CLASSES + '/CGM%.2i*.tfrec' % x for x in idxT])\n    FILENAMES_COMP_Healthy = tf.io.gfile.glob([GCS_PATH_CLASSES + '/Healthy%.2i*.tfrec' % x for x in idxT])\n    \n#     FILENAMES_2019_CBB = tf.io.gfile.glob([GCS_PATH_EXT_CLASSES + '/CBB%.2i*.tfrec' % x for x in idxT])\n#     FILENAMES_2019_CBSD = tf.io.gfile.glob([GCS_PATH_EXT_CLASSES + '/CBSD%.2i*.tfrec' % x for x in idxT])\n#     FILENAMES_2019_CGM = tf.io.gfile.glob([GCS_PATH_EXT_CLASSES + '/CGM%.2i*.tfrec' % x for x in idxT])\n#     FILENAMES_2019_Healthy = tf.io.gfile.glob([GCS_PATH_EXT_CLASSES + '/Healthy%.2i*.tfrec' % x for x in idxT])\n\n    TRAIN_FILENAMES = (FILENAMES_COMP + \n#                        FILENAMES_2019 + \n                       (10 * FILENAMES_COMP_CBB) + \n#                        (2 * FILENAMES_2019_CBB) + \n                       (5 * FILENAMES_COMP_CBSD) + \n#                        (2 * FILENAMES_2019_CBSD) + \n                       (4 * FILENAMES_COMP_CGM) + \n#                        (2 * FILENAMES_2019_CGM) + \n                       (4 * FILENAMES_COMP_Healthy) \n#                        (2 * FILENAMES_2019_Healthy)\n                      )\n    VALID_FILENAMES = tf.io.gfile.glob([GCS_PATH + '/Id_train%.2i*.tfrec' % x for x in idxV])\n    np.random.shuffle(TRAIN_FILENAMES)\n    ct_train = count_data_items(TRAIN_FILENAMES)\n    step_size = (ct_train // BATCH_SIZE)\n    total_steps = EPOCHS * step_size\n    \n    # Model\n    with strategy.scope():\n        encoder_reg = encoder_fn((None, None, CHANNELS))\n        model_reg = classifier_fn((None, None, CHANNELS), N_CLASSES, encoder_reg)\n        model_reg.summary()\n        \n        lr = lambda: cosine_schedule_with_warmup(tf.cast(optimizer.iterations, tf.float32), \n                                                 tf.cast(total_steps, tf.float32), \n                                                 tf.cast(warmup_steps, tf.float32), \n                                                 hold_max_steps, lr_start, lr_max, lr_min, num_cycles)\n        optimizer = optimizers.Adam(learning_rate=lr)\n        model_reg.compile(optimizer=optimizer, \n                          loss=losses.SparseCategoricalCrossentropy(), \n                          metrics=[metrics.SparseCategoricalAccuracy()])\n            \n    ## TRAIN\n    history_reg = model_reg.fit(x=get_dataset(TRAIN_FILENAMES, repeated=True, augment=True), \n                                validation_data=get_dataset(VALID_FILENAMES, ordered=True, cached=True), \n                                steps_per_epoch=step_size, \n                                epochs=EPOCHS,  \n                                verbose=1).history\n    model_path = f'model_reg_{fold}.h5'\n    model_reg.save_weights(model_path)\n    \n    ### RESULTS\n    print(f\"#### FOLD {fold+1} OOF Accuracy = {np.max(history_reg['val_sparse_categorical_accuracy']):.3f}\")\n\n    # OOF predictions\n    ds_valid = get_dataset(VALID_FILENAMES, ordered=True)\n    oof_reg_labels.append([target.numpy() for img, target in iter(ds_valid.unbatch())])\n    x_oof = ds_valid.map(lambda image, target: image)\n    oof_reg_pred.append(np.argmax(model_reg.predict(x_oof), axis=-1))\n    oof_reg_embed.append(encoder_reg.predict(x_oof)) # OOF embeddings","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#以下都为修改的","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow.keras import layers\n\ndef encoder_fn(input_shape):\n    inputs = layers.Input(shape=input_shape)\n    x = layers.experimental.preprocessing.Rescaling(1./255)(inputs)\n    x = layers.Conv2D(3, (3, 3), activation='relu')(x)\n    x = layers.MaxPooling2D((2, 2))(x)\n    x = layers.Conv2D(6, (3, 3), activation='relu')(x)\n    x = layers.MaxPooling2D((2, 2))(x)\n    x = layers.Conv2D(9, (3, 3), activation='relu')(x)\n    x = layers.Flatten()(x)  # 添加这一行\n    x = layers.Reshape((-1,))(x)  # 添加这一行\n    x = layers.Dense(128, activation='relu')(x)\n    outputs = layers.Dense(N_CLASSES, activation='softmax')(x)\n    model = Model(inputs=inputs, outputs=outputs)\n    return model\n\n\ndef create_train_filenames(gcs_path, idxT):\n    FILENAMES_COMP = create_filenames(gcs_path, idxT)\n    FILENAMES_COMP_CBB = create_filenames(gcs_path + '/classes', idxT) * 10\n    FILENAMES_COMP_CBSD = create_filenames(gcs_path + '/classes', idxT) * 5\n    FILENAMES_COMP_CGM = create_filenames(gcs_path + '/classes', idxT) * 4\n    FILENAMES_COMP_Healthy = create_filenames(gcs_path + '/classes', idxT) * 4\n    return FILENAMES_COMP + FILENAMES_COMP_CBB + FILENAMES_COMP_CBSD + FILENAMES_COMP_CGM + FILENAMES_COMP_Healthy\n\nskf = KFold(n_splits=N_FOLDS, shuffle=True, random_state=seed)\noof_reg_pred = []\noof_reg_labels = []\noof_reg_embed = []\n\nfor fold, (idxT, idxV) in enumerate(skf.split(np.arange(15))):\n    if fold >= FOLDS_USED:\n        break\n    print(f'FOLD: {fold+1}')\n    print(f'TRAIN: {idxT} VALID: {idxV}')\n\n    # Create train and validation sets\n    TRAIN_FILENAMES = create_train_filenames(GCS_PATH, idxT)\n    VALID_FILENAMES = create_filenames(GCS_PATH, idxV)\n    np.random.shuffle(TRAIN_FILENAMES)\n    ct_train = count_data_items(TRAIN_FILENAMES)\n    step_size = (ct_train // BATCH_SIZE)\n    total_steps = EPOCHS * step_size\n    \n# Model\nwith strategy.scope():\n    encoder_reg = encoder_fn((None, None, CHANNELS))\n    model_reg = classifier_fn((None, None, CHANNELS), N_CLASSES, encoder_reg)\n    model_reg.summary()\n\n    lr = lambda: cosine_schedule_with_warmup(tf.cast(optimizer.iterations, tf.float32), \n                                             tf.cast(total_steps, tf.float32), \n                                             tf.cast(warmup_steps, tf.float32), \n                                             hold_max_steps, lr_start, lr_max, lr_min, num_cycles)\n    optimizer = tf.keras.optimizers.Adam(learning_rate=lr)\n    model_reg.compile(optimizer=optimizer, \n                      loss=tf.keras.losses.SparseCategoricalCrossentropy(), \n                      metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])\n\n## TRAIN\n    history_reg = model_reg.fit(x=get_dataset(TRAIN_FILENAMES, repeated=True, augment=True), \n                                validation_data=get_dataset(VALID_FILENAMES, ordered=True, cached=True), \n                                steps_per_epoch=step_size, \n                                epochs=EPOCHS,  \n                                verbose=1).history\n    model_path = f'model_reg_{fold}.h5'\n    model_reg.save_weights(model_path)\n    \n    ### RESULTS\n    print(f\"#### FOLD {fold+1} OOF Accuracy = {np.max(history_reg['val_sparse_categorical_accuracy']):.3f}\")\n\n    # OOF predictions\n    ds_valid = get_dataset(VALID_FILENAMES, ordered=True)\n    oof_reg_labels.append([target.numpy() for img, target in iter(ds_valid.unbatch())])\n    x_oof = ds_valid.map(lambda image, target: image)\n    oof_reg_pred.append(np.argmax(model_reg.predict(x_oof), axis=-1))\n    oof_reg_embed.append(encoder_reg.predict(x_oof)) # OOF embeddings\n","metadata":{"execution":{"iopub.status.busy":"2024-05-29T05:43:12.735901Z","iopub.execute_input":"2024-05-29T05:43:12.736292Z","iopub.status.idle":"2024-05-29T05:43:12.854685Z","shell.execute_reply.started":"2024-05-29T05:43:12.736261Z","shell.execute_reply":"2024-05-29T05:43:12.853174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def create_filenames(gcs_path, idx):\n    return [f'{gcs_path}/Id_train%.2i*.tfrec' % x for x in idx]\n\ndef create_train_filenames(gcs_path, idxT):\n    FILENAMES_COMP = create_filenames(gcs_path, idxT)\n    FILENAMES_COMP_CBB = create_filenames(gcs_path + '/classes', idxT) * 10\n    FILENAMES_COMP_CBSD = create_filenames(gcs_path + '/classes', idxT) * 5\n    FILENAMES_COMP_CGM = create_filenames(gcs_path + '/classes', idxT) * 4\n    FILENAMES_COMP_Healthy = create_filenames(gcs_path + '/classes', idxT) * 4\n    return FILENAMES_COMP + FILENAMES_COMP_CBB + FILENAMES_COMP_CBSD + FILENAMES_COMP_CGM + FILENAMES_COMP_Healthy\n\nskf = KFold(n_splits=N_FOLDS, shuffle=True, random_state=seed)\noof_reg_pred = []\noof_reg_labels = []\noof_reg_embed = []\n\nfor fold, (idxT, idxV) in enumerate(skf.split(np.arange(15))):\n    if fold >= FOLDS_USED:\n        break\n    print(f'FOLD: {fold+1}')\n    print(f'TRAIN: {idxT} VALID: {idxV}')\n\n    # Create train and validation sets\n    TRAIN_FILENAMES = create_train_filenames(GCS_PATH, idxT)\n    VALID_FILENAMES = create_filenames(GCS_PATH, idxV)\n    np.random.shuffle(TRAIN_FILENAMES)\n    ct_train = count_data_items(TRAIN_FILENAMES)\n    step_size = (ct_train // BATCH_SIZE)\n    total_steps = EPOCHS * step_size\n    \n    # Model\n    with strategy.scope():\n        encoder_reg = encoder_fn((None, None, CHANNELS))\n        model_reg = classifier_fn((None, None, CHANNELS), N_CLASSES, encoder_reg)\n        model_reg.summary()\n        \n        lr = lambda: cosine_schedule_with_warmup(tf.cast(optimizer.iterations, tf.float32), \n                                                 tf.cast(total_steps, tf.float32), \n                                                 tf.cast(warmup_steps, tf.float32), \n                                                 hold_max_steps, lr_start, lr_max, lr_min, num_cycles)\n        optimizer = optimizers.Adam(learning_rate=lr)\n        model_reg.compile(optimizer=optimizer, \n                          loss=losses.SparseCategoricalCrossentropy(), \n                          metrics=[metrics.SparseCategoricalAccuracy()])\n            \n    ## TRAIN\n    history_reg = model_reg.fit(x=get_dataset(TRAIN_FILENAMES, repeated=True, augment=True), \n                                validation_data=get_dataset(VALID_FILENAMES, ordered=True, cached=True), \n                                steps_per_epoch=step_size, \n                                epochs=EPOCHS,  \n                                verbose=1).history\n    model_path = f'model_reg_{fold}.h5'\n    model_reg.save_weights(model_path)\n    \n    ### RESULTS\n    print(f\"#### FOLD {fold+1} OOF Accuracy = {np.max(history_reg['val_sparse_categorical_accuracy']):.3f}\")\n\n    # OOF predictions\n    ds_valid = get_dataset(VALID_FILENAMES, ordered=True)\n    oof_reg_labels.append([target.numpy() for img, target in iter(ds_valid.unbatch())])\n    x_oof = ds_valid.map(lambda image, target: image)\n    oof_reg_pred.append(np.argmax(model_reg.predict(x_oof), axis=-1))\n    oof_reg_embed.append(encoder_reg.predict(x_oof)) # OOF embeddings\n","metadata":{"execution":{"iopub.status.busy":"2024-05-29T05:25:50.748023Z","iopub.execute_input":"2024-05-29T05:25:50.748760Z","iopub.status.idle":"2024-05-29T05:25:50.809596Z","shell.execute_reply.started":"2024-05-29T05:25:50.748723Z","shell.execute_reply":"2024-05-29T05:25:50.808137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#以上都为修改","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# # Training (supervised contrastive learning)","metadata":{}},{"cell_type":"code","source":"skf = KFold(n_splits=N_FOLDS, shuffle=True, random_state=seed)\noof_pred = []; oof_labels = []; oof_embed = []\n\nfor fold,(idxT, idxV) in enumerate(skf.split(np.arange(15))):\n    if fold >= FOLDS_USED:\n        break\n    if tpu: tf.tpu.experimental.initialize_tpu_system(tpu)\n    K.clear_session()\n    print(f'\\nFOLD: {fold+1}')\n    print(f'TRAIN: {idxT} VALID: {idxV}')\n\n    # Create train and validation sets\n    FILENAMES_COMP = tf.io.gfile.glob([GCS_PATH + '/Id_train%.2i*.tfrec' % x for x in idxT])\n#     FILENAMES_2019 = tf.io.gfile.glob([GCS_PATH_EXT + '/Id_train%.2i*.tfrec' % x for x in idxT])\n\n    FILENAMES_COMP_CBB = tf.io.gfile.glob([GCS_PATH_CLASSES + '/CBB%.2i*.tfrec' % x for x in idxT])\n    FILENAMES_COMP_CBSD = tf.io.gfile.glob([GCS_PATH_CLASSES + '/CBSD%.2i*.tfrec' % x for x in idxT])\n    FILENAMES_COMP_CGM = tf.io.gfile.glob([GCS_PATH_CLASSES + '/CGM%.2i*.tfrec' % x for x in idxT])\n    FILENAMES_COMP_Healthy = tf.io.gfile.glob([GCS_PATH_CLASSES + '/Healthy%.2i*.tfrec' % x for x in idxT])\n    \n#     FILENAMES_2019_CBB = tf.io.gfile.glob([GCS_PATH_EXT_CLASSES + '/CBB%.2i*.tfrec' % x for x in idxT])\n#     FILENAMES_2019_CBSD = tf.io.gfile.glob([GCS_PATH_EXT_CLASSES + '/CBSD%.2i*.tfrec' % x for x in idxT])\n#     FILENAMES_2019_CGM = tf.io.gfile.glob([GCS_PATH_EXT_CLASSES + '/CGM%.2i*.tfrec' % x for x in idxT])\n#     FILENAMES_2019_Healthy = tf.io.gfile.glob([GCS_PATH_EXT_CLASSES + '/Healthy%.2i*.tfrec' % x for x in idxT])\n\n    TRAIN_FILENAMES = (FILENAMES_COMP + \n#                        FILENAMES_2019 + \n                       (10 * FILENAMES_COMP_CBB) + \n#                        (2 * FILENAMES_2019_CBB) + \n                       (5 * FILENAMES_COMP_CBSD) + \n#                        (2 * FILENAMES_2019_CBSD) + \n                       (4 * FILENAMES_COMP_CGM) + \n#                        (2 * FILENAMES_2019_CGM) + \n                       (4 * FILENAMES_COMP_Healthy) \n#                        (2 * FILENAMES_2019_Healthy)\n                      )    \n    VALID_FILENAMES = tf.io.gfile.glob([GCS_PATH + '/Id_train%.2i*.tfrec' % x for x in idxV])\n    np.random.shuffle(TRAIN_FILENAMES)\n       \n    ct_train = count_data_items(TRAIN_FILENAMES)\n    step_size = (ct_train // BATCH_SIZE)\n    warmup_steps = warmup_epochs * step_size\n    total_steps = EPOCHS_SCL * step_size\n\n    ### Pre-train the encoder\n    print('Pre-training the encoder using \"Supervised Contrastive\" Loss')\n    with strategy.scope():\n        encoder = encoder_fn((None, None, CHANNELS))\n        encoder_proj = add_projection_head((None, None, CHANNELS), encoder)\n        encoder_proj.summary()\n\n        lr = lambda: cosine_schedule_with_warmup(tf.cast(optimizer.iterations, tf.float32), \n                                                 tf.cast(total_steps, tf.float32), \n                                                 tf.cast(warmup_steps, tf.float32), \n                                                 hold_max_steps, lr_start, lr_max, lr_min, num_cycles)\n        \n        optimizer = optimizers.Adam(learning_rate=lr)\n        encoder_proj.compile(optimizer=optimizer, \n                             loss=SupervisedContrastiveLoss(temperature))\n        \n    history_enc = encoder_proj.fit(x=get_dataset(TRAIN_FILENAMES, repeated=True, augment=True), \n                                   validation_data=get_dataset(VALID_FILENAMES, ordered=True, cached=True), \n                                   steps_per_epoch=step_size, \n                                   batch_size=BATCH_SIZE, \n                                   epochs=EPOCHS_SCL,\n                                   verbose=2).history\n\n    \n    ### Train the classifier with the frozen encoder\n    print('Training the classifier with the frozen encoder')\n    warmup_steps = 1\n    total_steps = EPOCHS * step_size\n    \n    with strategy.scope():\n        model = classifier_fn((None, None, CHANNELS), N_CLASSES, encoder, trainable=False)\n        model.summary()\n\n        lr = lambda: cosine_schedule_with_warmup(tf.cast(optimizer.iterations, tf.float32), \n                                                 tf.cast(total_steps, tf.float32), \n                                                 tf.cast(warmup_steps, tf.float32), \n                                                 hold_max_steps, lr_start, lr_max, lr_min, num_cycles)\n        optimizer = optimizers.Adam(learning_rate=lr)\n        model.compile(optimizer=optimizer, \n                      loss=losses.SparseCategoricalCrossentropy(), \n                      metrics=[metrics.SparseCategoricalAccuracy()])\n    \n    history = model.fit(x=get_dataset(TRAIN_FILENAMES, repeated=True, augment=True), \n                        validation_data=get_dataset(VALID_FILENAMES, ordered=True, cached=True), \n                        steps_per_epoch=step_size, \n                        epochs=EPOCHS,  \n                        verbose=2).history\n    model_path = f'model_scl_{fold}.h5'\n    model.save_weights(model_path)\n    \n    \n    ### RESULTS\n    print(f\"#### FOLD {fold+1} OOF Accuracy = {np.max(history['val_sparse_categorical_accuracy']):.3f}\")\n\n    # OOF predictions\n    ds_valid = get_dataset(VALID_FILENAMES, ordered=True)\n    oof_labels.append([target.numpy() for img, target in iter(ds_valid.unbatch())])\n    x_oof = ds_valid.map(lambda image, target: image)\n    oof_pred.append(np.argmax(model.predict(x_oof), axis=-1))\n    oof_embed.append(encoder.predict(x_oof)) # OOF embeddings","metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-05-29T04:52:28.470843Z","iopub.status.idle":"2024-05-29T04:52:28.471327Z","shell.execute_reply.started":"2024-05-29T04:52:28.471091Z","shell.execute_reply":"2024-05-29T04:52:28.471112Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualize embeddings outputs\n\nSince we have randomness from many sources (model initialization, data augmentation, training, and t-SNE itself, it can get tricky to predict the output from the embedding visualizations, but what is expected here is that the embedding training using `supervised contrastive learning` will be much more clustered than the embedding trained using regular `cross-entropy` since the former rewards the model from \"pushing\" apart instances from different classes.\n\n\n## Regular training","metadata":{"papermill":{"duration":0.157699,"end_time":"2020-10-10T22:09:56.87745","exception":false,"start_time":"2020-10-10T22:09:56.719751","status":"completed"},"tags":[]}},{"cell_type":"code","source":"y_reg_true = np.concatenate(oof_reg_labels)\ny_reg_pred = np.concatenate(oof_reg_pred)\nembeddings_reg = np.concatenate(oof_reg_embed)\n\nvisualize_embeddings(embeddings_reg, y_reg_true)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:52:28.472672Z","iopub.status.idle":"2024-05-29T04:52:28.473117Z","shell.execute_reply.started":"2024-05-29T04:52:28.472856Z","shell.execute_reply":"2024-05-29T04:52:28.472873Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Supervised contrastive learning","metadata":{}},{"cell_type":"code","source":"y_true = np.concatenate(oof_labels)\ny_pred = np.concatenate(oof_pred)\nembeddings_scl = np.concatenate(oof_embed)\n\nvisualize_embeddings(embeddings_scl, y_true)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:52:28.475188Z","iopub.status.idle":"2024-05-29T04:52:28.475666Z","shell.execute_reply.started":"2024-05-29T04:52:28.475400Z","shell.execute_reply":"2024-05-29T04:52:28.475436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model evaluation\n\nNow we can evaluate the performance of the model, first, we can evaluate the usual metrics like, `accuracy`, `precision`, `recall`, and `f1-score`, `scikit-learn` provides the perfect function for this `classification_report`.\n\nWe are evaluating the model on the `OOF` predictions, it stands for `Out Of Fold`, since we are training using `K-Fold` our model will see all the data, and the correct way to evaluate each fold is by looking at the predictions that are not from that fold.\n\n## OOF metrics (regular training)","metadata":{"papermill":{"duration":0.180457,"end_time":"2020-10-10T22:09:58.291265","exception":false,"start_time":"2020-10-10T22:09:58.110808","status":"completed"},"tags":[]}},{"cell_type":"code","source":"print(classification_report(y_reg_true, y_reg_pred, target_names=CLASSES))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:52:28.477101Z","iopub.status.idle":"2024-05-29T04:52:28.477529Z","shell.execute_reply.started":"2024-05-29T04:52:28.477313Z","shell.execute_reply":"2024-05-29T04:52:28.477330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## OOF metrics (supervised contrastive learning)","metadata":{}},{"cell_type":"code","source":"print(classification_report(y_true, y_pred, target_names=CLASSES))","metadata":{"_kg_hide-input":true,"papermill":{"duration":5.399145,"end_time":"2020-10-10T22:10:49.740438","exception":false,"start_time":"2020-10-10T22:10:44.341293","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:52:28.478699Z","iopub.status.idle":"2024-05-29T04:52:28.479108Z","shell.execute_reply.started":"2024-05-29T04:52:28.478890Z","shell.execute_reply":"2024-05-29T04:52:28.478914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Confusion matrix  (regular training)\n\nLet's also take a look at the confusion matrix, this will give us an idea about what classes the model is mixing or having a hard time.","metadata":{"papermill":{"duration":0.165203,"end_time":"2020-10-10T22:10:50.079293","exception":false,"start_time":"2020-10-10T22:10:49.91409","status":"completed"},"tags":[]}},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 1, figsize=(20, 12))\ncfn_matrix = confusion_matrix(y_reg_true, y_reg_pred, labels=range(len(CLASSES)))\ncfn_matrix = (cfn_matrix.T / cfn_matrix.sum(axis=1)).T\ndf_cm = pd.DataFrame(cfn_matrix, index=CLASSES, columns=CLASSES)\nax = sns.heatmap(df_cm, cmap='Blues', annot=True, fmt='.2f', linewidths=.5).set_title('OOF', fontsize=30)\nplt.show()","metadata":{"_kg_hide-input":true,"papermill":{"duration":3.872244,"end_time":"2020-10-10T22:10:54.125651","exception":false,"start_time":"2020-10-10T22:10:50.253407","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:52:28.480700Z","iopub.status.idle":"2024-05-29T04:52:28.481125Z","shell.execute_reply.started":"2024-05-29T04:52:28.480893Z","shell.execute_reply":"2024-05-29T04:52:28.480917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Confusion matrix (supervised contrastive learning)","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(1, 1, figsize=(20, 12))\ncfn_matrix = confusion_matrix(y_true, y_pred, labels=range(len(CLASSES)))\ncfn_matrix = (cfn_matrix.T / cfn_matrix.sum(axis=1)).T\ndf_cm = pd.DataFrame(cfn_matrix, index=CLASSES, columns=CLASSES)\nax = sns.heatmap(df_cm, cmap='Blues', annot=True, fmt='.2f', linewidths=.5).set_title('OOF', fontsize=30)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2024-05-29T04:52:28.482379Z","iopub.status.idle":"2024-05-29T04:52:28.482734Z","shell.execute_reply.started":"2024-05-29T04:52:28.482558Z","shell.execute_reply":"2024-05-29T04:52:28.482574Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualize predictions\n\nFinally, it is a good practice to always inspect some of the model's prediction by looking at the data, this can give an idea if the model is getting some predictions wrong because the data is really hard, or if it is because the model is actually bad.\n\n\n### Class map\n```\n0: Cassava Bacterial Blight (CBB)\n1: Cassava Brown Streak Disease (CBSD)\n2: Cassava Green Mottle (CGM)\n3: Cassava Mosaic Disease (CMD)\n4: Healthy\n```","metadata":{"papermill":{"duration":0.168779,"end_time":"2020-10-10T22:10:58.678136","exception":false,"start_time":"2020-10-10T22:10:58.509357","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train_dataset = get_dataset(TRAINING_FILENAMES, ordered=True)\nx_samp, y_samp = dataset_to_numpy_util(train_dataset, 18)\n\nx_samp_1, y_samp_1 = x_samp[:9,:,:,:], y_samp[:9]\nsamp_preds_1 = model.predict(x_samp_1, batch_size=9)\ndisplay_9_images_with_predictions(x_samp_1, samp_preds_1, y_samp_1)\n\nx_samp_2, y_samp_2 = x_samp[9:,:,:,:], y_samp[9:]\nsamp_preds_2 = model.predict(x_samp_2, batch_size=9)\ndisplay_9_images_with_predictions(x_samp_2, samp_preds_2, y_samp_2)","metadata":{"_kg_hide-input":true,"papermill":{"duration":23.038156,"end_time":"2020-10-10T22:11:21.885383","exception":false,"start_time":"2020-10-10T22:10:58.847227","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-05-29T04:52:28.483961Z","iopub.status.idle":"2024-05-29T04:52:28.484388Z","shell.execute_reply.started":"2024-05-29T04:52:28.484183Z","shell.execute_reply":"2024-05-29T04:52:28.484202Z"},"trusted":true},"execution_count":null,"outputs":[]}]}