{"cells":[{"metadata":{},"cell_type":"markdown","source":"### Version information\n\n**Version 4** -- The original public version of the notebook.\n\n**Version 5** -- Two major changes were made: \n   1. As was pointed out by [Roman](https://www.kaggle.com/nroman), the hair images were originally designed for the 256x256 size, so they need to be scaled to use with images of different sizes. In this version we introduced a scaling factor for the dimensions of the hair images, so now they should work just fine with any input sizes.\n   2. It was pointed out by [Helgi](https://www.kaggle.com/helgith) that the TensorFlow code from **Version 4** was throwing an error when used in the graph mode. In this new version of the notebook, The TensorFlow code was tweaked to make it workable in the graph mode. An example is added to illustrate how to fetch a training batch and print it to the screen.\n   \n**Version 6** -- fixed some minor typo (thank you [Franko Sikic](https://www.kaggle.com/frankosikic)!). If you want to see how this augmentation can be included in your training pipeline take a look at **Versions 20** of the following public notebook of mine: \n\n[EfficientNet BN+Tabular Features TF CV5 512x512](https://www.kaggle.com/graf10a/efficientnet-bn-tabular-features-tf-cv5-512x512).\n\n**Version 9** -- modified the part illustrating how to incorporate this data augmentation into `tf.data.Dataset` API to show an example of how to deal with both images and tabular data. Please note that since I am running this notebook on CPU, I had to decrease the size of the shuffling buffer from 2048 to 512 in the definition of the `get_training_dataset` function below. \n\nAlso, the images shown in this last part might look a bit strange to you. This is because they were pre-processed with the Shades of Gray algorithm. This is totally optional, I decided to use this images just for fun. You can read more about the Shades of Gray algorithm in this discussion topic: [Shades of Gray prepossessed data](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/161719).\n\n**Version 11** -- Updated tfrecords (fixed color artifacts after Shades of Gray pre-processing).\n\n**Version 13** -- Added the batch version of [Chris Deotte's affine augmentation](https://www.kaggle.com/cdeotte/rotation-augmentation-gpu-tpu-0-96) using the approach from [this great kernel](https://www.kaggle.com/yihdarshieh/make-chris-deotte-s-data-augmentation-faster) by [Yih-Dar SHIEH](https://www.kaggle.com/yihdarshieh). We switched back to the original images (no color constant pre-processing). For more information see the following discussion topic: [Batch form of affine augmentations in Tensor Flow](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/169504).","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Motivation and acknowledgement\n\nThis notebook is based on the idea suggested by [Roman](https://www.kaggle.com/nroman) in the following discussion topic: [Advanced hair augmentation](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/159176). We first reproduce his result using the OpenCV library and illustrate it with some sample images. After that we re-write the OpenCV code in TensorFlow. The TensorFlow implementation of this technique makes it possible to use this agumentation with the `tf.data` API which is very well suited for tfrecords and TPU.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Libraries","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport gc\nimport cv2\nimport math\nimport random\nimport numpy as np\nimport pandas as pd\nfrom glob import glob\nimport tensorflow as tf\nfrom pathlib import Path\nimport matplotlib.pyplot as plt\nimport tensorflow.keras.backend as K\nfrom kaggle_datasets import KaggleDatasets\n\nprint(\"Tensorflow version \" + tf.__version__)\nAUTO = tf.data.experimental.AUTOTUNE","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Image paths","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"n_max=6     # the maximum number of hairs to augment\nim_size=512  # all images are resized to this size\n\nhair_images=glob('/kaggle/input/melanoma-hairs/*.png')\ntrain_images=glob('/kaggle/input/siim-isic-melanoma-classification/jpeg/train/*.jpg')\ntest_images=glob('/kaggle/input/siim-isic-melanoma-classification/jpeg/test/*.jpg')\n\nlen(hair_images), len(train_images), len(test_images)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Augmenting hair with OpenCV","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def hair_aug_ocv(input_img):\n    \n    img=input_img.copy()\n    # Randomly choose the number of hairs to augment (up to n_max)\n    n_hairs = random.randint(0, n_max)\n\n    # If the number of hairs is zero then do nothing\n    if not n_hairs:\n        return img, n_hairs\n\n    # The image height and width (ignore the number of color channels)\n    im_height, im_width, _ = img.shape \n\n    for _ in range(n_hairs):\n\n        # Read a random hair image\n        hair = cv2.imread(random.choice(hair_images)) \n        \n        # Rescale the hair image to the right size (256 -- original size)\n        scale=im_size/256\n        hair = cv2.resize(hair, (int(scale*hair.shape[1]), int(scale*hair.shape[0])), \n                          interpolation=cv2.INTER_AREA)       \n\n        # Flip it\n        # flipcode = 0: flip vertically\n        # flipcode > 0: flip horizontally\n        # flipcode < 0: flip vertically and horizontally    \n        hair = cv2.flip(hair, flipCode=random.choice([-1, 0, 1]))\n\n        # Rotate it\n        hair = cv2.rotate(hair, rotateCode=random.choice([cv2.ROTATE_90_CLOCKWISE,\n                                                          cv2.ROTATE_90_COUNTERCLOCKWISE,\n                                                          cv2.ROTATE_180\n                                                         ])\n                         )\n        \n        \n        # The hair image height and width (ignore the number of color channels)\n        h_height, h_width, _ = hair.shape\n\n        # The top left coord's of the region of interest (roi)  \n        # where the augmentation will be performed\n        roi_h0 = random.randint(0, im_height - h_height)\n        roi_w0 = random.randint(0, im_width - h_width)\n\n        # The region of interest\n        roi = img[roi_h0:(roi_h0 + h_height), roi_w0:(roi_w0 + h_width)]\n\n        # Convert the hair image to grayscale\n        hair2gray = cv2.cvtColor(hair, cv2.COLOR_BGR2GRAY)\n\n        # If the pixel value is smaller than the threshold (10), it is set to 0 (black), \n        # otherwise it is set to a maximum value (255, white).\n        # ret -- the list of thresholds (10 in this case)\n        # mask -- the thresholded image\n        # The original image must be a grayscale image\n        # https://docs.opencv.org/master/d7/d4d/tutorial_py_thresholding.html\n        ret, mask = cv2.threshold(hair2gray, 10, 255, cv2.THRESH_BINARY)\n\n        # Invert the mask\n        mask_inv = cv2.bitwise_not(mask)\n\n        # Bitwise AND won't be performed where mask=0\n        img_bg = cv2.bitwise_and(roi, roi, mask=mask_inv)\n        hair_fg = cv2.bitwise_and(hair, hair, mask=mask)\n        # Fixing colors\n        hair_fg = cv2.cvtColor(hair_fg, cv2.COLOR_BGR2RGB)\n        # Overlapping the image with the hair in the region of interest\n        dst = cv2.add(img_bg, hair_fg)\n        # Inserting the result in the original image\n        img[roi_h0:roi_h0 + h_height, roi_w0:roi_w0 + h_width] = dst\n        \n    return img, n_hairs","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Examples of hair augmentation with OpenCV","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def aug_examples(paths):\n\n    for img_path in paths:\n        # Read the image\n        img=cv2.imread(img_path)\n        # Fixing colors\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        # Resize to the desired size\n        img = cv2.resize(img , (im_size, im_size), interpolation = cv2.INTER_AREA )\n        # Creating an augmented image\n        img_aug, n_hairs = hair_aug_ocv(img)\n        \n        _, (ax1,ax2) = plt.subplots(1, 2)\n        \n        im_name=img_path.split('/')[-1].split('.')[0]    \n        ax1.set_title(f\"{im_name}\")            \n        ax2.set_title(f\"{im_name} with {n_hairs} {'hair' if n_hairs==1 else 'hairs'}\")\n        \n        ax1.imshow(img)\n        ax2.imshow(img_aug)\n        \n        plt.tight_layout()\n        plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"aug_examples(train_images[6:9])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"aug_examples(test_images[6:9])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Augmenting hair with TensorFlow","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\n# Select a sample image\nfname=test_images[7]\n# Read and decode the image\nbits = tf.io.read_file(fname)\n# dct_method='INTEGER_ACCURATE' produces the same result as OpenCV\nimg0 = tf.image.decode_jpeg(bits, channels=3, dct_method='INTEGER_ACCURATE')\n\nplt.imshow(img0.numpy() / 255)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def resize_and_crop_image(input_img):\n    \n    img=tf.identity(input_img)\n    # Resize and crop using \"fill\" algorithm:\n    # always make sure the resulting image\n    # is cut out from the source image so that\n    # it fills the TARGET_SIZE entirely with no\n    # black bars and a preserved aspect ratio.\n    w = tf.shape(img)[0] \n    h = tf.shape(img)[1]\n    tw = im_size\n    th = im_size\n    resize_crit = (w * th) / (h * tw)\n    img = tf.cond(resize_crit < 1,\n                  # if true\n                  lambda: tf.image.resize(img, [w*tw/w, h*tw/w],\n                                          #method='lanczos3',\n                                          #antialias=True\n                                         ),\n                  \n                  # if false\n                  lambda: tf.image.resize(img, [w*th/h, h*th/h],\n                                          #method='lanczos3',\n                                          #antialias=True\n                                         )\n                 )\n    \n    nw = tf.shape(img)[0]\n    nh = tf.shape(img)[1]\n    img = tf.image.crop_to_bounding_box(img,\n                                        (nw - tw) // 2,\n                                        (nh - th) // 2,\n                                        tw, th\n                                       )\n    \n    return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"img_cropped=resize_and_crop_image(input_img=img0)\n\n# Divide by 255 to bring it in the 0-1 range. plt.imshow()  \n# expects either 0 to 1 floats or 0 to 255 integers.\nplt.imshow(img_cropped.numpy()/255)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"hair_images_tf=tf.convert_to_tensor(hair_images)\nscale=tf.cast(im_size/256, dtype=tf.int32)\ntf.random.set_seed(42)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def hair_aug_tf(input_img):\n    # Copy the input image, so it won't be changed\n    img=tf.identity(input_img) \n    # Randomly choose the number of hairs to augment (up to n_max)\n    n_hairs = tf.random.uniform(shape=[], maxval=tf.constant(n_max)+1, \n                                dtype=tf.int32)\n\n    im_height = tf.shape(img)[0]\n    im_width = tf.shape(img)[1]\n    \n    if n_hairs == 0:\n        return img, n_hairs\n\n    for _ in tf.range(n_hairs):\n\n        # Read a random hair image\n        i=tf.random.uniform(shape=[], maxval=tf.shape(hair_images_tf)[0], \n                            dtype=tf.int32)\n        fname=hair_images_tf[i]\n\n        bits = tf.io.read_file(fname)\n        hair = tf.image.decode_jpeg(bits)\n        \n        # Rescale the hair image to the right size (256 -- original size)\n        new_width=scale*tf.shape(hair)[1]\n        new_height=scale*tf.shape(hair)[0]\n        hair = tf.image.resize(hair, [new_height, new_width])\n\n        \n        # Random flips of the hair image\n        hair = tf.image.random_flip_left_right(hair)\n        hair = tf.image.random_flip_up_down(hair)\n        # Random number of 90 degree rotations\n        n_rot=tf.random.uniform(shape=[], maxval=4,\n                                dtype=tf.int32)\n        hair = tf.image.rot90(hair, k=n_rot)\n\n        h_height = tf.shape(hair)[0]\n        h_width = tf.shape(hair)[1]\n\n        roi_h0 = tf.random.uniform(shape=[], maxval=im_height - h_height + 1, \n                                    dtype=tf.int32)\n        roi_w0 = tf.random.uniform(shape=[], maxval=im_width - h_width + 1, \n                                    dtype=tf.int32)\n\n\n        roi = img[roi_h0:(roi_h0 + h_height), roi_w0:(roi_w0 + h_width)]  \n\n        # Convert the hair image to grayscale \n        # (slice to remove the trainsparency channel)\n        hair2gray = tf.image.rgb_to_grayscale(hair[:, :, :3])\n\n        mask=hair2gray>10\n\n        img_bg = tf.multiply(roi, tf.cast(tf.image.grayscale_to_rgb(~mask),\n                                          dtype=tf.float32))\n        hair_fg = tf.multiply(tf.cast(hair[:, :, :3], dtype=tf.int32),\n                              tf.cast(tf.image.grayscale_to_rgb(mask), dtype=tf.int32)#uint8)\n                             )\n\n        dst = tf.add(img_bg, tf.cast(hair_fg, dtype=tf.float32))\n\n        paddings = tf.stack([\n            [roi_h0, im_height-(roi_h0 + h_height)], \n            [roi_w0, im_width-(roi_w0 + h_width)],\n            [0, 0]\n        ])\n\n        # Pad dst with zeros to make it the same shape as image.\n        dst_padded=tf.pad(dst, paddings, \"CONSTANT\")\n        # Create a boolean mask with zeros at the pixels of\n        # the augmentation segment and ones everywhere else\n        mask_img=tf.pad(tf.ones_like(dst), paddings, \"CONSTANT\")\n        mask_img=~tf.cast(mask_img, dtype=tf.bool)\n        # Make a hole in the original image at the location\n        # of the augmentation segment\n        img_hole=tf.multiply(img, tf.cast(mask_img, dtype=tf.float32))\n        # Inserting the augmentation segment in place of the hole\n        img=tf.add(img_hole, dst_padded)\n        \n    return img, n_hairs","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Examples of hair augmentation with TensorFlow","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def aug_examples_tf(paths):\n\n    for img_path in paths:\n        \n        # Read and decode the image\n        bits = tf.io.read_file(img_path)\n        # dct_method='INTEGER_ACCURATE' produces the same result as OpenCV\n        img = tf.image.decode_jpeg(bits, channels=3, dct_method='INTEGER_ACCURATE')\n        \n        # Resize and crop the image\n        img=resize_and_crop_image(img)  \n        # Creating an augmented image\n        img_aug, n_hairs = hair_aug_tf(img)\n        \n        _, (ax1,ax2) = plt.subplots(1, 2)\n        \n        im_name=img_path.split('/')[-1].split('.')[0]    \n        ax1.set_title(f\"{im_name}\")            \n        ax2.set_title(f\"{im_name} with {n_hairs} {'hair' if n_hairs==1 else 'hairs'}\")\n        \n        ax1.imshow(img/255)\n        ax2.imshow(img_aug/255)\n        \n        plt.tight_layout()\n        plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"aug_examples_tf(train_images[6:9])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"aug_examples(test_images[6:9])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Using it with tf.data.Dataset API ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Detect hardware, return appropriate distribution strategy\ntry:\n    # TPU detection. No parameters necessary if TPU_NAME environment \n    # variable is set. On Kaggle this is always the case.\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()  \n    print('Running on TPU ', tpu.master())\nexcept ValueError:\n    tpu = None\n\nif tpu:\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n    # default distribution strategy in Tensorflow. Works on CPU and single GPU.\n    strategy = tf.distribute.get_strategy() \n\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"CLASSES = ['benign', 'malignant']\nIMAGE_SIZE = [512, 512]\nBATCH_SIZE = 8 * strategy.num_replicas_in_sync\nNFOLDS=5","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tab_feats=['age_scaled',\n           'sex_female', \n           'sex_male', \n           'sex_unknown', \n           'site_head/neck', \n           'site_lower extremity', \n           'site_oral/genital',\n           'site_palms/soles',\n           'site_torso',\n           'site_unknown',\n           'site_upper extremity',\n#            'height',\n#            'width',\n          ]\n\nN_TAB_FEATS=len(tab_feats)\n\nprint(f\"The number of tabular features is {N_TAB_FEATS}.\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"GCS_PATH={}\n\n# GCS_PATH['train']=KaggleDatasets().get_gcs_path('siim-tfrec-cc-512-train')\n# GCS_PATH['test']=KaggleDatasets().get_gcs_path('siim-tfrec-cc-512-test')\nGCS_PATH['train']=KaggleDatasets().get_gcs_path('siim-512x512-tfrec-q95')\nGCS_PATH['test']=KaggleDatasets().get_gcs_path('siim-512x512-tfrec-q95-test')\n# Roman's images of hairs\nGCS_PATH['hairs']=KaggleDatasets().get_gcs_path('melanoma-hairs')\n\nprint(GCS_PATH['train'])\nprint(GCS_PATH['test'])\nprint(GCS_PATH['hairs'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ALL_TRAIN=tf.io.gfile.glob(GCS_PATH['train'] + '/*.tfrec')\n\nVAL_FNAMES={}\nfor fn in range(1, NFOLDS+1):\n    VAL_FNAMES[f\"fold_{fn}\"]=[path for path in ALL_TRAIN if f\"fold_{fn}\" in path]    \n    print(\"Fold\", f'{fn}:', len(VAL_FNAMES[f'fold_{fn}']), \"elements in total.\")\n    \nTRAIN_FNAMES={f'fold_{i}': list(set(ALL_TRAIN)-set(VAL_FNAMES[f'fold_{i}']))\n              for i in range(1, NFOLDS+1)}\n\nTEST_FNAMES = tf.io.gfile.glob(GCS_PATH['test'] + '/*.tfrec')\n\n# Roman's images of hairs\nhair_images=tf.io.gfile.glob(GCS_PATH['hairs'] + '/*.png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Below are the functions that we will be using to read and process the data from the `.tfrec` files.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def decode_image(image_data):\n    image = tf.image.decode_jpeg(image_data, channels=3)\n    # convert image to floats in [0, 1] range\n    image = tf.cast(image, tf.float32) / 255.0 \n    # explicit size needed for TPU\n    image = tf.reshape(image, [*IMAGE_SIZE, 3])\n    \n    return image","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def read_labeled_tfrecord(example):\n    LABELED_TFREC_FORMAT = {\n        # tf.string means bytestring\n        # shape [] means single element\n        ################################\n        # bytestring features\n        \"image\": tf.io.FixedLenFeature([], tf.string), \n        \"image_name\": tf.io.FixedLenFeature([], tf.string),\n        \"patient_id\": tf.io.FixedLenFeature([], tf.string),\n        \"benign_malignant\": tf.io.FixedLenFeature([], tf.string),\n        # integer features\n        \"age\": tf.io.FixedLenFeature([], tf.int64),\n        \"sex_female\": tf.io.FixedLenFeature([], tf.int64),        \n        \"sex_male\": tf.io.FixedLenFeature([], tf.int64),\n        \"sex_unknown\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_head/neck\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_lower extremity\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_oral/genital\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_palms/soles\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_torso\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_unknown\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_upper extremity\": tf.io.FixedLenFeature([], tf.int64),\n        \"height\": tf.io.FixedLenFeature([], tf.int64),\n        \"width\": tf.io.FixedLenFeature([], tf.int64),\n        \"target\": tf.io.FixedLenFeature([], tf.int64), \n        # float features\n        \"age_scaled\": tf.io.FixedLenFeature([], tf.float32),\n    }\n\n    example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n    # image data\n    image = decode_image(example['image']) \n    data={}\n    # bytestring features\n    data['image_name']=image_name=tf.cast(example['image_name'], tf.string)\n    data['patient_id']=tf.cast(example['patient_id'], tf.string)\n    # integer features\n    data['age']=tf.cast(example['age'], tf.int32)\n    data['sex_female']=tf.cast(example['sex_female'], tf.int32)\n    data['sex_male']=tf.cast(example['sex_male'], tf.int32)\n    data['sex_unknown']=tf.cast(example['sex_unknown'], tf.int32)\n    data['site_head/neck']=tf.cast(example['site_head/neck'], tf.int32)\n    data['site_lower extremity']=tf.cast(example['site_lower extremity'], tf.int32)\n    data['site_oral/genital']=tf.cast(example['site_oral/genital'], tf.int32)\n    data['site_palms/soles']=tf.cast(example['site_palms/soles'], tf.int32)\n    data['site_torso']=tf.cast(example['site_torso'], tf.int32)\n    data['site_unknown']=tf.cast(example['site_unknown'], tf.int32)\n    data['site_upper extremity']=tf.cast(example['site_upper extremity'], tf.int32)\n    # float features\n    data['age_scaled']=tf.cast(example['age_scaled'], tf.float32)\n    # target (integer)\n    label=tf.cast(example['target'], tf.int32)\n     # target (string)\n    label_name=tf.cast(example['benign_malignant'], tf.string)\n\n    return image, label, data, label_name","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def read_unlabeled_tfrecord(example):\n    UNLABELED_TFREC_FORMAT = {\n        # tf.string means bytestring\n        # shape [] means single element\n        ################################\n        # bytestring features\n        \"image\": tf.io.FixedLenFeature([], tf.string), \n        \"image_name\": tf.io.FixedLenFeature([], tf.string),\n        \"patient_id\": tf.io.FixedLenFeature([], tf.string),\n        # integer features\n        \"age\": tf.io.FixedLenFeature([], tf.int64),\n        \"sex_female\": tf.io.FixedLenFeature([], tf.int64),        \n        \"sex_male\": tf.io.FixedLenFeature([], tf.int64),\n        \"sex_unknown\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_head/neck\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_lower extremity\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_oral/genital\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_palms/soles\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_torso\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_unknown\": tf.io.FixedLenFeature([], tf.int64),\n        \"site_upper extremity\": tf.io.FixedLenFeature([], tf.int64),\n        \"height\": tf.io.FixedLenFeature([], tf.int64),\n        \"width\": tf.io.FixedLenFeature([], tf.int64), \n        # float features\n        \"age_scaled\": tf.io.FixedLenFeature([], tf.float32),\n    }\n\n    example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n    # image data\n    image = decode_image(example['image']) \n    data={}\n    # bytestring features\n    data['image_name']=image_name=tf.cast(example['image_name'], tf.string)\n    data['patient_id']=tf.cast(example['patient_id'], tf.string)\n    # integer features\n    data['age']=tf.cast(example['age'], tf.int32)\n    data['sex_female']=tf.cast(example['sex_female'], tf.int32)\n    data['sex_male']=tf.cast(example['sex_male'], tf.int32)\n    data['sex_unknown']=tf.cast(example['sex_unknown'], tf.int32)\n    data['site_head/neck']=tf.cast(example['site_head/neck'], tf.int32)\n    data['site_lower extremity']=tf.cast(example['site_lower extremity'], tf.int32)\n    data['site_oral/genital']=tf.cast(example['site_oral/genital'], tf.int32)\n    data['site_palms/soles']=tf.cast(example['site_palms/soles'], tf.int32)\n    data['site_torso']=tf.cast(example['site_torso'], tf.int32)\n    data['site_unknown']=tf.cast(example['site_unknown'], tf.int32)\n    data['site_upper extremity']=tf.cast(example['site_upper extremity'], tf.int32)\n    # float features\n    data['age_scaled']=tf.cast(example['age_scaled'], tf.float32)\n\n    return image, data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_dataset(filenames, labeled=True, ordered=False):\n    # Read from TFRecords. For optimal performance, reading from multiple files \n    # at once and disregarding data order. Order does not matter since we will \n    # be shuffling the data anyway.\n\n    ignore_order = tf.data.Options()\n    if not ordered:\n        # disable order, increase speed\n        ignore_order.experimental_deterministic = False\n\n    # automatically interleaves reads from multiple files\n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO)\n    # uses data as soon as it streams in, rather than in its original order\n    dataset = dataset.with_options(ignore_order)\n    # returns a dataset of (image, label) pairs if labeled=True \n    # or (image, id) pairs if labeled=False\n    dataset = dataset.map(read_labeled_tfrecord if labeled \n                          else read_unlabeled_tfrecord, num_parallel_calls=AUTO)\n    \n    return dataset","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def setup_input(image, label, data, label_name):\n    \n    tab_data=[tf.cast(data[tfeat], dtype=tf.float32) for tfeat in tab_feats]\n    \n    tabular=tf.stack(tab_data)\n    \n    return {'inp1': image, 'inp2':  tabular}, label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def data_augment(data, label):\n    # data augmentation. Thanks to the dataset.prefetch(AUTO) statement \n    # in the next function (below), this happens essentially for free on TPU. \n    # Data pipeline code is executed on the \"CPU\" part\n    # of the TPU while the TPU itself is computing gradients.\n    image = tf.image.random_flip_left_right(data['inp1'])\n    image = tf.image.random_flip_up_down(image)\n    \n    return {'inp1': image, 'inp2':  data['inp2']}, label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"hair_images_tf=tf.convert_to_tensor(hair_images)\nscale=tf.cast(IMAGE_SIZE[0]/256, dtype=tf.int32)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def hair_aug(data, label):\n    # Copy the input image, so it won't be changed\n    img=tf.identity(data['inp1']) \n    # Randomly choose the number of hairs to augment (up to n_max)\n    n_hairs = tf.random.uniform(shape=[], maxval=tf.constant(n_max)+1, \n                                dtype=tf.int32)\n    \n    im_height=tf.shape(img)[1]\n    im_width=tf.shape(img)[0]\n    \n    if n_hairs == 0:\n        return data, label\n\n    for _ in tf.range(n_hairs):\n\n        # Read a random hair image\n        i=tf.random.uniform(shape=[], maxval=tf.shape(hair_images_tf)[0], \n                            dtype=tf.int32)\n        fname=hair_images_tf[i]\n\n        bits = tf.io.read_file(fname)\n        hair = tf.image.decode_jpeg(bits)\n        \n        # Rescale the hair image to the right size (256 -- original size)\n        new_width=scale*tf.shape(hair)[1]\n        new_height=scale*tf.shape(hair)[0]\n        hair = tf.image.resize(hair, [new_height, new_width])\n\n        \n        # Random flips of the hair image\n        hair = tf.image.random_flip_left_right(hair)\n        hair = tf.image.random_flip_up_down(hair)\n        # Random number of 90 degree rotations\n        n_rot=tf.random.uniform(shape=[], maxval=4,\n                                dtype=tf.int32)\n        hair = tf.image.rot90(hair, k=n_rot)\n        \n        h_height=tf.shape(hair)[0]\n        h_width=tf.shape(hair)[1]\n        \n        roi_h0 = tf.random.uniform(shape=[], maxval=im_height - h_height + 1, \n                                    dtype=tf.int32)\n        roi_w0 = tf.random.uniform(shape=[], maxval=im_width - h_width + 1, \n                                    dtype=tf.int32)\n\n\n        roi = img[roi_h0:(roi_h0 + h_height), roi_w0:(roi_w0 + h_width)]  \n\n        # Convert the hair image to grayscale \n        # (slice to remove the trainsparency channel)\n        hair2gray = tf.image.rgb_to_grayscale(hair[:, :, :3])\n\n        mask=hair2gray>10\n\n        img_bg = tf.multiply(roi, tf.cast(tf.image.grayscale_to_rgb(~mask),\n                                          dtype=tf.float32))\n        hair_fg = tf.multiply(tf.cast(hair[:, :, :3], dtype=tf.int32),\n                              tf.cast(tf.image.grayscale_to_rgb(mask), \n                                      dtype=tf.int32\n                                      )\n                             )\n\n        dst = tf.add(img_bg, tf.cast(hair_fg, dtype=tf.float32)/255)\n\n        paddings = tf.stack([\n            [roi_h0, im_height-(roi_h0 + h_height)], \n            [roi_w0, im_width-(roi_w0 + h_width)],\n            [0, 0]\n        ])\n\n        # Pad dst with zeros to make it the same shape as image.\n        dst_padded=tf.pad(dst, paddings, \"CONSTANT\")\n        # Create a boolean mask with zeros at the pixels of\n        # the augmentation segment and ones everywhere else\n        mask_img=tf.pad(tf.ones_like(dst), paddings, \"CONSTANT\")\n        mask_img=~tf.cast(mask_img, dtype=tf.bool)\n        # Make a hole in the original image at the location\n        # of the augmentation segment\n        img_hole=tf.multiply(img, tf.cast(mask_img, dtype=tf.float32))\n        # Inserting the augmentation segment in place of the hole\n        img=tf.add(img_hole, dst_padded)\n        \n    return {'inp1': img, 'inp2':  data['inp2']}, label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_training_dataset(dataset):\n    # horizontal and vertical random flips\n    dataset = dataset.map(data_augment, num_parallel_calls=AUTO)\n    # advanced hair augmentation\n    dataset = dataset.map(hair_aug, num_parallel_calls=AUTO)\n    # the training dataset must repeat for several epochs\n    dataset = dataset.repeat()\n    dataset = dataset.shuffle(512)\n    dataset = dataset.batch(BATCH_SIZE)\n    # prefetch next batch while training (autotune prefetch buffer size)\n    dataset = dataset.prefetch(AUTO)\n    \n    return dataset","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualization utilities","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# numpy and matplotlib defaults\nnp.set_printoptions(threshold=15, linewidth=80)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def batch_to_numpy_images_and_labels(databatch, ds='train'):\n    if ds=='train':\n        data, labels = databatch\n        numpy_images = data['inp1'].numpy()\n        numpy_labels = labels.numpy()\n    else:\n        data = databatch\n        numpy_images = data['inp1'].numpy()\n        numpy_labels = [None for _ in enumerate(numpy_images)]\n\n    # If no labels, only image IDs, return None for labels (this is the case for test data)\n    return numpy_images, numpy_labels","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def title_from_label_and_target(label, correct_label):\n    if correct_label is None:\n        return CLASSES[label], True\n    correct = (label == correct_label)\n    return \"{} [{}{}{}]\".format(CLASSES[label], 'OK' if correct else 'NO', u\"\\u2192\" \n                                if not correct else '', \n                                CLASSES[correct_label] if not correct else ''), correct","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def display_one_image(image, title, subplot, red=False, titlesize=16):\n    plt.subplot(*subplot)\n    plt.axis('off')\n    plt.imshow(image)\n    if len(title) > 0:\n        plt.title(title, fontsize=int(titlesize) if not red else int(titlesize/1.2), \n                  color='red' if red else 'black', fontdict={'verticalalignment':'center'}, \n                  pad=int(titlesize/1.5)\n                 )\n    return (subplot[0], subplot[1], subplot[2]+1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def display_batch_of_images(databatch, predictions=None, ds='train'):\n    \"\"\"This will work with:\n    display_batch_of_images(images)\n    display_batch_of_images(images, predictions)\n    display_batch_of_images((images, labels))\n    display_batch_of_images((images, labels), predictions)\n    \"\"\"\n    # data\n    images, labels = batch_to_numpy_images_and_labels(databatch, ds=ds)\n    if labels is None:\n        labels = [None for _ in enumerate(images)]\n        \n    # auto-squaring: this will drop data that does  \n    # not fit into square or square-ish rectangle\n    rows = int(math.sqrt(len(images)))\n    cols = len(images)//rows\n        \n    # size and spacing\n    FIGSIZE = 13.0\n    SPACING = 0.1\n    subplot=(rows,cols,1)\n    if rows < cols:\n        plt.figure(figsize=(FIGSIZE,FIGSIZE/cols*rows))\n    else:\n        plt.figure(figsize=(FIGSIZE/rows*cols,FIGSIZE))\n    \n    # display\n    for i, (image, label) in enumerate(zip(images[:rows*cols], labels[:rows*cols])):\n        title = '' if label is None else CLASSES[label]\n        correct = True\n        if predictions is not None:\n            title, correct = title_from_label_and_target(predictions[i], label)\n        # magic formula tested to work from 1x1 to 10x10 images\n        dynamic_titlesize = FIGSIZE*SPACING/max(rows,cols)*40+3\n        subplot = display_one_image(image, title, subplot, \n                                     not correct, titlesize=dynamic_titlesize)\n    \n    #layout\n    plt.tight_layout()\n    if label is None and predictions is None:\n        plt.subplots_adjust(wspace=0, hspace=0)\n    else:\n        plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Dataset visualizations","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# Peek at training data\n\ntraining_dataset = load_dataset(TRAIN_FNAMES['fold_2'])\ntraining_dataset = training_dataset.map(setup_input, num_parallel_calls=AUTO)   \ntraining_dataset = get_training_dataset(training_dataset)\ntraining_dataset = training_dataset.unbatch().batch(20)\ntrain_batch = iter(training_dataset)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\n# run this cell again for next set of images\ndisplay_batch_of_images(next(train_batch))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Batch form of Chris Deotte's augmentation utilizing affine transformations\n\n(We just copy the following code from [Yih-Dar SHIEH's notebook](https://www.kaggle.com/yihdarshieh/make-chris-deotte-s-data-augmentation-faster). Note that the shear matrix looks different from the standard form as defined in [this Wikepedia article](https://en.wikipedia.org/wiki/Affine_transformation)).","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"ROT_ = 180.0\nSHR_ = 2.0\nHZOOM_ = 8.0\nWZOOM_ = 8.0\nHSHIFT_ = 8.0\nWSHIFT_ = 8.0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_batch_transformatioin_matrix(rotation, shear, \n                                     height_zoom, width_zoom, \n                                     height_shift, width_shift):\n  \n    \"\"\"Returns a tf.Tensor of shape (batch_size, 3, 3) with each element along the 1st axis being\n       an image transformation matrix (which transforms indicies).\n\n    Args:\n        rotation: 1-D Tensor with shape [batch_size].\n        shear: 1-D Tensor with shape [batch_size].\n        height_zoom: 1-D Tensor with shape [batch_size].\n        width_zoom: 1-D Tensor with shape [batch_size].\n        height_shift: 1-D Tensor with shape [batch_size].\n        width_shift: 1-D Tensor with shape [batch_size].\n        \n    Returns:\n        A 3-D Tensor with shape [batch_size, 3, 3].\n    \"\"\"    \n\n    # A trick to get batch_size\n    batch_size = tf.cast(tf.reduce_sum(tf.ones_like(rotation)), tf.int64)    \n    \n    # CONVERT DEGREES TO RADIANS\n    rotation = tf.constant(math.pi) * rotation / 180.0\n    shear = tf.constant(math.pi) * shear / 180.0\n\n    # shape = (batch_size,)\n    one = tf.ones_like(rotation, dtype=tf.float32)\n    zero = tf.zeros_like(rotation, dtype=tf.float32)\n    \n    # ROTATION MATRIX\n    c1 = tf.math.cos(rotation) # shape = (batch_size,)\n    s1 = tf.math.sin(rotation) # shape = (batch_size,)\n\n    # Intermediate matrix for rotation, shape = (9, batch_size) \n    rotation_matrix_temp = tf.stack([c1, s1, zero, -s1, c1, zero, zero, zero, one], axis=0)\n    # shape = (batch_size, 9)\n    rotation_matrix_temp = tf.transpose(rotation_matrix_temp)\n    # Fianl rotation matrix, shape = (batch_size, 3, 3)\n    rotation_matrix = tf.reshape(rotation_matrix_temp, shape=(batch_size, 3, 3))\n        \n    # SHEAR MATRIX\n    c2 = tf.math.cos(shear) # shape = (batch_size,)\n    s2 = tf.math.sin(shear) # shape = (batch_size,)\n    \n    # Intermediate matrix for shear, shape = (9, batch_size) \n    shear_matrix_temp = tf.stack([one, s2, zero, zero, c2, zero, zero, zero, one], axis=0)\n    # shape = (batch_size, 9)\n    shear_matrix_temp = tf.transpose(shear_matrix_temp)\n    # Fianl shear matrix, shape = (batch_size, 3, 3)\n    shear_matrix = tf.reshape(shear_matrix_temp, shape=(batch_size, 3, 3))    \n    \n    # ZOOM MATRIX\n    \n    # Intermediate matrix for zoom, shape = (9, batch_size) \n    zoom_matrix_temp = tf.stack([one / height_zoom, zero, zero, zero, one / width_zoom, zero, zero, zero, one], axis=0)\n    # shape = (batch_size, 9)\n    zoom_matrix_temp = tf.transpose(zoom_matrix_temp)\n    # Fianl zoom matrix, shape = (batch_size, 3, 3)\n    zoom_matrix = tf.reshape(zoom_matrix_temp, shape=(batch_size, 3, 3))\n    \n    # SHIFT MATRIX\n    \n    # Intermediate matrix for shift, shape = (9, batch_size) \n    shift_matrix_temp = tf.stack([one, zero, height_shift, zero, one, width_shift, zero, zero, one], axis=0)\n    # shape = (batch_size, 9)\n    shift_matrix_temp = tf.transpose(shift_matrix_temp)\n    # Fianl shift matrix, shape = (batch_size, 3, 3)\n    shift_matrix = tf.reshape(shift_matrix_temp, shape=(batch_size, 3, 3))    \n        \n    return tf.linalg.matmul(tf.linalg.matmul(rotation_matrix, shear_matrix), tf.linalg.matmul(zoom_matrix, shift_matrix))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def affine_aug(data, label):\n    \"\"\"Returns a tf.Tensor of the same shape as `images`, represented a batch of randomly transformed images.\n\n    Args:\n        images: 4-D Tensor with shape (batch_size, width, hight, depth).\n            Currently, `depth` can only be 3.\n        \n    Returns:\n        A 4-D Tensor with the same shape as `images`.\n    \"\"\" \n    images=data['inp1']\n\n    # input `images`: a batch of images [batch_size, dim, dim, 3]\n    # output: images randomly rotated, sheared, zoomed, and shifted\n    DIM = images.shape[1]\n    XDIM = DIM % 2  # fix for size 331\n    \n    # A trick to get batch_size\n    batch_size = tf.cast(tf.reduce_sum(tf.ones_like(images)) / (images.shape[1] * images.shape[2] * images.shape[3]), tf.int64)\n    \n    rot = ROT_ * tf.random.normal([batch_size], dtype='float32')\n    shr = SHR_ * tf.random.normal([batch_size], dtype='float32') \n    h_zoom = 1.0 #+ tf.random.normal([batch_size], dtype='float32') / HZOOM_\n    w_zoom = 1.0 #+ tf.random.normal([batch_size], dtype='float32') / WZOOM_\n    h_shift = HSHIFT_ * tf.random.normal([batch_size], dtype='float32') \n    w_shift = WSHIFT_ * tf.random.normal([batch_size], dtype='float32') \n  \n    # GET TRANSFORMATION MATRIX\n    # shape = (batch_size, 3, 3)\n    m = get_batch_transformatioin_matrix(rot, shr, h_zoom, w_zoom, h_shift, w_shift) \n\n    # LIST DESTINATION PIXEL INDICES\n    x = tf.repeat(tf.range(DIM // 2, -DIM // 2, -1), DIM)  # shape = (DIM * DIM,)\n    y = tf.tile(tf.range(-DIM // 2, DIM // 2), [DIM])  # shape = (DIM * DIM,)\n    z = tf.ones([DIM * DIM], dtype='int32')  # shape = (DIM * DIM,)\n    idx = tf.stack([x, y, z])  # shape = (3, DIM * DIM)\n    \n    # ROTATE DESTINATION PIXELS ONTO ORIGIN PIXELS\n    idx2 = tf.linalg.matmul(m, tf.cast(idx, dtype='float32'))  # shape = (batch_size, 3, DIM ** 2)\n    idx2 = K.cast(idx2, dtype='int32')  # shape = (batch_size, 3, DIM ** 2)\n    idx2 = K.clip(idx2, -DIM // 2 + XDIM + 1, DIM // 2)  # shape = (batch_size, 3, DIM ** 2)\n    \n    # FIND ORIGIN PIXEL VALUES\n    # shape = (batch_size, 2, DIM ** 2)\n    idx3 = tf.stack([DIM // 2 - idx2[:, 0, ], DIM // 2 - 1 + idx2[:, 1, ]], axis=1)  \n    \n    # shape = (batch_size, DIM ** 2, 3)\n    d = tf.gather_nd(images, tf.transpose(idx3, perm=[0, 2, 1]), batch_dims=1)\n        \n    # shape = (batch_size, DIM, DIM, 3)\n    new_images = tf.reshape(d, (batch_size, DIM, DIM, 3))\n\n    return {'inp1': new_images, 'inp2':  data['inp2']}, label","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here is the batch version of the augmentation that flips the image vertically and horizontaly. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def flip_aug(data, label):\n    # data augmentation. Thanks to the dataset.prefetch(AUTO) statement \n    # in the next function (below), this happens essentially for free on TPU. \n    # Data pipeline code is executed on the \"CPU\" part\n    # of the TPU while the TPU itself is computing gradients.\n    images = tf.image.random_flip_left_right(data['inp1'])\n    images = tf.image.random_flip_up_down(images)\n    \n    return {'inp1': images, 'inp2':  data['inp2']}, label","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Redefine `get_training_dataset` function to include the additional augmentations.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_training_dataset(dataset, do_flip_aug=True, do_affine_aug=True, do_hair_aug=True):\n\n    if do_hair_aug:\n      # advanced hair augmentation\n      dataset = dataset.map(hair_aug, num_parallel_calls=AUTO)\n  \n    # the training dataset must repeat for several epochs    \n    dataset = dataset.repeat()\n    dataset = dataset.shuffle(2048)\n    dataset = dataset.batch(BATCH_SIZE)\n\n    if do_flip_aug:\n      # horizontal and vertical random flips\n      dataset = dataset.map(flip_aug, num_parallel_calls=AUTO)\n\n    if do_affine_aug:\n      # affine transformations\n      dataset = dataset.map(affine_aug, num_parallel_calls=AUTO)\n\n    # prefetch next batch while training (autotune prefetch buffer size)\n    dataset = dataset.prefetch(AUTO)\n  \n    return dataset","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Peek at training data\n\ntraining_dataset = load_dataset(TRAIN_FNAMES['fold_4'])\ntraining_dataset = training_dataset.map(setup_input, num_parallel_calls=AUTO)   \ntraining_dataset = get_training_dataset(training_dataset)\ntraining_dataset = training_dataset.unbatch().batch(20)\ntrain_batch = iter(training_dataset)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\n# run this cell again for next set of images\ndisplay_batch_of_images(next(train_batch))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Below are examples of a training image randomly augmented 12 different times.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n\nrow = 3; col = 4;\n\ntraining_dataset = load_dataset(TRAIN_FNAMES['fold_1'])\ntraining_dataset = training_dataset.map(setup_input, num_parallel_calls=AUTO)   \n\nall_elements = get_training_dataset(training_dataset,\n                                    do_flip_aug=False, \n                                    do_affine_aug=False, \n                                    do_hair_aug=False).unbatch()\n\none_element = tf.data.Dataset.from_tensors( next(iter(all_elements)) )\naugmented_element = one_element.repeat().map(hair_aug).batch(row*col).map(flip_aug).map(affine_aug)\nfor (data, label) in augmented_element:\n    plt.figure(figsize=(15,int(15*row/col)))\n    for j in range(row*col):\n        plt.subplot(row,col,j+1)\n        plt.axis('off')\n        plt.imshow(data['inp1'][j,])\n    plt.show()\n    break","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}