{"cells":[{"metadata":{},"cell_type":"markdown","source":"This kernel applies the newly proposed [AugMix](https://arxiv.org/pdf/1912.02781v1.pdf) augmentation to the `128x128` training images. Thanks to Iafoss for the dataset.\n\n## AUGMIX: A SIMPLE DATA PROCESSING METHOD TO IMPROVE ROBUSTNESS AND UNCERTAINTY\n"},{"metadata":{},"cell_type":"markdown","source":"> Modern deep neural networks can achieve high accuracy when the training distribution and test distribution are identically distributed, but this assumption is frequently\nviolated in practice. When the train and test distributions are mismatched, accuracy can plummet. Currently there are few techniques that improve robustness to\nunforeseen data shifts encountered during deployment. In this work, we propose a\ntechnique to improve the robustness and uncertainty estimates of image classifiers.\nWe propose AUGMIX, a data processing technique that is simple to implement,\nadds limited computational overhead, and helps models withstand unforeseen corruptions. AUGMIX significantly improves robustness and uncertainty measures on\nchallenging image classification benchmarks, closing the gap between previous\nmethods and the best possible performance in some cases by more than half.\n\n![](https://storage.googleapis.com/groundai-web-prod/media/users/user_135639/project_400799/images/x1.png)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport cv2\n\nimport numpy as np\nfrom PIL import Image\nfrom PIL import ImageOps\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"TRAIN = '../input/grapheme-imgs-128x128/'","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"IMAGE_SIZE = 128\n\n\ndef int_parameter(level, maxval):\n  \"\"\"Helper function to scale `val` between 0 and maxval .\n  Args:\n    level: Level of the operation that will be between [0, `PARAMETER_MAX`].\n    maxval: Maximum value that the operation can have. This will be scaled to\n      level/PARAMETER_MAX.\n  Returns:\n    An int that results from scaling `maxval` according to `level`.\n  \"\"\"\n  return int(level * maxval / 10)\n\n\ndef float_parameter(level, maxval):\n  \"\"\"Helper function to scale `val` between 0 and maxval.\n  Args:\n    level: Level of the operation that will be between [0, `PARAMETER_MAX`].\n    maxval: Maximum value that the operation can have. This will be scaled to\n      level/PARAMETER_MAX.\n  Returns:\n    A float that results from scaling `maxval` according to `level`.\n  \"\"\"\n  return float(level) * maxval / 10.\n\n\ndef sample_level(n):\n  return np.random.uniform(low=0.1, high=n)\n\n\ndef autocontrast(pil_img, _):\n  return ImageOps.autocontrast(pil_img)\n\n\ndef equalize(pil_img, _):\n  return ImageOps.equalize(pil_img)\n\n\ndef posterize(pil_img, level):\n  level = int_parameter(sample_level(level), 4)\n  return ImageOps.posterize(pil_img, 4 - level)\n\n\ndef rotate(pil_img, level):\n  degrees = int_parameter(sample_level(level), 30)\n  if np.random.uniform() > 0.5:\n    degrees = -degrees\n  return pil_img.rotate(degrees, resample=Image.BILINEAR)\n\n\ndef solarize(pil_img, level):\n  level = int_parameter(sample_level(level), 256)\n  return ImageOps.solarize(pil_img, 256 - level)\n\n\ndef shear_x(pil_img, level):\n  level = float_parameter(sample_level(level), 0.3)\n  if np.random.uniform() > 0.5:\n    level = -level\n  return pil_img.transform((IMAGE_SIZE, IMAGE_SIZE),\n                           Image.AFFINE, (1, level, 0, 0, 1, 0),\n                           resample=Image.BILINEAR)\n\n\ndef shear_y(pil_img, level):\n  level = float_parameter(sample_level(level), 0.3)\n  if np.random.uniform() > 0.5:\n    level = -level\n  return pil_img.transform((IMAGE_SIZE, IMAGE_SIZE),\n                           Image.AFFINE, (1, 0, 0, level, 1, 0),\n                           resample=Image.BILINEAR)\n\n\ndef translate_x(pil_img, level):\n  level = int_parameter(sample_level(level), IMAGE_SIZE / 3)\n  if np.random.random() > 0.5:\n    level = -level\n  return pil_img.transform((IMAGE_SIZE, IMAGE_SIZE),\n                           Image.AFFINE, (1, 0, level, 0, 1, 0),\n                           resample=Image.BILINEAR)\n\n\ndef translate_y(pil_img, level):\n  level = int_parameter(sample_level(level), IMAGE_SIZE / 3)\n  if np.random.random() > 0.5:\n    level = -level\n  return pil_img.transform((IMAGE_SIZE, IMAGE_SIZE),\n                           Image.AFFINE, (1, 0, 0, 0, 1, level),\n                           resample=Image.BILINEAR)\n\n\naugmentations = [\n    autocontrast, equalize, posterize, rotate, solarize, shear_x, shear_y,\n    translate_x, translate_y\n]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# taken from https://www.kaggle.com/iafoss/image-preprocessing-128x128\nMEAN = [ 0.06922848809290576,  0.06922848809290576,  0.06922848809290576]\nSTD = [ 0.20515700083327537,  0.20515700083327537,  0.20515700083327537]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def normalize(image):\n  \"\"\"Normalize input image channel-wise to zero mean and unit variance.\"\"\"\n  image = image.transpose(2, 0, 1)  # Switch to channel-first\n  mean, std = np.array(MEAN), np.array(STD)\n  image = (image - mean[:, None, None]) / std[:, None, None]\n  return image.transpose(1, 2, 0)\n\n\ndef apply_op(image, op, severity):\n  image = np.clip(image * 255., 0, 255).astype(np.uint8)\n  pil_img = Image.fromarray(image)  # Convert to PIL.Image\n  pil_img = op(pil_img, severity)\n  return np.asarray(pil_img) / 255.\n\n\ndef augment_and_mix(image, severity=1, width=3, depth=1, alpha=1.):\n  \"\"\"Perform AugMix augmentations and compute mixture.\n  Args:\n    image: Raw input image as float32 np.ndarray of shape (h, w, c)\n    severity: Severity of underlying augmentation operators (between 1 to 10).\n    width: Width of augmentation chain\n    depth: Depth of augmentation chain. -1 enables stochastic depth uniformly\n      from [1, 3]\n    alpha: Probability coefficient for Beta and Dirichlet distributions.\n  Returns:\n    mixed: Augmented and mixed image.\n  \"\"\"\n  ws = np.float32(\n      np.random.dirichlet([alpha] * width))\n  m = np.float32(np.random.beta(alpha, alpha))\n\n  mix = np.zeros_like(image)\n  for i in range(width):\n    image_aug = image.copy()\n    depth = depth if depth > 0 else np.random.randint(1, 4)\n    for _ in range(depth):\n      op = np.random.choice(augmentations)\n      image_aug = apply_op(image_aug, op, severity)\n    # Preprocessing commutes since all coefficients are convex\n    #mix += ws[i] * normalize(image_aug)\n    mix = np.add(mix, ws[i] * normalize(image_aug), out=mix, casting=\"unsafe\")\n\n\n  mixed = (1 - m) * normalize(image) + m * mix\n  return mixed\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"n_imgs = 10\nimg_filenames = os.listdir(TRAIN)[:n_imgs]\nimg_filenames[:3]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def visualize(original_image,aug_image):\n    fontsize = 18\n    \n    f, ax = plt.subplots(1, 2, figsize=(8, 8))\n\n    ax[0].imshow(original_image, cmap='gray')\n    ax[0].set_title('Original image', fontsize=fontsize)\n    ax[1].imshow(aug_image,cmap='gray')\n    ax[1].set_title('Augmented image', fontsize=fontsize)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for file_name in img_filenames:\n    img = cv2.imread(TRAIN +file_name)\n    img_aug = augment_and_mix(img)\n    visualize(img, img_aug)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}