{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### Why this notebook\n- Most notebooks simply print the (random) outputs after augmentation and I could not find any interactive data augmentation notebooks to experiment how continuous changes in augmentation parameters affect the output images.\n- An interactive slider or \"simulation\" helps us visualise the augmentation properly and also decide which augmentation is relevant for our use-case instead of using them all by default just because they \"work\". This could also give us an idea about what parameter range we want to use for the given augmentation.\n- This notebook also allows other users to experiment with different augmentations as it was implemented to be easily extended and customisable.\n\n## Libraries\n`albumentations.augmentations.functional` and `kornia` methods are used in this notebook. Ideally this code could work with other kind of functions but this has not been tested.\n\n## Observations if you use this notebook\n- <span style=\"color:red\">This notebook might take some time to load up in view mode as it has many images so give it a few secs ;)</span>\n- Make sure that the range values are valid for each transformation, for example: \n    - For `kornia.enhance.posterize` the argument `bits` must be an integer between 1 and 8\n- __This notebook runs on a CPU instance, therefore some transformations will run slower than when using a GPU. To obtain a speed up from using a GPU one must first adapt the code, for example by copying the tensor images to the desired device first__.\n- When using this notebook focus on the bounding boxes (or targets) as they are what matter when we augmentate the image. One can zoom into these and then move the slider to see how each augmentation affect them. This will give you a hint of how the targets (COTS in this competition) will be changed according to each parameter. This way one can judge if our model will learn new useful information or not.","metadata":{}},{"cell_type":"code","source":"from typing import Callable, Dict, Tuple, Sequence, Optional, Union\nimport collections\n\nimport cv2\nimport random\nfrom pathlib import Path\nimport numpy as np\nimport xarray as xr\nimport pandas as pd\n\n# https://www.kaggle.com/product-feedback/138599#787336\nfrom plotly.offline import plot, iplot, init_notebook_mode\ninit_notebook_mode(connected=True)\nimport plotly.express as px\n\nimport torch\nimport kornia as kor\nimport albumentations.augmentations.functional as alb_aug\n\n# IMPORTANT: Fix seed for reproducibility\nseed = 37\nrandom.seed(seed)\nnp.random.seed(seed)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-output":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:14:51.731679Z","iopub.execute_input":"2022-02-05T22:14:51.732071Z","iopub.status.idle":"2022-02-05T22:14:57.69749Z","shell.execute_reply.started":"2022-02-05T22:14:51.731978Z","shell.execute_reply":"2022-02-05T22:14:57.696537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def fetch_images_and_boxes(\n        csv_file: str,\n        num_imgs: int,\n        min_num_boxes: int,\n        rescale_size: Optional[Tuple[int, int]] = None\n) -> Tuple[Sequence[np.ndarray], Sequence[Dict[str, float]]]:\n    \"\"\"Fetch images and their resp. boxes\"\"\"\n\n    def func_num_entries(_str: str):\n        # Convert string to struct and get the number of bounding boxes in it\n        return len(eval(_str))\n\n    # Load dataframe\n    df = pd.read_csv(csv_file)\n\n    # Filter df by a minimum number of annotations/boxes\n    anns = df['annotations'].apply(func_num_entries)\n    _df = df[anns >= min_num_boxes]\n    \n    # Sample rows from df\n    subdf = _df.iloc[random.sample(range(len(_df.index)), num_imgs)]\n    \n    img_list = []\n    bbox_list = []\n    \n    for _, row in subdf.iterrows():\n        video_num = row['video_id']\n        frame_num = row['video_frame']\n        base_path = Path(\"../input/tensorflow-great-barrier-reef/train_images\")\n        img_path = str(base_path / f\"video_{video_num}\" / f\"{frame_num}.jpg\")\n        img = cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB)\n        boxes = eval(row['annotations'])\n        if rescale_size is not None:\n            nw, nh = rescale_size\n            oh, ow = img.shape[:2]\n            img = cv2.resize(img, rescale_size)\n            for box in boxes:\n                box['x'] = int(box['x'] * nw / ow)\n                box['width'] = int(box['width'] * nw / ow)\n                box['y'] = int(box['y'] * nh / oh)\n                box['height'] = int(box['height'] * nh / oh)\n        img_list.append(img)\n        bbox_list.append(boxes)\n    return img_list, bbox_list\n\n\ndef show_bboxes(\n        _img: np.ndarray,\n        _bboxes: Dict[str, float],\n        _color: Tuple[int, int, int] = (255, 0, 0)\n) -> None:\n    \"\"\"In-place plotting of boxes on the given image\"\"\"\n    for _bbox in _bboxes:\n        x1, y1 = _bbox['x'], _bbox['y']\n        x2, y2 = _bbox['x'] + _bbox['width'], _bbox['y'] + _bbox['height']\n        cv2.rectangle(_img, (int(x1), int(y1)), (int(x2), int(y2)), _color, 2)\n\n\ndef show_images_with_slider(\n        images: Sequence[np.ndarray],\n        labels: Sequence[str],\n        slider_name: str,\n        title: str,\n) -> None:\n    \"\"\"Load and show slider figure\n    :param images: images to plot in order\n    :param labels: labels to show for each image\n    :param slider_name: slider value name\n    :param title: figure name\n    \"\"\"\n    stacked = np.stack(images, axis=0)\n    xrData = xr.DataArray(\n        data=stacked,\n        dims=[slider_name, 'row', 'col', 'rgb'],\n        coords={slider_name: labels}\n    )\n    # Hide the axes\n    layout_dict = dict(yaxis_visible=False, yaxis_showticklabels=False, xaxis_visible=False, xaxis_showticklabels=False)\n    fig = px.imshow(xrData, title=title, animation_frame=slider_name, width=800, height=600).update_layout(layout_dict)\n    fig.show()\n\n\ndef load_simulation(\n        base_img: Union[torch.Tensor, np.ndarray],\n        aug_function: Callable,\n        arg_name: str,\n        min_val: Union[float, Sequence] = 0,\n        max_val: Union[float, Sequence] = 1.,\n        val_type: type = float,\n        num_steps: int = 10,\n        aug_img_arg_name: str = 'input',\n        aug_kwargs: dict = None,\n        bboxes: Optional[Dict[str, float]] = None,\n        expand: bool = False,\n) -> None:\n    \"\"\"Load and show slider figure given the augmentation function and its parameters\n    :param base_img: initial tensor image to be augmentated\n    :param aug_function: augmentation callable to use\n    :param arg_name: augmentation argument name to vary\n    :param min_val: minimum value/vector to vary\n    :param max_val: maximum value/vector to vary\n    :param val_type: type to cast variation values into (int or float)\n    :param num_steps: number of step variation between minimum and maximum value\n    :param aug_img_arg_name: image argument name for the selected `aug_function`\n    :param aug_kwargs: other augmentation key word arguments\n    :param bboxes: bounding boxes to plot on all augmented images\n    :param expand: whether to expand the base image shape to simulate batches (required for some augmentations)\n    \"\"\"\n    aug_kwargs = aug_kwargs or {}\n\n    # Set value range for the parameter to vary\n    val_range = np.linspace(min_val, max_val, num_steps + 1, dtype=val_type)\n    \n    # Convert to tuples if necessary\n    if hasattr(val_range[0], '__len__'):\n        val_range = [tuple(val_type(_v) for _v in val) for val in val_range]\n    else:\n        val_range = [val_type(val) for val in val_range]\n    \n    # Add a dimension, simulating a batch size\n    if expand:\n        base_img = base_img[None, ...]\n    \n    aug_kwargs.update({aug_img_arg_name: base_img,}) \n    \n    # Compute variations according to the previous interval values\n    x_out = [aug_function(**{arg_name: val}, **aug_kwargs) for val in val_range]\n    \n    # Convert images to numpy arrays\n    images = [\n        np.ascontiguousarray(\n            (kor.utils.tensor_to_image(im) * 255 if isinstance(im, torch.Tensor) else im)\n            , dtype=np.uint8\n        ) \n        for im in x_out]\n    \n    # Plot boxes if given\n    if bboxes is not None:\n        for i, im in enumerate(images):\n            show_bboxes(im, bboxes)\n    \n    # Format values to show on the slider\n    def _format(val):\n        if isinstance(val, (float, np.inexact)):\n            return f\"{val:.2f}\"\n        elif isinstance(val, (int, np.integer)):\n            return str(val)\n        elif isinstance(val, (collections.abc.Sequence, np.ndarray)):\n            return str(tuple(_format(subval) for subval in val))\n        else:\n            raise Exception(f\"Invalid type {type(val)}\")\n            \n    # Show slider plot\n    show_images_with_slider(images, [_format(val) for val in val_range], arg_name, aug_function.__name__)\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:14:57.699317Z","iopub.execute_input":"2022-02-05T22:14:57.699559Z","iopub.status.idle":"2022-02-05T22:14:57.734236Z","shell.execute_reply.started":"2022-02-05T22:14:57.699529Z","shell.execute_reply":"2022-02-05T22:14:57.733182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Original image\nLoad and visualize the original image","metadata":{}},{"cell_type":"code","source":"# Fetch some images and their bounding boxes\nmin_num_boxes = 3\nnum_imgs = 3\ncsv_file = \"../input/tensorflow-great-barrier-reef/train.csv\"\n\n# Rescale images to accelerate computations\nimg_list, bbox_list = fetch_images_and_boxes(csv_file, num_imgs, min_num_boxes, rescale_size=(640, 360))\n\n# Set numpy array image and boxes\nnp_img, bboxes = img_list[0], bbox_list[0]\n\n# Set tensor image\ntensor_img = kor.utils.image_to_tensor(np_img)\ntensor_img = tensor_img.float() / 255.0\n\n# Plot image with boxes\ncopy_img = np_img.copy()\nshow_bboxes(copy_img, bboxes)\nfig = px.imshow(copy_img)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:14:57.735908Z","iopub.execute_input":"2022-02-05T22:14:57.736739Z","iopub.status.idle":"2022-02-05T22:14:59.616293Z","shell.execute_reply.started":"2022-02-05T22:14:57.736703Z","shell.execute_reply":"2022-02-05T22:14:59.615209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Augmentations\nSome of these augmentation you can find in `kornia` and some others in `albumentations`. Both libraries are easily integrable with `PyTorch` code. If you want to try different augmentations just make sure you use the right range value for sampling. `load_simulation` is well documented and can be easily adapted if it is ever needed.","metadata":{}},{"cell_type":"markdown","source":"# Brightness\nIn this example, one could argue that when `brightness_factor` is larger than $0.3\\sim 0.4$ then the COTS are barely distinguishable/visible, therefore not adding new information for the model.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.adjust_brightness,\n    'arg_name': 'brightness_factor',\n    'bboxes': bboxes\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:14:59.618388Z","iopub.execute_input":"2022-02-05T22:14:59.618852Z","iopub.status.idle":"2022-02-05T22:15:00.715206Z","shell.execute_reply.started":"2022-02-05T22:14:59.618814Z","shell.execute_reply":"2022-02-05T22:15:00.713973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Contrast\nIn this one, one could argue that values smaller than $0.5/sim 0.6$ then the COTS are too dark to be detected, therefore restricting our augmentation parameter between $0.6$ and $1$.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.adjust_contrast,\n    'arg_name': 'contrast_factor',\n    'bboxes': bboxes\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:00.716614Z","iopub.execute_input":"2022-02-05T22:15:00.716862Z","iopub.status.idle":"2022-02-05T22:15:01.686157Z","shell.execute_reply.started":"2022-02-05T22:15:00.716833Z","shell.execute_reply":"2022-02-05T22:15:01.684785Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Gamma\nHere the range could be set to $0.4$ to $1$.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.adjust_gamma,\n    'arg_name': 'gamma',\n    'bboxes': bboxes\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:01.687585Z","iopub.execute_input":"2022-02-05T22:15:01.687934Z","iopub.status.idle":"2022-02-05T22:15:02.714994Z","shell.execute_reply.started":"2022-02-05T22:15:01.687886Z","shell.execute_reply":"2022-02-05T22:15:02.713813Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Saturation\nFor saturation one could argue that grey images are useful therefore using the whole shown range to augmentate the images.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.adjust_saturation,\n    'arg_name': 'saturation_factor',\n    'bboxes': bboxes,\n    'min_val': 0,\n    'max_val': 3,\n    'num_steps': 20\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:02.716142Z","iopub.execute_input":"2022-02-05T22:15:02.716355Z","iopub.status.idle":"2022-02-05T22:15:05.217686Z","shell.execute_reply.started":"2022-02-05T22:15:02.716327Z","shell.execute_reply":"2022-02-05T22:15:05.216739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Posterize\nIn this case, one could say that only `bits` values above $4$ are acceptable.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.posterize,\n    'arg_name': 'bits',\n    'bboxes': bboxes,\n    'min_val': 1,\n    'max_val': 8,\n    'num_steps': 7,\n    'val_type': int\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:05.219246Z","iopub.execute_input":"2022-02-05T22:15:05.220287Z","iopub.status.idle":"2022-02-05T22:15:05.939314Z","shell.execute_reply.started":"2022-02-05T22:15:05.220233Z","shell.execute_reply":"2022-02-05T22:15:05.938307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Sharpness\nIn this one it might be a bit difficult to decide on a range for `factor` but at least one could say that this augmentation could help the model learn new characteristics of COTS.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.sharpness,\n    'arg_name': 'factor',\n    'bboxes': bboxes,\n    'min_val': 0,\n    'max_val': 10,\n    'num_steps': 10,\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:05.941062Z","iopub.execute_input":"2022-02-05T22:15:05.941572Z","iopub.status.idle":"2022-02-05T22:15:07.129647Z","shell.execute_reply.started":"2022-02-05T22:15:05.941526Z","shell.execute_reply":"2022-02-05T22:15:07.127958Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Hue\nOne could say that values between $-0.31$ and $0.31$  simulate different water colors as it varies the color between green and blue.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.adjust_hue,\n    'arg_name': 'hue_factor',\n    'bboxes': bboxes,\n    'min_val': -np.pi,\n    'max_val': np.pi,\n    'num_steps': 20\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:07.133953Z","iopub.execute_input":"2022-02-05T22:15:07.134378Z","iopub.status.idle":"2022-02-05T22:15:09.49176Z","shell.execute_reply.started":"2022-02-05T22:15:07.134301Z","shell.execute_reply":"2022-02-05T22:15:09.488346Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Solarize\nAs shown in the picture the values shown do no bring any new useful information so one could argue that this augmentation is not useful.. or one could go deeper and restrict the parameters between $0.9$ and $1$ to further experiment in this case.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.solarize,\n    'arg_name': 'thresholds',\n    'bboxes': bboxes,\n    'min_val': 0.,\n    'max_val': 1,\n    'val_type': float,\n    'num_steps': 10,\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:09.494525Z","iopub.execute_input":"2022-02-05T22:15:09.494992Z","iopub.status.idle":"2022-02-05T22:15:10.565338Z","shell.execute_reply.started":"2022-02-05T22:15:09.494924Z","shell.execute_reply":"2022-02-05T22:15:10.562667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Equalize CLAHE\nSimilar to `Solarize` one could further investigate if values between $1$ and $3.93$ bring new information.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.enhance.equalize_clahe,\n    'arg_name': 'clip_limit',\n    'bboxes': bboxes,\n    'min_val': 1,\n    'max_val': 45,\n    'num_steps': 15,\n}\n# Somehow this blur needs odd valued-kernels such as (3,3), (5,5), etc\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2022-02-05T22:15:10.567757Z","iopub.execute_input":"2022-02-05T22:15:10.568284Z","iopub.status.idle":"2022-02-05T22:15:12.4616Z","shell.execute_reply.started":"2022-02-05T22:15:10.568211Z","shell.execute_reply":"2022-02-05T22:15:12.45929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Box blur\nHere only $(2,2)$ kernels seem to useful as larger kernels only obscure the image, nevertheless one should be careful with small targets.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.filters.box_blur,\n    'arg_name': 'kernel_size',\n    'bboxes': bboxes,\n    'min_val': (1, 1),\n    'max_val': (11, 11),\n    'val_type': int,\n    'num_steps': 10,\n    'expand': True\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:12.46376Z","iopub.execute_input":"2022-02-05T22:15:12.464235Z","iopub.status.idle":"2022-02-05T22:15:13.650792Z","shell.execute_reply.started":"2022-02-05T22:15:12.464168Z","shell.execute_reply":"2022-02-05T22:15:13.649824Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Unsharp mask\nDue to the \"noisyness\" of the coral reefs, this augmentation brings no useful information for generalisation.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'input',\n    'base_img': tensor_img,\n    'aug_function': kor.filters.unsharp_mask,\n    'arg_name': 'kernel_size',\n    'bboxes': bboxes,\n    'min_val': (1, 1),\n    'max_val': (21, 21),\n    'val_type': int,\n    'num_steps': 10,\n    'aug_kwargs': {'sigma': (3, 3)},\n    'expand': True\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:13.652113Z","iopub.execute_input":"2022-02-05T22:15:13.652391Z","iopub.status.idle":"2022-02-05T22:15:14.866789Z","shell.execute_reply.started":"2022-02-05T22:15:13.652351Z","shell.execute_reply":"2022-02-05T22:15:14.862634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Median blur\nOne could argue that blur with kernel size 3 is useful but it seems that small targets just dissapear.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'img',\n    'base_img': np_img,\n    'aug_function': alb_aug.median_blur,\n    'arg_name': 'ksize',\n    'bboxes': bboxes,\n    'min_val': 1,\n    'max_val': 15,\n    'val_type': int,\n    'num_steps': 7,\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:14.869136Z","iopub.execute_input":"2022-02-05T22:15:14.869644Z","iopub.status.idle":"2022-02-05T22:15:16.000605Z","shell.execute_reply.started":"2022-02-05T22:15:14.869571Z","shell.execute_reply":"2022-02-05T22:15:15.995702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Fancy PCA\nVisually PCA seems to be a combination of some previous augmentation so one muss decide whether to use it or just ignore since it is already represented by other augmentations.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'img',\n    'base_img': np_img,\n    'aug_function': alb_aug.fancy_pca,\n    'arg_name': 'alpha',\n    'bboxes': bboxes,\n    'min_val': 0,\n    'max_val': 1,\n    'num_steps': 10,\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:16.003062Z","iopub.execute_input":"2022-02-05T22:15:16.004117Z","iopub.status.idle":"2022-02-05T22:15:17.248473Z","shell.execute_reply.started":"2022-02-05T22:15:16.004003Z","shell.execute_reply":"2022-02-05T22:15:17.247754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Camera sensor noise (ISO)\nISO seems to be useful to simulate camera noises. In this competition the cameras might not have that problem so one should discuss further if this augmentation is useful.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'img',\n    'base_img': np_img,\n    'aug_function': alb_aug.iso_noise,\n    'arg_name': 'intensity',\n    'bboxes': bboxes,\n    'min_val': 0,\n    'max_val': 1,\n    'num_steps': 10,\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:17.249555Z","iopub.execute_input":"2022-02-05T22:15:17.249888Z","iopub.status.idle":"2022-02-05T22:15:18.617912Z","shell.execute_reply.started":"2022-02-05T22:15:17.249858Z","shell.execute_reply":"2022-02-05T22:15:18.616357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Downscale\nThis augmentation could be useful for `scale` between $0.9$ and $1$ but again one should be careful with small labels.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'img',\n    'base_img': np_img,\n    'aug_function': alb_aug.downscale,\n    'arg_name': 'scale',\n    'bboxes': bboxes,\n    'min_val': 0.1,\n    'max_val': 1,\n    'num_steps': 19,\n    'aug_kwargs': {'interpolation': cv2.INTER_NEAREST}\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:18.620328Z","iopub.execute_input":"2022-02-05T22:15:18.620812Z","iopub.status.idle":"2022-02-05T22:15:19.59798Z","shell.execute_reply.started":"2022-02-05T22:15:18.620737Z","shell.execute_reply":"2022-02-05T22:15:19.597158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Image compression\nThis one is interesting as it a special `downscale`, therefore one should treat it similarly: small object might get harder to learn.","metadata":{}},{"cell_type":"code","source":"kwargs = {\n    'aug_img_arg_name': 'img',\n    'base_img': np_img,\n    'aug_function': alb_aug.image_compression,\n    'arg_name': 'quality',\n    'bboxes': bboxes,\n    'min_val': 100,\n    'max_val': 10,\n    'val_type': int,\n    'num_steps': 9,\n    'aug_kwargs': {'image_type': '.jpg'}\n}\nload_simulation(**kwargs)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-02-05T22:15:19.599055Z","iopub.execute_input":"2022-02-05T22:15:19.599431Z","iopub.status.idle":"2022-02-05T22:15:20.582507Z","shell.execute_reply.started":"2022-02-05T22:15:19.5994Z","shell.execute_reply":"2022-02-05T22:15:20.580235Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Summary\nAs seen from the previous figures, some of these augmentations are suitable for this challenge and some are not. As already mentioned before, it is important to see how these transformations affect the targets to detect, more specifically the COTS in this competition. As most already know, good generalisation and small object detection are important factors to improve our score, in that sense, these augmentations should be used carefully. Of course, we should not completely rely on visualizations to analyze augmentations, nevertheless they always are a good start to understand them.","metadata":{}},{"cell_type":"markdown","source":"# References\n- [Kornia: differentiable computer vision library for PyTorch](https://kornia.readthedocs.io/en/latest/)\n- [Albumentations: fast and flexible image augmentations](https://albumentations.ai/)\n- For plotting [plotly](https://plotly.com/python/getting-started/)","metadata":{}}]}