{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":63056,"databundleVersionId":9094797,"sourceType":"competition"},{"sourceId":23812,"sourceType":"datasetVersion","datasetId":17810},{"sourceId":7857373,"sourceType":"datasetVersion","datasetId":4608696}],"dockerImageVersionId":30042,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# CLAHE augmentations\n\nYou can look into my other notebooks to get more detailed augmentation insights.\nlist of my other notebooks - \n\n1. Understanding Image Augmentation - [link](https://www.kaggle.com/amritpal333/understanding-image-augmentation-ranzcr-comp)\n2. CLAHE augmentation  - [link](https://www.kaggle.com/amritpal333/clahe-augmentation-ranzcr-comp)\n3. Findings your own mean,standerd deviation for the dataset - [link](https://www.kaggle.com/amritpal333/using-custom-mean-std-ranzcr-comp)\n\n\n## Upvote the notebooks if you find them insightful.","metadata":{}},{"cell_type":"markdown","source":"# What is CLAHE?\n\n**Contrast Limited Adaptive Histogram Equalization (CLAHE)** to equalize images. \n\nCLAHE is a variant of Adaptive histogram equalization (AHE) which takes care of over-amplification of the contrast. \nOperates on small regions in the image, called tiles, rather than the entire image. \nThe neighboring tiles are then combined using bilinear interpolation to remove the artificial boundaries.\n\n### This algorithm can be applied to improve the contrast of images.\n\n","metadata":{}},{"cell_type":"markdown","source":"![CLAHE](https://docs.opencv.org/master/histogram_equalization.png)","metadata":{}},{"cell_type":"code","source":"#!pip install albumentations\nimport albumentations\n\nimport torch\nfrom albumentations import ( Compose, OneOf, Normalize, Resize, RandomResizedCrop, RandomCrop, HorizontalFlip, VerticalFlip, \n    RandomBrightness, RandomContrast, RandomBrightnessContrast, Rotate, ShiftScaleRotate, Cutout, IAAAdditiveGaussianNoise, Transpose, ToGray )\nfrom albumentations.pytorch import ToTensorV2\nfrom albumentations import ImageOnlyTransform\n\nimport matplotlib.pyplot as plt\n\nseed = 42\n\nimport pandas as pd\nimport os\nimport cv2\nfrom torch.utils.data import Dataset,DataLoader\nfrom tqdm import tqdm","metadata":{"_uuid":"af89a040-95d5-49ba-bb59-eed78dedff0a","_cell_guid":"7faae6ad-2c75-4b64-aa3f-fae0807a1aee","execution":{"iopub.status.busy":"2024-07-14T14:41:43.818579Z","iopub.execute_input":"2024-07-14T14:41:43.818948Z","iopub.status.idle":"2024-07-14T14:41:46.785561Z","shell.execute_reply.started":"2024-07-14T14:41:43.818913Z","shell.execute_reply":"2024-07-14T14:41:46.784720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Albumentation module\nhttps://albumentations.ai/docs/api_reference/augmentations/transforms","metadata":{}},{"cell_type":"code","source":"transform = albumentations.Compose([\n     albumentations.CLAHE(clip_limit=4.0, tile_grid_size=(8, 8), always_apply=None, p=0.5)\n])","metadata":{"_uuid":"8a98a696-c072-4ebb-ab18-89f167062a99","_cell_guid":"0a640f66-daf1-4038-b127-4d37c172a369","execution":{"iopub.status.busy":"2024-07-14T14:41:49.319663Z","iopub.execute_input":"2024-07-14T14:41:49.320151Z","iopub.status.idle":"2024-07-14T14:41:49.327284Z","shell.execute_reply.started":"2024-07-14T14:41:49.320105Z","shell.execute_reply":"2024-07-14T14:41:49.325925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def show_img(image_path):\n    # Read an image with OpenCV and convert it to the RGB colorspace\n    image = cv2.imread(image_path)\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    # Augment an image\n    transformed = transform(image=image)\n    transformed_image = transformed[\"image\"]\n\n\n    fig = plt.figure()\n    fig.set_size_inches(15, 15)\n\n    ax = plt.subplot(1,1, 1)\n    plt.tight_layout()\n    ax.axis('off')\n    plt.imshow(transformed_image)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T14:46:35.408831Z","iopub.execute_input":"2024-07-14T14:46:35.409174Z","iopub.status.idle":"2024-07-14T14:46:35.415882Z","shell.execute_reply.started":"2024-07-14T14:46:35.409144Z","shell.execute_reply":"2024-07-14T14:46:35.414978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"show_img(\"/kaggle/input/isic-2024-challenge/train-image/image/ISIC_0052094.jpg\")","metadata":{"execution":{"iopub.status.busy":"2024-07-14T14:47:48.178924Z","iopub.execute_input":"2024-07-14T14:47:48.179297Z","iopub.status.idle":"2024-07-14T14:47:48.308078Z","shell.execute_reply.started":"2024-07-14T14:47:48.179261Z","shell.execute_reply":"2024-07-14T14:47:48.307071Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Baseline","metadata":{}},{"cell_type":"code","source":"import os\n\ndef remove_files_in_folder(folder_path):\n    for filename in os.listdir(folder_path):\n        file_path = os.path.join(folder_path, filename)\n        try:\n            if os.path.isfile(file_path):\n                os.remove(file_path)\n        except Exception as e:\n            print(f\"Error while removing {file_path}: {e}\")","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:30:32.332749Z","iopub.execute_input":"2024-03-07T03:30:32.333127Z","iopub.status.idle":"2024-03-07T03:30:32.353210Z","shell.execute_reply.started":"2024-03-07T03:30:32.333090Z","shell.execute_reply":"2024-03-07T03:30:32.352084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\n\ntrain_path_dest = '/kaggle/working/train_image'\n\nos.makedirs(train_path_dest)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:36:37.785745Z","iopub.execute_input":"2024-03-07T03:36:37.786115Z","iopub.status.idle":"2024-03-07T03:36:37.791080Z","shell.execute_reply.started":"2024-03-07T03:36:37.786086Z","shell.execute_reply":"2024-03-07T03:36:37.790098Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"CLAHE function\n\n> clip_limit - upper threshold value for contrast limiting. If clip_limit is a single float value, the range will be (1, clip_limit). Default: (1, 4).\n\n> tile_grid_size  - size of grid for histogram equalization. Default: (8, 8).\n \n> p  \t- probability of applying the transform. Default: 0.5.\n \nTargets: image","metadata":{}},{"cell_type":"markdown","source":"# Reference images","metadata":{}},{"cell_type":"code","source":"def Preprocessing(source, dest):\n    for filename in os.listdir(source):\n        source_file = os.path.join(source, filename)\n        # Read an image with OpenCV and convert it to the RGB colorspace\n        image = cv2.imread(source_file)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n        # Augment an image\n        transformed = transform(image=image)\n        transformed_image = transformed[\"image\"]\n        \n        dest_file = os.path.join(dest, filename)\n        \n        cv2.imwrite(dest_file, transformed_image)\n\n        ","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:43:28.483340Z","iopub.execute_input":"2024-03-07T03:43:28.483709Z","iopub.status.idle":"2024-03-07T03:43:28.490992Z","shell.execute_reply.started":"2024-03-07T03:43:28.483679Z","shell.execute_reply":"2024-03-07T03:43:28.490067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"main_source = '/kaggle/input/isic-2024-challenge/train-image/image'","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:43:31.561881Z","iopub.execute_input":"2024-03-07T03:43:31.562245Z","iopub.status.idle":"2024-03-07T03:43:31.569643Z","shell.execute_reply.started":"2024-03-07T03:43:31.562212Z","shell.execute_reply":"2024-03-07T03:43:31.568762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Preprocessing(main_source, train_path_dest)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:43:51.468098Z","iopub.execute_input":"2024-03-07T03:43:51.468450Z","iopub.status.idle":"2024-03-07T03:49:12.243399Z","shell.execute_reply.started":"2024-03-07T03:43:51.468417Z","shell.execute_reply":"2024-03-07T03:49:12.242498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:49:33.992359Z","iopub.execute_input":"2024-03-07T03:49:33.992685Z","iopub.status.idle":"2024-03-07T03:49:33.997694Z","shell.execute_reply.started":"2024-03-07T03:49:33.992657Z","shell.execute_reply":"2024-03-07T03:49:33.996893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:50:49.828891Z","iopub.execute_input":"2024-03-07T03:50:49.829235Z","iopub.status.idle":"2024-03-07T03:50:49.835473Z","shell.execute_reply.started":"2024-03-07T03:50:49.829205Z","shell.execute_reply":"2024-03-07T03:50:49.834406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## How far can we go?","metadata":{}},{"cell_type":"markdown","source":"setting clim limit to 1,40\nand tile_grid_size to 15,15","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"These look good to use.","metadata":{}},{"cell_type":"markdown","source":"# Trying differnt combinations of image augmentation\n\n(I hope to find one that magically works!)","metadata":{}},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-06T16:51:50.126198Z","iopub.execute_input":"2024-03-06T16:51:50.126645Z","iopub.status.idle":"2024-03-06T16:51:52.956268Z","shell.execute_reply.started":"2024-03-06T16:51:50.126586Z","shell.execute_reply":"2024-03-06T16:51:52.954812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T01:21:35.656142Z","iopub.execute_input":"2024-03-07T01:21:35.656616Z","iopub.status.idle":"2024-03-07T01:21:36.418624Z","shell.execute_reply.started":"2024-03-07T01:21:35.656563Z","shell.execute_reply":"2024-03-07T01:21:36.417437Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T02:51:15.533172Z","iopub.execute_input":"2024-03-07T02:51:15.533553Z","iopub.status.idle":"2024-03-07T02:51:15.538897Z","shell.execute_reply.started":"2024-03-07T02:51:15.533517Z","shell.execute_reply":"2024-03-07T02:51:15.538166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T02:01:31.724493Z","iopub.execute_input":"2024-03-07T02:01:31.724860Z","iopub.status.idle":"2024-03-07T02:01:31.729272Z","shell.execute_reply.started":"2024-03-07T02:01:31.724826Z","shell.execute_reply":"2024-03-07T02:01:31.728500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:04:27.653247Z","iopub.execute_input":"2024-03-07T03:04:27.653601Z","iopub.status.idle":"2024-03-07T03:04:27.658494Z","shell.execute_reply.started":"2024-03-07T03:04:27.653572Z","shell.execute_reply":"2024-03-07T03:04:27.657398Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T02:57:01.320147Z","iopub.execute_input":"2024-03-07T02:57:01.320505Z","iopub.status.idle":"2024-03-07T02:57:01.327580Z","shell.execute_reply.started":"2024-03-07T02:57:01.320474Z","shell.execute_reply":"2024-03-07T02:57:01.326653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T02:57:04.265846Z","iopub.execute_input":"2024-03-07T02:57:04.266189Z","iopub.status.idle":"2024-03-07T02:57:04.270897Z","shell.execute_reply.started":"2024-03-07T02:57:04.266160Z","shell.execute_reply":"2024-03-07T02:57:04.270021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T02:58:03.797669Z","iopub.execute_input":"2024-03-07T02:58:03.798057Z","iopub.status.idle":"2024-03-07T02:58:03.803860Z","shell.execute_reply.started":"2024-03-07T02:58:03.798009Z","shell.execute_reply":"2024-03-07T02:58:03.803025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:03:48.190599Z","iopub.execute_input":"2024-03-07T03:03:48.190954Z","iopub.status.idle":"2024-03-07T03:03:48.195310Z","shell.execute_reply.started":"2024-03-07T03:03:48.190925Z","shell.execute_reply":"2024-03-07T03:03:48.194466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:04:30.377610Z","iopub.execute_input":"2024-03-07T03:04:30.377986Z","iopub.status.idle":"2024-03-07T03:06:48.361747Z","shell.execute_reply.started":"2024-03-07T03:04:30.377948Z","shell.execute_reply":"2024-03-07T03:06:48.360700Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:14:18.683191Z","iopub.execute_input":"2024-03-07T03:14:18.683542Z","iopub.status.idle":"2024-03-07T03:17:17.726631Z","shell.execute_reply.started":"2024-03-07T03:14:18.683513Z","shell.execute_reply":"2024-03-07T03:17:17.725483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:19:05.973653Z","iopub.execute_input":"2024-03-07T03:19:05.974202Z","iopub.status.idle":"2024-03-07T03:19:05.988154Z","shell.execute_reply.started":"2024-03-07T03:19:05.974155Z","shell.execute_reply":"2024-03-07T03:19:05.986993Z"},"trusted":true},"execution_count":null,"outputs":[]}]}