{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n<h2 style=\"text-align: center; font-family: Verdana; font-size: 24px; font-style: normal; font-weight: bold; text-decoration: underline; text-transform: none; letter-spacing: 2px; color: navy; background-color: #ffffff;\"> Underwater img enhancement</h2>\n\n\n<p align=\"center\">\n  \n</p>\n\n\n# 📌**Introduction:**\n<!-- <h1 style=\"font-family: times-new-roman\">📌Introduction</h1> -->\n> <p style=\"font-family: times-new-roman\">Robust recovery of lost colors in underwater images remains a challenging problem. We recently showed that this was partly due to the prevalent use of an atmospheric image formation model for underwater images. We proposed a physically accurate model that explicitly showed: 1) the attenuation coefficient of the signal is not uniform across the scene but depends on object range and reflectance, 2) the coefficient governing the increase in backscatter with distance differs from the signal attenuation coefficient. Here, we present a method that recovers color with the revised model using RGBD images. The Sea-thru method first calculates backscatter using the darkest pixels in the image and their known range information. Then, it uses an estimate of the spatially varying illuminant to obtain the range-dependent attenuation coefficient. Using more than 1,100 images from two optically different water bodies, which we make available, we show that our method outperforms those using the atmospheric model. Consistent removal of water will open up large underwater datasets to powerful computer vision and machine learning algorithms, creating exciting opportunities for the future of underwater exploration and conservation</p>\n\n# 📑 **About the Notebook:**\n> <p style=\"font-family: times-new-roman\">there were total 8 methods for image color Enhancement.</p>\n\n> - [8 Methods on Underwater Image Enhancement and Color Restoration]\n\n> ## ⭕ **Underwater Image Enhancement**\n> > - **CLAHE**: Contrast limited adaptive histogram equalization (1994)\n> > - **Fusion-Matlab**: Enhancing underwater images and videos by fusion (2012)\n> > - **GC**: Gamma Correction\n> > - **HE**: Image enhancement by histogram transformation (2011)\n> > - **ICM**: Underwater Image Enhancement Using an Integrated Colour Model (2007)\n> > - **UCM**: Enhancing the low-quality images using Unsupervised Colour Correction Method (2010)\n> > - **RayleighDistribution**: Underwater image quality enhancement through composition of dual-intensity images and Rayleigh-stretching (2014)\n> > - **RGHS**: Shallow-Water Image Enhancement Using Relative Global Histogram Stretching Based on Adaptive Parameter Acquisition (2018)\n\n\n\n\n\n\n> > - Dataset: http://csms.haifa.ac.il/profiles/tTreibitz/datasets/sea_thru/index.html\n\n> > <p align=\"center\">\n<img width = \"600\" src=\"https://i.postimg.cc/6qpZCTFK/sea-thru3.jpg\">\n</p>\n\n\n\n> ## 🧪 **Importance of image Pre-processing:**\n> > <p style=\"font-family: times-new-roman\"> Image preprocessing in one of the most important part of a Model building pipeline. It helps to improve the quality of your image, we apply different kind of filters and methods to enhance sertain aspect of the image, may be the given image is very blurry, or may be the image in very noisy, or maybe the image size across the data is not similar, or may be the labeled data is not accurate etc. In these cases we need to use Image preprocessing to clean the data. In some cases it also helps by decreasing model training time and increasing model inference speed.  </p>\n\n</b>\n\n\n> > > <p align=\"center\">\n<img src=\"https://i.imgur.com/wc9rY2J.png\">\n</p>\n\n> > > <p align=\"center\">\n<img src=\"https://i.imgur.com/ho1D0Ye.png\">\n</p>\n\n> > > <p align=\"center\">\n<img src=\"https://i.imgur.com/pwBNYJr.png\">\n</p>\n\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-11-28T13:07:14.292729Z","iopub.execute_input":"2021-11-28T13:07:14.293136Z","iopub.status.idle":"2021-11-28T13:07:15.581044Z","shell.execute_reply.started":"2021-11-28T13:07:14.293028Z","shell.execute_reply":"2021-11-28T13:07:15.580091Z"}}},{"cell_type":"markdown","source":"# 📚 **Importing Libraries:**","metadata":{}},{"cell_type":"code","source":"!pip install -q ffmpeg-python","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:58:49.362477Z","iopub.execute_input":"2022-05-08T11:58:49.36309Z","iopub.status.idle":"2022-05-08T11:59:00.074181Z","shell.execute_reply.started":"2022-05-08T11:58:49.362968Z","shell.execute_reply":"2022-05-08T11:59:00.073139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')\nimport cv2\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport random\nimport ffmpeg\nfrom IPython.display import Video\nfrom tqdm import tqdm\nimport tensorflow as tf\nimport tensorflow_addons as tfa\nimport logging\nfrom itertools import cycle\n\nlogging.disable(logging.WARNING)\nos.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' \n\n\nplt.style.use('ggplot')\ncm = sns.light_palette(\"green\", as_cmap=True)\npd.option_context('display.max_colwidth', 100)\ncolor_pal = plt.rcParams[\"axes.prop_cycle\"].by_key()[\"color\"]\ncolor_cycle = cycle(plt.rcParams[\"axes.prop_cycle\"].by_key()[\"color\"])","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:12.776898Z","iopub.execute_input":"2022-05-08T11:59:12.777302Z","iopub.status.idle":"2022-05-08T11:59:20.580532Z","shell.execute_reply.started":"2022-05-08T11:59:12.77726Z","shell.execute_reply":"2022-05-08T11:59:20.579523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# SEED EVERYTHING\nrandom.seed(hash(\"setting random seeds\") % 2**32 - 1)\nnp.random.seed(hash(\"improves reproducibility\") % 2**32 - 1)\n\n# config\nclass config:\n    BASE_DIR = \"../input/tensorflow-great-barrier-reef/train_images/\"","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:25.600543Z","iopub.execute_input":"2022-05-08T11:59:25.6009Z","iopub.status.idle":"2022-05-08T11:59:25.607483Z","shell.execute_reply.started":"2022-05-08T11:59:25.600864Z","shell.execute_reply":"2022-05-08T11:59:25.606122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔎 **Lets checkout the training data:**","metadata":{}},{"cell_type":"code","source":"img_og = plt.imread('../input/tensorflow-great-barrier-reef/train_images/video_1/9101.jpg')\nimg_9101 = cv2.imread('../input/tensorflow-great-barrier-reef/train_images/video_1/9101.jpg')","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:31.088852Z","iopub.execute_input":"2022-05-08T11:59:31.089236Z","iopub.status.idle":"2022-05-08T11:59:31.194794Z","shell.execute_reply.started":"2022-05-08T11:59:31.089195Z","shell.execute_reply":"2022-05-08T11:59:31.193699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv('../input/tensorflow-great-barrier-reef/train.csv')\ntrain_dir = \"../input/tensorflow-great-barrier-reef/train_images\"\ndf['image_path'] = train_dir + \"/video_\" + df['video_id'].astype(str) + \"/\" + df['video_frame'].astype(str) + \".jpg\"\ndf.head().style.set_properties(**{'background-color': 'black',\n                           'color': 'lawngreen',\n                           'border-color': 'white'})","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:33.194438Z","iopub.execute_input":"2022-05-08T11:59:33.194782Z","iopub.status.idle":"2022-05-08T11:59:33.433957Z","shell.execute_reply.started":"2022-05-08T11:59:33.194746Z","shell.execute_reply":"2022-05-08T11:59:33.432857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.info() # lets check more details about the data","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:36.12232Z","iopub.execute_input":"2022-05-08T11:59:36.123119Z","iopub.status.idle":"2022-05-08T11:59:36.154916Z","shell.execute_reply.started":"2022-05-08T11:59:36.123064Z","shell.execute_reply":"2022-05-08T11:59:36.15407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df[df.annotations.str.len() > 2].head(5).style.background_gradient(cmap=cm) # filling up the annotation column","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:38.866179Z","iopub.execute_input":"2022-05-08T11:59:38.867281Z","iopub.status.idle":"2022-05-08T11:59:38.915477Z","shell.execute_reply.started":"2022-05-08T11:59:38.867215Z","shell.execute_reply":"2022-05-08T11:59:38.914368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['annotations'] = df['annotations'].apply(eval)\ndf_train_v2 = df[df.annotations.str.len() > 0 ].reset_index(drop=True)\ndf_train_v2.head(5).style.background_gradient(cmap='Reds')","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:41.577893Z","iopub.execute_input":"2022-05-08T11:59:41.578938Z","iopub.status.idle":"2022-05-08T11:59:42.036181Z","shell.execute_reply.started":"2022-05-08T11:59:41.578881Z","shell.execute_reply":"2022-05-08T11:59:42.035311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# What is Sequence and its properties:","metadata":{}},{"cell_type":"code","source":"df_train_v2[\"no_of_bbox\"] = df_train_v2[\"annotations\"].apply(lambda x: len(x))\ndf_train_v2[\"sequence\"].value_counts(), len(df_train_v2[\"sequence\"].value_counts())","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:45.224388Z","iopub.execute_input":"2022-05-08T11:59:45.225399Z","iopub.status.idle":"2022-05-08T11:59:45.24233Z","shell.execute_reply.started":"2022-05-08T11:59:45.225338Z","shell.execute_reply":"2022-05-08T11:59:45.24078Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(3):\n    print(df_train_v2[\"sequence\"][df_train_v2[\"video_id\"] == i].unique(), \n          df_train_v2[\"sequence\"][df_train_v2[\"video_id\"] == i].nunique())","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:47.823047Z","iopub.execute_input":"2022-05-08T11:59:47.823867Z","iopub.status.idle":"2022-05-08T11:59:47.835455Z","shell.execute_reply.started":"2022-05-08T11:59:47.823813Z","shell.execute_reply":"2022-05-08T11:59:47.834488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Bounding box analysis in each video:","metadata":{}},{"cell_type":"code","source":"def plot_with_count(df,vid):\n    names = df[\"bbox_typ\"].to_list()\n    values = df[\"counts\"].to_list()\n\n    N = len(names)\n    menMeans = values\n    ind = np.arange(N)\n\n    plt.rcParams[\"figure.figsize\"] = [7.00, 3.50]\n    plt.rcParams[\"figure.autolayout\"] = True\n    fig, ax = plt.subplots(figsize=(15,6))\n\n    ax.bar(ind,menMeans,width=0.4)\n    plt.xticks(np.arange(0, N, step=1))\n    plt.title(f\"Number of bounding box VS Count of Bounding Box: Video{vid} \",fontsize=20)\n\n    plt.xlabel('Number of bounding box', fontsize=18)\n    plt.ylabel('Count', fontsize=16)\n\n    for index,data in enumerate(menMeans):\n        plt.text(x=index , y =data+1 , s=f\"{data}\" , fontdict=dict(fontsize=15))","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:56.160806Z","iopub.execute_input":"2022-05-08T11:59:56.161711Z","iopub.status.idle":"2022-05-08T11:59:56.171913Z","shell.execute_reply.started":"2022-05-08T11:59:56.161649Z","shell.execute_reply":"2022-05-08T11:59:56.171225Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vid = 0\ndf_vod0_bbox_cnt = df_train_v2[\"no_of_bbox\"][df_train_v2[\"video_id\"] == vid].value_counts().reset_index() # LEARNING .to_frame() and .reset_index()\ndf_vod0_bbox_cnt.columns = ['bbox_typ', 'counts']\nplot_with_count(df_vod0_bbox_cnt,vid)","metadata":{"execution":{"iopub.status.busy":"2022-05-08T11:59:59.713645Z","iopub.execute_input":"2022-05-08T11:59:59.714052Z","iopub.status.idle":"2022-05-08T12:00:00.084702Z","shell.execute_reply.started":"2022-05-08T11:59:59.714001Z","shell.execute_reply":"2022-05-08T12:00:00.08357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vid = 1\ndf_vod1_bbox_cnt = df_train_v2[\"no_of_bbox\"][df_train_v2[\"video_id\"] == vid].value_counts().reset_index() # LEARNING .to_frame() and .reset_index()\ndf_vod1_bbox_cnt.columns = ['bbox_typ', 'counts']\nplot_with_count(df_vod1_bbox_cnt,vid)","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:00:04.307364Z","iopub.execute_input":"2022-05-08T12:00:04.307681Z","iopub.status.idle":"2022-05-08T12:00:04.702199Z","shell.execute_reply.started":"2022-05-08T12:00:04.307648Z","shell.execute_reply":"2022-05-08T12:00:04.700931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vid = 2\ndf_vod2_bbox_cnt = df_train_v2[\"no_of_bbox\"][df_train_v2[\"video_id\"] == vid].value_counts().reset_index() # LEARNING .to_frame() and .reset_index()\ndf_vod2_bbox_cnt.columns = ['bbox_typ', 'counts']\nplot_with_count(df_vod2_bbox_cnt,vid)","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:00:08.049076Z","iopub.execute_input":"2022-05-08T12:00:08.049373Z","iopub.status.idle":"2022-05-08T12:00:08.483789Z","shell.execute_reply.started":"2022-05-08T12:00:08.049341Z","shell.execute_reply":"2022-05-08T12:00:08.483085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# https://www.kaggle.com/julian3833/reef-a-cv-strategy-subsequences\ndf = pd.read_csv(\"/kaggle/input/tensorflow-great-barrier-reef/train.csv\")\ndf['annotations'] = df['annotations'].apply(eval)\ndf['n_annotations'] = df['annotations'].str.len()\ndf['has_annotations'] = df['annotations'].str.len() > 0\ndf['has_2_or_more_annotations'] = df['annotations'].str.len() >= 2\ndf['doesnt_have_annotations'] = df['annotations'].str.len() == 0\ndf['image_path'] = config.BASE_DIR + \"video_\" + df['video_id'].astype(str) + \"/\" + df['video_frame'].astype(str) + \".jpg\"","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:00:11.586833Z","iopub.execute_input":"2022-05-08T12:00:11.587849Z","iopub.status.idle":"2022-05-08T12:00:12.010767Z","shell.execute_reply.started":"2022-05-08T12:00:11.587807Z","shell.execute_reply":"2022-05-08T12:00:12.009429Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_agg = df.groupby([\"video_id\", 'sequence']).agg({'sequence_frame': 'count', 'has_annotations': 'sum', 'doesnt_have_annotations': 'sum'})\\\n           .rename(columns={'sequence_frame': 'Total Frames', 'has_annotations': 'Frames with at least 1 object', 'doesnt_have_annotations': \"Frames with no object\"})\ndf_agg","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:00:15.539318Z","iopub.execute_input":"2022-05-08T12:00:15.539608Z","iopub.status.idle":"2022-05-08T12:00:15.569106Z","shell.execute_reply.started":"2022-05-08T12:00:15.539575Z","shell.execute_reply":"2022-05-08T12:00:15.567948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The point I want to make over here is that, In the pipeline we need to add more images with no lables, because there are many images in the private test case which has zero objects associated with it. But we cant actually infuse some unlabeled/no object image into the pipeline coz there are 80% of the total dataset present in that unlabeled/no object iamge categoury. So to do that we first need to choose a percentage in which we want to make the combiniation of both labeled and unlabeled image. So, say we deceide to take 6k img where 5k has object and 1k has not. Now the problem comes down to how to choose these 1k images. I would say choose few sequence and keep all of the unlabeled/no object images in the dataset and do that untill you get near 1k images. You might need to hand pich for each fold. I was thinking about using the fold CSVs generated from julian's NB and instade of filtering out all the unlabeled/no object images, keep some of them and train the whole network using that.","metadata":{}},{"cell_type":"markdown","source":"### Future work:\n- Add plots on distribution of height and width of the bbox provided\n- Add plots on distribution of area of the bbox provided","metadata":{}},{"cell_type":"markdown","source":"# 🎨 **Some Data Preparation:**","metadata":{}},{"cell_type":"code","source":"def RecoverCLAHE(sceneRadiance):\n    clahe = cv2.createCLAHE(clipLimit=7, tileGridSize=(14, 14))\n    for i in range(3):\n        sceneRadiance[:, :, i] = clahe.apply((sceneRadiance[:, :, i]))\n    return sceneRadiance\n\ndest_path1 = \"./clahe_img\"\nos.mkdir(dest_path1)\n\nfor img_path in tqdm(df_train_v2[\"image_path\"][0:400]):\n\n    image = plt.imread(img_path)\n    image_cv = cv2.imread(img_path)\n    img_clahe = RecoverCLAHE(image_cv)\n    file_name = img_path.split(\"/\")[-1]\n    \n    cv2.imwrite(dest_path1+\"/\"+file_name, img_clahe)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-05-08T12:00:21.038328Z","iopub.execute_input":"2022-05-08T12:00:21.038891Z","iopub.status.idle":"2022-05-08T12:01:06.166022Z","shell.execute_reply.started":"2022-05-08T12:00:21.038841Z","shell.execute_reply":"2022-05-08T12:01:06.165114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dest_path1 = \"./annot_img\"\nos.mkdir(dest_path1)\n\nidx = 0\nfor img_idx in tqdm(df_train_v2[\"image_path\"][0:400]):\n    file_name = img_idx.split(\"/\")[-1] \n    img_path = os.path.join(\"./clahe_img\",file_name)\n    image = plt.imread(img_path)\n\n\n    for i in range(len(df_train_v2[\"annotations\"][idx])):\n        file_name = img_path.split('/')[-1]\n        b_boxs = df_train_v2[\"annotations\"][idx][i]\n        x,y,w,h = b_boxs[\"x\"],b_boxs[\"y\"],b_boxs[\"width\"],b_boxs[\"height\"]\n\n        image = cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 3)\n        image = cv2.putText(image, 'starfish', (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36,255,12), 2)\n\n    cv2.imwrite(dest_path1+\"/\"+file_name, image)\n    idx +=1","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-05-08T12:01:16.181124Z","iopub.execute_input":"2022-05-08T12:01:16.182119Z","iopub.status.idle":"2022-05-08T12:01:35.02699Z","shell.execute_reply.started":"2022-05-08T12:01:16.182076Z","shell.execute_reply":"2022-05-08T12:01:35.02625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🧽 ** about the images:**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (12,15))\n\n\nplt.rcParams[\"figure.figsize\"] = [20.00, 10.50]\nplt.rcParams[\"figure.autolayout\"] = True\n\nfig, ax = plt.subplots(figsize=(20,6))\n\nax.imshow(plt.imread(\"../input/tensorflow-great-barrier-reef/train_images/video_0/1068.jpg\"));\nnewax = fig.add_axes([0.3,0.3,0.6,0.7], anchor='NE', zorder=1)\n\n\nnewax.axis('off')\nplt.show();","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:19:42.263Z","iopub.execute_input":"2022-05-08T12:19:42.263437Z","iopub.status.idle":"2022-05-08T12:19:42.910202Z","shell.execute_reply.started":"2022-05-08T12:19:42.2634Z","shell.execute_reply":"2022-05-08T12:19:42.909115Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Lets check we have images with same size or not:","metadata":{}},{"cell_type":"code","source":"img_sizes = []\nfor i in df_train_v2[\"image_path\"]:\n    img_sizes.append(plt.imread(i).shape)\n\nnp.unique(img_sizes)","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:01:55.928109Z","iopub.execute_input":"2022-05-08T12:01:55.92843Z","iopub.status.idle":"2022-05-08T12:05:30.678278Z","shell.execute_reply.started":"2022-05-08T12:01:55.928394Z","shell.execute_reply":"2022-05-08T12:05:30.677076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets check total number of images with annotations\nlen(df_train_v2)","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:05:40.148417Z","iopub.execute_input":"2022-05-08T12:05:40.148735Z","iopub.status.idle":"2022-05-08T12:05:40.15521Z","shell.execute_reply.started":"2022-05-08T12:05:40.148702Z","shell.execute_reply":"2022-05-08T12:05:40.154136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🦀 ** about the annotations:** ","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (12,15))\n\n\nplt.figure(figsize = (12,15))\n\n\nplt.rcParams[\"figure.figsize\"] = [20.00, 10.50]\nplt.rcParams[\"figure.autolayout\"] = True\n\nfig, ax = plt.subplots(figsize=(20,6))\n\nax.imshow(plt.imread(\"./annot_img/40.jpg\"))\n\nnewax = fig.add_axes([0.26,0.2,0.6,0.6], anchor='NE', zorder=1)\n\n\nnewax.axis('off')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:20:03.594805Z","iopub.execute_input":"2022-05-08T12:20:03.595167Z","iopub.status.idle":"2022-05-08T12:20:04.200947Z","shell.execute_reply.started":"2022-05-08T12:20:03.595126Z","shell.execute_reply":"2022-05-08T12:20:04.200243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📪 **80% of the data has not objects:** ","metadata":{}},{"cell_type":"code","source":"count_bbox = []\nfor i in df[\"annotations\"]:\n    count_bbox.append(len(i))\n    \nfrom collections import defaultdict\n\n\nbbox_dict = defaultdict(int)\n\nfor val in count_bbox:\n    bbox_dict[val] += 1\n    \n\nnames = list(bbox_dict.keys())\nvalues = list(bbox_dict.values())\n\nN = len(list(bbox_dict.values()))\nmenMeans = list(bbox_dict.values())\nind = np.arange(N)\n\nplt.rcParams[\"figure.figsize\"] = [20.00, 10.50]\nplt.rcParams[\"figure.autolayout\"] = True\nim = plt.imread('../input/random-images-dataset/Number of bounding box VS Count of Bounding Boxlittle one.png') # insert local path of the image.\nfig, ax = plt.subplots(figsize=(20,6))\n\nax.bar(ind,menMeans,width=0.4)\nplt.xticks(np.arange(0, N, step=1))\nplt.title(\"Number of bounding box VS Count of Bounding Box\",fontsize=20)\n\nplt.xlabel('Number of bounding box', fontsize=18)\nplt.ylabel('Count', fontsize=16)\nfor index,data in enumerate(menMeans):\n    plt.text(x=index , y =data+1 , s=f\"{data}\" , fontdict=dict(fontsize=20))\nnewax = fig.add_axes([0.3,0.35,0.6,0.5], anchor='NE', zorder=1)\nnewax.imshow(im)\n\nnewax.axis('off')\n\nplt.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-05-08T12:05:52.615699Z","iopub.execute_input":"2022-05-08T12:05:52.616883Z","iopub.status.idle":"2022-05-08T12:05:53.22986Z","shell.execute_reply.started":"2022-05-08T12:05:52.616819Z","shell.execute_reply":"2022-05-08T12:05:53.229096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> <font size=\"4\" face=\"verdana\">\n            <b>\"professor\" Squidward:</b>  Well seem like there are more empty bounding boxes, the fill ones.\n        </font>\n        \n<center><div class=\"alert alert-block alert-warning\" style=\"margin: 2em; line-height: 1.7em; font-family: Verdana;\">\n    There are alot of things to discover from this dataset. So keep a eye on the EDA part Im surely going to update that.\n</div></center>","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📊 **How to perform histogram equalization?**\nHistogram Equalization is a method of contrast adjustment based on the image's histogram.\n\n> In this image the pixel values are are between 0-255, but we will not find any pixel values which are exactly 0 or 255, it does not have any image which is pure white or pure black.If we apply the histogram equalization then it will reduce the color depth.Currently the minimum pixel value is 52 and the highest is 255.\nAfter you apply histogram equalization, you will find the the min pixel value now got transformed to zero and the max got converted to 255. So, notice again how the min and max values are equalized between 0 and 255, we also see less shade of gray.(view fig1 to 2)\n\n<img src=\"https://i.imgur.com/mWTXqZ4.png\" style=\"width:500px;height:250px;\">\n<center><p style=\"padding-left:380px;color:red\">Fig1:before applying histogram equalization</p></center>\n\n<img src=\"https://i.imgur.com/5diUO7c.png\" style=\"width:500px;height:250px;\">\n<center><p style=\"padding-left:380px;color:red\">Fig2: After applying histogram equalization</p></center>\n\n> Again, now we have a image on the left hand side and it's coresponding histogram(in red) on the right the black line is nothing but the cumulative of the pixel values.So, after we apply the Histogram equalizer that cumulative changes to linear step function. Notice that we don't literally flatten out the histogram we only just focus on the cumulative linear. And the real mathematics behind the Histogram equalization is just like that.(view fig 3 to 5)\n<img src=\"https://i.imgur.com/uIBnzbu.png\" style=\"width:500px;height:250px;\">\n\n<center><p style=\"padding-left:380px;color:red\">Fig3:before applying histogram equalization</p></center>\n\n<img src=\"https://i.imgur.com/fuIGrCi.png\" style=\"width:500px;height:250px;\">\n\n<center><p style=\"padding-left:380px;color:red\">Fig4: After applying histogram equalization</p></center>\n\n<img src=\"https://i.imgur.com/9q3NVIg.png\" style=\"width:500px;height:250px;\">\n<center><p style=\"padding-left:380px;color:red\">Fig5:Main difference </p></center>\n\nWe mainly use histogram equalization when we need to increase the contrast of the image.","metadata":{}},{"cell_type":"code","source":"def he_hsv(img_demo):\n    img_hsv = cv2.cvtColor(img_demo, cv2.COLOR_RGB2HSV)\n\n    # Histogram equalisation on the V-channel\n    img_hsv[:, :, 2] = cv2.equalizeHist(img_hsv[:, :, 2])\n\n    # convert image back from HSV to RGB\n    image_hsv = cv2.cvtColor(img_hsv, cv2.COLOR_HSV2RGB)\n    \n    return image_hsv","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:06:11.415805Z","iopub.execute_input":"2022-05-08T12:06:11.416392Z","iopub.status.idle":"2022-05-08T12:06:11.423342Z","shell.execute_reply.started":"2022-05-08T12:06:11.416346Z","shell.execute_reply":"2022-05-08T12:06:11.421903Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_img(img_dir,num_items,func,mode):\n    img_list = random.sample(os.listdir(img_dir), num_items)\n\n    for i in range(len(img_list)):\n        full_path = img_dir + '/' + img_list[i]\n        img_temp1 = plt.imread(full_path)\n        img_temp_cv = cv2.imread(full_path)\n        plt.figure(figsize=(20,15))\n        plt.subplot(1,2,1)\n        plt.imshow(img_temp1);\n        plt.subplot(1,2,2)\n        if mode == 'plt':\n            plt.imshow(func(img_temp1));\n        elif mode == 'cv2':\n            plt.imshow(func(img_temp_cv));","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:06:14.031429Z","iopub.execute_input":"2022-05-08T12:06:14.031862Z","iopub.status.idle":"2022-05-08T12:06:14.040607Z","shell.execute_reply.started":"2022-05-08T12:06:14.031811Z","shell.execute_reply":"2022-05-08T12:06:14.039316Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,he_hsv,\"plt\")","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:06:17.182766Z","iopub.execute_input":"2022-05-08T12:06:17.184353Z","iopub.status.idle":"2022-05-08T12:06:23.421721Z","shell.execute_reply.started":"2022-05-08T12:06:17.184286Z","shell.execute_reply":"2022-05-08T12:06:23.420627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🧩 **HE code from the repo:**\n\n- below is the code from the repo\n- check out the Code from the repo [here](https://github.com/wangyanckxx/Single-Underwater-Image-Enhancement-and-Color-Restoration/tree/master/Underwater%20Image%20Enhancement/HE )\n- It is basically allpying Histogram Equalizers in each channel","metadata":{}},{"cell_type":"code","source":"def RecoverHE(sceneRadiance):\n    for i in range(3):\n        sceneRadiance[:, :, i] =  cv2.equalizeHist(sceneRadiance[:, :, i])\n    return sceneRadiance\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,RecoverHE,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:06:38.259869Z","iopub.execute_input":"2022-05-08T12:06:38.260582Z","iopub.status.idle":"2022-05-08T12:06:44.007516Z","shell.execute_reply.started":"2022-05-08T12:06:38.260546Z","shell.execute_reply":"2022-05-08T12:06:44.005598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔮 **CLAHE: without repo code:**\n\n> Contrast Limited AHE (CLAHE) differs from adaptive histogram equalization in its contrast limiting. In the case of CLAHE, the contrast limiting procedure is applied to each neighborhood from which a transformation function is derived. CLAHE was developed to prevent the over amplification of noise that adaptive histogram equalization can give rise to.\n\n> We apply CLAHE to color images, where usually it is applied on the luminance channel and the results after equalizing only the luminance channel of an HSV image are much better than equalizing all the channels of the BGR image. \n\n<div class=\"alert alert-block alert-info\">\n<b>Note:</b> Each pixel in a image has brightness level, called luminance. This value is between 0 to 1, where 0 means complete darkness (black), and 1 is brightest (white)\n</div>\n\n","metadata":{}},{"cell_type":"code","source":"def clahe_hsv(img):\n    hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n\n    h, s, v = hsv_img[:,:,0], hsv_img[:,:,1], hsv_img[:,:,2]\n    clahe = cv2.createCLAHE(clipLimit = 15.0, tileGridSize = (20,20))\n    v = clahe.apply(v)\n\n    hsv_img = np.dstack((h,s,v))\n\n    rgb = cv2.cvtColor(hsv_img, cv2.COLOR_HSV2RGB)\n    \n    return rgb\n\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,clahe_hsv,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:06:51.156258Z","iopub.execute_input":"2022-05-08T12:06:51.156608Z","iopub.status.idle":"2022-05-08T12:06:56.055687Z","shell.execute_reply.started":"2022-05-08T12:06:51.156556Z","shell.execute_reply":"2022-05-08T12:06:56.054793Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🥏 **CLAHE: with repo code:**","metadata":{}},{"cell_type":"code","source":"def RecoverCLAHE(sceneRadiance):\n    clahe = cv2.createCLAHE(clipLimit=7, tileGridSize=(14, 14))\n    for i in range(3):\n\n        \n        sceneRadiance[:, :, i] = clahe.apply((sceneRadiance[:, :, i]))\n\n\n    return sceneRadiance\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,RecoverCLAHE,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:07:00.303331Z","iopub.execute_input":"2022-05-08T12:07:00.303665Z","iopub.status.idle":"2022-05-08T12:07:04.984268Z","shell.execute_reply.started":"2022-05-08T12:07:00.303625Z","shell.execute_reply":"2022-05-08T12:07:04.983264Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> <font size=\"4\" face=\"verdana\">\n            From the image, it seems like all the rocks, plants, and other underwater objects are on the ground and sunlight falling upon them, not inside a sea.\n        </font>","metadata":{}},{"cell_type":"markdown","source":"# 🪀 **GC: without repo code:**\n\n> Different camera or video recorder devices do not correctly capture luminance. (they are not linear) Different display devices (monitor, phone screen, TV) do not display luminance correctly neither. So, one needs to correct them, therefore the gamma correction function. Gamma correction function is a function that maps luminance levels to compensate the non-linear luminance effect of display devices (or sync it to human perceptive bias on brightness). For example check out the below image,\n\n\n<p align=\"center\">\n    <img width=\"600\" src=\"https://i.imgur.com/2GLA50Q.png\">\n</p>\n\n\n<div class=\"alert alert-block alert-info\">\n<b>Source:</b> https://medium.com/giscle/how-we-utilized-gamma-correction-for-increasing-our-training-data-47c16a040adc\n<br>\n<b>Check out more about GC, here :</b> https://en.wikipedia.org/wiki/Gamma_correction\n</div>\n","metadata":{}},{"cell_type":"code","source":"def gamma_enhancement(image,gamma):\n    R = 255.0\n    return (R * np.power(image.astype(np.uint32)/R, gamma)).astype(np.uint8)\n\nplt.figure(figsize=(20,15))\nplt.subplot(2,2,1)\nplt.imshow(img_og);\nplt.subplot(2,2,2)\nplt.imshow(gamma_enhancement(img_9101,1/0.6))\n\nplt.subplot(2,2,3)\nplt.imshow(img_og);\nplt.subplot(2,2,4)\nplt.imshow(gamma_enhancement(img_og,1/0.6))\n","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:07:21.761526Z","iopub.execute_input":"2022-05-08T12:07:21.761962Z","iopub.status.idle":"2022-05-08T12:07:24.391244Z","shell.execute_reply.started":"2022-05-08T12:07:21.761918Z","shell.execute_reply":"2022-05-08T12:07:24.390174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🖌 **GC: with repo code:**","metadata":{}},{"cell_type":"code","source":"def RecoverGC(sceneRadiance):\n    sceneRadiance = sceneRadiance/255.0\n    \n    for i in range(3):\n        sceneRadiance[:, :, i] =  np.power(sceneRadiance[:, :, i] / float(np.max(sceneRadiance[:, :, i])), 3.2)\n    sceneRadiance = np.clip(sceneRadiance*255, 0, 255)\n    sceneRadiance = np.uint8(sceneRadiance)\n    return sceneRadiance\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,RecoverGC,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:07:32.066327Z","iopub.execute_input":"2022-05-08T12:07:32.06687Z","iopub.status.idle":"2022-05-08T12:07:38.629807Z","shell.execute_reply.started":"2022-05-08T12:07:32.066824Z","shell.execute_reply":"2022-05-08T12:07:38.628648Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📍 **ICM: with repo code**","metadata":{}},{"cell_type":"code","source":"import numpy as np\n\ndef global_stretching(img_L,height, width):\n    I_min = np.min(img_L)\n    I_max = np.max(img_L)\n    I_mean = np.mean(img_L)\n\n    array_Global_histogram_stretching_L = np.zeros((height, width))\n    for i in range(0, height):\n        for j in range(0, width):\n            p_out = (img_L[i][j] - I_min) * ((1) / (I_max - I_min))\n            array_Global_histogram_stretching_L[i][j] = p_out\n\n    return array_Global_histogram_stretching_L\n\ndef stretching(img):\n    height = len(img)\n    width = len(img[0])\n    for k in range(0, 3):\n        Max_channel  = np.max(img[:,:,k])\n        Min_channel  = np.min(img[:,:,k])\n        for i in range(height):\n            for j in range(width):\n                img[i,j,k] = (img[i,j,k] - Min_channel) * (255 - 0) / (Max_channel - Min_channel)+ 0\n    return img\n\nfrom skimage.color import rgb2hsv,hsv2rgb\nimport numpy as np\n\n\n\ndef  HSVStretching(sceneRadiance):\n    height = len(sceneRadiance)\n    width = len(sceneRadiance[0])\n    img_hsv = rgb2hsv(sceneRadiance)\n    h, s, v = cv2.split(img_hsv)\n    img_s_stretching = global_stretching(s, height, width)\n\n    img_v_stretching = global_stretching(v, height, width)\n\n    labArray = np.zeros((height, width, 3), 'float64')\n    labArray[:, :, 0] = h\n    labArray[:, :, 1] = img_s_stretching\n    labArray[:, :, 2] = img_v_stretching\n    img_rgb = hsv2rgb(labArray) * 255\n\n    \n\n    return img_rgb\n\ndef sceneRadianceRGB(sceneRadiance):\n\n    sceneRadiance = np.clip(sceneRadiance, 0, 255)\n    sceneRadiance = np.uint8(sceneRadiance)\n\n    return sceneRadiance","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-05-08T12:07:47.814967Z","iopub.execute_input":"2022-05-08T12:07:47.815518Z","iopub.status.idle":"2022-05-08T12:07:48.071327Z","shell.execute_reply.started":"2022-05-08T12:07:47.815477Z","shell.execute_reply":"2022-05-08T12:07:48.069993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def RecoverICM(img1):\n    img = stretching(img1)\n    sceneRadiance = sceneRadianceRGB(img)\n    sceneRadiance = HSVStretching(sceneRadiance)\n    sceneRadiance = sceneRadianceRGB(sceneRadiance)\n    \n    return sceneRadiance\n\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,RecoverICM,\"cv2\")","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-05-08T12:07:52.657797Z","iopub.execute_input":"2022-05-08T12:07:52.658129Z","iopub.status.idle":"2022-05-08T12:09:27.740068Z","shell.execute_reply.started":"2022-05-08T12:07:52.658091Z","shell.execute_reply":"2022-05-08T12:09:27.739103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🎯 **Main Working Code:**\n> I am using **`tfa.image.equalize`** for the preprocessing. You can also add this like that in the data augmentation pipeline. Im passing this as a function in the **`plot_img_tf`** function.","metadata":{}},{"cell_type":"code","source":"def plot_img_tf(img_dir,num_items,func,mode):\n    img_list = random.sample(os.listdir(img_dir), num_items)\n    full_path = img_dir + '/' + img_list[0]\n    img_temp_plt = plt.imread(full_path)\n    img_temp_cv = cv2.imread(full_path)\n    if mode==\"plt\":\n        \n        img_stack = np.hstack((img_temp_plt,func(img_temp_plt)))\n        plt.figure(figsize=(20,15))\n        plt.imshow(img_stack);\n        plt.title(\"Original Image VS Enhanced Image\",fontsize=25)\n        plt.axis(\"off\")\n        plt.show()\n    if mode==\"cv2\":\n        \n        img_stack = np.hstack((img_temp_cv,func(img_temp_cv)))\n        plt.figure(figsize=(20,15))\n        plt.imshow(img_stack);\n        plt.title(\"Original Image VS Enhanced Image\",fontsize=25)\n        plt.axis(\"off\")\n        plt.show()\n    \n    \n    for i in range(1, len(img_list)):\n        full_path = img_dir + '/' + img_list[i]\n        img_temp_plt = plt.imread(full_path)\n        img_temp_cv = cv2.imread(full_path)\n        if mode==\"plt\":\n            img_stack = np.hstack((img_temp_plt,func(img_temp_plt)));\n            plt.figure(figsize=(20,15))\n            plt.imshow(img_stack);\n            plt.axis(\"off\")\n            plt.show()\n        if mode==\"cv2\":\n            img_stack = np.hstack((img_temp_cv,func(img_temp_cv)));\n            plt.figure(figsize=(20,15))\n            plt.imshow(img_stack);\n            plt.axis(\"off\")\n            plt.show()\n\nimg_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items = 4\nplot_img_tf(img_dir,num_items,tfa.image.equalize,\"plt\")","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:09:27.742078Z","iopub.execute_input":"2022-05-08T12:09:27.742928Z","iopub.status.idle":"2022-05-08T12:09:32.399173Z","shell.execute_reply.started":"2022-05-08T12:09:27.742879Z","shell.execute_reply":"2022-05-08T12:09:32.398126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📺 **Lets checkout How it looks like as a video:**","metadata":{}},{"cell_type":"code","source":"(\n    ffmpeg.input('./clahe_img/*.jpg', pattern_type='glob', framerate=25)\n    .output('img_movie.mp4')\n    .run()\n)","metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-05-08T12:10:16.632764Z","iopub.execute_input":"2022-05-08T12:10:16.634154Z","iopub.status.idle":"2022-05-08T12:10:16.720611Z","shell.execute_reply.started":"2022-05-08T12:10:16.634091Z","shell.execute_reply":"2022-05-08T12:10:16.719361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"(\n    ffmpeg.input('./annot_img/*.jpg', pattern_type='glob', framerate=25)\n    .output('annot_movie.mp4')\n    .run()\n)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-05-08T12:11:19.685601Z","iopub.execute_input":"2022-05-08T12:11:19.685924Z","iopub.status.idle":"2022-05-08T12:11:54.940282Z","shell.execute_reply.started":"2022-05-08T12:11:19.685892Z","shell.execute_reply":"2022-05-08T12:11:54.939198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Video(\"./img_movie.mp4\")","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:10:27.997492Z","iopub.execute_input":"2022-05-08T12:10:27.997828Z","iopub.status.idle":"2022-05-08T12:10:28.006166Z","shell.execute_reply.started":"2022-05-08T12:10:27.99779Z","shell.execute_reply":"2022-05-08T12:10:28.004787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Video(\"./annot_movie.mp4\")","metadata":{"execution":{"iopub.status.busy":"2022-05-08T12:11:59.43744Z","iopub.execute_input":"2022-05-08T12:11:59.437795Z","iopub.status.idle":"2022-05-08T12:11:59.445551Z","shell.execute_reply.started":"2022-05-08T12:11:59.437759Z","shell.execute_reply":"2022-05-08T12:11:59.444382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}