{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

Tensorflow - Help Protect the Great Barrier Reef

\n

Look-sea: Underwater img enhancement

\n\n\n

\n \n

\n\n\n# ๐Ÿ“Œ**Introduction:**\n\n>

Seems like Patrick the Star is lost, and Spongebob, Mr. Krabs, Squidward, and the others need our help to find him. To help them out Kaggle is also trying their best and has put together a very informative dataset the dataset is created based on the places where Patrick most certainly be going, and we need to take the help of AI and computer vision and perform Object detection to find him from that dataset. We still don't know whether Patrick is lost or has left the city because of the last night's argument between him and Spongebob. Well, this will stay as a mystery until we find Patrik. Spongebob, Mr. Krabs, Squidward, and Sandy are all very concerned and doing their best to find Patrick. So let's do our best too, and get Patrick back to his friends.

\n\n# ๐Ÿ“‘ **About the Notebook:**\n>

As per the Discussion thread I posted earlier, I said that I will post the results. So, to do that Im creating this NB. Here is the discussion thread. After doing playing with data for a bit it was clear that the images were extreamly hezzy because of the thickness of the water, and most of the data are colored in greenish blue. After seeing a Disussion thread on this, I started searching some techniques for underwater image enhancement. After sometime, I stumbled upon this repo which talks about some techiques for Underwater color restoration and color enhancement, there were total 8 methods for image color Enhancement.Don't know how important Color Restoration would be [please let me know if you think color restoration is also important] but Image Enhancement would be very much required. The mentioned techniques are,

\n\n> - [8 Methods on Underwater Image Enhancement and Color Restoration, With Code](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/291063)\n\n> ## โญ• **Underwater Image Enhancement**\n> > - **CLAHE**: Contrast limited adaptive histogram equalization (1994)\n> > - **Fusion-Matlab**: Enhancing underwater images and videos by fusion (2012)\n> > - **GC**: Gamma Correction\n> > - **HE**: Image enhancement by histogram transformation (2011)\n> > - **ICM**: Underwater Image Enhancement Using an Integrated Colour Model (2007)\n> > - **UCM**: Enhancing the low-quality images using Unsupervised Colour Correction Method (2010)\n> > - **RayleighDistribution**: Underwater image quality enhancement through composition of dual-intensity images and Rayleigh-stretching (2014)\n> > - **RGHS**: Shallow-Water Image Enhancement Using Relative Global Histogram Stretching Based on Adaptive Parameter Acquisition (2018)\n\n> >

From these 8 methods I was able to perform, 4 and 3 of them are implemented in two different ways, one is done using the repo's code and the other is implemetnted by me. Results are good but not too good. Because there are papers like sea-thru, FUnIE-GAN and Water-GAN which yields much better results. Check about the GANs here

\n\n> ## ๐Ÿ‘€ **Sea-thru**: \n> >

This is a great paper, not only because of its good results, also the way the problem is tackled. Most of the time people ends up training large model with a huge pile of data for days to get a decent result. but this paper tries to break down the problem to its core components and solves it. This paper has a extensive use of depth map. Itried to find the trained wights but it seems like its not available. the main concept this paper goes like this,

\n\n> > -

As a result of their research, Akkaynak and Treibitz contend that wavelengths of observed colors depend on speed, distance, and the original color of objects. As with the speed of light through water, the source hue is constant. In this case, if we know the color of an object and its distance from us, we can reverse-engineer the formula that produces the observed color.

\n\n\n> > To know more check out these two links,\n> > - Article: [Sea-Thru: Removing Water from Underwater Images](https://towardsdatascience.com/sea-thru-removing-water-from-underwater-images-935288e13f7d)\n> >- Paper: [Sea-Thru: A Method for Removing Water From Underwater Images](https://openaccess.thecvf.com/content_CVPR_2019/html/Akkaynak_Sea-Thru_A_Method_for_Removing_Water_From_Underwater_Images_CVPR_2019_paper.html)\n> > - Github: https://github.com/hainh/sea-thru\n> > - Dataset: http://csms.haifa.ac.il/profiles/tTreibitz/datasets/sea_thru/index.html\n\n> >

\n\n

\n\n\n\n> ## ๐Ÿงช **Importance of image Pre-processing:**\n> >

Image preprocessing in one of the most important part of a Model building pipeline. It helps to improve the quality of your image, we apply different kind of filters and methods to enhance sertain aspect of the image, may be the given image is very blurry, or may be the image in very noisy, or maybe the image size across the data is not similar, or may be the labeled data is not accurate etc. In these cases we need to use Image preprocessing to clean the data. In some cases it also helps by decreasing model training time and increasing model inference speed. One of the incredible image preprocessing that I saw, was the hair removal in the Melanoma Classification comp.

\n\n> ## ๐ŸŽ **Main aim of this Notebook:**\n> > -

I tried doing EDA for the provided dataset. I plan to dig much dipper and understand the data in a much better way.

\n> > -

I tried different approaches to get a decent preprocessed image as mentioned above. The best preprocessed Images this Notebook offers are shown below. We will go through all the methods(except few) and then look into the working code. This is certainly not my work, it is taken from @M.Innat's comment from

\n\n> > >
  • Fast underwater image enhancement for Improved Visual Perception [github+Paper]
    • \n\n> > >

    \n\n

    \n\n> > >

    \n\n

    \n\n> > >

    \n\n

    \n\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-11-28T13:07:14.292729Z","iopub.execute_input":"2021-11-28T13:07:14.293136Z","iopub.status.idle":"2021-11-28T13:07:15.581044Z","shell.execute_reply.started":"2021-11-28T13:07:14.293028Z","shell.execute_reply":"2021-11-28T13:07:15.580091Z"}}},{"cell_type":"markdown","source":"# ๐Ÿ“š **Importing Libraries:**","metadata":{}},{"cell_type":"code","source":"!pip install -q ffmpeg-python","metadata":{"execution":{"iopub.status.busy":"2021-11-29T00:55:21.755174Z","iopub.execute_input":"2021-11-29T00:55:21.755548Z","iopub.status.idle":"2021-11-29T00:55:30.973081Z","shell.execute_reply.started":"2021-11-29T00:55:21.755503Z","shell.execute_reply":"2021-11-29T00:55:30.971885Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import cv2\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport random\nimport ffmpeg\nfrom IPython.display import Video\nfrom tqdm import tqdm\nimport tensorflow as tf\nimport tensorflow_addons as tfa\nimport logging\nlogging.disable(logging.WARNING)\nos.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' \nimport warnings\nwarnings.filterwarnings('ignore')\n\n# plt.style.use('ggplot')\ncm = sns.light_palette(\"green\", as_cmap=True)\npd.option_context('display.max_colwidth', 100)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T19:01:57.223773Z","iopub.execute_input":"2021-12-01T19:01:57.224082Z","iopub.status.idle":"2021-12-01T19:01:57.234898Z","shell.execute_reply.started":"2021-12-01T19:01:57.22405Z","shell.execute_reply":"2021-12-01T19:01:57.234102Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# SEED EVERYTHING\nrandom.seed(hash(\"setting random seeds\") % 2**32 - 1)\nnp.random.seed(hash(\"improves reproducibility\") % 2**32 - 1)","metadata":{"execution":{"iopub.status.busy":"2021-12-01T18:48:08.337894Z","iopub.execute_input":"2021-12-01T18:48:08.338889Z","iopub.status.idle":"2021-12-01T18:48:08.343348Z","shell.execute_reply.started":"2021-12-01T18:48:08.338845Z","shell.execute_reply":"2021-12-01T18:48:08.342775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ”Ž **Lets checkout the training data:**","metadata":{}},{"cell_type":"code","source":"img_og = plt.imread('../input/tensorflow-great-barrier-reef/train_images/video_1/9101.jpg')\nimg_9101 = cv2.imread('../input/tensorflow-great-barrier-reef/train_images/video_1/9101.jpg')","metadata":{"execution":{"iopub.status.busy":"2021-12-01T19:02:03.363622Z","iopub.execute_input":"2021-12-01T19:02:03.363979Z","iopub.status.idle":"2021-12-01T19:02:03.415784Z","shell.execute_reply.started":"2021-12-01T19:02:03.363942Z","shell.execute_reply":"2021-12-01T19:02:03.414992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv('../input/tensorflow-great-barrier-reef/train.csv')\ntrain_dir = \"../input/tensorflow-great-barrier-reef/train_images\"\ndf['image_path'] = train_dir + \"/video_\" + df['video_id'].astype(str) + \"/\" + df['video_frame'].astype(str) + \".jpg\"\ndf.head().style.set_properties(**{'background-color': 'black',\n 'color': 'lawngreen',\n 'border-color': 'white'})","metadata":{"execution":{"iopub.status.busy":"2021-11-29T00:55:32.232371Z","iopub.execute_input":"2021-11-29T00:55:32.232602Z","iopub.status.idle":"2021-11-29T00:55:32.551239Z","shell.execute_reply.started":"2021-11-29T00:55:32.232573Z","shell.execute_reply":"2021-11-29T00:55:32.550432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.info() # lets check more details about the data","metadata":{"execution":{"iopub.status.busy":"2021-11-29T00:55:32.552639Z","iopub.execute_input":"2021-11-29T00:55:32.552888Z","iopub.status.idle":"2021-11-29T00:55:32.576449Z","shell.execute_reply.started":"2021-11-29T00:55:32.552858Z","shell.execute_reply":"2021-11-29T00:55:32.575512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df[df.annotations.str.len() > 2].head(5).style.background_gradient(cmap=cm) # filling up the annotation column","metadata":{"execution":{"iopub.status.busy":"2021-11-29T00:55:32.577868Z","iopub.execute_input":"2021-11-29T00:55:32.578321Z","iopub.status.idle":"2021-11-29T00:55:32.615071Z","shell.execute_reply.started":"2021-11-29T00:55:32.578273Z","shell.execute_reply":"2021-11-29T00:55:32.614107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['annotations'] = df['annotations'].apply(eval)\ndf_train_v2 = df[df.annotations.str.len() > 0 ].reset_index(drop=True)\ndf_train_v2.head(5).style.background_gradient(cmap='Reds')","metadata":{"execution":{"iopub.status.busy":"2021-11-29T00:55:32.616179Z","iopub.execute_input":"2021-11-29T00:55:32.616402Z","iopub.status.idle":"2021-11-29T00:55:32.864651Z","shell.execute_reply.started":"2021-11-29T00:55:32.616374Z","shell.execute_reply":"2021-11-29T00:55:32.863743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐ŸŽจ **Some Data Preparation:**","metadata":{}},{"cell_type":"code","source":"def RecoverCLAHE(sceneRadiance):\n clahe = cv2.createCLAHE(clipLimit=7, tileGridSize=(14, 14))\n for i in range(3):\n sceneRadiance[:, :, i] = clahe.apply((sceneRadiance[:, :, i]))\n return sceneRadiance\n\ndest_path1 = \"./clahe_img\"\nos.mkdir(dest_path1)\n\nfor img_path in tqdm(df_train_v2[\"image_path\"][0:400]):\n\n image = plt.imread(img_path)\n image_cv = cv2.imread(img_path)\n img_clahe = RecoverCLAHE(image_cv)\n file_name = img_path.split(\"/\")[-1]\n \n cv2.imwrite(dest_path1+\"/\"+file_name, img_clahe)","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:02:09.218144Z","iopub.execute_input":"2021-11-29T01:02:09.218477Z","iopub.status.idle":"2021-11-29T01:02:55.35283Z","shell.execute_reply.started":"2021-11-29T01:02:09.218446Z","shell.execute_reply":"2021-11-29T01:02:55.351881Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dest_path1 = \"./annot_img\"\nos.mkdir(dest_path1)\n\nidx = 0\nfor img_idx in tqdm(df_train_v2[\"image_path\"][0:400]):\n file_name = img_idx.split(\"/\")[-1] \n img_path = os.path.join(\"./clahe_img\",file_name)\n image = plt.imread(img_path)\n\n# idx = int(img_path.split(\"/\")[-1].split(\".\")[0])\n for i in range(len(df_train_v2[\"annotations\"][idx])):\n file_name = img_path.split('/')[-1]\n b_boxs = df_train_v2[\"annotations\"][idx][i]\n x,y,w,h = b_boxs[\"x\"],b_boxs[\"y\"],b_boxs[\"width\"],b_boxs[\"height\"]\n\n image = cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 3)\n image = cv2.putText(image, 'starfish', (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36,255,12), 2)\n\n cv2.imwrite(dest_path1+\"/\"+file_name, image)\n idx +=1\n# print(img_path)","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:02:55.354549Z","iopub.execute_input":"2021-11-29T01:02:55.354894Z","iopub.status.idle":"2021-11-29T01:03:14.34231Z","shell.execute_reply.started":"2021-11-29T01:02:55.354856Z","shell.execute_reply":"2021-11-29T01:03:14.341203Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿงฝ **Lets see that SpongeBob thinks about the images:**","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (12,15))\n\n\nplt.rcParams[\"figure.figsize\"] = [20.00, 10.50]\nplt.rcParams[\"figure.autolayout\"] = True\nim = plt.imread('../input/random-images-dataset/Wow (1).png') # insert local path of the image.\nfig, ax = plt.subplots(figsize=(20,6))\n# plt.figure(figsize=(20,6))\nax.imshow(plt.imread(\"../input/tensorflow-great-barrier-reef/train_images/video_0/40.jpg\"));\nnewax = fig.add_axes([0.3,0.3,0.6,0.7], anchor='NE', zorder=1)\nnewax.imshow(im);\n\nnewax.axis('off')\nplt.show();","metadata":{"execution":{"iopub.status.busy":"2021-11-29T00:55:32.865998Z","iopub.execute_input":"2021-11-29T00:55:32.86625Z","iopub.status.idle":"2021-11-29T00:55:33.714502Z","shell.execute_reply.started":"2021-11-29T00:55:32.86622Z","shell.execute_reply":"2021-11-29T00:55:33.711353Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Lets check we have images with same size or not:","metadata":{}},{"cell_type":"code","source":"img_sizes = []\nfor i in df_train_v2[\"image_path\"]:\n img_sizes.append(plt.imread(i).shape)\n\nnp.unique(img_sizes)\n# seems like the same","metadata":{"execution":{"iopub.status.busy":"2021-11-29T00:55:33.718553Z","iopub.execute_input":"2021-11-29T00:55:33.719225Z","iopub.status.idle":"2021-11-29T00:59:01.587427Z","shell.execute_reply.started":"2021-11-29T00:55:33.719177Z","shell.execute_reply":"2021-11-29T00:59:01.586048Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets check total number of images with annotations\nlen(df_train_v2)","metadata":{"execution":{"iopub.status.busy":"2021-11-29T00:59:01.590264Z","iopub.execute_input":"2021-11-29T00:59:01.590648Z","iopub.status.idle":"2021-11-29T00:59:01.598401Z","shell.execute_reply.started":"2021-11-29T00:59:01.590593Z","shell.execute_reply":"2021-11-29T00:59:01.597696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿฆ€ **Lets see what Mr: krabe has to say about the annotations:** ","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (12,15))\n# plt.imshow(plt.imread(\"./annot_img/40.jpg\"))\n\nplt.figure(figsize = (12,15))\n\n\nplt.rcParams[\"figure.figsize\"] = [20.00, 10.50]\nplt.rcParams[\"figure.autolayout\"] = True\nim = plt.imread('../input/random-images-dataset/Wow.png') # insert local path of the image.\nfig, ax = plt.subplots(figsize=(20,6))\n# plt.figure(figsize=(20,6))\n# ax.imshow(plt.imread(\"../input/tensorflow-great-barrier-reef/train_images/video_0/16.jpg\"))\nax.imshow(plt.imread(\"./annot_img/40.jpg\"))\n\nnewax = fig.add_axes([0.26,0.2,0.6,0.6], anchor='NE', zorder=1)\nnewax.imshow(im)\n\nnewax.axis('off')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:03:14.343622Z","iopub.execute_input":"2021-11-29T01:03:14.343859Z","iopub.status.idle":"2021-11-29T01:03:15.002112Z","shell.execute_reply.started":"2021-11-29T01:03:14.343831Z","shell.execute_reply":"2021-11-29T01:03:15.001203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ“ช **80% of the data is not annotated:** ","metadata":{}},{"cell_type":"code","source":"count_bbox = []\nfor i in df[\"annotations\"]:\n count_bbox.append(len(i))\n \nfrom collections import defaultdict\n\n\nbbox_dict = defaultdict(int)\n\nfor val in count_bbox:\n bbox_dict[val] += 1\n \n# data = {'milk': 60, 'water': 10}\nnames = list(bbox_dict.keys())\nvalues = list(bbox_dict.values())\n\nN = len(list(bbox_dict.values()))\nmenMeans = list(bbox_dict.values())\nind = np.arange(N)\n\nplt.rcParams[\"figure.figsize\"] = [20.00, 10.50]\nplt.rcParams[\"figure.autolayout\"] = True\nim = plt.imread('../input/random-images-dataset/Number of bounding box VS Count of Bounding Boxlittle one.png') # insert local path of the image.\nfig, ax = plt.subplots(figsize=(20,6))\n# plt.figure(figsize=(20,6))\nax.bar(ind,menMeans,width=0.4)\nplt.xticks(np.arange(0, N, step=1))\nplt.title(\"Number of bounding box VS Count of Bounding Box\",fontsize=20)\n# fig.suptitle('test title', fontsize=20)\nplt.xlabel('Number of bounding box', fontsize=18)\nplt.ylabel('Count', fontsize=16)\nfor index,data in enumerate(menMeans):\n plt.text(x=index , y =data+1 , s=f\"{data}\" , fontdict=dict(fontsize=20))\nnewax = fig.add_axes([0.3,0.35,0.6,0.5], anchor='NE', zorder=1)\nnewax.imshow(im)\n\nnewax.axis('off')\n# for index,data in enumerate(menMeans):\n# plt.text(x=index , y =data+1 , s=f\"{data}\" , fontdict=dict(fontsize=20))\n# plt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:03:15.003777Z","iopub.execute_input":"2021-11-29T01:03:15.004053Z","iopub.status.idle":"2021-11-29T01:03:15.551032Z","shell.execute_reply.started":"2021-11-29T01:03:15.004022Z","shell.execute_reply":"2021-11-29T01:03:15.54987Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> \n \"professor\" Squidward: Well seem like there are more empty bounding boxes, the fill ones.\n \n \n
    \n There are alot of things to discover from this dataset. So keep a eye on the EDA part Im surely going to update that.\n
    ","metadata":{}},{"cell_type":"markdown","source":"# ๐Ÿ“Š **How to perform histogram equalization?**\nHistogram Equalization is a method of contrast adjustment based on the image's histogram.\n\n> In this image the pixel values are are between 0-255, but we will not find any pixel values which are exactly 0 or 255, it does not have any image which is pure white or pure black.If we apply the histogram equalization then it will reduce the color depth.Currently the minimum pixel value is 52 and the highest is 255.\nAfter you apply histogram equalization, you will find the the min pixel value now got transformed to zero and the max got converted to 255. So, notice again how the min and max values are equalized between 0 and 255, we also see less shade of gray.(view fig1 to 2)\n\n\n

    Fig1:before applying histogram equalization

    \n\n\n

    Fig2: After applying histogram equalization

    \n\n> Again, now we have a image on the left hand side and it's coresponding histogram(in red) on the right the black line is nothing but the cumulative of the pixel values.So, after we apply the Histogram equalizer that cumulative changes to linear step function. Notice that we don't literally flatten out the histogram we only just focus on the cumulative linear. And the real mathematics behind the Histogram equalization is just like that.(view fig 3 to 5)\n\n\n

    Fig3:before applying histogram equalization

    \n\n\n\n

    Fig4: After applying histogram equalization

    \n\n\n

    Fig5:Main difference

    \n\nWe mainly use histogram equalization when we need to increase the contrast of the image.","metadata":{}},{"cell_type":"code","source":"def he_hsv(img_demo):\n img_hsv = cv2.cvtColor(img_demo, cv2.COLOR_RGB2HSV)\n\n # Histogram equalisation on the V-channel\n img_hsv[:, :, 2] = cv2.equalizeHist(img_hsv[:, :, 2])\n\n # convert image back from HSV to RGB\n image_hsv = cv2.cvtColor(img_hsv, cv2.COLOR_HSV2RGB)\n \n return image_hsv","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:05:45.949927Z","iopub.execute_input":"2021-11-29T01:05:45.950775Z","iopub.status.idle":"2021-11-29T01:05:45.956255Z","shell.execute_reply.started":"2021-11-29T01:05:45.950721Z","shell.execute_reply":"2021-11-29T01:05:45.955354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_img(img_dir,num_items,func,mode):\n img_list = random.sample(os.listdir(img_dir), num_items)\n\n for i in range(len(img_list)):\n full_path = img_dir + '/' + img_list[i]\n img_temp1 = plt.imread(full_path)\n img_temp_cv = cv2.imread(full_path)\n plt.figure(figsize=(20,15))\n plt.subplot(1,2,1)\n plt.imshow(img_temp1);\n plt.subplot(1,2,2)\n if mode == 'plt':\n plt.imshow(func(img_temp1));\n elif mode == 'cv2':\n plt.imshow(func(img_temp_cv));","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:05:46.501583Z","iopub.execute_input":"2021-11-29T01:05:46.502274Z","iopub.status.idle":"2021-11-29T01:05:46.508734Z","shell.execute_reply.started":"2021-11-29T01:05:46.502233Z","shell.execute_reply":"2021-11-29T01:05:46.507962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"vid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,he_hsv,\"plt\")","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:05:47.696866Z","iopub.execute_input":"2021-11-29T01:05:47.697334Z","iopub.status.idle":"2021-11-29T01:05:53.185504Z","shell.execute_reply.started":"2021-11-29T01:05:47.697283Z","shell.execute_reply":"2021-11-29T01:05:53.184498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿงฉ **HE code from the repo:**\n\n- below is the code from the repo\n- check out the Code from the repo [here](https://github.com/wangyanckxx/Single-Underwater-Image-Enhancement-and-Color-Restoration/tree/master/Underwater%20Image%20Enhancement/HE )\n- It is basically allpying Histogram Equalizers in each channel","metadata":{}},{"cell_type":"code","source":"def RecoverHE(sceneRadiance):\n for i in range(3):\n sceneRadiance[:, :, i] = cv2.equalizeHist(sceneRadiance[:, :, i])\n return sceneRadiance\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,RecoverHE,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:05:53.187103Z","iopub.execute_input":"2021-11-29T01:05:53.187346Z","iopub.status.idle":"2021-11-29T01:05:57.932475Z","shell.execute_reply.started":"2021-11-29T01:05:53.187317Z","shell.execute_reply":"2021-11-29T01:05:57.931015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ”ฎ **CLAHE: without repo code:**\n\n> Contrast Limited AHE (CLAHE) differs from adaptive histogram equalization in its contrast limiting. In the case of CLAHE, the contrast limiting procedure is applied to each neighborhood from which a transformation function is derived. CLAHE was developed to prevent the over amplification of noise that adaptive histogram equalization can give rise to.\n\n> We apply CLAHE to color images, where usually it is applied on the luminance channel and the results after equalizing only the luminance channel of an HSV image are much better than equalizing all the channels of the BGR image. \n\n
    \nNote: Each pixel in a image has brightness level, called luminance. This value is between 0 to 1, where 0 means complete darkness (black), and 1 is brightest (white)\n
    \n\n","metadata":{}},{"cell_type":"code","source":"def clahe_hsv(img):\n hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n\n h, s, v = hsv_img[:,:,0], hsv_img[:,:,1], hsv_img[:,:,2]\n clahe = cv2.createCLAHE(clipLimit = 15.0, tileGridSize = (20,20))\n v = clahe.apply(v)\n\n hsv_img = np.dstack((h,s,v))\n\n rgb = cv2.cvtColor(hsv_img, cv2.COLOR_HSV2RGB)\n \n return rgb\n\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,clahe_hsv,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:05:57.934118Z","iopub.execute_input":"2021-11-29T01:05:57.934579Z","iopub.status.idle":"2021-11-29T01:06:02.513658Z","shell.execute_reply.started":"2021-11-29T01:05:57.934537Z","shell.execute_reply":"2021-11-29T01:06:02.512585Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿฅ **CLAHE: with repo code:**","metadata":{}},{"cell_type":"code","source":"def RecoverCLAHE(sceneRadiance):\n clahe = cv2.createCLAHE(clipLimit=7, tileGridSize=(14, 14))\n for i in range(3):\n\n # sceneRadiance[:, :, i] = cv2.equalizeHist(sceneRadiance[:, :, i])\n sceneRadiance[:, :, i] = clahe.apply((sceneRadiance[:, :, i]))\n\n\n return sceneRadiance\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,RecoverCLAHE,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:06:02.515586Z","iopub.execute_input":"2021-11-29T01:06:02.515874Z","iopub.status.idle":"2021-11-29T01:06:07.300997Z","shell.execute_reply.started":"2021-11-29T01:06:02.51583Z","shell.execute_reply":"2021-11-29T01:06:07.297929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> \n From the image, it seems like all the rocks, plants, and other underwater objects are on the ground and sunlight falling upon them, not inside a sea.\n ","metadata":{}},{"cell_type":"markdown","source":"# ๐Ÿช€ **GC: without repo code:**\n\n> Different camera or video recorder devices do not correctly capture luminance. (they are not linear) Different display devices (monitor, phone screen, TV) do not display luminance correctly neither. So, one needs to correct them, therefore the gamma correction function. Gamma correction function is a function that maps luminance levels to compensate the non-linear luminance effect of display devices (or sync it to human perceptive bias on brightness). For example check out the below image,\n\n\n

    \n \n

    \n\n\n
    \nSource: https://medium.com/giscle/how-we-utilized-gamma-correction-for-increasing-our-training-data-47c16a040adc\n
    \nCheck out more about GC, here : https://en.wikipedia.org/wiki/Gamma_correction\n
    \n","metadata":{}},{"cell_type":"code","source":"def gamma_enhancement(image,gamma):\n R = 255.0\n return (R * np.power(image.astype(np.uint32)/R, gamma)).astype(np.uint8)\n\nplt.figure(figsize=(20,15))\nplt.subplot(2,2,1)\nplt.imshow(img_og);\nplt.subplot(2,2,2)\nplt.imshow(gamma_enhancement(img_9101,1/0.6))\n\nplt.subplot(2,2,3)\nplt.imshow(img_og);\nplt.subplot(2,2,4)\nplt.imshow(gamma_enhancement(img_og,1/0.6))\n","metadata":{"execution":{"iopub.status.busy":"2021-12-01T19:10:53.530859Z","iopub.execute_input":"2021-12-01T19:10:53.531192Z","iopub.status.idle":"2021-12-01T19:10:55.569629Z","shell.execute_reply.started":"2021-12-01T19:10:53.531158Z","shell.execute_reply":"2021-12-01T19:10:55.568481Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ–Œ **GC: with repo code:**","metadata":{}},{"cell_type":"code","source":"def RecoverGC(sceneRadiance):\n sceneRadiance = sceneRadiance/255.0\n # clahe = cv2.createCLAHE(clipLimit=2, tileGridSize=(2, 2))\n for i in range(3):\n sceneRadiance[:, :, i] = np.power(sceneRadiance[:, :, i] / float(np.max(sceneRadiance[:, :, i])), 3.2)\n sceneRadiance = np.clip(sceneRadiance*255, 0, 255)\n sceneRadiance = np.uint8(sceneRadiance)\n return sceneRadiance\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,RecoverGC,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:06:09.869395Z","iopub.execute_input":"2021-11-29T01:06:09.869732Z","iopub.status.idle":"2021-11-29T01:06:15.691607Z","shell.execute_reply.started":"2021-11-29T01:06:09.869685Z","shell.execute_reply":"2021-11-29T01:06:15.690621Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ“ **ICM: with repo code**","metadata":{}},{"cell_type":"code","source":"import numpy as np\n\ndef global_stretching(img_L,height, width):\n I_min = np.min(img_L)\n I_max = np.max(img_L)\n I_mean = np.mean(img_L)\n\n array_Global_histogram_stretching_L = np.zeros((height, width))\n for i in range(0, height):\n for j in range(0, width):\n p_out = (img_L[i][j] - I_min) * ((1) / (I_max - I_min))\n array_Global_histogram_stretching_L[i][j] = p_out\n\n return array_Global_histogram_stretching_L\n\ndef stretching(img):\n height = len(img)\n width = len(img[0])\n for k in range(0, 3):\n Max_channel = np.max(img[:,:,k])\n Min_channel = np.min(img[:,:,k])\n for i in range(height):\n for j in range(width):\n img[i,j,k] = (img[i,j,k] - Min_channel) * (255 - 0) / (Max_channel - Min_channel)+ 0\n return img\n\nfrom skimage.color import rgb2hsv,hsv2rgb\nimport numpy as np\n\n\n\ndef HSVStretching(sceneRadiance):\n height = len(sceneRadiance)\n width = len(sceneRadiance[0])\n img_hsv = rgb2hsv(sceneRadiance)\n h, s, v = cv2.split(img_hsv)\n img_s_stretching = global_stretching(s, height, width)\n\n img_v_stretching = global_stretching(v, height, width)\n\n labArray = np.zeros((height, width, 3), 'float64')\n labArray[:, :, 0] = h\n labArray[:, :, 1] = img_s_stretching\n labArray[:, :, 2] = img_v_stretching\n img_rgb = hsv2rgb(labArray) * 255\n\n # img_rgb = np.clip(img_rgb, 0, 255)\n\n return img_rgb\n\ndef sceneRadianceRGB(sceneRadiance):\n\n sceneRadiance = np.clip(sceneRadiance, 0, 255)\n sceneRadiance = np.uint8(sceneRadiance)\n\n return sceneRadiance","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:06:15.693071Z","iopub.execute_input":"2021-11-29T01:06:15.693385Z","iopub.status.idle":"2021-11-29T01:06:15.901522Z","shell.execute_reply.started":"2021-11-29T01:06:15.69335Z","shell.execute_reply":"2021-11-29T01:06:15.90085Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def RecoverICM(img1):\n img = stretching(img1)\n sceneRadiance = sceneRadianceRGB(img)\n sceneRadiance = HSVStretching(sceneRadiance)\n sceneRadiance = sceneRadianceRGB(sceneRadiance)\n \n return sceneRadiance\n\n\nvid_0_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items1 = 4\nplot_img(vid_0_dir,num_items1,RecoverICM,\"cv2\")","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:06:15.902686Z","iopub.execute_input":"2021-11-29T01:06:15.90328Z","iopub.status.idle":"2021-11-29T01:07:10.061965Z","shell.execute_reply.started":"2021-11-29T01:06:15.903239Z","shell.execute_reply":"2021-11-29T01:07:10.061013Z"},"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐ŸŽฏ **Main Working Code:**\n> I am using **`tfa.image.equalize`** for the preprocessing. You can also add this like that in the data augmentation pipeline. Im passing this as a function in the **`plot_img_tf`** function.","metadata":{}},{"cell_type":"code","source":"def plot_img_tf(img_dir,num_items,func,mode):\n img_list = random.sample(os.listdir(img_dir), num_items)\n full_path = img_dir + '/' + img_list[0]\n img_temp_plt = plt.imread(full_path)\n img_temp_cv = cv2.imread(full_path)\n if mode==\"plt\":\n \n img_stack = np.hstack((img_temp_plt,func(img_temp_plt)))\n plt.figure(figsize=(20,15))\n plt.imshow(img_stack);\n plt.title(\"Original Image VS Enhanced Image\",fontsize=25)\n plt.axis(\"off\")\n plt.show()\n if mode==\"cv2\":\n \n img_stack = np.hstack((img_temp_cv,func(img_temp_cv)))\n plt.figure(figsize=(20,15))\n plt.imshow(img_stack);\n plt.title(\"Original Image VS Enhanced Image\",fontsize=25)\n plt.axis(\"off\")\n plt.show()\n \n \n for i in range(1, len(img_list)):\n full_path = img_dir + '/' + img_list[i]\n img_temp_plt = plt.imread(full_path)\n img_temp_cv = cv2.imread(full_path)\n if mode==\"plt\":\n img_stack = np.hstack((img_temp_plt,func(img_temp_plt)));\n plt.figure(figsize=(20,15))\n plt.imshow(img_stack);\n plt.axis(\"off\")\n plt.show()\n if mode==\"cv2\":\n img_stack = np.hstack((img_temp_cv,func(img_temp_cv)));\n plt.figure(figsize=(20,15))\n plt.imshow(img_stack);\n plt.axis(\"off\")\n plt.show()\n\nimg_dir = \"../input/tensorflow-great-barrier-reef/train_images/video_0\"\nnum_items = 4\nplot_img_tf(img_dir,num_items,tfa.image.equalize,\"plt\")","metadata":{"execution":{"iopub.status.busy":"2021-12-01T18:51:12.843677Z","iopub.execute_input":"2021-12-01T18:51:12.844069Z","iopub.status.idle":"2021-12-01T18:51:17.325348Z","shell.execute_reply.started":"2021-12-01T18:51:12.844029Z","shell.execute_reply":"2021-12-01T18:51:17.324413Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ๐Ÿ“บ **Lets checkout How it looks like as a video:**","metadata":{}},{"cell_type":"code","source":"(\n ffmpeg.input('./clahe_img/*.jpg', pattern_type='glob', framerate=25)\n .output('img_movie.mp4')\n .run()\n)","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:07:10.063293Z","iopub.execute_input":"2021-11-29T01:07:10.063588Z","iopub.status.idle":"2021-11-29T01:07:41.058033Z","shell.execute_reply.started":"2021-11-29T01:07:10.063553Z","shell.execute_reply":"2021-11-29T01:07:41.056911Z"},"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"(\n ffmpeg.input('./annot_img/*.jpg', pattern_type='glob', framerate=25)\n .output('annot_movie.mp4')\n .run()\n)","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:07:41.062204Z","iopub.execute_input":"2021-11-29T01:07:41.06257Z","iopub.status.idle":"2021-11-29T01:08:11.910885Z","shell.execute_reply.started":"2021-11-29T01:07:41.062521Z","shell.execute_reply":"2021-11-29T01:08:11.909751Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Video(\"./img_movie.mp4\")","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:08:11.913542Z","iopub.execute_input":"2021-11-29T01:08:11.914252Z","iopub.status.idle":"2021-11-29T01:08:11.92228Z","shell.execute_reply.started":"2021-11-29T01:08:11.914199Z","shell.execute_reply":"2021-11-29T01:08:11.921193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Video(\"./annot_movie.mp4\")","metadata":{"execution":{"iopub.status.busy":"2021-11-29T01:08:11.924438Z","iopub.execute_input":"2021-11-29T01:08:11.925362Z","iopub.status.idle":"2021-11-29T01:08:11.935063Z","shell.execute_reply.started":"2021-11-29T01:08:11.925313Z","shell.execute_reply":"2021-11-29T01:08:11.934016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}