{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### Import packages","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib import image\nfrom PIL import Image\nimport cv2","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-03T17:30:54.581787Z","iopub.execute_input":"2022-02-03T17:30:54.582166Z","iopub.status.idle":"2022-02-03T17:30:54.799539Z","shell.execute_reply.started":"2022-02-03T17:30:54.582058Z","shell.execute_reply":"2022-02-03T17:30:54.798704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Loading data train.csv","metadata":{}},{"cell_type":"code","source":"df_train = pd.read_csv('../input/tensorflow-great-barrier-reef/train.csv')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-03T17:30:54.801174Z","iopub.execute_input":"2022-02-03T17:30:54.801394Z","iopub.status.idle":"2022-02-03T17:30:54.862262Z","shell.execute_reply.started":"2022-02-03T17:30:54.801368Z","shell.execute_reply":"2022-02-03T17:30:54.861426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Dataframe description\nMetadata for each image in the training set indexed by the unique image ids, comprising both sequence and bounding box information.\n* video_id - ID number of the video the image was part of. The video ids are not meaningfully ordered.\n* video_frame - The frame number of the image within the video. Expect to see occasional gaps in the frame number from when the diver surfaced.\n* sequence - ID of a gap-free subset of a given video. The sequence ids are not meaningfully ordered.\n* sequence_frame - The frame number within a given sequence.\n* image_id - ID code for the image, in the format '{video_id}-{video_frame}'\n* annotations - The bounding boxes of any starfish detections in a string format that can be evaluated directly with Python. Does not use the same format as the predictions you will submit. Not available in test.csv. A bounding box is described by the pixel coordinate (x_min, y_min) of its upper left corner within the image together with its width and height in pixels.","metadata":{}},{"cell_type":"markdown","source":"Initial parte of data.","metadata":{}},{"cell_type":"code","source":"df_train[15:20]","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-03T17:30:54.863287Z","iopub.execute_input":"2022-02-03T17:30:54.863517Z","iopub.status.idle":"2022-02-03T17:30:54.883570Z","shell.execute_reply.started":"2022-02-03T17:30:54.863491Z","shell.execute_reply":"2022-02-03T17:30:54.882513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Final part of data.","metadata":{}},{"cell_type":"code","source":"df_train.tail()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-03T17:30:54.885089Z","iopub.execute_input":"2022-02-03T17:30:54.885404Z","iopub.status.idle":"2022-02-03T17:30:54.897808Z","shell.execute_reply.started":"2022-02-03T17:30:54.885353Z","shell.execute_reply":"2022-02-03T17:30:54.897276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Showing the first image.","metadata":{}},{"cell_type":"code","source":"img_0 = plt.imread(\"../input/tensorflow-great-barrier-reef/train_images/video_0/0.jpg\")\nplt.imshow(img_0)\nplt.show()","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-03T17:30:54.899835Z","iopub.execute_input":"2022-02-03T17:30:54.900199Z","iopub.status.idle":"2022-02-03T17:30:55.279197Z","shell.execute_reply.started":"2022-02-03T17:30:54.900166Z","shell.execute_reply":"2022-02-03T17:30:55.278375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Color system","metadata":{}},{"cell_type":"code","source":"image_0 = Image.open(\"../input/tensorflow-great-barrier-reef/train_images/video_0/0.jpg\")\nprint(image_0.mode)","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-03T17:30:55.280433Z","iopub.execute_input":"2022-02-03T17:30:55.281068Z","iopub.status.idle":"2022-02-03T17:30:55.287444Z","shell.execute_reply.started":"2022-02-03T17:30:55.281015Z","shell.execute_reply":"2022-02-03T17:30:55.286923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Data type and dimension (width, height, canal)","metadata":{}},{"cell_type":"code","source":"print(type(img_0))\nprint(img_0.dtype)\nprint(img_0.shape)","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-03T17:30:55.288221Z","iopub.execute_input":"2022-02-03T17:30:55.288904Z","iopub.status.idle":"2022-02-03T17:30:55.296542Z","shell.execute_reply.started":"2022-02-03T17:30:55.288875Z","shell.execute_reply":"2022-02-03T17:30:55.295945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### The structure of an image\n\nWhen we look at an image, its smallest unit is called a pixel. The pixel is represented by three 8 bits numbers associated with the Red, Green and Blue (RGB) colors, where each color is a channel ranging from 0 to 255. Therefore, the color of a pixel corresponds to a combination of that range of channels.\nThe lowest value in this range (0) corresponds to the black color, the highest (255) represents the white color.","metadata":{"execution":{"iopub.status.busy":"2022-01-02T19:33:55.337786Z","iopub.execute_input":"2022-01-02T19:33:55.338069Z","iopub.status.idle":"2022-01-02T19:33:55.730185Z","shell.execute_reply.started":"2022-01-02T19:33:55.33804Z","shell.execute_reply":"2022-01-02T19:33:55.725678Z"}}},{"cell_type":"markdown","source":"As seen in the cell above, an image is nothing more than arrays <class 'numpy.ndarray'>, and we can observe this using the numpy library.","metadata":{}},{"cell_type":"code","source":"print(img_0)","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-02-03T17:30:55.297567Z","iopub.execute_input":"2022-02-03T17:30:55.297915Z","iopub.status.idle":"2022-02-03T17:30:55.306748Z","shell.execute_reply.started":"2022-02-03T17:30:55.297888Z","shell.execute_reply":"2022-02-03T17:30:55.306159Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It is possible to see that for each line in the matrix above there are 3 values, and these values correspond to the hue of the RGB colors, and that each line represents a pixel.","metadata":{}},{"cell_type":"markdown","source":"### Histogram equalization\n\nA common treatment when using images is their equalization from the interpretation of their histograms, thus ensuring an adequate pre-processing of the image.","metadata":{}},{"cell_type":"markdown","source":"This histogram represents the colors in pixels within a given range representing the three color channels.","metadata":{}},{"cell_type":"code","source":"plt.hist(img_0.ravel(), 256, [0, 256])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:30:55.307974Z","iopub.execute_input":"2022-02-03T17:30:55.308504Z","iopub.status.idle":"2022-02-03T17:30:56.001816Z","shell.execute_reply.started":"2022-02-03T17:30:55.308466Z","shell.execute_reply":"2022-02-03T17:30:56.001036Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, let's see how the histograms are divided into the blue, green and red channels. With this separation we can better understand the composition and distribution of colors.","metadata":{}},{"cell_type":"code","source":"blue, green, red = cv2.split(img_0)","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:30:56.002874Z","iopub.execute_input":"2022-02-03T17:30:56.003070Z","iopub.status.idle":"2022-02-03T17:30:56.024794Z","shell.execute_reply.started":"2022-02-03T17:30:56.003044Z","shell.execute_reply":"2022-02-03T17:30:56.024024Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,5))\nplt.subplot(131)\nplt.hist(blue.ravel(), 256, [0, 256])\nplt.title('Blue histogram')\n\nplt.subplot(132)\nplt.hist(green.ravel(), 256, [0, 256])\nplt.title('Green histogram')\n\nplt.subplot(133)\nplt.hist(red.ravel(), 256, [0, 256])\nplt.title('Red histogram')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:30:56.026064Z","iopub.execute_input":"2022-02-03T17:30:56.026282Z","iopub.status.idle":"2022-02-03T17:30:57.738596Z","shell.execute_reply.started":"2022-02-03T17:30:56.026257Z","shell.execute_reply":"2022-02-03T17:30:57.737752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Converting color scale\n\nFor some image classification algorithms, transforming the image to a gray scale is often one of the first steps. Below we can see the original image, in grayscale and its equalized form.","metadata":{}},{"cell_type":"code","source":"image_0_gray = cv2.imread(\"../input/tensorflow-great-barrier-reef/train_images/video_0/0.jpg\", 0)\nimage_0_eq_hist = cv2.equalizeHist(image_0_gray)","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:30:57.739999Z","iopub.execute_input":"2022-02-03T17:30:57.740423Z","iopub.status.idle":"2022-02-03T17:30:57.770857Z","shell.execute_reply.started":"2022-02-03T17:30:57.740380Z","shell.execute_reply":"2022-02-03T17:30:57.769862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,5))\n\nplt.subplot(131)\nplt.imshow(img_0)\nplt.title('Original image')\n\nplt.subplot(132)\nplt.imshow(image_0_gray, cmap=plt.cm.gray)\nplt.title('Grayscale image')\n\nplt.subplot(133)\nplt.imshow(image_0_eq_hist, cmap=plt.cm.gray)\nplt.title('Image with the equalizer function')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:30:57.772172Z","iopub.execute_input":"2022-02-03T17:30:57.772416Z","iopub.status.idle":"2022-02-03T17:30:58.621132Z","shell.execute_reply.started":"2022-02-03T17:30:57.772386Z","shell.execute_reply":"2022-02-03T17:30:58.620358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we have the distribution of light, medium and dark gray tones that make up the image.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(20, 5))\n\nplt.subplot(121)\nplt.hist(image_0_gray.ravel(), 256, [0, 256])\nplt.title('Grayscale histogram')\n\nplt.subplot(122)\nplt.hist(image_0_eq_hist.ravel(), 256, [0, 256])\nplt.title('Equalized image histogram')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:30:58.623884Z","iopub.execute_input":"2022-02-03T17:30:58.624111Z","iopub.status.idle":"2022-02-03T17:30:59.803439Z","shell.execute_reply.started":"2022-02-03T17:30:58.624082Z","shell.execute_reply":"2022-02-03T17:30:59.802801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Image treatment","metadata":{}},{"cell_type":"markdown","source":"#### Blur smoothing filter\n\nOne of the most common treatments when working with images is the use of smoothing filters, in order to try to reduce unwanted graininess in the image.\nThe appearance of noise in the image is very common when it is obtained, for example, in low light environments.","metadata":{}},{"cell_type":"code","source":"filtered_image = cv2.blur(img_0, (3, 3))","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:30:59.804546Z","iopub.execute_input":"2022-02-03T17:30:59.805207Z","iopub.status.idle":"2022-02-03T17:30:59.820223Z","shell.execute_reply.started":"2022-02-03T17:30:59.805156Z","shell.execute_reply":"2022-02-03T17:30:59.819492Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The above function of opencv is responsible for applying a filter on the original image, the command (3, 3) generates a blur based on a pixel in relation to its neighbors. Below we can see the original image, which has a certain level of noise and is smoothed without losing its sharpness and image detail.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(20,5))\n\nplt.subplot(121)\nplt.imshow(img_0)\nplt.title('Original image')\n\nplt.subplot(122)\nplt.imshow(filtered_image)\nplt.title('Filtered image')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:30:59.821335Z","iopub.execute_input":"2022-02-03T17:30:59.821548Z","iopub.status.idle":"2022-02-03T17:31:00.489837Z","shell.execute_reply.started":"2022-02-03T17:30:59.821515Z","shell.execute_reply":"2022-02-03T17:31:00.488975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Gaussian filter\n\nAnother widely used filter is the Gaussian.","metadata":{}},{"cell_type":"code","source":"filtered_image_gaus = cv2.GaussianBlur(img_0, (5, 5), 2)","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:31:00.490909Z","iopub.execute_input":"2022-02-03T17:31:00.491122Z","iopub.status.idle":"2022-02-03T17:31:00.504247Z","shell.execute_reply.started":"2022-02-03T17:31:00.491094Z","shell.execute_reply":"2022-02-03T17:31:00.503515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the case of the Gaussian filter, we have to pass a third parameter, which defines that the number of neighboring pixels to be considered must be equal on the x and y axis.\nBelow we can see the original image and the filter image. We can see that it has a lower quality compared to when the blur function is used.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(20,5))\n\nplt.subplot(121)\nplt.imshow(img_0)\nplt.title('Original image')\n\nplt.subplot(122)\nplt.imshow(filtered_image_gaus)\nplt.title('Filtered image')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:31:00.505224Z","iopub.execute_input":"2022-02-03T17:31:00.505427Z","iopub.status.idle":"2022-02-03T17:31:01.177941Z","shell.execute_reply.started":"2022-02-03T17:31:00.505402Z","shell.execute_reply":"2022-02-03T17:31:01.177068Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Image Rotation\n\nRotating the image is also a very common technique used in image recognition models.\nRotating the image by 10 degrees.","metadata":{}},{"cell_type":"code","source":"row, column, canal = img_0.shape\n\nrotation_matrix = cv2.getRotationMatrix2D((row/2, column/2), 10, 1)\n\nrotated_image = cv2.warpAffine(img_0, rotation_matrix,(column, row))\n\nfig = plt.figure(figsize=(10,8))\nplt.imshow(rotated_image)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:31:01.179138Z","iopub.execute_input":"2022-02-03T17:31:01.179383Z","iopub.status.idle":"2022-02-03T17:31:01.585057Z","shell.execute_reply.started":"2022-02-03T17:31:01.179355Z","shell.execute_reply":"2022-02-03T17:31:01.584339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_16 = plt.imread(\"../input/tensorflow-great-barrier-reef/train_images/video_0/1011.jpg\")","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:33:22.889724Z","iopub.execute_input":"2022-02-03T17:33:22.889999Z","iopub.status.idle":"2022-02-03T17:33:22.922929Z","shell.execute_reply.started":"2022-02-03T17:33:22.889966Z","shell.execute_reply":"2022-02-03T17:33:22.921884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"red = (0, 0, 255) #'x': 559, 'y': 213, 'width': 50, 'height': 32\ncv2.rectangle(image_16, (559, 213), (559 + 50, 213 + 32), red, 2)\nfig = plt.figure(figsize=(10,8))\nplt.imshow(image_16)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-02-03T17:33:24.863686Z","iopub.execute_input":"2022-02-03T17:33:24.863970Z","iopub.status.idle":"2022-02-03T17:33:25.265115Z","shell.execute_reply.started":"2022-02-03T17:33:24.863939Z","shell.execute_reply":"2022-02-03T17:33:25.264355Z"},"trusted":true},"execution_count":null,"outputs":[]}]}