{"cells":[{"metadata":{"_uuid":"1fe0140ae484f8ae8506f01b93042fd7c81f04a6"},"cell_type":"markdown","source":"# Table of Contents\n* What Are Color Spaces?\n* Simple Segmentation Using Color Spaces\n  * Color Spaces and Reading Images in OpenCV\n  * Visualizing Whale in RGB Color Space\n  * Visualizing Whale in HSV Color Space\n  * Picking Out a Range"},{"metadata":{"_uuid":"d3e0f263bc6a5fe786b094f4d08296879f78555e"},"cell_type":"markdown","source":"## What Are Color Spaces?\n\nIn the most common color space, **RGB** (Red Green Blue), colors are represented in terms of their red, green, and blue components. In more technical terms, **RGB** describes a color as a tuple of three components. Each component can take a value between 0 and 255, where the tuple (0, 0, 0) represents black and (255, 255, 255) represents white.\n\nRGB is considered an “additive” color space, and colors can be imagined as being produced from shining quantities of red, blue, and green light onto a black background.\n\nHere are a few more examples of colors in RGB:\n<hr>\n    **Color \tRGB value**\n    <hr>\n    Red \t255, 0, 0\n    <hr>\n    Orange \t255, 128, 0\n    <hr>\n    Blue     0,0,255\n    <hr>\n\nRGB is one of the five major color space models, each of which has many offshoots. There are so many color spaces because different color spaces are useful for different purposes.\n\nIn the printing world, **CMYK** is useful because it describes the color combinations required to produce a color from a white background. While the 0 tuple in RGB is black, in CMYK the 0 tuple is white. Our printers contain ink canisters of cyan, magenta, yellow, and black.\n\n**HSV** and **HSL** are descriptions of hue, saturation, and brightness/luminance, which are particularly useful for identifying contrast in images. These color spaces are frequently used in color selection tools in software and for web design.\n\nIn reality, color is a continuous phenomenon, meaning that there are an infinite number of colors. Color spaces, however, represent color through discrete structures (a fixed number of whole number integer values), which is acceptable since the human eye and perception are also limited. Color spaces are fully able to represent all the colors we are able to distinguish between."},{"metadata":{"_uuid":"13174be89b78ad59ab87b40a6a8df0b0f9e12161"},"cell_type":"markdown","source":"## Simple Segmentation Using Color Spaces\n### Color Spaces and Reading Images in OpenCV"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nfrom mpl_toolkits.mplot3d import Axes3D\nfrom matplotlib import cm\nfrom matplotlib import colors\n\nimport cv2\n\nimport os\nprint(os.listdir(\"../input\"))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f3f9f477307ba6a7c95dc224e1ba874adcefeb45"},"cell_type":"code","source":"img_path='../input/train'\nwhale_image_list=os.listdir(img_path)\nwhale_image_list[10:21]","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"markdown","source":"Look at all the color space conversions OpenCV provides"},{"metadata":{"trusted":true,"_uuid":"da98703e4317b8ba7112f693ea264f0b4473d092"},"cell_type":"code","source":"flags = [i for i in dir(cv2) if i.startswith('COLOR_')]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"162a92df548f9f3d0be44b1339187673000f2c98"},"cell_type":"code","source":"len(flags)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c265bbf4b707df06abca28a5fe42f16b5ef1ac80"},"cell_type":"code","source":"flags[30:40]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"37c831de1ff14fe05e3c2446d6e89fd8173e1d04"},"cell_type":"markdown","source":"**Load an Image**"},{"metadata":{"trusted":true,"_uuid":"09b2fa51921f27f910719813da9e4148379b982e"},"cell_type":"code","source":"whale = cv2.imread(img_path+'/'+whale_image_list[30])\nplt.imshow(whale)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ad2dd7f2950793f77d0db6f0c52b543678df74e9"},"cell_type":"markdown","source":"### Changing Color-space "},{"metadata":{"_uuid":"2f72e21da2742fa6d34561f6208effada0f3611a"},"cell_type":"markdown","source":" In fact,**OpenCV** by default reads images in **BGR** format. You can use the function `cv2.cvtColor(input_image, flag)` ( where flag determines the type of conversion) to change color-space."},{"metadata":{"_uuid":"e88afe2bd915c2220198ac207db63cba454b39fc"},"cell_type":"markdown","source":"**Convert from BGR to RGB color-space**"},{"metadata":{"trusted":true,"_uuid":"97a0b15726a76f664f0acade0b5c6ed51fa67eec"},"cell_type":"code","source":"whale_rgb = cv2.cvtColor(whale, cv2.COLOR_BGR2RGB)\nplt.imshow(whale_rgb)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"133aa382f2682747a85d138dfaea5efd5fbb4408"},"cell_type":"markdown","source":"**Convert from BGR to GRAY color-space**"},{"metadata":{"trusted":true,"_uuid":"e2b3c28acc45948e746416fca02a156aecbb4fda"},"cell_type":"code","source":"whale_gray = cv2.cvtColor(whale, cv2.COLOR_BGR2GRAY)\nplt.imshow(whale_gray)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d01cf0d6c28c70c13c1d4ca6f9198ccdbb2f4c22"},"cell_type":"markdown","source":"**Convert from BGR to HSV color-space**"},{"metadata":{"trusted":true,"_uuid":"90eab066e93238745b7420d255419ae97fedc7cc"},"cell_type":"code","source":"whale_hsv = cv2.cvtColor(whale, cv2.COLOR_BGR2HSV)\nplt.imshow(whale_hsv)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f3bad62eedcb6641a2426522e7ed6cadd9f5f09d"},"cell_type":"markdown","source":"### Separate the channels of an RGB image "},{"metadata":{"_uuid":"29ef3ad298feaf5e2a6c50eaa091caea009ff8dc"},"cell_type":"markdown","source":"**R channel**"},{"metadata":{"trusted":true,"_uuid":"e302a74f54838b8a24d4a4a2ddafb48e9f43046e"},"cell_type":"code","source":"red = whale_rgb.copy()\n# set blue and green channels to 0\nred[:, :, 1] = 0\nred[:, :, 2] = 0\n# RGB - Red\nplt.imshow(red)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"aede6afff662862af13965f33703a10143ffef0e"},"cell_type":"markdown","source":"**G channel**"},{"metadata":{"trusted":true,"_uuid":"0da5a5bad69c7b7aff502bbcf0dabcec0fdf6174"},"cell_type":"code","source":"green = whale_rgb.copy()\n# set blue and red channels to 0\ngreen[:, :, 0] = 0\ngreen[:, :, 2] = 0\n# RGB - Green\nplt.imshow(green)\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"259fc62e9d88745803a4bfe5f8f738400a9959ea"},"cell_type":"markdown","source":"**B channel**"},{"metadata":{"trusted":true,"_uuid":"79d2aba626e31745f62ca9b7e23eea5cee11bf56"},"cell_type":"code","source":"blue = whale_rgb.copy()\n# set green and red channels to 0\nblue[:, :, 0] = 0\nblue[:, :, 1] = 0\n# RGB - Blue\nplt.imshow(blue)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b37fbc8d8a79ca2d9a91f830388c50cf96be46a1"},"cell_type":"markdown","source":"### Visualizing Whale in RGB Color Space\n\n**HSV** is a good choice of color space for segmenting by color, but to see why, let’s compare the image in both RGB and HSV color spaces by visualizing the color distribution of its pixels. A 3D plot shows this quite nicely, with each axis representing one of the channels in the color space."},{"metadata":{"trusted":true,"_uuid":"90a1a511ae0f5f551424fd22e7dbfa6cae33a9f7"},"cell_type":"code","source":"r, g, b = cv2.split(whale_rgb)\nfig = plt.figure()\naxis = fig.add_subplot(1, 1, 1, projection=\"3d\")\npixel_colors = whale_rgb.reshape((np.shape(whale_rgb)[0]*np.shape(whale_rgb)[1], 3))\nnorm = colors.Normalize(vmin=-1.,vmax=1.)\nnorm.autoscale(pixel_colors)\npixel_colors = norm(pixel_colors).tolist()\naxis.scatter(r.flatten(), g.flatten(), b.flatten(), facecolors=pixel_colors, marker=\".\")\naxis.set_xlabel(\"Red\")\naxis.set_ylabel(\"Green\")\naxis.set_zlabel(\"Blue\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3e65d2ea3ce44737f907273c6b23e25c5e6d9d80"},"cell_type":"markdown","source":"### Visualizing Whale in HSV Color Space\n\nHSV stands for Hue, Saturation, and Value (or brightness), and is a cylindrical color space. The colors, or hues, are modeled as an angular dimension rotating around a central, vertical axis, which represents the value channel. Values go from dark (0 at the bottom) to light at the top. The third axis, saturation, defines the shades of hue from least saturated, at the vertical axis, to most saturated furthest away from the center:"},{"metadata":{"trusted":true,"_uuid":"3d86e74460dfcc189caccfc06a8aa26bc9f34e00"},"cell_type":"code","source":"# whale_hsv = cv2.cvtColor(whale_rgb, cv2.COLOR_RGB2HSV)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"20538884c1362db5e48344a384b00b9c78fcbcd2"},"cell_type":"code","source":"h, s, v = cv2.split(whale_hsv)\nfig = plt.figure()\naxis = fig.add_subplot(1, 1, 1, projection=\"3d\")\n\naxis.scatter(h.flatten(), s.flatten(), v.flatten(), facecolors=pixel_colors, marker=\".\")\naxis.set_xlabel(\"Hue\")\naxis.set_ylabel(\"Saturation\")\naxis.set_zlabel(\"Value\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5b503358b8dc89df94d27e6624a9b555c842d5ec"},"cell_type":"markdown","source":"In HSV space, blue’s oranges are much more localized and visually separable. The saturation and value of the blue do vary, but they are mostly located within a small range along the hue axis. This is the key point that can be leveraged for segmentation."},{"metadata":{"_uuid":"6f11b5ac20990610ee352770af2fecfddb2e18a9"},"cell_type":"markdown","source":"### Picking Out a Range\n\nLet’s threshold whale just based on a simple range of blues."},{"metadata":{"trusted":true,"_uuid":"ae7e18bc791f0fca0bf4d0a5b736943da554a764"},"cell_type":"code","source":"# define range of blue color in HSV\nlower_blue = np.array([110,50,50])\nupper_blue = np.array([130,255,255])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"44e8fd94924fecc07b111557b8aa024b7e50ecc1"},"cell_type":"markdown","source":"A simple way to display the colors in Python is to make small square images of the desired color and plot them in Matplotlib."},{"metadata":{"trusted":true,"_uuid":"dc877b1e74e8706e60a07193069b6a25d70a4c48"},"cell_type":"code","source":"from matplotlib.colors import hsv_to_rgb\nlo_square = np.full((10, 10, 3), lower_blue, dtype=np.uint8) / 255.0\ndo_square = np.full((10, 10, 3), upper_blue, dtype=np.uint8) / 255.0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"20cd1ad751fd428edadc8d7e717a658c7a89cd4e"},"cell_type":"code","source":"plt.subplot(1, 2, 1)\nplt.imshow(hsv_to_rgb(do_square))\nplt.subplot(1, 2, 2)\nplt.imshow(hsv_to_rgb(lo_square))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6b8a3ff473b7abd32c779d423c237bcbad0c6c4e"},"cell_type":"markdown","source":"Once you get a decent color range, you can use cv2.inRange() to try to threshold Whale. inRange() takes three parameters: the image, the lower range, and the higher range. It returns a binary mask (an ndarray of 1s and 0s) the size of the image where values of 1 indicate values within the range, and zero values indicate values outside:"},{"metadata":{"trusted":true,"_uuid":"1b2ebd9a8658209a8f77047c137e57ab170c5c46"},"cell_type":"code","source":"mask = cv2.inRange(whale_hsv, lower_blue, upper_blue)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0cd7ae9cb8eaf90c7e94594384e9a046506e017b"},"cell_type":"code","source":"result = cv2.bitwise_and(whale_rgb, whale_rgb, mask=mask)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"88dbd910be2782de57f75fdbb27745877e78584f"},"cell_type":"code","source":"plt.figure(figsize=(15,20))\nplt.subplot(1, 2, 1)\nplt.imshow(mask, cmap=\"gray\")\nplt.subplot(1, 2, 2)\nplt.imshow(result)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f651e1c21a74ef3581dff2fb22f2985643a4604c"},"cell_type":"code","source":"light_white = (0, 0, 200)\ndark_white = (145, 60, 255)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d9b4851a976670fad53b05b0c75a189150d2f208"},"cell_type":"code","source":"lw_square = np.full((10, 10, 3), light_white, dtype=np.uint8) / 255.0\ndw_square = np.full((10, 10, 3), dark_white, dtype=np.uint8) / 255.0\n\nplt.subplot(1, 2, 1)\nplt.imshow(hsv_to_rgb(lw_square))\nplt.subplot(1, 2, 2)\nplt.imshow(hsv_to_rgb(dw_square))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0f63a684cdbf01c4f20cb99816bc411bce3e69a6"},"cell_type":"code","source":"mask_white = cv2.inRange(whale_hsv, light_white, dark_white)\nresult_white = cv2.bitwise_and(whale_hsv, whale_hsv, mask=mask_white)\nplt.figure(figsize=(15,20))\nplt.subplot(1, 2, 1)\nplt.imshow(mask_white, cmap=\"gray\")\nplt.subplot(1, 2, 2)\nplt.imshow(result_white)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d42859ae4aeedf194d9ea2c6d3539df1e4187765"},"cell_type":"markdown","source":"Not bad! Now you can combine the masks. Adding the two masks together results in 1 values wherever there is blue or white,"},{"metadata":{"trusted":true,"_uuid":"6c64a84dce282be74a2370ca4721069ac3b642be"},"cell_type":"code","source":"final_mask = mask + mask_white\n\nfinal_result = cv2.bitwise_and(whale_rgb, whale_rgb, mask=final_mask)\nplt.figure(figsize=(15,20))\nplt.subplot(1, 2, 1)\nplt.imshow(final_mask, cmap=\"gray\")\nplt.subplot(1, 2, 2)\nplt.imshow(final_result)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"50ce8dd1195e3741f80ddf2500607f8682608ff5"},"cell_type":"markdown","source":"A Gaussian blur is an image filter that uses a kind of function called a Gaussian to transform each pixel in the image. It has the result of smoothing out image noise and reducing detail."},{"metadata":{"trusted":true,"_uuid":"0596549f7fea9c224505de997910421d137fb0f0"},"cell_type":"code","source":"blur = cv2.GaussianBlur(final_result, (7, 7), 0)\nplt.imshow(blur)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"80b7ff41ab875e0c7219dfada28c40f68e87330e"},"cell_type":"markdown","source":"### Segmentation Generalize to Whale’s Relatives"},{"metadata":{"trusted":true,"_uuid":"d24a3d827251ef7ca9181682fd7fa42a55300699"},"cell_type":"code","source":"whales_friends = []\nfor whale in whale_image_list[:10]:\n   friend = cv2.cvtColor(cv2.imread(img_path +'/'+ whale), cv2.COLOR_BGR2RGB)\n   whales_friends.append(friend)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ad67ceebe4a76f07b84c3b7768bff5ee1dbebd59"},"cell_type":"code","source":"def segment_whale(image):\n    ''' Attempts to segment the whale out of the provided image '''\n\n    # Convert the image into HSV\n    hsv_image = cv2.cvtColor(image, cv2.COLOR_RGB2HSV)\n\n    # Set the blue range\n    lower_blue = np.array([110,50,50])\n    upper_blue = np.array([130,255,255])\n\n    # Apply the blue mask \n    mask = cv2.inRange(hsv_image, lower_blue, upper_blue)\n\n    # Set a white range\n    light_white = (0, 0, 200)\n    dark_white = (145, 60, 255)\n\n    # Apply the white mask\n    mask_white = cv2.inRange(hsv_image, light_white, dark_white)\n\n    # Combine the two masks\n    final_mask = mask + mask_white\n    result = cv2.bitwise_and(image, image, mask=final_mask)\n\n    # Clean up the segmentation using a blur\n    blur = cv2.GaussianBlur(result, (7, 7), 0)\n    return blur","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"62e56027f3268ce0a081bc2fe7c6dbdf4aa0f5f5"},"cell_type":"code","source":"results = [segment_whale(whale) for whale in whales_friends]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bfd7ba426d4ddb2baa64598eb067dfbb7fc1d48f"},"cell_type":"code","source":"\nfor i in range(10):\n    plt.figure(figsize=(15,20))\n    plt.subplot(1, 2, 1)\n    plt.imshow(whales_friends[i])\n    plt.subplot(1, 2, 2)\n    plt.imshow(results[i])\n    plt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}