{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"As the title of this notebook suggests, you may need to display masks over images during competitions or otherwise. Here are some simple functions that can help you do this.\n\nDisplaying masks on top of images is not very complicated in itself. However, this can quickly turn into a puzzle for the beginner and waste precious time which takes him away from his main concern.","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\nimport os\nimport matplotlib.pyplot as plt\nfrom skimage.measure import label, regionprops\nimport cv2\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **1. Data import**","metadata":{}},{"cell_type":"markdown","source":"First of all, for the needs of this notebook we need to load images as well as information about the masks to display. To do this I chose to use the data from the \"cloud organization\" competition.\n\nAs shown in the output of the code above, an image is associated with several RLE codes (in this case one per label). We will have to transform these RLE codes into masks.\nFinally, note that an RLE code can be \"NaN\", ie the class corresponding to the label is not present in the image.","metadata":{}},{"cell_type":"code","source":"repBase = '../input/understanding_cloud_organization'\nrepTrain = '../input/understanding_cloud_organization/train_images'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv(os.path.join(repBase, 'train.csv'))\ndf['image'] = df['Image_Label'].apply(lambda x: x.split('_')[0])\ndf['label'] = df['Image_Label'].apply(lambda x: x.split('_')[1])\ndf = df.drop(['Image_Label'], axis=1)\ndf.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **2. From RLE to mask**","metadata":{}},{"cell_type":"markdown","source":"Still for the needs of the notebook we define below a simple function in order to convert an RLE encoding into a mask.\n\nA mask is made up of 0's and 1's, so it's relatively easy to describe it in a much more compact form than a matrix. This is the goal of RLE encoding (run-length encoding). For more details : https://fr.wikipedia.org/wiki/Run-length_encoding","metadata":{}},{"cell_type":"code","source":"def rleToMask(rle: str, shape: tuple  =(1400, 2100)) -> np.ndarray:\n    \"\"\"\n    Converting an RLE encoding to a mask\n\n     Parameter\n     ----------\n     rle   : RLE encoding to convert \n     shape : mask shape\n\n     Return\n     ----------\n     np.array : mask\n    \"\"\"\n\n    width, height = shape[:2]\n    \n    mask= np.zeros( width*height ).astype(np.uint8)\n    \n    array = np.asarray([int(x) for x in rle.split()])\n    starts = array[0::2]\n    lengths = array[1::2]\n\n    current_position = 0\n    for index, start in enumerate(starts):\n        mask[int(start):int(start+lengths[index])] = 1\n        current_position += lengths[index]\n        \n    return mask.reshape(height, width).T","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **3. Mask over image**","metadata":{}},{"cell_type":"markdown","source":"We first define some constants, in particular the colors associated with each of the classes as well as the name of the image on which we are going to work.\nFollowing this we load our image then extract the list of RLE codes associated with it.","metadata":{}},{"cell_type":"code","source":"colors = [(0,0,255), (255,0,0), (0,255,0), (255,255,0)]\nimage_name = '7405a00.jpg'\n\nrles = df[df['image']==image_name]['EncodedPixels'].reset_index(drop=True)\nimage_start = plt.imread(os.path.join(repTrain, image_name))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here is our original image :","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots()\nax.imshow(image_start) \nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### 3.1. Colored mask","metadata":{}},{"cell_type":"markdown","source":"For our first example, we are going to display the masks transparently over the image. Each time, we take the image resulting from the previous superposition as support.","metadata":{}},{"cell_type":"code","source":"def maskInColor(image : np.ndarray, \n                mask : np.ndarray, \n                color : tuple = (0,0,255), \n                alpha : float=0.2) -> np.ndarray:\n    \"\"\"\n    Overlay mask on image\n\n     Parameter\n     ----------\n     image : image on which we want to overlay the mask \n     mask  : mask to process\n     color : color we want to apply on mask\n     alpha : opacity coefficient\n\n     Return\n     ----------\n     np.array : result of layering\n    \"\"\"\n    \n    image = np.array(image)\n    H,W,C = image.shape\n    mask    = mask.reshape(H,W,1)\n    overlay = image.astype(np.float32)\n    overlay =  255-(255-overlay)*(1-mask*alpha*color/255 )\n    overlay = np.clip(overlay,0,255)\n    overlay = overlay.astype(np.uint8)\n    return overlay","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here is the result :\n\nThis method is efficient and very visual, it has the advantage of respecting the mask since the reader can distinguish its exact shape. The problem, if any, can appear when two masks overlap. You will notice in this case a mixture of colors which is sometimes difficult to read.","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots()\n\nimage = np.copy(image_start)         #reset the working image\nfor k in range(4):                   #We have 4 classes in this dataset\n    rle = rles[k]                    #initialize the current RLE code\n    if not isinstance(rle, float):   #it's not a 'NaN' RLE\n        mask = rleToMask(rles[k])\n        image = maskInColor(image, mask, color=colors[k], alpha=0.3)\n\nax.imshow(image) \nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### 3.2. Bounding box","metadata":{}},{"cell_type":"markdown","source":"In our second example we are going to draw a frame (bounding box) around the mask.","metadata":{}},{"cell_type":"code","source":"def trace_boundingBox(image : np.ndarray,\n                      mask : np.ndarray,\n                      color : tuple = (0,0,255),\n                      width : int = 10):\n    \"\"\"\n    Draw a bounding box on image\n\n     Parameter\n     ----------\n     image : image on which we want to draw the box \n     mask  : mask to process\n     color : color we want to use to draw the box edges\n     width : box edges's width\n\n    \"\"\"\n    \n    lbl = label(mask)\n    props = regionprops(lbl)\n    for prop in props:\n        coin1 = (prop.bbox[3], prop.bbox[2])\n        coin2 = (prop.bbox[1], prop.bbox[0])\n        cv2.rectangle(image, coin2, coin1, color, width)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The resulting image is clear. In addition there is no more problem with the superposition of 2 masks. However, unlike the previous example, we no longer distinguish the exact shape of the mask.","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots()\n\nimage = np.copy(image_start)         #reset the working image\nfor k in range(4):                   #We have 4 classes in this dataset\n    rle = rles[k]                    #initialize the current RLE code\n    if not isinstance(rle, float):   #it's not a 'NaN' RLE\n        mask = rleToMask(rles[k])\n        trace_boundingBox(image, mask, color=colors[k])\n\nax.imshow(image) \nplt.show()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### 3.3. Mask inlay","metadata":{}},{"cell_type":"markdown","source":"This third and last example will consist in coloring the part of the image corresponding to the mask. This action is based on a threshold pixel value.\n\nIn our example we want to color the clouds (which are white) localized under the mask. The idea is to create a mask by isolating the area of the image corresponding to the original mask, convert it to black and white and then apply a filter on the value of the pixels. Then all we have to do is to color our new mask as we did in the first example.","metadata":{}},{"cell_type":"code","source":"def cloudInColor(image : np.ndarray, \n                 mask : np.ndarray, \n                 color : tuple = (0,0,255),\n                 alpha : float = 0.7, \n                 threshold : int = 90) -> np.ndarray:\n    \"\"\"\n    Draw a bounding box on image\n\n     Parameter\n     ----------\n     image : image on which we want to colorize parts \n     mask  : mask to process\n     color : color we want to use to colorize image\n     alpha : opacity coefficient\n     threshold : pixel value threshold to apply\n     \n     Return\n     ----------\n     np.array : result of layering\n    \"\"\"    \n    imZone = cv2.bitwise_and(image, image, mask=mask)\n    image_gray = cv2.cvtColor(imZone, cv2.COLOR_RGB2GRAY)\n    (thresh, blackAndWhiteImage) = cv2.threshold(image_gray, \n                                                 threshold, \n                                                 255, \n                                                 cv2.THRESH_BINARY)\n    inlay = maskInColor(image, blackAndWhiteImage, color=color, alpha=alpha)\n    return inlay","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The result is rather aesthetic, I'll let you judge by yourself:","metadata":{}},{"cell_type":"code","source":"\nfig, ax = plt.subplots()\n\nimage = np.copy(image_start)         #reset the working image\nfor k in range(4):                   #We have 4 classes in this dataset\n    rle = rles[k]                    #initialize the current RLE code\n    if not isinstance(rle, float):   #it's not a 'NaN' RLE\n        mask = rleToMask(rles[k])\n        image = cloudInColor(image, mask, color=colors[k], alpha=0.7, threshold=90)\n\nax.imshow(image) \nplt.show()\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Hope these few functions will be helpful for someone ...","metadata":{}}]}