{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# 2. Understanding and plotting rle bounding boxes\n### Airbus Ship Detection Challenge - A quick overview for computer vision noobs\n\n&nbsp;\n\n\nHi, and welcome! This is the second kernel of the series `Airbus Ship Detection Challenge - A quick overview for computer vision noobs.` In this short kernel we will explain the run-length encoded bounding boxes, translate the rle code into a list of pixels with pure python and plot that list of pixels as a mask on top of the pictures with matplotlib.\n\n\nThe full series consist of the following notebooks:\n1. [Loading and visualizing the images](https://www.kaggle.com/julian3833/1-loading-and-visualizing-the-images)\n2. *[Understanding and plotting rle bounding boxes](https://www.kaggle.com/julian3833/2-understanding-and-plotting-rle-bounding-boxes)*\n3. [Basic exploratory analysis](https://www.kaggle.com/julian3833/3-basic-exploratory-analysis)\n4. [Exploring public models](https://www.kaggle.com/julian3833/4-exploring-models-shared-by-the-community)\n5. [1.0 submission: submitting the test file](https://www.kaggle.com/julian3833/5-1-0-submission-submitting-the-test-file)\n\nThis is an ongoing project, so expect more notebooks to be added to the series soon. Actually, we are currently working on the following ones:\n* Understanding and exploiting the data leak\n* A quick overview of image segmentation domain\n* Jumping into Pytorch\n* Understanding U-net\n* Proposing a simple improvement to U-net model"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"<a id='understand'></a>\n# 1. Understanding run-length encoding\n\n&nbsp;\n\nThere is no clear information about this encoding on the Challenge's Data tab - may be it's too obvious? : in any case, it is definitely new for us. There are some comments about the encoding  on the [Evaluation](https://www.kaggle.com/c/airbus-ship-detection#evaluation) tab and - yes - there is an entry on [wikipedia](https://es.wikipedia.org/wiki/Run-length_encoding) explaining the  `run-length encoding` idea.  RLE, for short, is a simple morse-like representation of shapes in 2d images. In this case, what's encoded are some rectangular shapes - the bounding boxes - where the ships are located in the respective images.\n\nThe encoded string looks like this: `start, length, start, length, ...` , where each pair of (`start`, `length`) draws a line of `length` pixeles starting from position `start.`  The `start` position, in turn, is not a  `(x, y)` coordinate but an index of the 1-d array resulting of flattening the 2-d image into a rows-after-row 1-d sequence of pixels.  Knowing the shape of the images we can just unfold this 1-d representating into a 2-dimensions mask using  `//` and `%`.  \n\nLet's start by checking a csv for a rle code example. It's stored in the column `EncodedPixels`:"},{"metadata":{"trusted":true,"_uuid":"56e29075fe310de8b6e22ef379c405173762fd69"},"cell_type":"code","source":"import PIL\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\ndf = pd.read_csv(\"../input/train_ship_segmentations_v2.csv\", index_col=0).dropna()\ndisplay(df.head())\ndf['EncodedPixels']['000155de5.jpg']","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9078ed7e1091c696d4eebe026a4a476df444054c"},"cell_type":"markdown","source":"Ok: let's parse it with pure python. In the next cell, we just map this string into a list of (`start`, `length`) pairs:"},{"metadata":{"trusted":true,"_uuid":"410aaa9104553cf74ec50457a067f3222d719dc4"},"cell_type":"code","source":"# turn rle example into a list of ints\nrle = [int(i) for i in df['EncodedPixels']['000155de5.jpg'].split()]\n# turn list of ints into a list of (`start`, `length`) `pairs`\npairs = list(zip(rle[0:-1:2], rle[1::2])) \npairs[:3]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"780b53dea6a232c53e08c2c700a9af7a62855ed6"},"cell_type":"markdown","source":"On the other hand, we can trivially encode and decode from a `start` scalar position like `264661` into a 2-d coordinate in our 768$\\times$768 pictures using `%`, `//` and `*`:"},{"metadata":{"trusted":true,"_uuid":"f2a72ab37e0462899dbdb24b5db2ff5d33ea72ad"},"cell_type":"code","source":"start = pairs[0][0]\nprint(f\"Original start position: {start}\")\n\ncoordinate = (start % 768, start // 768)\nprint(f\"Maps to this coordinate: {coordinate}\")\n\nback = 768 * coordinate[1] + coordinate[0]\nprint(f\"And back: {back}\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"790ba043ac474353c29050af2c93780a0b51bc7a"},"cell_type":"markdown","source":"With this in mind, we can map the list of (`start`, `length`) pairs into a list of `pixels` very easily in one line.\nThere are some python gotchas so let's comment a little what does this line do:\n1. Map each pair (`start`, `length`) into a list of `positions` [`start`, `start + 1`, `...` `start + length`] using <span style='color:green'>range</span>\n2. Flatten those lists using a `nested for` (note: python's [nested for](https://stackoverflow.com/questions/17657720/python-list-comprehension-double-for/17657966) looks weird)\n3. Map the list of  `positions` into a list of (`x`, `y`) `coordinates` using `%` and `//` as explained above."},{"metadata":{"trusted":true,"_uuid":"e6073893360e7690fc07753364786248ea245d1e"},"cell_type":"code","source":"pixels = [(pixel_position % 768, pixel_position // 768) \n                            for start, length in pairs \n                            for pixel_position in range(start, start + length)]\npixels[:3]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6afb59e1ecf1b48d36d0700eea687a30c38c42f9"},"cell_type":"markdown","source":"Finally, the following function puts it all together, translating from the RLE string into a list of pixels in a (768, 768) image:"},{"metadata":{"trusted":true,"_uuid":"65db28adfa7322033c485416ad05ccba9951fbe8"},"cell_type":"code","source":"def rle_to_pixels(rle_code):\n    '''\n    Transforms a RLE code string into a list of pixels of a (768, 768) canvas\n    '''\n    rle_code = [int(i) for i in rle_code.split()]\n    pixels = [(pixel_position % 768, pixel_position // 768) \n                 for start, length in list(zip(rle_code[0:-1:2], rle_code[1::2])) \n                 for pixel_position in range(start, start + length)]\n    return pixels\n\n# First three pixels of this particular bounding box:\nrle_to_pixels(df['EncodedPixels']['000155de5.jpg'])[0:3]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"26d8f91a3995fd9f59b80958e134481bb8fa0468"},"cell_type":"markdown","source":"# 2. Plotting the bounding boxes as a mask"},{"metadata":{"_uuid":"10e7a5085e8c196f4e468e79b87481770d0ba42d"},"cell_type":"markdown","source":"The first thing we can do with this list of pixels is to plot them on a monochrome (768, 768) map.  To do that, we need to create a `0` and `1` mask matrix and plot it using [imshow](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.imshow.html)  as we covered in the [previous kernel](https://www.kaggle.com/julian3833/1-loading-and-visualizing-images) of the series:"},{"metadata":{"trusted":true,"_uuid":"7b19a83cd041e9c247bef6fe945c8d19c6f1e05a"},"cell_type":"code","source":"# Create a matrix of shape (768, 768) full of zeros\ncanvas = np.zeros((768, 768))\n\n# numpy arrays can't be indexed by a list of pairs [(x1, y1), (x2, y2)]\n# but it can be indexed with a tuple with ([x1, x2,..., xn], [y1, y2... yn])\n# tuple(zip(*)) does exactly this map.... \n# ref: https://stackoverflow.com/questions/28491230/indexing-a-numpy-array-with-a-list-of-tuples\ncanvas[tuple(zip(*pixels))] = 1\n\nplt.imshow(canvas);","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6515baff434efeddb8083d4717386539fd8fa824"},"cell_type":"markdown","source":"Running the cell below, you can get some random samples of the bounding boxes (Press Ctrl+Enter to run and stay in the cell):"},{"metadata":{"trusted":true,"_uuid":"39eff2da586a0ef59e4c83eccffd8850f1c6a03e"},"cell_type":"code","source":"canvas = np.zeros((768, 768))\npixels = rle_to_pixels(np.random.choice(df['EncodedPixels']))\ncanvas[tuple(zip(*pixels))] = 1\nplt.imshow(canvas);","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e9fd5b4123f1777cbeef40a63355faf0cf1f5cbc"},"cell_type":"markdown","source":"# 3. Masking the images with the bounding boxes"},{"metadata":{"_uuid":"bf29b9398b3b98ddf1395444257128ebed33197c"},"cell_type":"markdown","source":"To finish, we will plot these masks over the corresponding images.  We can apply a mask to an image by just overriding some colors for the relevant pixels (the ones obtained with `rle_to_pixels()`). In this case we saturate completely the red and green coordinates and leave the blue one as it was.\n\nThis cell uses some simple functions (open, array, imshow) covered on the [previous kernel](https://www.kaggle.com/julian3833/1-loading-and-visualizing-the-images) of the series."},{"metadata":{"trusted":true,"_uuid":"107061425d36fa92ebb34781ba4a15a418b9c774"},"cell_type":"code","source":"# An image may have more than one row in the df, \n# Meaning that the image has more than one ship present\n# Here we merge those n-ships into the a continuos rle-code for the image....\ndf = df.groupby(\"ImageId\")[['EncodedPixels']].agg(lambda rle_codes: ' '.join(rle_codes)).reset_index()\n\nload_img = lambda filename: np.array(PIL.Image.open(f\"../input/train_v2/{filename}\"))\n\ndef apply_mask(image, mask):\n    for x, y in mask:\n        image[x, y, [0, 1]] = 255\n    return image\n\nimg = load_img(df.loc[0, 'ImageId'])\nmask_pixels = rle_to_pixels(df.loc[0, 'EncodedPixels'])\nimg = apply_mask(img, mask_pixels)\nplt.imshow(img);","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"97321728a72a468739624ef45c5b5eebd3544714"},"cell_type":"markdown","source":"To summarize, we present some random examples of masked pictures on a grid  (we covered subplot on [this](https://www.kaggle.com/julian3833/1-loading-and-visualizing-the-images) kernel):"},{"metadata":{"trusted":true,"_uuid":"1f0572e96f7563165923d8d468b52f903c68bd5a"},"cell_type":"code","source":"w = 6\nh = 6\n\n_, axes_list = plt.subplots(h, w, figsize=(2*w, 2*h))\n\nfor axes in axes_list:\n    for ax in axes:\n        ax.axis('off')\n        row_index = np.random.randint(len(df)) # take a random row from the df\n        ax.imshow(apply_mask(load_img(df.loc[row_index, 'ImageId']), rle_to_pixels(df.loc[row_index, 'EncodedPixels'])))\n        ax.set_title(df.loc[row_index, 'ImageId'])\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f9e970af955aca764a635d7a99d407d66bac73c6"},"cell_type":"markdown","source":"### References\n* [Airbus ship data vizualization](https://www.kaggle.com/meaninglesslives/airbus-ship-detection-data-visualization) - a nice data visualization and exploratory data analysis kernel. We didn't copy code from there, but it helped us to quick start the project.\n\n### What's next?\nYou can check the [next kernel](https://www.kaggle.com/julian3833/3-basic-exploratory-analysis) of the series, where we explore the data and present the class imbalance problem of the dataset."}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}