{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction\nA desirable image augmentation is the horizontal flipping. This means\n- flipping the image horizontally\n- changing the labels for each car as...\n  - y=-y\n  - yaw=-yaw (the actual yaw as defined e.g. [here](https://www.google.com/imgres?imgurl=https%3A%2F%2Fwww.researchgate.net%2Fpublication%2F283951857%2Ffigure%2Ffig2%2FAS%3A319897696522260%401453280962401%2FRoll-pitch-yaw-angles-of-cars-and-other-land-based-vehicles-10.png&imgrefurl=https%3A%2F%2Fwww.researchgate.net%2Ffigure%2FRoll-pitch-yaw-angles-of-cars-and-other-land-based-vehicles-10_fig2_283951857&docid=sZPJ9uehdFLqlM&tbnid=D-qwg3dzlKNduM%3A&vet=10ahUKEwjot8PcyYXnAhVVPcAKHZxcCpgQMwhbKAYwBg..i&w=600&h=401&bih=846&biw=1707&q=yaw%20car%20&ved=0ahUKEwjot8PcyYXnAhVVPcAKHZxcCpgQMwhbKAYwBg&iact=mrc&uact=8))\n  - roll=-roll\n\n## Problem\nHowever, as [some people have noticed](https://www.kaggle.com/c/pku-autonomous-driving/discussion/125591), there is an issue: If you project the xyz coordinates to 2D, the resulting uv-coordinates will not match the cars in the image anymore. The reason is, that the principal point of the camera does not lie exactly at the image center, but slightly off.\n\n## Solution\nInstead of a simple horizontal flipping, we need to flip the image exactly at the camera principal point (see [explanation of principal point](http://ksimek.github.io/2013/08/13/intrinsic/)). Therefore, I provide the function below. It flips the image with a precision of 0.5 pixels without using interpolation."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\nimport cv2\n\ndef flip_hor_at_u(img, cx, flag_plot=False):\n    # determine line for flipping rounded to 0.5 pixel\n    cx_rounded = np.round(cx * 2).astype(np.int)\n    u_flip = cx_rounded / 2\n\n    # determine new width\n    height, width, nchannels = img.shape\n    if cx_rounded % 2 == 1:\n        # if flipping line lies between two pixels...\n        width_left = np.ceil(u_flip)\n        width_right = np.floor(width - u_flip)\n        width_new = 2 * max(width_left, width_right)\n        pad_left = width_new / 2 - width_left\n        pad_right = width_new / 2 - width_right\n    else:\n        # if flipping line lies at a pixel...\n        width_left = np.round(u_flip)\n        width_right = np.round(width - u_flip - 1)\n        width_new = 2 * max(width_left, width_right) + 1\n        pad_left = (width_new - 1) / 2 - width_left\n        pad_right = (width_new - 1) / 2 - width_right\n\n    # create new image and flip horizontally\n    bg = img.mean(1, keepdims=True).astype(img.dtype)\n    bg_left = np.repeat(bg, pad_left, axis=1)\n    bg_right = np.repeat(bg, pad_right, axis=1)\n    img_padded = np.hstack((bg_left, img, bg_right))\n    img_padded_flipped = img_padded[:, ::-1, :]\n\n    # crop back to org size s.t. cx=const\n    dim_right = width_new-pad_right\n    img_cropped = img_padded_flipped[:,\n                  pad_left.astype(np.int):dim_right.astype(np.int)\n                  :]\n    width_cropped = img_cropped.shape[1]\n    assert width_cropped== width, \"width changed during flipping ?!\"\n\n    # plot images\n    if flag_plot:\n        fig_width, fig_height = max(4,width/100), max(6, 2*height/100)\n        fig,ax = plt.subplots(2,1, sharey=True, sharex=True, figsize=(fig_width,fig_height))\n        ax[0].imshow(img)\n        ax[1].imshow(img_cropped)\n        for axi in ax:\n            axi.vlines(u_flip, 0, height-1)\n        plt.show()\n\n    return img_cropped","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Examples\n## Small dummy ones, to see effect"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# flip image at u=2\nimg = np.random.rand(4, 6, 3) * 255\nimg = img.astype(np.uint8)\nimg_flipped = flip_hor_at_u(img, cx=2, flag_plot=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# flip image at u=1.5\nimg = np.random.rand(4, 6, 3) * 255\nimg = img.astype(np.uint8)\nimg_flipped = flip_hor_at_u(img, cx=1.5, flag_plot=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Actual images used in competition"},{"metadata":{"trusted":true},"cell_type":"code","source":"path_img = '/kaggle/input/pku-autonomous-driving/train_images/ID_00ac30455.jpg'\ncam_K = np.array([[2304.5479, 0, 1686.2379],\n                  [0, 2305.8757, 1354.9849],\n                  [0, 0, 1]], dtype=np.float32)\ncx = cam_K[0, 2]\nimg = cv2.imread(path_img)[:,:,::-1] #BGR to RGB\nimg_flipped = flip_hor_at_u(img, cx, flag_plot=True)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}