{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Hi everyone! I've written this data generation process for my own solution but it needs a separate training process since it creates a labeled object detection data. But I don't really have the required time or computing power for a solution like this. So I decided to focus on a trickier solution. But I think this approach might be helpful for you since it gets **~0.71** on the training set.","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport cv2\nimport matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2022-04-05T14:00:23.006591Z","iopub.execute_input":"2022-04-05T14:00:23.006925Z","iopub.status.idle":"2022-04-05T14:00:23.355947Z","shell.execute_reply.started":"2022-04-05T14:00:23.006841Z","shell.execute_reply":"2022-04-05T14:00:23.355274Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So I think almost everyone noticed that the colors are reversed where the shapes overlap. This is actually pretty similar concept with the bit-wise operation called **XOR**.\n\nThat is a logic gate that processes a pair input like:\n\n| A | B | Out |\n|:-:|:-:|:---:|\n| 0 | 0 | 0   |\n| 0 | 1 | 1   |\n| 1 | 0 | 1   |\n| 1 | 1 | 0   |\n\nLet's see that on an example.","metadata":{}},{"cell_type":"code","source":"SIZE = 1500\n\nblanks = np.ones(shape=[SIZE, SIZE, 3]) * 255\nstart_x = 200\nstart_y = 200\nw = 600\nh = 800\ncolor_mask = 0\ncolor = (color_mask, color_mask, color_mask)\ncv2.rectangle(blanks, (start_x, start_y), (start_x + w, start_y + h), color, -1)\n\n\nblanks2 = np.ones(shape=[SIZE, SIZE, 3]) * 255\nstart_x = 400\nstart_y = 400\ncv2.rectangle(blanks2, (start_x, start_y), (start_x + w, start_y + h), color, -1)\n    \nfig, ax = plt.subplots(2, figsize=(10,10))\nax[0].imshow(blanks)\nax[1].imshow(blanks2)","metadata":{"execution":{"iopub.status.busy":"2022-04-05T14:00:23.357566Z","iopub.execute_input":"2022-04-05T14:00:23.357807Z","iopub.status.idle":"2022-04-05T14:00:25.351225Z","shell.execute_reply.started":"2022-04-05T14:00:23.357779Z","shell.execute_reply":"2022-04-05T14:00:25.349527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We created two black rectangles. Let's merge them with bit-wise XOR.","metadata":{}},{"cell_type":"code","source":"xor_merged = cv2.bitwise_xor(blanks, blanks2)\nfig, ax = plt.subplots(1, figsize=(6,6))\n_ = plt.imshow(xor_merged)","metadata":{"execution":{"iopub.status.busy":"2022-04-05T14:00:25.352262Z","iopub.execute_input":"2022-04-05T14:00:25.352483Z","iopub.status.idle":"2022-04-05T14:00:26.305276Z","shell.execute_reply.started":"2022-04-05T14:00:25.352457Z","shell.execute_reply":"2022-04-05T14:00:26.304520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Seems familiar? 💀 Let's extend this implementation with random locations, sizes and rotations.","metadata":{}},{"cell_type":"code","source":"SIZE = 1500\n\nblanks = np.ones(shape=[SIZE, SIZE, 3]) * 255\n\nrects = []\n\nfor i in range(5):\n    start_x = int(np.random.rand() * 1000)\n    start_y = int(np.random.rand() * 1000)\n    w = int(np.random.rand() * 1500) + 400\n    h = int(np.random.rand() * 1500) + 400\n    color_mask = 0\n    color = (color_mask, color_mask, color_mask)\n    \n    rect = blanks.copy()\n    cv2.rectangle(rect, (start_x, start_y), (start_x + w, start_y + h), color, -1)\n    \n    rect_rot = int(np.random.rand() * 45)\n    \n    M = cv2.getRotationMatrix2D((start_x+w/2, start_y+h/2), rect_rot, 1.0)\n    rotated = cv2.warpAffine(rect, M, (1500,1500))\n    \n    rects.append(rotated)\n\nfor rect_i in range(len(rects)):\n    if rect_i < len(rects) - 1:\n        rects[rect_i + 1] = cv2.bitwise_xor(rects[rect_i], rects[rect_i + 1])\n        \nfig, ax = plt.subplots(1, figsize=(6,6))\n_ = plt.imshow(rects[-1])","metadata":{"execution":{"iopub.status.busy":"2022-04-05T14:00:26.307142Z","iopub.execute_input":"2022-04-05T14:00:26.307340Z","iopub.status.idle":"2022-04-05T14:00:27.617334Z","shell.execute_reply.started":"2022-04-05T14:00:26.307316Z","shell.execute_reply":"2022-04-05T14:00:27.616508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I think we are getting there. The data also has circular shapes. Let's do it.","metadata":{}},{"cell_type":"code","source":"circles = [rects[-1]]\nfor i in range(5):\n    rad = int(np.random.rand() * 500) + 150\n    cent_x = int(np.random.rand() * 1500)\n    cent_y = int(np.random.rand() * 1500)\n    color_mask = 0\n    color = (color_mask, color_mask, color_mask)\n    circ = blanks.copy()\n    cv2.circle(circ, (cent_x, cent_y), rad, color, -1)\n    circles.append(circ)\n\nfor circle_i in range(len(circles)):\n    if circle_i < len(circles) - 1:\n        circles[circle_i + 1] = cv2.bitwise_xor(circles[circle_i], circles[circle_i + 1])\n\nfig, ax = plt.subplots(1, figsize=(6,6))\n_ = plt.imshow(circles[-1])","metadata":{"execution":{"iopub.status.busy":"2022-04-05T14:00:27.618545Z","iopub.execute_input":"2022-04-05T14:00:27.618998Z","iopub.status.idle":"2022-04-05T14:00:28.784058Z","shell.execute_reply.started":"2022-04-05T14:00:27.618955Z","shell.execute_reply":"2022-04-05T14:00:28.783259Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We kind of replicated the noisy background in the competition. Finally, we can add our digits.","metadata":{}},{"cell_type":"code","source":"digit_img = cv2.imread(\"../input/ultramnistish-custom-data-generation/5_05668.pgm\")\ndigit_img[digit_img<128] = 0\ndigit_img[digit_img>=128] = 255\ndigit_img = cv2.resize(digit_img, (750, 750), interpolation=cv2.INTER_NEAREST)\nfig, ax = plt.subplots(1, figsize=(5,5))\n_ = plt.imshow(digit_img)","metadata":{"execution":{"iopub.status.busy":"2022-04-05T14:00:28.785324Z","iopub.execute_input":"2022-04-05T14:00:28.786005Z","iopub.status.idle":"2022-04-05T14:00:29.095913Z","shell.execute_reply.started":"2022-04-05T14:00:28.785957Z","shell.execute_reply":"2022-04-05T14:00:29.095122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's add this digit to our pre-generated background image. To do so, we can do a bitwise trick here. We know that the numbers are shown in black in the raw MNIST data we are using. We can mask only the black pixels, and reverse these coordinates in the original image. Let's see what happens when we do that.","metadata":{}},{"cell_type":"code","source":"h, w, c = digit_img.shape\n\nxmin = np.random.randint(0, SIZE-w, 1)[0]\nymin = np.random.randint(0, SIZE-h, 1)[0]\nxmax = xmin + w\nymax = ymin + h\n\nfor i in range(w):\n    for j in range(h):\n        x = xmin + i\n        y = ymin + j\n        if digit_img[j][i].sum() == 0:\n            circles[-1][y][x] = 255 - circles[-1][y][x]\n            \nfig, ax = plt.subplots(1, figsize=(6,6))\n_ = plt.imshow(circles[-1])","metadata":{"execution":{"iopub.status.busy":"2022-04-05T14:00:29.097053Z","iopub.execute_input":"2022-04-05T14:00:29.097284Z","iopub.status.idle":"2022-04-05T14:00:35.323387Z","shell.execute_reply.started":"2022-04-05T14:00:29.097255Z","shell.execute_reply":"2022-04-05T14:00:35.322517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And that's it! You can generate your own custom object detection datasets since you know your bounding box properties of your digits! I know the code is a bit sloppy, I will try to write in a more understandable and simple way when I find time.","metadata":{}}]}