{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Resizing & cropping images\n\nMultiple people have pointed out that saving the daaframes to .feather made loading times much faster. There are already multiple datasets and notebooks for this competitions and I mostly based myself on [Hanjoon Choe's very nice notebook](https://www.kaggle.com/hanjoonchoe/resize-and-load-with-feather-format-much-faster) for this one.\n\nTheir are two objectives here:\n- make loading times faster with feather\n- resizing images to reduce the size of the dataset.\n\nOn that second point however, I've seen some people cropping images or resizing the images, but what I saw so far seemd to be rather random.\n\nIn this notebook, using OpenCV2, I'm making a bounding bow around each character to crop only the character, but still have all of it. However, there is a tricky part as these characters are composed on multiple strokes that sometimes do not touch each other. This is taken into account, by simply making a bigger bounding box englobing all the different ones.\n\nThere probably are faster ways of doing this, however since this is only supposed to be done once to prepare and save the data, I went for fastest implementation rather than run time.\n\nI hope you might find this notebook and it's outputs useful!\n\n\n**The dataset can be found [here](https://www.kaggle.com/maxlenormand/cropped-resized-bengaliai-images)**"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport cv2\n\nimport time\nimport os\nfrom tqdm import tqdm\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"start_time = time.time()\ndf_0 = pd.read_parquet('/kaggle/input/bengaliai-cv19/train_image_data_0.parquet')\ncurrent_time = time.time()\nprint(f\"Shape: {df_0.shape} (took {time.time() - start_time}sec to load)\")\n\ndf_1 = pd.read_parquet('/kaggle/input/bengaliai-cv19/train_image_data_1.parquet')\ncurrent_time = time.time()\nprint(f\"Shape: {df_1.shape} (took {time.time() - current_time}sec to load)\")\n\ndf_2 = pd.read_parquet('/kaggle/input/bengaliai-cv19/train_image_data_2.parquet')\ncurrent_time = time.time()\nprint(f\"Shape: {df_2.shape} (took {time.time() - current_time}sec to load)\")\n\ndf_3 = pd.read_parquet('/kaggle/input/bengaliai-cv19/train_image_data_3.parquet')\n\nprint(f\"It took: {time.time() - start_time} to load all 4 datasets\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"HEIGHT = 137\nWIDTH = 236\n\nCROP_SIZE = 100","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"original_img_size = HEIGHT * WIDTH\n\ncropped_img_size = CROP_SIZE * CROP_SIZE\n\nprint(f\"Original shape of images: {original_img_size}\\nCropped & resized shape of images: {cropped_img_size}\")\nprint(f\"Reduction fatio: {np.round(original_img_size/cropped_img_size, 3)}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We reduced the image size by **3.23** times.\n\nThat's not that bad. Especially if, like me, you don't have access to any good GPUs. Reducing the size efficiently while keeping the most information is particularly important and interesting."},{"metadata":{"trusted":true},"cell_type":"code","source":"def crop_and_resize_images(df, resized_df, resize_size = CROP_SIZE):\n    cropped_imgs = {}\n    for img_id in tqdm(range(df.shape[0])):\n        img = resized_df[img_id]\n        _, thresh = cv2.threshold(img, 30, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)\n        contours, _ = cv2.findContours(thresh,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)[-2:]\n        \n        idx = 0 \n        ls_xmin = []\n        ls_ymin = []\n        ls_xmax = []\n        ls_ymax = []\n        for cnt in contours:\n            idx += 1\n            x,y,w,h = cv2.boundingRect(cnt)\n            ls_xmin.append(x)\n            ls_ymin.append(y)\n            ls_xmax.append(x + w)\n            ls_ymax.append(y + h)\n        xmin = min(ls_xmin)\n        ymin = min(ls_ymin)\n        xmax = max(ls_xmax)\n        ymax = max(ls_ymax)\n\n        roi = img[ymin:ymax,xmin:xmax]\n        resized_roi = cv2.resize(roi, (resize_size, resize_size))\n        cropped_imgs[df.image_id[img_id]] = resized_roi.reshape(-1)\n        \n    resized = pd.DataFrame(cropped_imgs).T.reset_index()\n    resized.columns = resized.columns.astype(str)\n    resized.rename(columns={'index':'image_id'},inplace=True)\n    return resized #out_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"resized = df_0.iloc[:, 1:].values.reshape(-1, HEIGHT, WIDTH)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"cropped_df = crop_and_resize_images(df_0, resized, CROP_SIZE)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We get a dataframe just like the original ones, but with a CROP_SIZE * CROP_SIZE number of columns + the image_id column. This significantly reduced the image size"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"cropped_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cropped_df.to_feather(\"train_data_0.feather\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can now save it to feather for later"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_df = pd.read_feather(\"train_data_0.feather\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"resized_sample = sample_df.iloc[:, 1:].values.reshape(-1, CROP_SIZE, CROP_SIZE)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If we simply takes the bounding box, we would end up with multiple different ones for each independant symbol in the image. This isn't what we want. To show that this notebook takes this into account, here is one resized & cropped example with multiple independant symbols in it:"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig, (ax0, ax1) = plt.subplots(1, 2, figsize=(10, 15))\nax0.imshow(resized[329], cmap='Greys')\nax0.set_title('Original image')\nax1.imshow(resized_sample[329], cmap='Greys')\nax1.set_title('Resized & cropped image')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can do all three other datasets"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"del resized\ndel cropped_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"start = time.time()\n\n# dataset 1\nresized_1 = df_1.iloc[:, 1:].values.reshape(-1, HEIGHT, WIDTH)\ncropped_df_1 = crop_and_resize_images(df_1, resized_1, CROP_SIZE)\ncropped_df_1.to_feather(\"train_data_1.feather\")\ndel resized_1\ndel cropped_df_1\nprint(f\"Saved cropped & resized df_1 to feather in {time.time() - start}sec\")\ncurrent = time.time()\n\n# dataset 2\nresized_2 = df_2.iloc[:, 1:].values.reshape(-1, HEIGHT, WIDTH)\ncropped_df_2 = crop_and_resize_images(df_2, resized_2, CROP_SIZE)\ncropped_df_2.to_feather(\"train_data_2.feather\")\ndel resized_2\ndel cropped_df_2\nprint(f\"Saved cropped & resized df_2 to feather in {time.time() - current}sec\")\ncurrent = time.time()\n\n# dataset 3\nresized_3 = df_3.iloc[:, 1:].values.reshape(-1, HEIGHT, WIDTH)\ncropped_df_3 = crop_and_resize_images(df_3, resized_3, CROP_SIZE)\ncropped_df_3.to_feather(\"train_data_3.feather\")\ndel resized_3\ndel cropped_df_3\nprint(f\"Saved cropped & resized df_3 to feather in {time.time() - current}sec\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}