{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Some statistics to choose the input resolution for your InChI predictor\n\nAnalysis of train and test images (random sample) in the [Bristol-Myers Squibb – Molecular Translation Competition](https://www.kaggle.com/c/bms-molecular-translation) to find a suitable input image ratio and resolution.\n\n**Credits: I adapted the crop function from https://www.kaggle.com/markwijkhuizen/advanced-image-cleaning-and-tfrecord-generation (great TFRecord kernel!)**","metadata":{}},{"cell_type":"markdown","source":"Hi everyone!\n\nI've seen different choices of the image resolution and w/h ratio so far, some use squares, some rectangles. I did this analysis to learn more about the images we are given, especially after they are cropped. Note that image width and height are swapped if height > width, for orginal versions as well as cropped versions.\n\n**In the end you can find a summary with the fractions of images that need to be 'shrinked' after cropping for different input resolutions together with the mean 'shrink factor' and more statistics for each resolution.**\n\nIn this summary width / height ratios of around 2 seems to work best. What ratio and image resolution did you choose as input for your InChI prediction model? What were the reasons?\n\nFeel free to comment below and / or leave a vote if you find this kernel helpful :)","metadata":{}},{"cell_type":"code","source":"DEBUG = False\nIMAGE_NUM = 1000 if DEBUG else 1_300_000\nEXAMPLE_NUM = 2 if DEBUG else 5\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom tqdm.notebook import tqdm\ntqdm.pandas()\n\nimport cv2\nimport imageio\nimport os\nimport sys\nimport re\nimport seaborn as sns\nimport time\nimport random\nimport pickle\n       \nSEED = round(time.time())\nprint(f'SEED: {SEED}')\nos.environ['PYTHONHASHSEED'] = str(SEED)\nrandom.seed(SEED)\nnp.random.seed(SEED)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sorting again after sampling seems to make the file access slightly faster for large IMAGE_NUM\ntrain_ids = pd.read_csv('/kaggle/input/bms-molecular-translation/train_labels.csv', dtype={'image_id': 'string', 'InChI': 'string'}).sample(n=IMAGE_NUM).sort_values(by='image_id', ignore_index=True).image_id\ntest_ids = pd.read_csv('/kaggle/input/bms-molecular-translation/sample_submission.csv', usecols=['image_id'], dtype={'image_id': 'string'}).sample(n=IMAGE_NUM).sort_values(by='image_id', ignore_index=True).image_id","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Adapted crop function","metadata":{}},{"cell_type":"markdown","source":"The crop function from the original source above was adapted to ignore noise pixels and thus crop the real molecule structure only without removing the noise first.","metadata":{}},{"cell_type":"code","source":"def crop(img, contour_min_size=2, small_stuff_size=2, small_stuff_dist=5, pad_pixels=1, debug=False, my_figsize=(12,6), horizontal=True):\n    \n    # idea: pad with contour_min_size pixels just in case we cut off\n    #       a small part of the structure that is separated by a missing pixel\n    \n    # rotate counter clockwise to get horizontal images\n    h, w = img.shape\n    if h > w:\n        img = np.rot90(img)\n    \n    if debug:\n        if horizontal:\n            fig, ax = plt.subplots(1,2, figsize=my_figsize)\n        else:\n            fig, ax = plt.subplots(2,1, figsize=my_figsize)\n        ax[0].imshow(img)\n        ax[0].set_title(f'original image, shape: {img.shape}', size=16)\n        \n    _, thresh = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)\n    contours, _ = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)[-2:]\n    \n    small_stuff = []\n    \n    x_min0, y_min0, x_max0, y_max0 = np.inf, np.inf, 0, 0\n    for cnt in contours:\n        if len(cnt) < contour_min_size:  # ignore contours under contour_min_size pixels\n            continue\n        x, y, w, h = cv2.boundingRect(cnt)\n        if w <= small_stuff_size and h <= small_stuff_size:  # collect position of small contours starting with contour_min_size pixels\n            small_stuff.append([x, y, x+w, y+h])\n            continue\n        x_min0 = min(x_min0, x)\n        y_min0 = min(y_min0, y)\n        x_max0 = max(x_max0, x + w)\n        y_max0 = max(y_max0, y + h)\n        \n    x_min, y_min, x_max, y_max = x_min0, y_min0, x_max0, y_max0\n    \n    # enlarge the found crop box if it cuts out small stuff that is very close by\n    for i in range(len(small_stuff)):\n        if small_stuff[i][0] < x_min0 and small_stuff[i][0] + small_stuff_dist >= x_min0:\n             x_min = small_stuff[i][0]\n        if small_stuff[i][1] < y_min0 and small_stuff[i][1] + small_stuff_dist >= y_min0:\n             y_min = small_stuff[i][1]\n        if small_stuff[i][2] > x_max0 and small_stuff[i][2] - small_stuff_dist <= x_max0:\n             x_max = small_stuff[i][2]\n        if small_stuff[i][3] > y_max0 and small_stuff[i][3] - small_stuff_dist <= y_max0:\n             y_max = small_stuff[i][3]\n                             \n    if pad_pixels > 0:  # make sure we get the crop within a valid range\n        y_min = max(0, y_min-pad_pixels)\n        y_max = min(img.shape[0], y_max+pad_pixels)\n        x_min = max(0, x_min-pad_pixels)\n        x_max = min(img.shape[1], x_max+pad_pixels)\n        \n    img_cropped = img[y_min:y_max, x_min:x_max]\n    \n    if debug:\n        ax[1].imshow(img_cropped)\n        ax[1].set_title(f'cropped image, shape: {img_cropped.shape}', size=16)\n        plt.show()\n    \n    return img_cropped","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def check_cropping(image_id, folder='train', my_figsize=(12,6), horizontal=True):\n    print(f'{folder}/{image_id}')\n    file_path =  f'/kaggle/input/bms-molecular-translation/{folder}/{image_id[0]}/{image_id[1]}/{image_id[2]}/{image_id}.png'\n    img = 255 - cv2.imread(file_path, cv2.IMREAD_GRAYSCALE)\n    img = crop(img, debug=True, my_figsize=my_figsize, horizontal=horizontal)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Check cropped train images","metadata":{}},{"cell_type":"code","source":"dummy = [check_cropping(image_id, folder='train') for image_id in train_ids[:EXAMPLE_NUM]]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Check cropped test images","metadata":{}},{"cell_type":"code","source":"dummy = [check_cropping(image_id, folder='test') for image_id in test_ids[:EXAMPLE_NUM]]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Image analysis function\n\nImage width and height are swapped if height > width, for orginal versions as well as cropped versions.","metadata":{}},{"cell_type":"code","source":"pd.set_option('display.float_format', lambda x: '%.2f' % x)\n\ndef analyse_img_sizes(image_ids, folder='train', plots=False, w_large=500, h_large=250, very_large_factor=1.5):\n    ws = []\n    hs = []\n    ws_c = []\n    hs_c = []\n    fs = []\n    for image_id in tqdm(image_ids):\n        file_path =  f'/kaggle/input/bms-molecular-translation/{folder}/{image_id[0]}/{image_id[1]}/{image_id[2]}/{image_id}.png'\n        file_size = os.path.getsize(file_path) \n        fs.append(file_size)\n        img = 255 - cv2.imread(file_path, cv2.IMREAD_GRAYSCALE)  # '255 -' need for cropping to work\n\n        h, w = img.shape\n        if h > w:\n            h, w = w, h\n        ws.append(w)\n        hs.append(h)\n\n        img_cropped = crop(img)\n        h_c, w_c = img_cropped.shape\n        if h_c > w_c:\n            h_c, w_c = w_c, h_c\n        ws_c.append(w_c)\n        hs_c.append(h_c)\n\n    img_info = pd.DataFrame({'image_id': image_ids, 'file_size': fs, 'width': ws, 'width_crop': ws_c, 'height': hs, 'height_crop': hs_c})\n    \n    img_info['area'] = img_info.width * img_info.height\n    img_info['area_crop'] = img_info.width_crop * img_info.height_crop\n    img_info['ratio'] = img_info.width / img_info.height\n    img_info['ratio_crop'] = img_info.width_crop / img_info.height_crop\n        \n    img_info_large = img_info.loc[np.logical_or(img_info.width_crop > w_large, img_info.height_crop > h_large),:]\n    \n    img_info_very_large = img_info.loc[np.logical_or(img_info.width_crop > very_large_factor*w_large, img_info.height_crop > very_large_factor*h_large),:]\n        \n    print(f'statistics for all images')\n    display(img_info.describe())\n    print()\n    print(f\"statistics for 'large' images with cropped width > {w_large} or height > {h_large} ({len(img_info_large)/len(img_info)*100:.3}%):\")\n    display(img_info_large.describe())\n    print()\n    print(f\"statistics for 'very large' images with cropped width > {very_large_factor*w_large} or height > {very_large_factor*h_large} ({len(img_info_very_large)/len(img_info)*100:.3}%):\")\n    display(img_info_very_large.describe())\n    \n    if plots:\n        print()\n        print(f\"plots for 'large' and 'very large' images only\")\n        plot_info =  img_info_large\n        sns.jointplot(data=plot_info, x='width', y='height', kind='hist')\n        sns.jointplot(data=plot_info, x='file_size', y='area', kind='hist')\n        sns.jointplot(data=plot_info, x='width', y='width_crop', kind='hist')\n        sns.jointplot(data=plot_info, x='height', y='height_crop', kind='hist')\n        sns.jointplot(data=plot_info, x='width_crop', y='height_crop', kind='hist')\n        sns.jointplot(data=plot_info, x='ratio', y='ratio_crop', kind='hist')\n        sns.jointplot(data=plot_info, x='area_crop', y='ratio_crop', kind='hist')\n\n    return img_info","metadata":{"_kg_hide-input":false,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train image statistics ","metadata":{}},{"cell_type":"code","source":"train_img_info = analyse_img_sizes(train_ids, folder='train', plots=not DEBUG)\n\nwith open('train_img_info.pkl', 'wb') as handle:\n    pickle.dump(train_img_info, handle)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Test image statistics ","metadata":{}},{"cell_type":"code","source":"test_img_info = analyse_img_sizes(test_ids, folder='test', plots=not DEBUG)\n\nwith open('test_img_info.pkl', 'wb') as handle:\n    pickle.dump(test_img_info, handle)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Images with extremly low width or height\n\nThere are some images with extremly low height after cropping. Checking if crop function made a mistake... Seems legit.","metadata":{}},{"cell_type":"code","source":"def plot_extreme_images(img_info, folder='train', my_figsize = (20, 10)):\n    img_info_width = img_info.sort_values(by='width_crop', ignore_index=True)[:EXAMPLE_NUM]\n    img_info_height = img_info.sort_values(by='height_crop', ignore_index=True)[:EXAMPLE_NUM]\n    \n    print('very low height images (after swapping if height > width)')\n    [check_cropping(image_id, folder=folder, my_figsize=my_figsize, horizontal=False) for image_id in img_info_height.image_id]\n    \n    print('very low width images (after swapping if height > width)')\n    [check_cropping(image_id, folder=folder, my_figsize=my_figsize) for image_id in img_info_width.image_id]\n\nplot_extreme_images(train_img_info)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_extreme_images(test_img_info, folder='test')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Find best input resolution","metadata":{}},{"cell_type":"code","source":"input_ratios = [1, 1.25, 1.5, 1.75, 1.9, 2, 2.1, 2.25, 2.5]\n\ndef get_res(pixels, ratio):\n    pixels = pixels**0.5\n    ratio = ratio**0.5\n    return (round(pixels*ratio), round(pixels/ratio))\n\nbase_pixels = 320*320\ninput_sizes = [get_res(base_pixels, r) for r in input_ratios]\n\nbase_pixels = 448*256\ninput_sizes += [get_res(base_pixels, r) for r in input_ratios]\n\nbase_pixels = 512*256\ninput_sizes += [get_res(base_pixels, r) for r in input_ratios]\n\nbase_pixels = 384*384\ninput_sizes += [get_res(base_pixels, r) for r in input_ratios]\n\npixels = [w*h for w, h in input_sizes]\ninput_ratios = [w/h for w, h in input_sizes]\n\ndef calc_shrink_factors(current_width, current_height, input_size):\n    if current_width < input_size[0] and current_height < input_size[1]:\n        return 1\n    else:\n        return max(input_size[0]/current_width, input_size[1]/current_height)\n\ndef check_resolutions(img_info):\n\n    mean_shrink_factors = []  # mean shrink factor (largest of the two factors to decrease image width and/or height to fit the image into the input size, 1 if image fits already)\n    rms_shrink_factors = []  # root mean square \n    mean_shrink_factors_over_1 = []\n    rms_shrink_factors_over_1 = []\n    fraction_shrinked = []\n\n    for input_size in input_sizes:\n        shrink_factors = np.array([calc_shrink_factors(train_img_info.width_crop[i], train_img_info.height_crop[i], input_size) for i in range(len(train_img_info))])\n        mean_shrink_factors.append(np.mean(shrink_factors))\n        rms_shrink_factors.append(np.mean(shrink_factors**2)**0.5)\n        temp = shrink_factors>1\n        fraction_shrinked.append(np.mean(temp))\n        mean_shrink_factors_over_1.append(np.mean(shrink_factors[temp]))\n        rms_shrink_factors_over_1.append(np.mean(shrink_factors[temp]**2)**0.5)\n        \n    return(pd.DataFrame({'resolution': input_sizes, 'pixels': pixels, 'input_ratio': input_ratios, 'frac_shrinked': fraction_shrinked, \n                        'mean_shr_factor': mean_shrink_factors, 'rms_shr_factor': rms_shrink_factors, 'mean_shr_fac_over_1': mean_shrink_factors_over_1, \n                        'rms_shr_fac_over_1': rms_shrink_factors_over_1}))\n\npd.set_option('display.float_format', lambda x: '%.3f' % x)\n\nprint('train images')\ndisplay(check_resolutions(train_img_info))\n\nprint('test images')\ndisplay(check_resolutions(test_img_info))","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}