{"cells":[{"metadata":{},"cell_type":"markdown","source":"# BMS Mol.Tr.: Approaches (EDA, Denoise, Baseline)\n![](https://storage.googleapis.com/kaggle-competitions/kaggle/22422/logos/header.png?t=2021-02-03-02-05-31)\n\nIn a technology-forward world, sometimes the best and easiest tools are still pen and paper. Organic chemists frequently draw out molecular work with the Skeletal formula, a structural notation used for centuries. Recent publications are also annotated with machine-readable chemical descriptions (InChI), but there are decades of scanned documents that can't be automatically searched for specific chemical depictions. Automated recognition of optical chemical structures, with the help of machine learning, could speed up research and development efforts.\n\nUnfortunately, most public data sets are too small to support modern machine learning models. Existing tools produce 90% accuracy but only under optimal conditions. Historical sources often have some level of image corruption, which reduces performance to near zero. In these cases, time-consuming, manual work is required to reliably convert scanned chemical structure images into a machine-readable format.\n\nBristol-Myers Squibb is a global biopharmaceutical company working to transform patients' lives through science. Their mission is to discover, develop, and deliver innovative medicines that help patients prevail over serious diseases."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nfrom tqdm.auto import tqdm\ntqdm.pandas()\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom sklearn.model_selection import train_test_split\nfrom collections import Counter, defaultdict\nimport cv2, os\nimport skimage.io as io\n\nimport Levenshtein\n\n# ignoring warnings\nimport warnings\nwarnings.simplefilter(\"ignore\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Fast look at the data"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"labels = pd.read_csv('../input/bms-molecular-translation/train_labels.csv')\nss = pd.read_csv('../input/bms-molecular-translation/sample_submission.csv', index_col = 0)\n\nprint('Labels:\\t\\t Len: {};\\tUnique values: {}'.format(\n    len(labels.InChI), labels.InChI.nunique()))\nprint('Samp. subm.:\\t Len: {};\\tUnique values: {}'.format(\n    len(ss.InChI), ss.InChI.nunique()))\n\nprint('*'*60)\nprint('-'*30, 'Labels head', '-'*30)\nprint(labels.head(5))\nprint('-'*25, 'Sample submission head', '-'*25)\nprint(ss.head(5))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check NaN values\nlabels.isna().sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Add all training paths to labels df\nlabels['path'] = labels['image_id'].progress_apply(\n    lambda x: \"../input/bms-molecular-translation/train/{}/{}/{}/{}.png\".format(\n        x[0], x[1], x[2], x))\nlabels.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Sample of 50k images\n\nWe use **io.imread** to read images, as this method is faster than **cv2.imread**."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_sample = labels.sample(50000)\ntrain_sample['img_tensor'] = train_sample['path'].progress_apply(lambda x: io.imread(x))\ntrain_sample = train_sample.reset_index()\ntrain_sample.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In this approach, we save tensors for 50k images, but in the future, when modeling starts, it will be too costly."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize = (15, 15))\nfor i in range(10):\n    image = train_sample.iloc[i, 4]\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    \n    plt.subplot(5, 2, i + 1)\n    plt.imshow(image)\n    plt.title(train_sample.loc[i, 'InChI'][:70] + '...', size = 9)\n    plt.axis('off')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(figsize = (15, 15))\nfor i in range(10):\n    image = 255 - train_sample.iloc[i, 4]\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n    \n    plt.subplot(5, 2, i + 1)\n    plt.imshow(image)\n    plt.title(train_sample.loc[i, 'InChI'][:70] + '...', size = 9)\n    plt.axis('off')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Shape columns\ntrain_sample['img_height'] = train_sample['img_tensor'].progress_apply(lambda x: np.shape(x)[0])\ntrain_sample['img_width'] = train_sample['img_tensor'].progress_apply(lambda x: np.shape(x)[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Image shapes distribution"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"sns.jointplot(x = train_sample['img_width'].astype('float32'), \n              y = train_sample['img_height'].astype('float32'),\n              height = 8, color = '#930077')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The data contains both very small and very large images. Now, let's look at the labels."},{"metadata":{"trusted":true},"cell_type":"code","source":"label_lengths = labels['InChI'].progress_apply(lambda x: len(x))","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"sns.set_style(\"whitegrid\")\nplt.figure(figsize = (10, 6))\nplt.title('Distribution of label length', fontsize = '15')\nsns.kdeplot(label_lengths, fill = True, color = '#930077', \n            edgecolor = 'black', alpha = 0.9)\nplt.xlabel('InChlI length')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The data contains both fairly short and very long formulas."},{"metadata":{"trusted":true},"cell_type":"code","source":"label_splited = labels['InChI'].progress_apply(lambda x: x.split('/'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"num_elem = label_splited.apply(lambda x: len(x))\nnum_elem_count = pd.DataFrame(num_elem.value_counts()).reset_index().sort_values('index')\nnum_elem_count['perc'] = round(num_elem_count['InChI'] / num_elem_count['InChI'].sum() * 100, 3)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"sns.set_style(\"whitegrid\")\nfig, ax = plt.subplots(figsize = (10, 6))\nplt.title('Distribution of labels by number of parts', fontsize = 15)\nsns.barplot(num_elem_count['index'], num_elem_count['InChI'], fill = True, \n            color = '#930077', edgecolor = 'black', alpha = 0.9, ax = ax)\nfor idx, i in enumerate(ax.patches):\n    ax.annotate(\"{} %\".format(num_elem_count.iloc[idx, 2]), (i.get_x() + i.get_width() / 2, i.get_height() + 20000),\n                 ha = 'center', fontsize = 12)\nax.set_xlabel('InChlI number of parts')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The dominant number of labels has 4 parts - almost 80% of 2,424,186. Also, a fairly large part - ~ 17% - is quite long.\n\nLet's look at the most common parts."},{"metadata":{"trusted":true},"cell_type":"code","source":"part_count = Counter([part for i in label_splited for part in i]).most_common()\n\nparts, counts = [], []\nfor part, count in part_count[1:]:\n    parts.append(part)\n    counts.append(count)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We exclude the first part because it is common for all formulas."},{"metadata":{"trusted":true},"cell_type":"code","source":"print('The first part is common for all formulas (must be True): {}'.format(\n    [i[1] for i in part_count][0] == len(labels)))\npart_count[0]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"sns.set_style(\"whitegrid\")\n\nplt.figure(figsize = (20, 2))\nplt.title('TOP-3 most common parts', fontsize = 12)\nsns.barplot(counts[:3], parts[:3], fill = True, \n            color = '#930077', edgecolor = 'black', alpha = 0.9)\nplt.xlabel('Frequency', fontsize = 12)\n\nplt.figure(figsize = (20, 8))\nfrom_ = 4\nto = 18\nfor i in range(2):\n    plt.subplot(1, 2, i + 1)\n    plt.title('Most common parts ({}-{})'.format(from_, to), fontsize = 12)\n    sns.barplot(counts[from_-1:to], parts[from_-1:to], fill = True, \n                color = '#930077', edgecolor = 'black', alpha = 0.9)\n    plt.xlabel('Frequency', fontsize = 12)\n    from_ += 15\n    to += 12\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print('Total unique values in the data: %i' % len(parts))\nprint('Parts with only one value: %i' % len([i for i in counts if i == 1]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Image denoising\n\nThere are many ways to deal with image noise. However, most of them will not be suitable for our task, since they try to delete noise throughout the image and are too redundant in terms of changes. \n\nSome examples of such approaches are presented below."},{"metadata":{"trusted":true},"cell_type":"code","source":"def image_viz(image, title, figsize=(15,8)):\n    \"\"\"\n    Function for image visualization.\n    Takes image tensor, plot title (label) and figsize.\n    \"\"\"\n    plt.figure(figsize = figsize)\n    plt.imshow(image)\n    plt.title(title, size = 16)\n    plt.axis('off')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"image = io.imread(labels.iloc[2, 2])\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\ntitle = 'Original image with noise'\n\nimage_viz(image, title)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from scipy import ndimage\n\nimage = io.imread(labels.iloc[2, 2])\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\nimage_blur = ndimage.gaussian_filter(image, 2)\ntitle = 'Blurred image (bad example)'\n\nimage_viz(image_blur, title)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Gaussian blur (or gaussian smoothing) looks expectedly bad. The principle of total variation denoising (according to the Rudin, Fatemi and Osher algorithm that was proposed by Chambolle) does not look much better."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from skimage.restoration import (denoise_tv_chambolle)\n\nimage = io.imread(labels.iloc[2, 2])\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\nimage_restorated = denoise_tv_chambolle(image, weight = 0.33, multichannel = True)\ntitle = 'Restorated image (still bad example)'\n\nimage_viz(image_restorated, title)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The best solution would be an algorithm that traverses the image and removes only those elements that are less than a certain size (in our case, the points that represent noise). An elegant solution to this problem is presented on [Stack Overflow](https://stackoverflow.com/questions/48681465/how-do-i-remove-the-dots-noise-without-damaging-the-text)"},{"metadata":{"trusted":true},"cell_type":"code","source":"def image_denoising(img_path, dot_size = 2):\n    \"\"\"\n    Source: https://stackoverflow.com/questions/48681465/how-do-i-remove-the-dots-noise-without-damaging-the-text\n    Function for removing noise in the form of small dots. \n    The input takes the path to the image.\n    Increase 'dot_size' parameter to increase the size of areas (dots) to be removed\n    \"\"\"\n    image = io.imread(img_path)\n    _, BW = cv2.threshold(image, 127, 255, cv2.THRESH_BINARY_INV)\n    nlabels, labels, stats, _ = \\\n        cv2.connectedComponentsWithStats(BW, None, None, None, \n                                         8, cv2.CV_32S)\n    sizes = stats[1:, -1]\n    image2 = np.zeros((labels.shape), np.uint8)\n    for i in range(0, nlabels - 1):\n        if sizes[i] >= dot_size: \n            image2[labels == i + 1] = 255\n    image = cv2.bitwise_not(image2)\n    return image","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"denoise_image = image_denoising(labels.iloc[2, 2])\ntitle = 'Image without dots (the best example)'\n\nimage_viz(denoise_image, title)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Simple baseline\n\nBefore we start doing serious modeling, it is necessary to establish a level above which our model will be considered effective. For this task, there are many different approaches to create a baseline. We'll start with the \"most frequent part\" principle. Over time, we may replace it with a more thoughtful and correct baseline, but for a start, I think it's a good idea."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"print('Unique values in sample submission: %i' % ss.nunique())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The baseline accuracy established by the competition is 109.6. For this, the same short label is assigned to all test images."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"ss.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Data Frame with label parts\nlabel_parts = pd.DataFrame.from_records(label_splited.values)\nlabel_parts.columns = np.array(label_parts.columns + 1)\nlabel_parts = label_parts.add_prefix('Part_')\n\nlabel_parts.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I think it would be a good idea to take a look at the variation in the length of each part. To do this, let's calculate some basic statistics."},{"metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"means = []\nvariations = []\nmins = []\nmaxs = []\nfor i in range(len(label_parts.columns)):\n    lengths = label_parts.iloc[:, i].dropna().progress_apply(lambda x: len(x))\n    means.append(round(lengths.mean(), 2))\n    variations.append(round(lengths.var(), 2))\n    mins.append(lengths.min())\n    maxs.append(lengths.max())\n\ndf_stat = pd.DataFrame({'Part': label_parts.columns,\n                        'Mean length': means,\n                        'Length variation': variations,\n                        'Count': label_parts.count().values,\n                        'Min length': mins,\n                        'Max length': maxs})","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"all_lengths = label_parts.progress_apply(lambda x: [len(i) for i in x.dropna()])\n\nsns.set_style(\"whitegrid\")\nplt.figure(figsize = (20, 20))\nfor i in range(11):\n    plt.subplot(4, 3, i + 1)\n    plt.title('Distribution of length (Part_{})'.format(i+1), fontsize = '10')\n    sns.kdeplot(all_lengths[i], fill = True, color = '#930077', \n                edgecolor = 'black', alpha = 0.9)\n    plt.xlabel('')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"df_stat","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Wow, the variation of the third part is extremely big (min length is 4 and max length is 267)! This is not good news, but let's not worry about it for now.\n\nIt's time to create our new formula! It will be assembled from the most frequent parts (what will chemists say about this Frankenstein?)."},{"metadata":{"trusted":true},"cell_type":"code","source":"baseline_label = ''\nn = 7\nfor i in range(n):\n    baseline_label += '/' + label_parts.iloc[:, i].value_counts().index[0]\nbaseline_label = baseline_label[1:]\n\nprint('Baseline {} part label:'.format(n))\nprint(baseline_label)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def Levenshtein_dist(y_true, y_pred):\n    \"\"\"\n    Function that calculates the average Levenshtein distance for all data.\n    Takes arrays of true (y_true) and predicted (y_pred) labels as input.\n    \"\"\"\n    values = []\n    for y_true, y_pred in zip(y_true, y_pred):\n        values.append(Levenshtein.distance(y_true, y_pred))\n    return np.mean(values)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print('Train Levenshtein distance: %.4f' \n      % Levenshtein_dist(labels['InChI'], [baseline_label] * len(labels)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Not bad. It looks better than the basic competition baseline. I think this is a good starting level that we need to exceed."},{"metadata":{"trusted":true},"cell_type":"code","source":"ss['InChI'] = [baseline_label] * len(ss['InChI'])\nss.to_csv(\"submission.csv\")\nss","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## WORK IN PROGRESS..."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}