\nBefore understanding the competition we need to understand what is InChi.\n\nInChi (prononounced \"en-chee\") stands for International Chemical Identifier. \nIt is a computer based standard for representing chemical structures created by chemists around the world under IUPAC.\n\nSo InChi is a way to query chemical structures using rule based key. \nThis key is divided into different layers which shows following things. \n\n1. Chemical Formula.\n2. How the structure is connected.\n3. Hydrogen atoms in these structure.\n4. Various properties of bonds etc.\n\nsee video below to understand InChi better.\n
\n\n\n\nAs you can see in the video each molecular structure can be represented using InChi. \nAs New compounds are created InChi for that compound is also created along with it. \nBut previously created structures do not have InChi and there are millions of such structures \nso creating InChi for them manually is not possible. \n\nSo, we have to create a machine learning model which creates InChi from Images.\n\n
\n\n\nThere are two \"train\" and \"test\" which contains folder in three layers. \n\ntrain_labels.csv has two columns image_id and InChi code for that image. \nfirst three character of image id shows path of the folder in which image exist. \n\n**Note: some images have some form of corruption**\n\nFor example :- image_id: 000011a64c74 has a path train/0/0/0/000011a64c74.png. \nSame for images in test data\n\nsample_submission also contains same two columns image_id and InChi."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_path = '../input/bms-molecular-translation/train/'\ntest_path = '../input/bms-molecular-translation/test/'\n\ntrain_data= dt.fread('../input/bms-molecular-translation/train_labels.csv').to_pandas()\nsample = dt.fread('../input/bms-molecular-translation/train_labels.csv').to_pandas()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(\"{0}Number of rows in train data: {1}{2}\\n{0}Number of columns in train data: {1}{3}\".format(y_,r_,train_data.shape[0],train_data.shape[1]))\nprint(\"{0}Number of rows in sample data: {1}{2}\\n{0}Number of columns in sample data: {1}{3}\".format(c_,r_,sample.shape[0],sample.shape[1]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
\n
2. Understanding Metrics 📐
\n
\n\nIn this competition submission is evaluated on basis of mean of [Levenshtein distance](https://en.wikipedia.org/wiki/Levenshtein_distance) distance.\n\nIt is used to measure difference between two strings.\n\nNumber of single-character edits (insertions, deletions or substitutions) between 2 strings is levenshtein distance\n\n
\n
Formula for Levenshtein distance
\n
\n\n\n\nwere tail of x is string of all but first character in x and x[n] is nth character of x."},{"metadata":{"trusted":true},"cell_type":"code","source":"def levenshtein_distance(s, t):\n rows = len(s)+1\n cols = len(t)+1\n distance = np.zeros((rows,cols),dtype = int)\n\n for i in range(1, rows):\n for k in range(1,cols):\n distance[i][0] = i\n distance[0][k] = k\n\n for col in range(1, cols):\n for row in range(1, rows):\n if s[row-1] == t[col-1]:\n cost = 0\n else:\n cost = 1\n distance[row][col] = min(distance[row-1][col] + 1, \n distance[row][col-1] + 1, \n distance[row-1][col-1] + cost)\n\n return distance[row][col]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"levenshtein_distance('InChI=1S/C13H20OS/c1-9(2)8-15-13-6-5-10(3)7-12(13)11(4)14/h5-7,9,11,14H,8H2,1-4H3',\n 'InChI=1S/C13H20OS/c1-9(2)8-15-13-6-5-10(3)7-12(13)11(4)14/h5-7,9,11,4H,8H2,1-4H3')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Can you find that one difference ? 😉"},{"metadata":{},"cell_type":"markdown","source":"
"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def show_some_images(df=train_data):\n n = np.random.choice(range(len(df)),size=20)\n plt.figure(figsize=(20,20))\n for i,j in enumerate(n):\n plt.subplot(5,4,i+1)\n show_image(df['image_id'][j],df['InChI'][j])\n plt.axis('off')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"show_some_images()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
\n
5. Distribution of shape of images
\n
"},{"metadata":{"trusted":true},"cell_type":"code","source":"df = train_data.sample(1000)\nheights, widths, percentages = [], [], []\n\nfor image_id in df['image_id']:\n path = get_image_path(image_id)\n image = cv2.imread(path)\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n percentage = len(image[np.where((image==[0,0,0]).all(axis=2))])/len(image[np.where((image!=[0,0,0]).all(axis=2))])\n heights.append(image.shape[0])\n widths.append(image.shape[1])\n percentages.append(percentage)\n\ndf['height'] = heights\ndf['width'] = widths\ndf['black_pixels_%']= percentages","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def distribution1(feature,color1,color2,df=train_data):\n plt.figure(figsize=(15,7))\n \n plt.subplot(121)\n dist = sns.distplot(df[feature],color=color1)\n a = dist.patches\n xy = [(a[i].get_x() + a[i].get_width() / 2,a[i].get_height()) \\\n for i in range(1,len(a)-1) if (a[i].get_height() > a[i-1].get_height() and a[i].get_height() > a[i+1].get_height())]\n \n for i,j in xy:\n dist.annotate(\n s=f\"{i:.3f}\",\n xy=(i,j), \n xycoords='data',\n ha='center', \n va='center', \n fontsize=11, \n color='black',\n xytext=(0,7), \n textcoords='offset points',\n )\n \n qnt = df[feature].quantile([.25, .5, .75]).reset_index(level=0).to_numpy()\n plt.subplot(122)\n box = sns.boxplot(df[feature],color=color2)\n for i,j in qnt:\n box.annotate(str(j)[:4],xy= (j-.05,-0.01),horizontalalignment='center')\n \n print(\"{}Max value of {} is: {} {:.2f} \\n{}Min value of {} is: {} {:.2f}\\n{}Mean of {} is: {}{:.2f}\\n{}Standard Deviation of {} is:{}{:.2f}\"\\\n .format(y_,feature,r_,df[feature].max(),g_,feature,r_,df[feature].min(),b_,feature,r_,df[feature].mean(),m_,feature,r_,df[feature].std()))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntemp_df = train_data.sample(n=100000)\n\nelement_counts = dict()\n\ndef get_main_layer(row):\n return row['InChI'].split('/')[1]\n\ndef get_element_counts(row):\n elements = list()\n numbers = list()\n number = \"\"\n element = \"\"\n for char in row['MainLayer']:\n if (ord(char) >=65 and ord(char) <= 90) or (ord(char) >= 97 and ord(char) <= 122):\n element += char\n if number != \"\":\n numbers.append(int(number))\n if element_counts.get(element):\n element_counts[element] += int(number)\n else:\n element_counts[element] = int(number)\n number = \"\"\n else:\n number += char\n if element != \"\":\n elements.append(element)\n if not element_counts.get(element):\n element_counts[element] = 1\n element = \"\"\n row['elements'] = elements\n row['counts'] = numbers\n return row\n \ntemp_df['MainLayer'] = temp_df.apply(lambda x: get_main_layer(x),axis=1)\ntemp_df = temp_df.apply(lambda x: get_element_counts(x),axis=1)\n\nelement_counts = pd.DataFrame({\"elements\":element_counts.keys(),'counts':element_counts.values()})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
\n
6.1 count of elements in 10000 samples
\n
"},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.bar(element_counts,x='elements',y='counts')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Obviously C, H, O, N, B, F, S are most common elements in structure."},{"metadata":{},"cell_type":"markdown","source":"
\n
7. 3D models
\n
"},{"metadata":{},"cell_type":"markdown","source":"you need cid of structure for getting 3D model which you can get from [here](https://pubchem.ncbi.nlm.nih.gov/#query=InChI%3D1S%2FC3H6O%2Fc1-3(2)4%2Fh1-2H3)"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data['InChI'][3]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def show_3d_models(cid):\n view = py3Dmol.view(width=600, height=1000, query=cid, viewergrid=(2,1), linked=False)\n view.setStyle({'stick': {}}, viewer=(0,0))\n view.setStyle({'sphere': {}}, viewer=(1,0))\n view.setBackgroundColor('#1AD40D', viewer=(0,0))\n view.setBackgroundColor('#1AD40D', viewer=(1,0))\n view.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"show_image(train_data['image_id'][3],train_data['InChI'][3])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cid = 'cid:120539154'\n\nshow_3d_models(cid)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 🚧 Work in Progress 🚧"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}
![](\"https://storage.googleapis.com/kaggle-competitions/kaggle/22422/logos/header.png?t=2021-02-03-02-05-31\")
![](\"https://www.inchi-trust.org/wp/wp-content/uploads/2014/05/techfaq_4.2.png\")
"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install py3Dmol","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Importing Libraries 📗"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import os\nimport gc\nimport cv2\nimport sys\nimport time\nimport tqdm\nimport random\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport datatable as dt\nimport matplotlib.pyplot as plt\nplt.style.use('fivethirtyeight')\n\nimport py3Dmol \n\nimport plotly.express as px\nimport plotly.graph_objs as go\nimport plotly.figure_factory as ff\n\nfrom sklearn.decomposition import PCA\nfrom sklearn.impute import SimpleImputer\nfrom sklearn.model_selection import train_test_split\n\nfrom sklearn.svm import SVC\nfrom xgboost import XGBClassifier\nfrom lightgbm import LGBMClassifier\nfrom sklearn.naive_bayes import GaussianNB\nfrom sklearn.ensemble import BaggingClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.ensemble import AdaBoostClassifier\nfrom sklearn.ensemble import ExtraTreesClassifier\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.linear_model import LogisticRegression\n\nfrom sklearn.manifold import TSNE\n\nfrom colorama import Fore, Back, Style\ny_ = Fore.YELLOW\nr_ = Fore.RED\ng_ = Fore.GREEN\nb_ = Fore.BLUE\nm_ = Fore.MAGENTA\nc_ = Fore.CYAN\nsr_ = Style.RESET_ALL\n\nimport warnings\nwarnings.filterwarnings('ignore')\n\nimport lightgbm as lgb\n\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nimport torch.nn.functional as F\nfrom sklearn.metrics import roc_auc_score\nfrom torch.utils.data import TensorDataset, DataLoader\nfrom sklearn.model_selection import KFold, StratifiedKFold","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"![](\"https://wikimedia.org/api/rest_v1/media/math/render/svg/10554aecc5e56da9be4657acd75b9a67b5e8b394\")
\n\nwere tail of x is string of all but first character in x and x[n] is nth character of x."},{"metadata":{"trusted":true},"cell_type":"code","source":"def levenshtein_distance(s, t):\n rows = len(s)+1\n cols = len(t)+1\n distance = np.zeros((rows,cols),dtype = int)\n\n for i in range(1, rows):\n for k in range(1,cols):\n distance[i][0] = i\n distance[0][k] = k\n\n for col in range(1, cols):\n for row in range(1, rows):\n if s[row-1] == t[col-1]:\n cost = 0\n else:\n cost = 1\n distance[row][col] = min(distance[row-1][col] + 1, \n distance[row][col-1] + 1, \n distance[row-1][col-1] + cost)\n\n return distance[row][col]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"levenshtein_distance('InChI=1S/C13H20OS/c1-9(2)8-15-13-6-5-10(3)7-12(13)11(4)14/h5-7,9,11,14H,8H2,1-4H3',\n 'InChI=1S/C13H20OS/c1-9(2)8-15-13-6-5-10(3)7-12(13)11(4)14/h5-7,9,11,4H,8H2,1-4H3')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Can you find that one difference ? 😉"},{"metadata":{},"cell_type":"markdown","source":"