{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction\n\n* The InChI identifier tells us about chemical substance in terms of layers information.\n* So, one of approaches to construct InChI descriptions is to determine all layers one by one.\n* To determine all layeres, it should be need to split train InChI into sublayers.\n\n## Aim of this notebook\n* In this notebook, I describe a bit of information about InChI in training dataset and arrange data.\n* The arranged dataset are avarable from [here](https://www.kaggle.com/wineplanetary/bms-arranged-label)\n\n## Reference and Acknowledgements\n* https://en.wikipedia.org/wiki/International_Chemical_Identifier\n\n## Version\n\nversion 6 : add total numbers of atoms\n\nversion 5 : atom order in appendix\n\nversion 4 : detected which and how many atoms are contained in each chemical formulas and listed in csv\n\nversion 1-3 : initial"},{"metadata":{},"cell_type":"markdown","source":"## Import libraries and load train labels"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport re\nimport itertools\nimport numpy as np\nimport pandas as pd\nfrom tqdm.notebook import tqdm","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_label_path = \"../input/bms-molecular-translation/train_labels.csv\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df = pd.read_csv(train_label_path)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Understanding InChI format\n\n## Version number\n\n* Every InChI starts with the string \"InChI=\" followed by the version number.\n* All InChI in train dataset have value \"InChI=1S\".\n* \"S\" means standard InChIs."},{"metadata":{"trusted":true},"cell_type":"code","source":"version_list = [inchi.split(\"/\")[0] for inchi in tqdm(df[\"InChI\"])]\nprint(\"A version number is always %s\" % set(version_list))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Chemical formula\n\n* In the next layer, InChIs show chemical formula which is the only sublayer that must occur in every InChI.\n* Every chemical substance in train dataset consists of atoms in ['B', 'Br', 'C', 'Cl', 'F', 'H', 'I', 'N', 'O', 'P', 'S', 'Si']"},{"metadata":{"trusted":true},"cell_type":"code","source":"chemical_formula_list = [inchi.split(\"/\")[1] for inchi in tqdm(df[\"InChI\"])]\natom_list_org = [re.split(\"\\d+\", chemical_formula) for chemical_formula in tqdm(chemical_formula_list)]\nbounded_atom_list = set(itertools.chain.from_iterable(atom_list_org))\natom_list = []\nfor bounded_atom in bounded_atom_list:\n    before_char = \"\"\n    for char in bounded_atom:\n        if char.isupper():\n            if before_char.isupper():\n                atom_list.append(before_char)\n            before_char = char\n        elif char.islower():\n            atom_list.append(before_char+char)\n            before_char = \"\"\nprint(\"atoms including in training chemical substances are follows %s\" % set(atom_list))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Other layers\n\n* Other layers also have indispensable information about chemical substances.\n* In InChI, these layers always start with some specific prefix. For example, atom connections sublayers have to start with prefix \"c\"\n* Every InChI layer and sublayer in training dataset starts with one of prefixes ['b', 'c', 'h', 'i', 'm', 's', 't']"},{"metadata":{"trusted":true},"cell_type":"code","source":"prefix_list = [layer[0] for inchi in tqdm(df[\"InChI\"]) for layer in inchi.split(\"/\")[2:]]\nprint(\"prefixes used in training InChI are follows %s\" % set(prefix_list))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Arange Dataset for Training\n\n* Then I split train dataset in some sublayers.\n* I detectd which and how many atoms are contained in each chemical formulas.\n* I convert image_id into absolute path of kaggle notebook."},{"metadata":{"trusted":true},"cell_type":"code","source":"# id to path\ndef id2path(image_id):\n    return \"../input/bms-molecular-translation/train/%s/%s/%s/%s.png\" % (image_id[0], image_id[1], image_id[2], image_id)\n\ndf[\"image_path\"] = df[\"image_id\"].apply(id2path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# separate into elements\nall_df = df.copy()\nprefix_list = [\"c\", \"h\", \"b\", \"t\", \"m\", \"s\", \"i\"]\nformula_list = []\nprefix_val_lists = {prefix: [] for prefix in prefix_list}\nprefix_val_lists.update({\"%s_flg\" % prefix: [] for prefix in prefix_list})\n\nfor inchi in tqdm(df[\"InChI\"]):\n    text_list = inchi.split(\"/\")\n    formula_list.append(text_list[1])\n    for prefix in prefix_list:\n        for text in text_list:\n            if text.startswith(prefix):\n                prefix_val_lists[prefix].append(text)\n                prefix_val_lists[\"%s_flg\" % prefix].append(1)\n                break\n        else:\n            prefix_val_lists[prefix].append(\"\")\n            prefix_val_lists[\"%s_flg\" % prefix].append(0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# reduce formula into atoms\natom_list = [\"C\", \"H\", \"B\", \"Br\", \"Cl\", \"F\", \"I\", \"N\", \"O\", \"P\", \"S\", \"Si\", \"total\"]\ndef split_atom(formula):\n    atom_dict = {atom: 0 for atom in atom_list}\n    now_atom = \"\"\n    now_num = \"\"\n    total_atom = 0\n    for char in formula+\"E\":\n        if char.isupper():\n            if now_atom != \"\":\n                if now_num == \"\":\n                    atom_dict[now_atom] = int(1)\n                    total_atom += 1\n                    now_atom = char\n                else:\n                    atom_dict[now_atom] = int(now_num)\n                    total_atom += int(now_num)\n                    now_atom = char\n                    now_num = \"\"\n            else:\n                now_atom = char\n        elif char.islower():\n            now_atom += char\n        else:\n            if now_atom != \"\":\n                now_num += char\n    atom_dict[\"total\"] = total_atom\n    return atom_dict\n\natom_num_list = [split_atom(inchi.split(\"/\")[1]) for inchi in tqdm(df[\"InChI\"])]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"split_df = df.copy()\nsplit_df[\"formula\"] = formula_list\nfor prefix in prefix_list:\n    split_df[prefix] = prefix_val_lists[prefix]\n    split_df[\"%s_flg\" % prefix] = prefix_val_lists[\"%s_flg\" % prefix]\narranged_df = pd.concat([split_df, pd.DataFrame(atom_num_list)], axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"arranged_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"arranged_df.to_csv(\"arranged_bms_train_labels.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Appendix1 : Atom order\n\nDoes formula part have strict atom ordering?\n\n* A chemical formula of organic matter has strict ordering.\n* \"C\" is placed first, \"H\" is next, then placed alphabetically.\n* The code below shows chemical formulas in train data are ordering strictly by expected order.\n* Note if a chemical formula does not contain \"C\", then formulas are ordered by alphabetically."},{"metadata":{"trusted":true},"cell_type":"code","source":"expected_atom_order = [\"C\", \"H\", \"B\", \"Br\", \"Cl\", \"F\", \"I\", \"N\", \"O\", \"P\", \"S\", \"Si\"]\natom_only_formula_list = [re.sub(\"\\d+\", \"\", formula) for formula in tqdm(chemical_formula_list)]\nfor atom_only_formula, atom_dict in zip(tqdm(atom_only_formula_list), atom_num_list):\n    expected_formula = \"\"\n    for atom in expected_atom_order:\n        if atom_dict[atom] != 0:\n            expected_formula += atom\n    if expected_formula != atom_only_formula:\n        print(\"Unexpected Order !: expected=%s, true=%s\" % (expected_formula, atom_only_formula))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}