{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Chemical formula prediction with less than 1 Levenshtein distance\n\n## Introduction\n* InChI descirbes many molecular information in terms of layers.\n* So one of approaches to construct InChI descriptions is to determine all layers one by one.\n* I first determined molecular chiralities from molecular images using CNN (see this [notebook](https://www.kaggle.com/wineplanetary/step-by-step-detection-1-99-acc-chirality)).\n* Next, I determine chemical formulas using CNN.\n* The dataset used in this notebook is avairable from [here](https://www.kaggle.com/wineplanetary/bms-arranged-label), which is produced by [this notebook](https://www.kaggle.com/wineplanetary/understanding-inchi-format-and-arrange-train-label)\n\n\n## Chemical Formula\n* I have already reduced all InChIs in train dataset into atoms by [this notebook](https://www.kaggle.com/wineplanetary/understanding-inchi-format-and-arrange-train-label)\n* I determine chemical formulas from images by solving multivariate regression problem\n* I achieved that <span style=\"color: red; font-weight: bold;\">less than 1 Levenshtein distance</span>  of chemical formulas with half of train dataset!\n* More dataset will decrease the Levenshtein distance.\n\n## References and Acknowledgements\n* Dataset\n * https://www.kaggle.com/wineplanetary/bms-arranged-label\n* Notebook\n * https://www.kaggle.com/wineplanetary/understanding-inchi-format-and-arrange-train-label\n * https://www.kaggle.com/wineplanetary/step-by-step-detection-1-99-acc-chirality"},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport glob\nimport random\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom tqdm.notebook import tqdm\nfrom sklearn.model_selection import train_test_split\n\nimport tensorflow as tf\n\nimport keras\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.experimental import CosineDecay\nfrom tensorflow.keras.callbacks import ModelCheckpoint\nfrom tensorflow.keras.models import Sequential, load_model\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.layers import GlobalAveragePooling2D, Dense, Conv2D, BatchNormalization, MaxPool2D\n\nimport Levenshtein","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class CFG:\n    seed = 12345\n    batch_size = 32\n    init_lr = 1e-3\n    epochs = 5\n    img_size = 380\n    class_mode = \"raw\"\n    n_CLASS = 12\n    interpolation = \"nearest\"\n    color_mode = \"grayscale\"\n    shuffle = True\n    num_data = 1000 # change to 1000000\n    test_size = 0.05","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# seed\ntf.random.set_seed(CFG.seed)\nnp.random.seed(CFG.seed)\nrandom.seed(CFG.seed)\nos.environ[\"PYTHONHASHSEED\"] = str(CFG.seed)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_dir = \"../input/bms-molecular-translation/train\"\nchbtmspath = \"../input/bms-arranged-label/arranged_bms_train_labels.csv\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_org = pd.read_csv(chbtmspath)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# I was truncate data to 1000000 due to the calculation time, but more data would be increase your accuracy\natom_list = [\"C\", \"H\", \"B\", \"Br\", \"Cl\", \"F\", \"I\", \"N\", \"O\", \"P\", \"S\", \"Si\"]\ndata = data_org[[\"image_path\", *atom_list]].copy()\ndata = data.sample(n=CFG.num_data, random_state=CFG.seed)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def train_trans_func(image):\n    return image / 255.\n\ndef val_trans_func(image):\n    return image / 255.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"datagen_train = ImageDataGenerator(preprocessing_function = train_trans_func)\ndatagen_val = ImageDataGenerator(preprocessing_function = val_trans_func)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_train_set(train):\n    train_set = datagen_train.flow_from_dataframe(train,\n                                                  directory = None,\n                                                  seed = CFG.seed,\n                                                  x_col = \"image_path\",\n                                                  y_col = atom_list,\n                                                  target_size = (CFG.img_size, CFG.img_size),\n                                                  class_mode = CFG.class_mode,\n                                                  interpolation = CFG.interpolation,\n                                                  shuffle = CFG.shuffle,\n                                                  color_mode = CFG.color_mode,\n                                                  batch_size = CFG.batch_size)\n    return train_set\n    \ndef create_val_set(val):\n    val_set = datagen_val.flow_from_dataframe(val,\n                                              directory = None,\n                                              seed=CFG.seed,\n                                              x_col = \"image_path\",\n                                              y_col = atom_list,\n                                              target_size = (CFG.img_size, CFG.img_size),\n                                              class_mode = CFG.class_mode,\n                                              interpolation = CFG.interpolation,\n                                              shuffle = CFG.shuffle,\n                                              color_mode = CFG.color_mode,\n                                              batch_size = CFG.batch_size)\n    return val_set","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train, val = train_test_split(data, test_size=CFG.test_size, random_state=CFG.seed)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"valid_set = create_val_set(val)\ntrain_set = create_train_set(train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_model():\n    model = Sequential()\n    model.add(Conv2D(16, 3, activation=\"relu\", padding=\"same\", input_shape=(CFG.img_size, CFG.img_size, 1)))\n    model.add(Conv2D(16, 3, activation=\"relu\", padding=\"same\"))\n    model.add(BatchNormalization())\n    model.add(MaxPool2D(pool_size=(2, 2), strides=None, padding=\"valid\"))\n    model.add(Conv2D(32, 3, activation=\"relu\", padding=\"same\"))\n    model.add(Conv2D(32, 3, activation=\"relu\", padding=\"same\"))\n    model.add(BatchNormalization())\n    model.add(MaxPool2D(pool_size=(2, 2), strides=None, padding=\"valid\"))\n    model.add(Conv2D(64, 3, activation=\"relu\", padding=\"same\"))\n    model.add(Conv2D(64, 3, activation=\"relu\", padding=\"same\"))\n    model.add(BatchNormalization())\n    model.add(MaxPool2D(pool_size=(2, 2), strides=None, padding=\"valid\"))\n    model.add(Conv2D(64, 3, activation=\"relu\", padding=\"same\"))\n    model.add(Conv2D(64, 3, activation=\"relu\", padding=\"same\"))\n    model.add(BatchNormalization())\n    model.add(GlobalAveragePooling2D())\n    model.add(Dense(CFG.n_CLASS, activation=\"relu\"))\n    return model\n\nmodel = create_model()\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"step_size_train = train_set.n // train_set.batch_size\nstep_size_valid = valid_set.n // valid_set.batch_size","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = create_model()\nmodel.load_weights(\"../input/bms-models/bms_formula_model.h5\")\n\nlr = CosineDecay(initial_learning_rate = CFG.init_lr,\n                 decay_steps = step_size_train * CFG.epochs)\n\nmodel.compile(optimizer = Adam(learning_rate=lr),\n              loss=\"mean_squared_error\",\n              metrics=[\"mean_squared_error\"])\n\ncheckpoint_cb = ModelCheckpoint(\"bms_formula_best_model.h5\",\n                                save_best_only=True,\n                                monitor=\"val_loss\",\n                                mode=\"min\")\n\nhistory = model.fit(train_set,\n                    validation_data = valid_set,\n                    epochs = CFG.epochs,\n                    batch_size = CFG.batch_size,\n                    steps_per_epoch = step_size_train,\n                    validation_steps = step_size_valid,\n                    callbacks=[checkpoint_cb])\n\nmodel.save(\"bms_formula_model.h5\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## check model accuracy in a simple way"},{"metadata":{"trusted":true},"cell_type":"code","source":"def arr2formula(atom_arr):\n    formula = \"\"\n    for atom, num in zip(atom_list, atom_arr):\n        if num > 1:\n            formula += \"%s%s\" % (atom, int(num))\n        elif num == 1:\n            formula += atom\n    return formula\n\nlsval = 0.\n\nfor i, (test, formula_list) in enumerate(tqdm(train_set)):\n    if i >= len(train_set):\n        break\n    predicts = np.round(model.predict(test))\n    for predict, formula in zip(predicts, formula_list):\n        predict_formula = arr2formula(predict)\n        true_formula = arr2formula(formula)\n        if i < 10:\n            print(\"predicted, true = %s, %s\" % (predict_formula, true_formula))\n        lsval += Levenshtein.distance(predict_formula, true_formula) / (CFG.num_data * (1 - CFG.test_size))\n\nprint(\"The Levenshtein distance of train data is %s\" % lsval)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}