{"cells":[{"metadata":{},"cell_type":"markdown","source":"# About\n\n`InChI` string can be split by `/` into some parts (max number of parts in training data is 11). The first part is the format which is uniform string(`InChI=1S`) in this competition, and the second part is **chemical formula** which represents **the number of atoms** in each molecular.\n\nLet me take `InChI=1S/C13H20OS/c1-9(2)8-15-13-6-5-10(3)7-12(13)11(4)14/h5-7,9,11,14H,8H2,1-4H3`(`image_id`: `000011a64c74`) as an example.\nThe second part is `C13H20OS`, which means that the molecular have 13`Carbon`s, 20`Hydrogen`s, a `Oxygen` and a `Sulfur`.\n\nAll the parts of `InChI` have **variable length** except for the second one (chemical formula) because the kind of atoms in training data is **limited** to 12(`B`, `Br`, `C`, `Cl`, `F`, `H`, `I`, `N`, `O`, `P`, `S`, and `Si`). Therefore, we can represents a chemical formula by a **fixed length** vector and treat chemical formula prediction task as **multi-output regression task**.\n\nI'm sharing **tranining process** of the task in another notebook(Note: I used **only 4%** of data for training):  \nhttps://www.kaggle.com/ttahara/bms-mt-chemical-formula-regression-training\n  \n@wineplanetary 's notebook( [Step by Step 2: LS dist < 1 chemical formula](https://www.kaggle.com/wineplanetary/step-by-step-2-ls-dist-1-chemical-formula) ) have already showed us that chemical formula prediction is relatively easy. Thanks!  \n\nIn this notebook, to make the difference, I try solving the competition task by utilizing predicted chemical formula.\n\n<br>\n\nNOTE: I skipped calculating OOF score because of Memory Error. Please see [1st version](https://www.kaggle.com/ttahara/bms-mt-chemical-formula-regression-inference?scriptVersionId=57399601)."},{"metadata":{},"cell_type":"markdown","source":"# Prepare"},{"metadata":{},"cell_type":"markdown","source":"## import"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport re\nimport gc\nimport sys\nimport yaml\nimport copy\nimport random\nimport shutil\nimport typing as tp\nfrom pathlib import Path\n\nimport Levenshtein\nimport numpy as np\nimport pandas as pd\nfrom tqdm.notebook import tqdm, trange\n\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.metrics import mean_squared_error\nfrom sklearn.preprocessing import StandardScaler\n\n\nfrom matplotlib import pyplot as plt\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import cv2\nimport albumentations\nfrom albumentations.core.transforms_interface import ImageOnlyTransform, DualTransform\nfrom albumentations.pytorch import ToTensorV2\n\nimport torch\nfrom torch import nn\nfrom torch.utils import data\n\nfrom cuml.cluster import KMeans\n\nsys.path.append(\"../input/timm-pytorch-image-models/pytorch-image-models-master\")\nimport timm","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ROOT = Path.cwd().parent\nINPUT = ROOT / \"input\"\nDATA = INPUT / \"bms-molecular-translation\"\nTRAIN = DATA / \"train\"\nTRAIN_224 = INPUT / \"bms-molecular-224px-jpg-padded\" / \"train\"\nTEST = DATA / \"test\"\nTEST_224 = INPUT / \"bms-molecular-224px-jpg-padded\" / \"test\"\n\nTRAINING_OUTPUTS = INPUT / \"bms-mt-chemical-formula-regression-training\"\nTMP = ROOT / \"tmp\"\nTMP.mkdir(exist_ok=True)\n\nRANDAM_SEED = 1086\nDEBUG_RUN = False\n\nFOLDS = [0, 1, 2, 3, 4]\nN_FOLD = len(FOLDS)\n\nTARGETS = [\n    'B', 'Br', 'C', 'Cl',\n    'F', 'H', 'I', 'N',\n    'O', 'P', 'S', 'Si']\nN_TARGETS = len(TARGETS)\nMAX_INCHI_N_SPLITS = 11\n\nN_CLUSTER = 1000","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## read competition data and the chemical formula prediction result"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(DATA / \"train_labels.csv\")\nsmpl_sub = pd.read_csv(DATA / \"sample_submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"if DEBUG_RUN:\n    smpl_sub = smpl_sub.iloc[:12800,].reset_index(drop=True)\nprint(smpl_sub.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_fml = pd.read_pickle(TRAINING_OUTPUTS / \"train_formula_mlskf_5fold.pkl\")\noof_fml = pd.read_csv(TRAINING_OUTPUTS / \"oof_prediction.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## preprocess"},{"metadata":{},"cell_type":"markdown","source":"### split InChi into 11 parts"},{"metadata":{"trusted":true},"cell_type":"code","source":"train[\"InChI_length\"] = train[\"InChI\"].str.len()\ntrain[\"InChI_n_splits\"] = train[\"InChI\"].str.count(\"/\") + 1\n\nprint(train[[\"InChI_length\", \"InChI_n_splits\"]].describe())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"split_inchi_list = [[] for inchi_pos in range(MAX_INCHI_N_SPLITS)] \n\nfor inchi_str, n_splits in tqdm(train[[\"InChI\", \"InChI_n_splits\"]].values):\n    split_inchi = inchi_str.split(\"/\")\n    \n    for inchi_pos in range(n_splits):\n        split_inchi_list[ inchi_pos].append(split_inchi[inchi_pos])\n        \n    for inchi_pos in range(n_splits, MAX_INCHI_N_SPLITS):\n        split_inchi_list[inchi_pos].append(\"\")\n        \n\nfor inchi_pos in range(MAX_INCHI_N_SPLITS):\n    train[f\"InChI_{inchi_pos}\"] = split_inchi_list[inchi_pos]\n\ndel split_inchi_list","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Analysis of chemical formula prediction result"},{"metadata":{},"cell_type":"markdown","source":"## correlation between predicted and target value for atoms"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig = plt.figure(figsize=(6 * 4, 6 * 3))\nfig.subplots_adjust(wspace=0.4, hspace=0.6)\n\nfor i, atom in enumerate(TARGETS):\n    ax = fig.add_subplot(3, 4, i + 1)\n    pred = oof_fml[atom].values\n    target = train_fml[atom].values\n    mse = mean_squared_error(target, pred)\n    corr = np.corrcoef([target, pred])[0, 1]\n    ax.scatter(pred, target, alpha=0.3)\n    ax.set_xlabel(\"Predicted Value\")\n    ax.set_ylabel(\"Target Value\")\n    ax.set_title(\n        f\"atom: {atom}, oof_mse: {mse:.3f}, corr: {corr:.3f}\",)\n    \ndel pred; del target\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## levenshtein distance for chemical formula"},{"metadata":{"trusted":true},"cell_type":"code","source":"def arr2formula(atom_arr):\n    \"\"\"\n    Convert predicted number of atoms to chemical formula\n    \n    atom_arr: 1d array shape of (12, )\n    \"\"\"\n    atom_arr = np.round(atom_arr).astype(int)\n    \n    # order when C exists: [\"C\", \"H\", 'B', 'Br', 'Cl', 'F', 'I', 'N', 'O', 'P', 'S', 'Si']\n    idx_order1 = [2, 5, 0, 1, 3, 4, 6, 7, 8, 9, 10, 11]\n   #  order when C doesn't exist: ['B', 'Br', \"C\", 'Cl', 'F', 'H', 'I', 'N', 'O', 'P', 'S', 'Si']\n    idx_order2 = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]\n\n    idx_order = idx_order1 if atom_arr[2]  > 0 else idx_order2\n        \n    elem_list = []\n    for idx in idx_order:\n        if atom_arr[idx] < 1:\n            continue\n        elif atom_arr[idx] == 1:\n            elem_list.append(TARGETS[idx])\n        else:\n            elem_list.append(f\"{TARGETS[idx]}{atom_arr[idx]}\")\n    \n    formula = \"\".join(elem_list)\n    return formula","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"oof_fml[\"pred_formula\"] = [arr2formula(atom_arr) for atom_arr in tqdm(oof_fml[TARGETS].values)]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# oof_fml[\"LS_dist\"] =  [\n#     Levenshtein.distance(fml, pred_fml) for fml, pred_fml in tqdm(oof_fml[[\"formula\", \"pred_formula\"]].values)]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# print(oof_fml[[\"formula\", \"pred_formula\", \"LS_dist\"]].head(10))\n\nprint(\n\"\"\"\n         formula   pred_formula  LS_dist\n0       C13H20OS       C13H20OS        0\n1       C21H30O4       C20H30O4        1\n2     C24H23N5O4     C23H23N5O4        1\n3    C17H24N2O4S    C17H23N2O4S        1\n4    C10H19N3O2S   C10H18BrN3O2        4\n5  C19H22Br2N2O2  C19H22Br2N2O2        0\n6    C17H10BrN3O      C17H10N3O        2\n7    C21H21N5O2S    C21H21N5O2S        0\n8    C13H18N2O5S    C13H18N2O5S        0\n9   C13H15BrN2O3   C13H15BrN2O3        0\n\"\"\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# mode_formula = oof_fml.formula.mode()[0]\n# print(\"mode formula:\", mode_formula)\n# ls_dist_by_mode = sum([\n#     Levenshtein.distance(formula, mode_formula) for formula in oof_fml.formula]) / len(oof_fml)\n\n# print(f\"oof Levenshtein distance for Chemical formula **by mode**: {ls_dist_by_mode:.4f}\")\n\nprint(\n\"\"\"\nmode formula: C15H22N2O2\noof Levenshtein distance for Chemical formula **by mode**: 5.3457\n\"\"\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# print(f\"oof Levenshtein distance for chemical formula by regression model: {oof_fml['LS_dist'].mean():.4f}\")\n\n\nprint(\"oof Levenshtein distance for chemical formula by regression model: 0.9503\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"oof Levenshtein distance for chemical formula is 0.95.(You may get 5.3457 if you use mode string(`C15H22N2O2`). )\n\nPlease note that this result arises from the model which was trained **by only 4% of training data**.  \nI've already confirmed in my local environment that models trained by more data can achinve more precise result ;)"},{"metadata":{},"cell_type":"markdown","source":"# Clustering Apprach by predicted number of atoms\n\nI think chemical formula can limit `InChI`'s degrees of freedom.\n\nFor example, total number of atoms excluding `H` has relatively high correlation with length of the whole `InChI` and `InchI_2`(which represents atom connections) as the following plots.\n\nOne possible approach that comes to mind right away is clustering moleculars by predicted chemical formula (number of atoms).\nMaybe, using mode string of each cluster can get a better result than mode string of all training data."},{"metadata":{"trusted":true},"cell_type":"code","source":"# # true value\ntrain_fml[\"n_atoms_ex_H\"] = train_fml[\"n_atoms\"] - train_fml[\"H\"]\n\n3 # pred value\noof_fml[\"n_atoms\"] = oof_fml[TARGETS].sum(axis=1)\noof_fml[\"n_atoms_ex_H\"] = oof_fml[\"n_atoms\"] - oof_fml[\"H\"]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig = plt.figure(figsize=(12, 5))\n\nn_atoms_ex_H = train_fml[\"n_atoms_ex_H\"].values\ninchi_len = train[\"InChI_length\"].values\ninchi_2_len = train[\"InChI_2\"].str.len().values\n\nax = fig.add_subplot(121)\ncorr = np.corrcoef(n_atoms_ex_H, inchi_len)[0, 1]\nax.set_title(f\"Total number of atoms ex.H vs. InChI length(corr: {corr:.4f})\")\nax.set_xlabel(\"number of atoms ex, H\")\nax.set_ylabel(\"InChI length\")\n_ = ax.scatter(n_atoms_ex_H, inchi_len, alpha=0.3)\n\nax = fig.add_subplot(122)\ncorr = np.corrcoef(n_atoms_ex_H, inchi_2_len)[0, 1]\nax.set_title(f\"Total number of atoms ex. H vs. InChI_2 length(corr: {corr:.4f})\")\nax.set_xlabel(\"number of atoms ex. H\")\nax.set_ylabel(\"InChI_2 length\")\n_ = ax.scatter(n_atoms_ex_H, inchi_2_len, alpha=0.3)\n\ndel n_atoms_ex_H; del inchi_len; del inchi_2_len;","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here I use 3 values (predicted number of atoms of `C`, `H` and total number of atoms **excluding `H`**) as a feature vector."},{"metadata":{"trusted":true},"cell_type":"code","source":"def calc_each_LS(y_true, y_pred, n_targets=11):\n    \n    res_list = []\n    for col_idx in trange(n_targets, desc='1st loop'):\n        mean_LS = 0\n        for row_idx in trange(len(y_pred), desc='2nd loop', leave=False):\n            mean_LS += Levenshtein.distance(y_true[row_idx, col_idx], y_pred[row_idx, col_idx])\n        mean_LS /= len(y_pred)\n        res_list.append(mean_LS)\n    \n    return res_list\n\n\ndef join_inchi_parts(pred):\n    inchi_cols = [f\"InChI_{i}\" for i in range(11)]\n    jonied_list = []\n    for inchi_parts in tqdm(pred[inchi_cols].values):\n        jonied_list.append(\"/\".join([s for s in inchi_parts if s != \"\"]))\n        \n    pred[\"InChI\"] = jonied_list\n    return pred","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nX = oof_fml[[\"C\", \"H\", \"n_atoms_ex_H\"]].values\nprint(X.shape)\n\nclustering_pipe = Pipeline([\n    ('scaler', StandardScaler()),\n    (\"kmeans\", KMeans(n_clusters=N_CLUSTER, random_state=1086)),\n])\n\noof_fml[\"cluster\"] = clustering_pipe.fit_predict(X)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ninchi_cols = [f\"InChI_{i}\" for i in range(11)]\n\nmode_by_cluster = train.groupby(oof_fml[\"cluster\"])[inchi_cols].agg(lambda x:x.value_counts().index[0])\n\noof_pred = train[[\"image_id\"]].copy()\noof_pred[\"cluster\"] = oof_fml[\"cluster\"]\noof_pred[\"InChI_0\"] = \"InChI=1S\"\noof_pred[\"InChI_1\"] = oof_fml[\"pred_formula\"]\n\nfor i in range(2, 11):\n    oof_pred[f\"InChI_{i}\"] = oof_pred[\"cluster\"].map(mode_by_cluster[f\"InChI_{i}\"])\n\noof_pred = join_inchi_parts(oof_pred)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del X; del oof_fml; del train_fml;\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"# oof_LS_each = calc_each_LS(\n#     train[inchi_cols + [\"InChI\"]].values, oof_pred[inchi_cols + [\"InChI\"]].values, 12)\n\n# print(pd.DataFrame(\n#     {\"parts\" : inchi_cols + [\"InChI\"], \"mean_LS\": oof_LS_each}))\n\nprint(\n\"\"\"\n       parts       mean_LS\n0    InChI_0  0.000000e+00\n1    InChI_1  9.503252e-01\n2    InChI_2  4.487596e+01\n3    InChI_3  1.796458e+01\n4    InChI_4  1.557185e+00\n5    InChI_5  3.846986e-01\n6    InChI_6  3.263887e-01\n7    InChI_7  2.152764e-02\n8    InChI_8  4.488104e-04\n9    InChI_9  3.011320e-05\n10  InChI_10  8.250192e-07\n11     InChI  6.529676e+01\n\"\"\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# print(f\"CV score by regression and clustering: {oof_LS_each[-1]:.4f}\")\n\nprint(\"CV score by regression and clustering: 65.2968\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"oof_pred.to_csv(\"oof_prediction.csv\", index=False)\ndel oof_pred; del train\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Inference Test Data"},{"metadata":{},"cell_type":"markdown","source":"## definition"},{"metadata":{},"cell_type":"markdown","source":"### model"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"class BasicImageModel(nn.Module):\n    \n    def __init__(\n        self, base_name, dims_head: tp, pretrained=False\n    ):\n        \"\"\"Initialize\"\"\"\n        self.base_name = base_name\n        super(BasicImageModel, self).__init__()\n        \n        # # prepare backbone\n        if hasattr(timm.models, base_name):\n            # # # load base model\n            base_model = timm.create_model(base_name, pretrained=pretrained)\n            in_features = base_model.num_features\n            # # remove head classifier\n            base_model.reset_classifier(0)\n        else:\n            raise NotImplementedError\n\n        self.backbone = base_model\n        \n        # # prepare head clasifier\n        if dims_head[0] is None:\n            dims_head[0] = in_features\n\n        layers_list = []\n        for i in range(len(dims_head) - 2):\n            in_dim, out_dim = dims_head[i: i + 2]\n            layers_list.extend([\n                nn.Linear(in_dim, out_dim),\n                nn.ReLU(), nn.Dropout(0.5),])\n        layers_list.append(\n            nn.Linear(dims_head[-2], dims_head[-1]))\n        self.head = nn.Sequential(*layers_list)\n\n    def forward(self, x):\n        \"\"\"Forward\"\"\"\n        h = self.backbone(x)\n        h = self.head(h)\n        return h","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### image dataset"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"class ImageTransformBase:\n    \"\"\"\n    Base Image Transform class.\n\n    Args:\n        data_augmentations: List of tuple(method: str, params :dict), each elems pass to albumentations\n    \"\"\"\n\n    def __init__(self, data_augmentations: tp.List[tp.Tuple[str, tp.Dict]]):\n        \"\"\"Initialize.\"\"\"\n        augmentations_list = [\n            self._get_augmentation(aug_name)(**params)\n            for aug_name, params in data_augmentations]\n        self.data_aug = albumentations.Compose(augmentations_list)\n\n    def __call__(self, pair: tp.Tuple[np.ndarray]) -> tp.Tuple[np.ndarray]:\n        \"\"\"You have to implement this by task\"\"\"\n        raise NotImplementedError\n\n    def _get_augmentation(self, aug_name: str) -> tp.Tuple[ImageOnlyTransform, DualTransform]:\n        \"\"\"Get augmentations from albumentations\"\"\"\n        if hasattr(albumentations, aug_name):\n            return getattr(albumentations, aug_name)\n        else:\n            return eval(aug_name)\n\n\nclass ImageTransformForCls(ImageTransformBase):\n    \"\"\"Data Augmentor for Classification Task.\"\"\"\n\n    def __init__(self, data_augmentations: tp.List[tp.Tuple[str, tp.Dict]]):\n        \"\"\"Initialize.\"\"\"\n        super(ImageTransformForCls, self).__init__(data_augmentations)\n\n    def __call__(self, in_arrs: tp.Tuple[np.ndarray]) -> tp.Tuple[np.ndarray]:\n        \"\"\"Apply Transform.\"\"\"\n        img, label = in_arrs\n        augmented = self.data_aug(image=img)\n        img = augmented[\"image\"]\n\n        return img, label","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"class LabeledImageDataset(data.Dataset):\n    \"\"\"Dataset class for (image, label) pairs\"\"\"\n\n    def __init__(\n        self,\n        file_list: tp.List[\n            tp.Tuple[tp.Union[str, Path], tp.Union[int, float, np.ndarray]]],\n        transform_list: tp.List[tp.Dict],\n    ):\n        \"\"\"Initialize\"\"\"\n        self.file_list = file_list\n        self.transform = ImageTransformForCls(transform_list)\n\n    def __len__(self):\n        \"\"\"Return Num of Images.\"\"\"\n        return len(self.file_list)\n\n    def __getitem__(self, index):\n        \"\"\"Return transformed image and mask for given index.\"\"\"\n        img_path, label = self.file_list[index]\n        img = self._read_image_as_array(img_path)\n        \n        img, label = self.transform((img, label))\n        return img, label\n\n    def _read_image_as_array(self, path: str):\n        \"\"\"Read image file and convert into numpy.ndarray\"\"\"\n        img_arr = cv2.imread(str(path))\n        img_arr = cv2.cvtColor(img_arr, cv2.COLOR_BGR2RGB)\n        return img_arr","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"class TestImageDataset(data.Dataset):\n    \"\"\"Dataset class for (image, label) pairs\"\"\"\n\n    def __init__(\n        self,\n        file_list: tp.List[\n            tp.Tuple[tp.Union[str, Path], tp.Union[int, float, np.ndarray], tp.Tuple[int, int]]],\n        transform_list: tp.List[tp.Dict],\n    ):\n        \"\"\"Initialize\"\"\"\n        self.file_list = file_list\n        self.transform = ImageTransformForCls(transform_list)\n        self.fix_transform = albumentations.Compose([\n            albumentations.Transpose(always_apply=True),\n            albumentations.VerticalFlip(always_apply=True)])\n\n    def __len__(self):\n        \"\"\"Return Num of Images.\"\"\"\n        return len(self.file_list)\n\n    def __getitem__(self, index):\n        \"\"\"Return transformed image and mask for given index.\"\"\"\n        img_path, label, (height, width) = self.file_list[index]\n        img = self._read_image_as_array(img_path)\n        \n        if height > width:\n            img = self.fix_transform(image=img)['image']\n        \n        img, label = self.transform((img, label))\n        return img, label\n\n    def _read_image_as_array(self, path: str):\n        \"\"\"Read image file and convert into numpy.ndarray\"\"\"\n        img_arr = cv2.imread(str(path))\n        img_arr = cv2.cvtColor(img_arr, cv2.COLOR_BGR2RGB)\n        return img_arr","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### get XXX"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def get_test_loader(\n    stgs: tp.Dict,\n    test_df: pd.DataFrame,\n    dataset_class: data.Dataset\n):\n    \"\"\"Create DataLoader\"\"\"\n    test_img_size_info = pd.read_csv(\"../input/bms-mt-image-size-info/train_image_size.csv\")\n    \n    test_file_list = list(zip(\n        # # cropped and padded image path \n        [\n            TEST_224 / f\"{img_id}.jpg\"\n            for img_id in test_df[\"image_id\"].values],\n        # # dummy label\n        [-1] * len(test_df),\n        # # image size info \n        test_img_size_info[[\"height\", \"width\"]].values.tolist()\n    ))\n\n    test_dataset = dataset_class(\n        test_file_list, **stgs[\"dataset\"][\"val\"])\n    test_loader = data.DataLoader(\n        test_dataset, **stgs[\"loader\"][\"val\"])\n\n    return test_loader","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### inference utils"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"def set_random_seed(seed: int = 42, deterministic: bool = False):\n    \"\"\"Set seeds\"\"\"\n    random.seed(seed)\n    np.random.seed(seed)\n    os.environ[\"PYTHONHASHSEED\"] = str(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)  # type: ignore\n    torch.backends.cudnn.deterministic = deterministic  # type: ignore","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def run_inference_loop(stgs, model, loader, device):\n    model.to(device)\n    model.eval()\n    pred_list = []\n    with torch.no_grad():\n        for x, _ in tqdm(loader):\n            if stgs[\"globals\"][\"use_amp\"]:\n                with torch.cuda.amp.autocast(): \n                    y = model(x.to(device))\n            else:\n                y = model(x.to(device))\n            pred_list.append(y.detach().cpu().numpy())\n        \n        pred_arr = np.concatenate(pred_list)\n        del pred_list\n    return pred_arr","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## inference chemical formula"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_preds_arr = np.zeros((N_FOLD, len(smpl_sub), N_TARGETS))    \n\nfor fold_id in range(N_FOLD):\n    print(f\"[fold {fold_id}]\")\n    tmp_dir = TRAINING_OUTPUTS / f\"fold{fold_id}\"\n    with open(tmp_dir / \"settings.yml\", \"r\") as fr:\n        tmp_stgs = yaml.safe_load(fr)\n    device = torch.device(tmp_stgs[\"globals\"][\"device\"])\n    \n    # # get data_loader\n    # test_loader = get_test_loader(tmp_stgs, smpl_sub, LabeledImageDataset)\n    test_loader = get_test_loader(tmp_stgs, smpl_sub, TestImageDataset)\n\n    # # get and load model\n    model_path = TRAINING_OUTPUTS / f\"./best_loss_model_fold{fold_id}.pth\"\n    tmp_stgs[\"model\"][\"params\"][\"pretrained\"] = False\n    model = BasicImageModel(**tmp_stgs[\"model\"][\"params\"])\n    model.load_state_dict(torch.load(model_path, map_location=device))\n\n    # # inference test\n    test_pred = run_inference_loop(tmp_stgs, model, test_loader, device)\n    test_preds_arr[fold_id] = test_pred\n    \n    del model; del test_pred;\n    torch.cuda.empty_cache()\n    gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_pred_fml = test_preds_arr.mean(axis=0)\ndel test_preds_arr\n\ntest_pred = smpl_sub[[\"image_id\"]].copy()\nfor i, atom in enumerate(TARGETS):\n    test_pred[atom] = test_pred_fml[:, i]\n    \ndel test_pred_fml\n    \ntest_pred[\"pred_formula\"] = [\n    arr2formula(atom_arr) for atom_arr in tqdm(test_pred[TARGETS].values)]\n\ntest_pred[\"n_atoms\"] = test_pred[TARGETS].sum(axis=1)\ntest_pred[\"n_atoms_ex_H\"] = test_pred[\"n_atoms\"] - test_pred[\"H\"]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## apply clustering"},{"metadata":{"trusted":true},"cell_type":"code","source":"X_test = test_pred[[\"C\", \"H\", \"n_atoms_ex_H\"]].values\n\ntest_pred[\"cluster\"] = clustering_pipe.predict(X_test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## make submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"sub = smpl_sub[[\"image_id\"]].copy()\n\nsub[\"InChI_0\"] = \"InChI=1S\"\nsub[\"InChI_1\"] = test_pred[\"pred_formula\"]\nfor i in range(2, 11):\n    sub[f\"InChI_{i}\"] = test_pred[\"cluster\"].map(mode_by_cluster[f\"InChI_{i}\"])\n\nsub = join_inchi_parts(sub)\nsub.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub[[\"image_id\", \"InChI\"]].to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}