{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Bristol-Myers Squibb – Molecular Translation - Exploratory Data Analysis\n\nQuick Exploratory Data Analysis for [Bristol-Myers Squibb – Molecular Translation](https://www.kaggle.com/c/bms-molecular-translation) challenge \n\nIn this competition, you are provided with images of chemicals, with the objective of predicting the corresponding International Chemical Identifier (InChI) text string of the image. The images provided (both in the training data as well as the test data) may be rotated to different angles, be at various resolutions, and have different noise levels."},{"metadata":{},"cell_type":"markdown","source":"![](https://storage.googleapis.com/kaggle-competitions/kaggle/22422/logos/header.png)"},{"metadata":{},"cell_type":"markdown","source":"\n\n
\n

Quick Navigation

\n\n* [Overview](#1)\n* [Data Visualization](#2)\n \n\n* [Competition Metric](#100)\n* [Submission](#101)"},{"metadata":{},"cell_type":"markdown","source":"\n

Overview

"},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nimport cv2\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**train_labels.csv** - ground truth InChi labels for the training images"},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN_LABELS_PATH = \"../input/bms-molecular-translation/train_labels.csv\"\n\ndf_train_labels = pd.read_csv(TRAIN_LABELS_PATH, index_col=0)\ndf_train_labels","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**train/** - the training images, arranged in a 3-level folder structure by image_id"},{"metadata":{"trusted":true},"cell_type":"code","source":"def convert_image_id_2_path(image_id: str) -> str:\n return \"../input/bms-molecular-translation/train/{}/{}/{}/{}.png\".format(\n image_id[0], image_id[1], image_id[2], image_id \n )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def visualize_train_batch(image_ids, labels):\n plt.figure(figsize=(16, 12))\n \n for ind, (image_id, label) in enumerate(zip(image_ids, labels)):\n plt.subplot(3, 3, ind + 1)\n image = cv2.imread(convert_image_id_2_path(image_id))\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n plt.imshow(image)\n# print(f\"{ind}: {label}\")\n plt.title(f\"{label[:30]}...\", fontsize=10)\n plt.axis(\"off\")\n \n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def visualize_train_image(image_id, label):\n plt.figure(figsize=(10, 8))\n \n image = cv2.imread(convert_image_id_2_path(image_id))\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n plt.imshow(image)\n plt.title(f\"{label}\", fontsize=14)\n plt.axis(\"off\")\n \n plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n

Data Visualization

"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_row = df_train_labels.sample(5)\nfor i in range(5):\n visualize_train_image(\n sample_row.index[i], sample_row[\"InChI\"][i]\n )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp_df = df_train_labels[:9]\nimage_ids = tmp_df.index\nlabels = tmp_df[\"InChI\"].values\n\nvisualize_train_batch(image_ids, labels)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp_df = df_train_labels.sample(9)\nimage_ids = tmp_df.index\nlabels = tmp_df[\"InChI\"].values\n\nvisualize_train_batch(image_ids, labels)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def visualize_batch_without_labels(image_ids):\n plt.figure(figsize=(16, 16))\n \n for ind, image_id in enumerate(image_ids):\n plt.subplot(5, 5, ind + 1)\n image = cv2.imread(convert_image_id_2_path(image_id))\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n\n plt.imshow(image)\n plt.axis(\"off\")\n \n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp_df = df_train_labels.sample(25)\nimage_ids = tmp_df.index\n\nvisualize_batch_without_labels(image_ids)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp_df = df_train_labels.sample(1000)\nh_shape = []\nw_shape = []\nfor image_id in tmp_df.index:\n image = cv2.imread(convert_image_id_2_path(image_id))\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n h_shape.append(image.shape[0])\n w_shape.append(image.shape[1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(16, 5))\nplt.subplot(1, 3, 1)\nplt.hist(np.array(h_shape) * np.array(w_shape), bins=50)\nplt.xticks(rotation=45)\nplt.title(\"Area Image Distribution\", fontsize=14)\nplt.subplot(1, 3, 2)\nplt.hist(h_shape, bins=50)\nplt.title(\"Height Image Distribution\", fontsize=14)\nplt.subplot(1, 3, 3)\nplt.hist(w_shape, bins=50)\nplt.title(\"Width Image Distribution\", fontsize=14);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**test/** - the test images, arranged in the same folder structure as train/"},{"metadata":{},"cell_type":"markdown","source":"\n

Competition Metric

"},{"metadata":{},"cell_type":"markdown","source":"Submissions are evaluated on the mean [Levenshtein distance](https://en.wikipedia.org/wiki/Levenshtein_distance) between the InChi strings you submit and the ground truth InChi values."},{"metadata":{},"cell_type":"markdown","source":"The Levenshtein distance between two strings **a,b** (of length **|a|** and **|b|** respectively) is given by **lev(a,b)** where"},{"metadata":{},"cell_type":"markdown","source":"$${\\displaystyle \\qquad \\operatorname {lev} (a,b)={\\begin{cases}|a|&{\\text{ if }}|b|=0,\\\\|b|&{\\text{ if }}|a|=0,\\\\\\operatorname {lev} (\\operatorname {tail} (a),\\operatorname {tail} (b))&{\\text{ if }}a[0]=b[0]\\\\1+\\min {\\begin{cases}\\operatorname {lev} (\\operatorname {tail} (a),b)\\\\\\operatorname {lev} (a,\\operatorname {tail} (b))\\\\\\operatorname {lev} (\\operatorname {tail} (a),\\operatorname {tail} (b))\\\\\\end{cases}}&{\\text{ otherwise.}}\\end{cases}}}$$"},{"metadata":{},"cell_type":"markdown","source":"where the **tail** of some string **x** is a string of all but the first character of **x**, and **x[n]** is the **n**th character of the string **x**, starting with character 0.\n\nNote that the first element in the minimum corresponds to deletion (from **a** to **b**), the second to insertion and the third to replacement."},{"metadata":{},"cell_type":"markdown","source":"\nThe image from Understanding the Levenshtein Distance Equation for Beginners"},{"metadata":{},"cell_type":"markdown","source":"\n

Submission

"},{"metadata":{},"cell_type":"markdown","source":"**sample_submission.csv** - a sample submission file in the correct format"},{"metadata":{"trusted":true},"cell_type":"code","source":"SAMPLE_SUBMISSION_PATH = \"../input/bms-molecular-translation/sample_submission.csv\"\n\ndf_sample_submission = pd.read_csv(SAMPLE_SUBMISSION_PATH, index_col=0)\ndf_sample_submission","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_sample_submission.to_csv(\"submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}