{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"","metadata":{"_uuid":"4c60c7a4-edb0-4e26-ad8e-c25d93d55314","_cell_guid":"326706e3-485d-4fd6-b6e7-6ac143ea07cf","trusted":true}},{"cell_type":"markdown","source":"## Table of Contents\n* [Introduction](#introduction)\n* [Details of the Problem](#details-of-the-problem)\n* [Visualise Random Images with Boundary Box](#visualise-random-images)\n* [Preparing Dataset for Training](#preparing-dataset-training)\n* [Create Model - ResNet50(Faster R-CNN)](#create-model-resnet50)\n* [Preparing Model for Training](#preparing-model-for-training)\n* [Training](#let's-train-it)\n* [Calculation on unlabeled data](#calculation-on-unlabeled-data)","metadata":{"_uuid":"d901bc2a-cb1a-4db5-b455-17dee0dfc935","_cell_guid":"424d435d-1ec8-48a0-8ec9-2930261bcbc0","trusted":true}},{"cell_type":"markdown","source":"## Introduction","metadata":{"_uuid":"f1010527-b0d7-48e2-89f7-405fa7c17274","_cell_guid":"741c0dfa-402b-4843-9984-dec0fc032ae8","trusted":true}},{"cell_type":"markdown","source":"Recognizing molecules and composing the full inchi solution seemed to be a very complex task.\nInchi has several levels of information. Being self-assured is not a successful mindset.\nI decided to accomplish the first part of inchi, the molecular formula.\nSo I concentrated my efforts on uncovering the molecular formula.\nMy problem is to recognize atoms, make some calculations, return molecular formula.\nSo the problem is mostly about object detection. \n\nSeems very easy. \nBut there are a lot of options. Which objects are the most useful, what objects should be recognized?\nThere are many of them. I thought about it for a long time. Finally, I realized that I am capable of accomplishing of something very simple.\nComplexity makes the code much larger. \nMy model is to recognize atoms from the depiction of a molecule.","metadata":{"_uuid":"ee331556-2331-453f-8784-b47c5a8d7ba1","_cell_guid":"422b492d-d4f7-4d3e-8ac6-b55afa5aaa08","trusted":true}},{"cell_type":"markdown","source":"### Details of the Problem","metadata":{"_uuid":"8eb3b89e-1d98-4d1a-a9e2-c3991c238dd2","_cell_guid":"fb5981af-054e-4c71-b4d7-e774dfde5324","trusted":true}},{"cell_type":"markdown","source":"\nAll organic molecules have one of the twelve atoms:\n\n'C', 'H', 'O', 'S', 'N', 'Br', 'F', 'Cl', 'P', 'Si', 'B', 'I'","metadata":{"_uuid":"80cea96e-26d5-4084-a2f9-b03e9ce87e98","_cell_guid":"e8db5637-95c5-4c43-a51e-8f859ff0847c","trusted":true}},{"cell_type":"code","source":"import os\nimport cv2\nimport time\nimport pandas as pd\nimport numpy as np\nimport re\nimport csv\n\nfrom PIL import Image\nimport torch\nimport torchvision\nimport torchvision.transforms as T\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor\nfrom torch.utils.data import DataLoader, Dataset\n\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport pathlib","metadata":{"_uuid":"400b55bb-f5d9-4871-847c-228884b7d3f1","_cell_guid":"eaaf4c0c-1760-4d2a-bb40-5632eb217689","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:17.981552Z","iopub.execute_input":"2021-06-03T11:05:17.981872Z","iopub.status.idle":"2021-06-03T11:05:17.987472Z","shell.execute_reply.started":"2021-06-03T11:05:17.98184Z","shell.execute_reply":"2021-06-03T11:05:17.986383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"EPOCHS = 1 #100\nTHRESHOLD = 0.5\n\nCSV_OUT = 'submission.csv'\nDIR_INPUT = \"/kaggle/input/extendedlabels/\"\nDIR_IMAGES = DIR_INPUT + \"images/train/\"\n\nDIR_TEST_IMAGES = \"/kaggle/input/bms-molecular-translation/train/0/0/0/\"\n# DIR_TEST_IMAGES = \"/kaggle/input/bms-molecular-translation/test/0/0/\"\n\n\n### Loading Dataset\ndf = pd.read_csv(DIR_INPUT + \"images/train_labels.csv\")\ndisplay(df.head())\ndf.describe()","metadata":{"_uuid":"f79f58e6-a39c-4809-9110-e03cb8067466","_cell_guid":"abb2b3f2-692d-42f8-b140-a511debf0c3b","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:18.019354Z","iopub.execute_input":"2021-06-03T11:05:18.019588Z","iopub.status.idle":"2021-06-03T11:05:18.063068Z","shell.execute_reply.started":"2021-06-03T11:05:18.019566Z","shell.execute_reply":"2021-06-03T11:05:18.062171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Null Values, Unique Values\n\nunq_values = df[\"filename\"].unique()\nprint(\"Total Records: \", len(df))\nprint(\"Unique Images: \", len(unq_values))\n\nnull_values = df.isnull().sum(axis=0)\nprint(\"\\n> Null Values in each column <\")\nprint(null_values)\n\n### Total Classes\n\nclasses = df[\"class\"].unique()\nprint(\"Total Classes: \", len(classes))\nprint(\"\\n> Classes <\\n\", classes)\n\n### Visualizing Class Distribution\n\nplt.figure(figsize=(14,8))\nplt.title('Class Distribution', fontsize=20)\nsns.countplot(x=\"class\", data=df)","metadata":{"_uuid":"10d4b004-90ee-4630-bb97-0b057edbd8ac","_cell_guid":"b8a238e6-efcf-437b-bbb3-cf292e0282a0","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:18.064591Z","iopub.execute_input":"2021-06-03T11:05:18.064939Z","iopub.status.idle":"2021-06-03T11:05:18.299315Z","shell.execute_reply.started":"2021-06-03T11:05:18.064906Z","shell.execute_reply":"2021-06-03T11:05:18.297856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualise Random Images with Boundary Box","metadata":{"_uuid":"95d41a00-c382-4f7c-b99c-92533ada3f6a","_cell_guid":"49de6224-2ec7-494b-896f-368fdf0de7fc","trusted":true}},{"cell_type":"markdown","source":"For labeling I used 2 tools, easy to use.\n\n[labelImg](https://github.com/tzutalin/labelImg)\n\n[xml_to_csv](https://github.com/EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10/blob/master/xml_to_csv.py)","metadata":{"_uuid":"f13a97f6-ab72-442d-9158-5efb5b95935b","_cell_guid":"da7a3d20-234d-4de7-a7b4-c4d5de6a53ac","trusted":true}},{"cell_type":"code","source":"### Function to plot image\n\ndef plot_img(image_name, image_dir, df_labels, verbose=1):\n \n fig, ax = plt.subplots(1, 2, figsize = (14, 14))\n ax = ax.flatten()\n \n df = df_labels\n bbox = df[df['filename'] == image_name]\n img_path = os.path.join(image_dir, image_name)\n \n image = cv2.imread(img_path, cv2.IMREAD_COLOR)\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n image /= 255.0\n image2 = image\n \n ax[0].set_title('Original Image')\n ax[0].imshow(image)\n \n for idx, row in bbox.iterrows():\n x1 = row['xmin']\n y1 = row['ymin']\n x2 = row['xmax']\n y2 = row['ymax']\n label = row['class']\n if verbose == 1:\n print(x1, y1, x2, y2, label)\n else: \n pass\n \n cv2.rectangle(image2, (int(x1),int(y1)), (int(x2),int(y2)), (255,0,0), 1)\n font = cv2.FONT_HERSHEY_SIMPLEX\n cv2.putText(image2, label, (int(x1),int(y1-10)), font, 0.5, (255,0,0), 1)\n \n ax[1].set_title('Image with Boundary Box')\n ax[1].imshow(image2)\n\n plt.show()","metadata":{"_uuid":"bc693624-80df-4fb4-908d-27b45e6a7a96","_cell_guid":"2f9499fc-7b98-4981-9d54-0c321fda25b9","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:18.302147Z","iopub.execute_input":"2021-06-03T11:05:18.302572Z","iopub.status.idle":"2021-06-03T11:05:18.318023Z","shell.execute_reply.started":"2021-06-03T11:05:18.302528Z","shell.execute_reply":"2021-06-03T11:05:18.316158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Pass any image name as parameter\ncount = 0\nfor image in unq_values:\n if len(image) > 9: \n plot_img(image, DIR_IMAGES, df, verbose=0)\n count += 1\n if count > 5:\n break","metadata":{"_uuid":"d500b005-6e22-441d-9c6b-382df5e25771","_cell_guid":"db2ee00c-a444-40b5-a726-16bc89efc453","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:18.320215Z","iopub.execute_input":"2021-06-03T11:05:18.320644Z","iopub.status.idle":"2021-06-03T11:05:20.715786Z","shell.execute_reply.started":"2021-06-03T11:05:18.320601Z","shell.execute_reply":"2021-06-03T11:05:20.714922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Preparing Dataset for Training","metadata":{"_uuid":"19236082-3c79-470a-87bc-99ed24e4d42f","_cell_guid":"0433ba8d-2b44-49e9-a9ff-62ca872c47d0","trusted":true}},{"cell_type":"code","source":"### Class <-> Int\n\n_classes = np.insert(classes, 0, 'background', axis=0) # adding a background class for Faseter R-CNN\nclass_to_int = {_classes[i] : i for i in range(len(_classes))}\nint_to_class = {i : _classes[i] for i in range(len(_classes))}\nprint(\"class_to_int : \\n\", class_to_int)\nprint(\"\\nint _to_class : \\n\", int_to_class)","metadata":{"_uuid":"3c21e2bf-8ff4-4bd2-bfd4-63f001360338","_cell_guid":"5f6847ea-3025-4fad-90af-32642394d2ab","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:20.717068Z","iopub.execute_input":"2021-06-03T11:05:20.717474Z","iopub.status.idle":"2021-06-03T11:05:20.725436Z","shell.execute_reply.started":"2021-06-03T11:05:20.717423Z","shell.execute_reply":"2021-06-03T11:05:20.723955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Creating Data (Labels & Targets) for Faster R-CNN\n\ndef get_transform():\n return T.Compose([T.ToTensor()])\n\nclass MoleculeDetectionDataset(Dataset):\n \n def __init__(self, dataframe, image_dir, mode='train', transforms=None ):\n # if mode='train' then dataframe is a dataframe with labels(classes and rectangles)\n # if mode = 'test' then dataframe = None, model will be used for prediction \\\n # Example for training:\n # MoleculeDetectionDataset(image_dir, transforms=function(), dataframe=df_labels, mode='train')\n \n super().__init__()\n \n self.paths = None\n \n if mode == 'train':\n self.image_names = dataframe[\"filename\"].unique()\n \n else: # mode == 'test'\n if dataframe == None:\n paths = [x for x in list(pathlib.Path(image_dir).rglob('*.png'))]\n self.paths = paths\n self.image_names = np.asarray([x.name for x in paths], dtype=object)\n \n else:\n raise ValueError(\"Ignore input parameter 'dataframe' in a 'test' mode\")\n \n self.df = dataframe\n self.image_dir = image_dir\n self.transforms = transforms\n self.mode = mode\n \n def __getitem__(self, index: int):\n \n\n if self.mode == 'train':\n \n # Retrive Image name and its records (x1, y1, x2, y2, classname) from df\n image_name = self.image_names[index]\n\n # Loading Image\n image = cv2.imread(self.image_dir + image_name, cv2.IMREAD_COLOR)\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n image /= 255.0\n \n # # # # # # # #\n \n records = self.df[self.df[\"filename\"] == image_name]\n # Get bounding box co-ordinates for each box\n boxes = records[['xmin','ymin','xmax','ymax']].values\n \n # Getting labels for each box\n temp_labels = records[['class']].to_numpy() #.value\n labels = []\n for label in temp_labels:\n label = class_to_int[label[0]]\n labels.append(label)\n \n # Converting boxes & labels into torch tensor\n boxes = torch.as_tensor(boxes, dtype=torch.float32)\n labels = torch.as_tensor(labels, dtype=torch.int64)\n \n # Creating target\n target = {}\n target['boxes'] = boxes\n target['labels'] = labels\n \n # Transforms\n if self.transforms:\n image = self.transforms(image)\n \n return image, target, image_name\n \n elif self.mode == 'test':\n \n # Retrive Image name and its records (x1, y1, x2, y2, classname) from df\n image_path = self.paths[index]\n image_name = image_path.name\n\n # Loading Image\n image = cv2.imread(str(image_path), cv2.IMREAD_COLOR) \n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32) \n image /= 255.0\n \n #-#-#-#-#-#-#\n \n if self.transforms:\n image = self.transforms(image)\n \n return image, image_name\n \n def __len__(self):\n return len(self.image_names)\n \n\n# test\ndataset = MoleculeDetectionDataset(image_dir=DIR_IMAGES, \n transforms=get_transform(), \n dataframe=df, \n mode='train' )\ndataset.__getitem__(0)","metadata":{"_uuid":"d1df06a5-7843-4fa8-8ef9-02d850c35925","_cell_guid":"75ee0d94-69ad-4cb6-94c9-f8d397f107f4","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:20.727015Z","iopub.execute_input":"2021-06-03T11:05:20.727586Z","iopub.status.idle":"2021-06-03T11:05:20.758575Z","shell.execute_reply.started":"2021-06-03T11:05:20.727319Z","shell.execute_reply":"2021-06-03T11:05:20.757665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### Preparing data for Train & Validation\n\ndef collate_fn(batch):\n return tuple(zip(*batch))\n\n# Dataset object\ndataset = MoleculeDetectionDataset(dataframe=df, \n image_dir=DIR_IMAGES, \n transforms=get_transform(),\n mode='train')\n\nindices = torch.randperm(len(dataset)).tolist()\n\ntrain_dataset = torch.utils.data.Subset(dataset, indices)\n\n# Preparing data loaders\ntrain_data_loader = DataLoader(\n train_dataset,\n batch_size=1, \n shuffle=True,\n num_workers=4, \n collate_fn=collate_fn\n)","metadata":{"_uuid":"297b9875-2134-4986-b4e8-44c05e5eed50","_cell_guid":"05cc12a7-e7ed-4f72-a59e-d6a62581ec06","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:20.7612Z","iopub.execute_input":"2021-06-03T11:05:20.761647Z","iopub.status.idle":"2021-06-03T11:05:20.769752Z","shell.execute_reply.started":"2021-06-03T11:05:20.761616Z","shell.execute_reply":"2021-06-03T11:05:20.768822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Create Model - ResNet50(Faster R-CNN)","metadata":{"_uuid":"812b1bf0-013d-4a39-937a-06d5f20272f2","_cell_guid":"7cd3ebd4-13c1-41bb-9bb3-fc026e772f74","trusted":true}},{"cell_type":"code","source":"### Utilize GPU if available\n\n# device = torch.device('cpu') \n# device = torch.device('cuda') \ndevice = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')\ntorch.cuda.empty_cache()","metadata":{"_uuid":"52a8c430-5582-48c8-9f32-a3803610623c","_cell_guid":"4a374784-9405-4ceb-87d9-dc4ec91ad91d","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:20.7719Z","iopub.execute_input":"2021-06-03T11:05:20.772599Z","iopub.status.idle":"2021-06-03T11:05:20.88222Z","shell.execute_reply.started":"2021-06-03T11:05:20.772562Z","shell.execute_reply":"2021-06-03T11:05:20.881307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"ResNet is one the latest successful architectures, probably the better solution is DenseNet. I compared the scores of neural networks. \n\nFor everybody who is new to Faster R-CNN, I insert a couple of links. \n\n[Faster R-CNN: Towards Real-Time ObjectDetection with Region Proposal Networks](https://arxiv.org/pdf/1506.01497.pdf)\n\n[TorchVision](https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html)","metadata":{"_uuid":"cdeb2fd1-6952-45ba-a931-5e35ad08069c","_cell_guid":"2589a3b3-0133-4ff6-bcc2-c0df6196bc2a","trusted":true}},{"cell_type":"code","source":"### Create / load model\n\n# Faster - RCNN Model - pretrained on COCO\nmodel = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\nnum_classes = len(class_to_int)\n\n# get number of input features for the classifier\nin_features = model.roi_heads.box_predictor.cls_score.in_features\n\n# replace the pre-trained head with a new one\nmodel.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)","metadata":{"_uuid":"61cb3900-c61f-45e4-84c5-9513bd047e6c","_cell_guid":"3e34d97c-3c21-45a0-abc8-2d910df4d3c7","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:20.883778Z","iopub.execute_input":"2021-06-03T11:05:20.884154Z","iopub.status.idle":"2021-06-03T11:05:21.591088Z","shell.execute_reply.started":"2021-06-03T11:05:20.884118Z","shell.execute_reply":"2021-06-03T11:05:21.590192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Preparing Model for Training","metadata":{"_uuid":"129e5e1c-452f-44b6-ab42-b28e4cd37f46","_cell_guid":"2eb391f1-e18d-4fec-9fdb-713974b840b6","trusted":true}},{"cell_type":"code","source":"### Preparing model for training\n\n#Retriving all trainable parameters from model (for optimizer)\nparams = [p for p in model.parameters() if p.requires_grad]\n\n#Defininig Optimizer\n# optimizer = torch.optim.SGD(params, lr = 0.25, momentum = 0.9)\noptimizer = torch.optim.SGD(params, lr = 0.005, momentum = 0.9)\n\n\nmodel.to(device)\n\n#No of epochs\nepochs = EPOCHS # 30","metadata":{"_uuid":"0766fc4c-7f79-42a6-9903-ea7434eabbdf","_cell_guid":"79ea3109-b0d2-4eeb-a7b3-809f0baf9fa8","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:21.592837Z","iopub.execute_input":"2021-06-03T11:05:21.593355Z","iopub.status.idle":"2021-06-03T11:05:21.654732Z","shell.execute_reply.started":"2021-06-03T11:05:21.593316Z","shell.execute_reply":"2021-06-03T11:05:21.65402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training","metadata":{"_uuid":"4b8297dd-b45b-4f1b-be4d-21a1cc7e67a1","_cell_guid":"6505e2bb-db21-498d-b515-5d5f88d40e0b","trusted":true}},{"cell_type":"code","source":"### Training model\nimport tensorflow as tf\n\nPATH_LOAD = pathlib.Path(\"/kaggle/input/modelsave/checkpoint_save_330ext2\")\n\n\n## Save/Load state_dict()\n# model.load_state_dict(torch.load(PATH_LOAD))\n# model.eval()\n\n## Save/Load Entire Model\n# model = torch.load(PATH_LOAD)\n# model.eval()\n# model.train()\n\ndef load_checkpoint(PATH, \n model, \n optimizer):\n checkpoint = torch.load(PATH, map_location=device)\n \n model.load_state_dict(checkpoint['model_state_dict'])\n optimizer.load_state_dict(checkpoint['optimizer_state_dict'])\n epoch = checkpoint['epoch']\n loss = checkpoint['loss']\n \n return epoch, loss\n\n## Loading a checkpoint\n# preferred\nepoch, loss = load_checkpoint(PATH_LOAD, model, optimizer)\n\nitr = 1\ntotal_train_loss = []\n\nfor epoch in range(epochs):\n \n start_time = time.time()\n train_loss = []\n\n #Retriving Mini-batch\n for images, targets, image_names in train_data_loader:\n \n #Loading images & targets on device\n images = list(image.to(device) for image in images)\n targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n \n #Forward propagation\n out = model(images, targets)\n losses = sum(loss for loss in out.values())\n \n #Reseting Gradients\n optimizer.zero_grad()\n \n #Back propagation\n losses.backward()\n torch.nn.utils.clip_grad_norm_(model.parameters(), 1)\n optimizer.step()\n \n #Average loss\n loss_value = losses.item()\n train_loss.append(loss_value)\n\n \n if itr % 25 == 0:\n print(f\"\\n Iteration #{itr} loss: {out} \\n\")\n\n itr += 1\n \n epoch_train_loss = np.mean(train_loss)\n total_train_loss.append(epoch_train_loss)\n print(f'Epoch train loss is {epoch_train_loss:.4f}')\n\n \n time_elapsed = time.time() - start_time\n print(\"Time elapsed: \",time_elapsed)","metadata":{"_uuid":"771e9710-b29f-4483-8388-3560ebfd6cf3","_cell_guid":"ec246643-4d74-42f2-8387-deb9fb6e7f75","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:21.655937Z","iopub.execute_input":"2021-06-03T11:05:21.656288Z","iopub.status.idle":"2021-06-03T11:05:37.687819Z","shell.execute_reply.started":"2021-06-03T11:05:21.656253Z","shell.execute_reply":"2021-06-03T11:05:37.686879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots()\nfig.suptitle('Training Metrics')\n\naxes.set_ylabel(\"Loss\", fontsize=14)\naxes.plot(total_train_loss)\n\nplt.show()","metadata":{"_uuid":"52aecc89-1b1a-43e4-9c4a-81eb535cbbfc","_cell_guid":"d052847a-c406-4e53-8513-2488bd4e8454","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:37.689521Z","iopub.execute_input":"2021-06-03T11:05:37.689892Z","iopub.status.idle":"2021-06-03T11:05:37.809719Z","shell.execute_reply.started":"2021-06-03T11:05:37.689847Z","shell.execute_reply":"2021-06-03T11:05:37.809015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sum([param.nelement() for param in model.parameters()])","metadata":{"_uuid":"7c385816-38e2-4020-83c4-a1433ee0a4ab","_cell_guid":"7f08e22f-0948-4679-9af9-34af03ec4900","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:37.810853Z","iopub.execute_input":"2021-06-03T11:05:37.811184Z","iopub.status.idle":"2021-06-03T11:05:37.821648Z","shell.execute_reply.started":"2021-06-03T11:05:37.81115Z","shell.execute_reply":"2021-06-03T11:05:37.820728Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pathlib \nimport tensorflow as tf\n\npathlib.Path(\"/kaggle/working/model\").mkdir(parents=True, exist_ok=True)\n\n# Saving a model\nPATH_SAVE = pathlib.Path('/kaggle/working/model/checkpoint_save')\n\n# ## Saving only a checkpoint\ntorch.save({\n 'epoch': epoch,\n 'model_state_dict': model.state_dict(),\n 'optimizer_state_dict': optimizer.state_dict(),\n 'loss': epoch_train_loss,},\n PATH_SAVE)","metadata":{"_uuid":"b9e88007-af8e-4a44-b13d-1e3f65091289","_cell_guid":"e59d4ed9-93db-451e-aa08-87840c69af9f","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:37.822888Z","iopub.execute_input":"2021-06-03T11:05:37.823283Z","iopub.status.idle":"2021-06-03T11:05:38.545054Z","shell.execute_reply.started":"2021-06-03T11:05:37.823248Z","shell.execute_reply":"2021-06-03T11:05:38.54416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Calculation on Unlabeled Data","metadata":{"_uuid":"14edf68d-21b6-4c3e-8ccc-e483d4010853","_cell_guid":"5035ea77-0ef6-4d45-9014-b082466ad823","trusted":true}},{"cell_type":"code","source":"### Preparing data for Tests\n# I try to make validation on dataset, based on code from training\n# The difference is only in the mode of data and model\n\n# Dataset object\ndataset = MoleculeDetectionDataset(dataframe=None, \n image_dir=DIR_TEST_IMAGES, \n mode='test', \n transforms=get_transform())\n\nindices = torch.randperm(len(dataset)).tolist()\n\nvalid_dataset = torch.utils.data.Subset(dataset, indices)\n\nvalid_data_loader = DataLoader(\n valid_dataset,\n batch_size=4, #1,\n shuffle=True,\n num_workers=4, #1,\n collate_fn=collate_fn\n)","metadata":{"_uuid":"48527efe-c240-41ef-af17-a51d4a0736e2","_cell_guid":"1ba6521f-998c-42b6-9228-60d2c937be8f","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:38.546488Z","iopub.execute_input":"2021-06-03T11:05:38.546805Z","iopub.status.idle":"2021-06-03T11:05:38.652114Z","shell.execute_reply.started":"2021-06-03T11:05:38.546769Z","shell.execute_reply":"2021-06-03T11:05:38.651385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"threshold = THRESHOLD # unnecessary parameter\n\n# Indicator of iterations\nitr = 1\n\n# This sets model for prediction mode, not for training\nmodel.eval()\n\nmodel.to(device) \n\nlist_ = []\ndict_row = {}\nj = 0\n \nstart_time = time.time()\n \n#Retriving Mini-batch\nfor images, image_names in valid_data_loader:\n \n #Loading images & targets on device\n images = list(image.to(device) for image in images)\n # Prediction based on already trained model\n out = model(images)\n \n for i in range(len(out)):\n # Converting tensors to array\n boxes = out[i]['boxes'].data.cpu().numpy()\n scores = out[i]['scores'].data.cpu().numpy()\n labels = out[i]['labels'].data.cpu().numpy()\n\n # Thresholding\n boxes_th = boxes[scores >= threshold].astype(np.int32)\n scores_th = scores[scores >= threshold]\n\n # int_to_class - labels\n labels_th = []\n for x in range(len(labels)):\n if scores[x] > threshold:\n labels_th.append(int_to_class[labels[x]])\n\n # Appending results to csv\n for y in range(len(boxes_th)):\n\n # Bboxes, classname & image name\n xmin = boxes_th[y][0]\n ymin = boxes_th[y][1]\n xmax = boxes_th[y][2]\n ymax = boxes_th[y][3]\n class_name = labels_th[y]\n\n # Creating row for df\n dict_row[j] = {\"filename\": image_names[i], \"xmin\" : xmin, \"ymin\" : ymin, \n \"xmax\" : xmax, \"ymax\" : ymax, \"class\" : class_name}\n j += 1\n\n list_.append(image_names[i])\n \n itr += 1\n \n if itr % 50 == 0:\n print('itr = ', itr)\n\nsubmission = pd.DataFrame.from_dict(dict_row, \"index\") # very fast approach \ntime_elapsed = time.time() - start_time\nprint(\"Time elapsed: \",time_elapsed)","metadata":{"_uuid":"697e3881-0991-4799-afc8-7b4013d58a58","_cell_guid":"71de1752-5259-485d-98d3-8e91954126d7","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:05:38.653364Z","iopub.execute_input":"2021-06-03T11:05:38.653834Z","iopub.status.idle":"2021-06-03T11:06:33.473634Z","shell.execute_reply.started":"2021-06-03T11:05:38.653796Z","shell.execute_reply":"2021-06-03T11:06:33.472667Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# list(set(list_))","metadata":{"_uuid":"208222d1-7598-4176-a2d2-c9ff52f45270","_cell_guid":"1c7ca8f0-a65a-4758-9dc1-fc71944d12c9","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:33.475093Z","iopub.execute_input":"2021-06-03T11:06:33.475576Z","iopub.status.idle":"2021-06-03T11:06:33.480255Z","shell.execute_reply.started":"2021-06-03T11:06:33.475527Z","shell.execute_reply":"2021-06-03T11:06:33.479326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('threshold = ', threshold)\ndisplay(submission.describe())\ndisplay(submission.head())","metadata":{"_uuid":"39b693a5-a2a1-41dc-b39d-031ce3d5a321","_cell_guid":"bb856ef5-147f-4d7a-a5ce-f32fedfaeaf7","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:33.481455Z","iopub.execute_input":"2021-06-03T11:06:33.482034Z","iopub.status.idle":"2021-06-03T11:06:33.518634Z","shell.execute_reply.started":"2021-06-03T11:06:33.481955Z","shell.execute_reply":"2021-06-03T11:06:33.517701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Hydrogen in molecular formula.** Hydrogen can't be detected from the picture, there is no signs for hydrogen. I don't mean chemical groups such as NH, NH2, OH, ... Hydrogen that is connected to carbon has no signs. It should be calculated.\n\nCaclulation of hydrogen needs information about structure of a molecule. So, calculation of hydrogen is a complex task, much harder, than caclulation of carbon atoms!\n\nI think, it is too complex to solve it. That is why I didn't print hydrogen amount.","metadata":{"_uuid":"d5a674da-4543-407f-8bc3-e98f696c7811","_cell_guid":"ed00a1af-9fa8-4ce9-b69f-a3f988851add","trusted":true}},{"cell_type":"code","source":"def submission_to_inchi(submission):\n classes = ['h', 'p', 'n', 'o', 'c', 'cl', 's', 'b', 'si', 'i', 'f', 'br']\n #CHBBrClFINOPSSi\n ordered_output = ['c', 'h', 'b', 'br', 'cl', 'f', 'i', 'n', 'o', 'p', 's', 'si']\n \n list_output = [] # (file, inchi)\n \n files = submission['filename'].unique().tolist()\n for file in files:\n df = submission.loc[submission['filename'] == file]\n labels = df['class'].tolist()\n dictionary = {}\n \n for label in classes:\n if labels.count(label) == 0:\n continue\n elif label == 'h':\n dictionary[label] = ''\n elif labels.count(label) == 1:\n dictionary[label] = ''\n else:\n dictionary[label] = labels.count(label)\n \n formula = ''\n for item in ordered_output:\n if item in dictionary:\n formula += item.title()\n formula += str(dictionary[item])\n \n m = re.search(r'(.*)\\.[^.]+$', file)\n file_name = m.group(1)\n \n list_output.append((file_name, 'InChI=1S/' + formula + '/c1/h'))\n \n return list_output","metadata":{"_uuid":"8f4302c7-8a46-4ef2-9bdf-70fead3f5dc5","_cell_guid":"d8a67861-c604-4ddd-a3ec-b137b58ae0fe","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:33.521918Z","iopub.execute_input":"2021-06-03T11:06:33.522272Z","iopub.status.idle":"2021-06-03T11:06:33.530937Z","shell.execute_reply.started":"2021-06-03T11:06:33.522236Z","shell.execute_reply":"2021-06-03T11:06:33.529867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Write inchi to disk\nlist_submission = submission_to_inchi(submission)\n\n\nwith open(CSV_OUT, 'w') as out:\n csv_out = csv.writer(out, quotechar=None)\n csv_out.writerow(['image_id', 'InChI'])\n for row in list_submission:\n csv_out.writerow(row)","metadata":{"_uuid":"1d5f52f6-6126-4a37-8f64-7252ff43a092","_cell_guid":"6083712e-6803-46bc-8357-52899da721f0","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:33.532583Z","iopub.execute_input":"2021-06-03T11:06:33.533105Z","iopub.status.idle":"2021-06-03T11:06:34.823193Z","shell.execute_reply.started":"2021-06-03T11:06:33.533067Z","shell.execute_reply":"2021-06-03T11:06:34.822351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## Displaying Test Images\nfilepath_image_list = list(pathlib.Path(DIR_TEST_IMAGES).rglob('*.png'))\nfilepath_image_list.sort()\nimage_test = [x.name for x in filepath_image_list]\n\ncount = 0\nfor image in image_test:\n if len(image) > 9: \n plot_img(image, DIR_TEST_IMAGES, submission, verbose=0)\n print(image)\n count += 1\n # limit output files to ...\n if count > 5:\n break","metadata":{"_uuid":"16afb4e4-f2cc-4ce2-abd4-ff3e9cb3ce78","_cell_guid":"a3523fb4-4d22-418e-ab2a-5c296186658e","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:34.824398Z","iopub.execute_input":"2021-06-03T11:06:34.824735Z","iopub.status.idle":"2021-06-03T11:06:36.619354Z","shell.execute_reply.started":"2021-06-03T11:06:34.824697Z","shell.execute_reply":"2021-06-03T11:06:36.618384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"_uuid":"6457a709-008f-4338-b878-dc6eb4b49fb2","_cell_guid":"ec603e1a-5322-4ea4-aaef-fe692e16228e","collapsed":false,"jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"raise SystemExit(\"Stop right there!\")","metadata":{"_uuid":"8a360ec3-0014-449f-92ab-4ecd9e78965b","_cell_guid":"b9f1d308-d49c-4381-9bbb-2a4ba7ed4945","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:36.620547Z","iopub.execute_input":"2021-06-03T11:06:36.620878Z","iopub.status.idle":"2021-06-03T11:06:36.627536Z","shell.execute_reply.started":"2021-06-03T11:06:36.620841Z","shell.execute_reply":"2021-06-03T11:06:36.626028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import re\nimport pandas as pd\nimport unittest\n\nclass Test02(unittest.TestCase):\n def test_01(self):\n submission = pd.DataFrame({'filename': [\n '003781e47d63.png', \n '003781e47d63.png',\n '003781e47d63.png',\n '002781e47d63.png',\n '003781e47d63.png'],\n 'class': ['c', 'c', 'h', 'h', 'c'],\n 'xmin': [159, 191, 197, 159, 75],\n 'ymin': [115, 41, 114, 44, 112],\n 'xmax': [187, 225, 229, 186, 108],\n 'ymax': [142, 72, 143, 76, 140]\n })\n \n list_ = submission_to_inchi(submission)\n print(list_)\n \n def test_02(self):\n submission = pd.DataFrame({'filename': [\n '003781e47d63.png', \n '003781e47d63.png',\n '003781e47d63.png',\n '003781e47d63.png',\n '003781e47d63.png'],\n 'class': ['c', 'c', 'p', 'p', 'c'],\n 'xmin': [159, 191, 197, 159, 75],\n 'ymin': [115, 41, 114, 44, 112],\n 'xmax': [187, 225, 229, 186, 108],\n 'ymax': [142, 72, 143, 76, 140]\n })\n \n list_ = submission_to_inchi(submission)\n print(list_)\n \n \nunittest.main(argv=[''], verbosity=2, exit=False)","metadata":{"_uuid":"95e32756-9acb-4143-a5aa-8e75acb7ba2e","_cell_guid":"65b24266-1b3c-4173-88c9-189bbb3f1e33","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:36.628756Z","iopub.status.idle":"2021-06-03T11:06:36.629732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"_uuid":"ae534fe6-09cf-4a49-a865-95614f0b6e59","_cell_guid":"dae60a4b-8779-4cf6-81ba-d1b8632a585f","collapsed":false,"jupyter":{"outputs_hidden":false},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv('submission.csv', index=False)","metadata":{"_uuid":"262cf8b5-e17b-45e1-8837-04edce90c590","_cell_guid":"ba8ec574-0f1d-480c-beb5-5a607b09e229","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:36.631256Z","iopub.status.idle":"2021-06-03T11:06:36.631902Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Clear the directory\n\nimport os\nimport glob\n\nfiles = glob.glob('/kaggle/working/*')\nfor f in files:\n os.remove(f)","metadata":{"_uuid":"3977648f-ede5-4e83-a3aa-e532bbd694dd","_cell_guid":"0fe04b32-ec8e-43de-946c-5e29803958f1","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2021-06-03T11:06:36.633543Z","iopub.status.idle":"2021-06-03T11:06:36.634128Z"},"trusted":true},"execution_count":null,"outputs":[]}]}