I'm trying hybrid method using logical process and deep learning. I think this method has possibility to get high score if it's doing well, and we can see where it's doing well and not. Actually, I haven't success by this method….If you are interested in this strategy, please advise to my method's issue.
\nMy procedure is as follows:
\n(1) To detect kinds of atoms and bonds by tensorflow object detection (OD).
\n - kinds of atoms: by other discussion, kinds of atoms are almost fixed.
\n B,Br,Cl,F,I,N,O,P,S,Su (visually expressed), C (cross section of bond or edge of bond),
\n H (unexpressed) in the train/test images
\n -kinds of bonds: single, double, triple (here, I only detect only double and triple bonding by OD)
\n - I tagged several hundred images.
(2) From bounding box of OD, To calculate position (center of bounding box, x,y coordinations) \n of atoms and double/triple bond.
\n(3) To number atoms from the left.
\n (I want to show image, but this board does not receive image..)
(4) from the result of (3), To make bonding matrix of the subject molecule.
\n -bonding matrix: the matrix which have lows and columns of all atoms number
\n whose value means 0:no bond, 1:single bond, 2:duble, 3:triple
\n -On all atoms in the molecule, to search neighbor atoms by distance calculated by position of
\n atoms. and to judge boding between subject atom and neighbor atom as whether this is no bond, single, double, or triple by CNN.
\n -the CNN receives images of area which is defined by (atom1.x, atom1.y)-(atom2.x, atom2.y).
\n here '.x' and '.y' means x/y coordination.
\n -CNN has to be trained before by image of bonds made by the train molecule image
(5) From the bonding matrix, To make 'rdkit mol block format file'
\n -rdkit mol block format need number of atoms and bonds, and bond kind between atoms,
\n which can be made by bonding matrix.
(6) To make Inchl from the 'mol block format file' by rdkit
\n -rdkit know writing rule of Inchl and where exist hydrogen bonding etc which is not expressed
\n in molecule image explicitly.
I could do (1)-(3) by not so bad precision. but in CNN judging bond kind of (4), I could not
\nget good precision as of now.
I think CNN-Attention model (end to end model) do similar things in neural network .
", "messages": [ { "id": "1282930", "postDate": "04/24/2021 12:13:01", "content": "I'm trying hybrid method using logical process and deep learning. I think this method has possibility to get high score if it's doing well, and we can see where it's doing well and not. Actually, I haven't success by this method….If you are interested in this strategy, please advise to my method's issue.
\nMy procedure is as follows:
\n(1) To detect kinds of atoms and bonds by tensorflow object detection (OD).
\n - kinds of atoms: by other discussion, kinds of atoms are almost fixed.
\n B,Br,Cl,F,I,N,O,P,S,Su (visually expressed), C (cross section of bond or edge of bond),
\n H (unexpressed) in the train/test images
\n -kinds of bonds: single, double, triple (here, I only detect only double and triple bonding by OD)
\n - I tagged several hundred images.
(2) From bounding box of OD, To calculate position (center of bounding box, x,y coordinations) \n of atoms and double/triple bond.
\n(3) To number atoms from the left.
\n (I want to show image, but this board does not receive image..)
(4) from the result of (3), To make bonding matrix of the subject molecule.
\n -bonding matrix: the matrix which have lows and columns of all atoms number
\n whose value means 0:no bond, 1:single bond, 2:duble, 3:triple
\n -On all atoms in the molecule, to search neighbor atoms by distance calculated by position of
\n atoms. and to judge boding between subject atom and neighbor atom as whether this is no bond, single, double, or triple by CNN.
\n -the CNN receives images of area which is defined by (atom1.x, atom1.y)-(atom2.x, atom2.y).
\n here '.x' and '.y' means x/y coordination.
\n -CNN has to be trained before by image of bonds made by the train molecule image
(5) From the bonding matrix, To make 'rdkit mol block format file'
\n -rdkit mol block format need number of atoms and bonds, and bond kind between atoms,
\n which can be made by bonding matrix.
(6) To make Inchl from the 'mol block format file' by rdkit
\n -rdkit know writing rule of Inchl and where exist hydrogen bonding etc which is not expressed
\n in molecule image explicitly.
I could do (1)-(3) by not so bad precision. but in CNN judging bond kind of (4), I could not
\nget good precision as of now.
I think CNN-Attention model (end to end model) do similar things in neural network .
", "rawMarkdown": "I'm trying hybrid method using logical process and deep learning. I think this method has possibility to get high score if it's doing well, and we can see where it's doing well and not. Actually, I haven't success by this method....If you are interested in this strategy, please advise to my method's issue.\n\nMy procedure is as follows:\n**(1) To detect kinds of atoms and bonds by tensorflow object detection (OD).**\n - kinds of atoms: by other discussion, kinds of atoms are almost fixed.\n B,Br,Cl,F,I,N,O,P,S,Su (visually expressed), C (cross section of bond or edge of bond), \n H (unexpressed) in the train/test images\n -kinds of bonds: single, double, triple (here, I only detect only double and triple bonding by OD)\n - I tagged several hundred images.\n\n**(2) From bounding box of OD, To calculate position (center of bounding box, x,y coordinations) \n of atoms and double/triple bond.**\n\n**(3) To number atoms from the left.**\n (I want to show image, but this board does not receive image..)\n\n**(4) from the result of (3), To make bonding matrix of the subject molecule.**\n -bonding matrix: the matrix which have lows and columns of all atoms number \n whose value means 0:no bond, 1:single bond, 2:duble, 3:triple\n -On all atoms in the molecule, to search neighbor atoms by distance calculated by position of\n atoms. and to judge boding between subject atom and neighbor atom as whether this is no bond, single, double, or triple by CNN. \n -the CNN receives images of area which is defined by (atom1.x, atom1.y)-(atom2.x, atom2.y).\n here '.x' and '.y' means x/y coordination.\n -CNN has to be trained before by image of bonds made by the train molecule image\n\n**(5) From the bonding matrix, To make 'rdkit mol block format file'**\n -rdkit mol block format need number of atoms and bonds, and bond kind between atoms,\n which can be made by bonding matrix.\n\n**(6) To make Inchl from the 'mol block format file' by rdkit**\n -rdkit know writing rule of Inchl and where exist hydrogen bonding etc which is not expressed\n in molecule image explicitly.\n\nI could do (1)-(3) by not so bad precision. but in CNN judging bond kind of (4), I could not\nget good precision as of now.\n\nI think CNN-Attention model (end to end model) do similar things in neural network .", "votes": null }, { "id": "1283002", "postDate": "04/24/2021 13:41:54", "content": "Since the image is crappy, several hundred training images are far from enough. I would assume 100x of your hand curated training set. In addition, you also need to train the single bond bbox to reconstruct the full graph. Absence of bbox signal != bbox is absent.
\nHowever, in many situations, bond may be missing (e.g., single bond -> nothing, double bond -> single bond, triple bond -> double/single bond). This makes the training extremely difficult. Fixing these errors are more dependent on the molecular context, e.g., a missing bond appears in a aromatic ring or breaks the molecule. Finally, all the bbox models need to achieve very high accuracy in order to get a decent LB score, because even one bond/atom misrecognition will inflate LD by 30-100 (depending on the molecular size).
", "rawMarkdown": "Since the image is crappy, several hundred training images are far from enough. I would assume 100x of your hand curated training set. In addition, you also need to train the single bond bbox to reconstruct the full graph. Absence of bbox signal != bbox is absent.\n\nHowever, in many situations, bond may be missing (e.g., single bond -> nothing, double bond -> single bond, triple bond -> double/single bond). This makes the training extremely difficult. Fixing these errors are more dependent on the molecular context, e.g., a missing bond appears in a aromatic ring or breaks the molecule. Finally, all the bbox models need to achieve very high accuracy in order to get a decent LB score, because even one bond/atom misrecognition will inflate LD by 30-100 (depending on the molecular size).", "votes": null }, { "id": "1283875", "postDate": "04/25/2021 11:09:15", "content": "Thank you for your comment. correct !
\nI actually made several thousands of tagged \"bond\" image to train CNN to judge bonding.
\nbut when the CNN judge the image between atoms, the image include sometime some noise such as
\nbond between neighbor atom1 - neighbor atom2 (not subject atom - neighbor atom) when actually there is no bond between subject atom - neighbor atom. this reduce CNN judgement precision.
Also, \"in many situations, bond may be missing\" is correct. When I tagged the train image, I found many this kinds of missing in benzene Ring. Yes, in this case, my current procedure does not work.
\nI though pix2pix or auto encoder might be needed to recover the image/bond before process (1)
\n….but in my next step.
Yes, this procedure need 100% precision in (1) and (4) to make one molecule Inchl. As a result of my trial, this is not so easy.
\nIf end to end model (RNN-attention captioning model etc) can automatically make these procedure
\nand rule only by the data and is robust for the noise, this is very impressive for me. this means neural network might go beyond my procedure. I need to learn more about it. Thank you
Since the image is crappy, several hundred training images are far from enough. I would assume 100x of your hand curated training set. In addition, you also need to train the single bond bbox to reconstruct the full graph. Absence of bbox signal != bbox is absent.
\nHowever, in many situations, bond may be missing (e.g., single bond -> nothing, double bond -> single bond, triple bond -> double/single bond). This makes the training extremely difficult. Fixing these errors are more dependent on the molecular context, e.g., a missing bond appears in a aromatic ring or breaks the molecule. Finally, all the bbox models need to achieve very high accuracy in order to get a decent LB score, because even one bond/atom misrecognition will inflate LD by 30-100 (depending on the molecular size).
", "votes": null, "replies": [ { "id": 1283875, "author_name": "kevin20181209", "author_url": "", "post_date": "04/25/2021 11:09:15", "content": "Thank you for your comment. correct !
\nI actually made several thousands of tagged \"bond\" image to train CNN to judge bonding.
\nbut when the CNN judge the image between atoms, the image include sometime some noise such as
\nbond between neighbor atom1 - neighbor atom2 (not subject atom - neighbor atom) when actually there is no bond between subject atom - neighbor atom. this reduce CNN judgement precision.
Also, \"in many situations, bond may be missing\" is correct. When I tagged the train image, I found many this kinds of missing in benzene Ring. Yes, in this case, my current procedure does not work.
\nI though pix2pix or auto encoder might be needed to recover the image/bond before process (1)
\n….but in my next step.
Yes, this procedure need 100% precision in (1) and (4) to make one molecule Inchl. As a result of my trial, this is not so easy.
\nIf end to end model (RNN-attention captioning model etc) can automatically make these procedure
\nand rule only by the data and is robust for the noise, this is very impressive for me. this means neural network might go beyond my procedure. I need to learn more about it. Thank you