{ "id": 229984, "title": "Dacon Top Solution Outline", "url": "/competitions/bms-molecular-translation/discussion/229984", "author_name": "", "post_date": "2021-04-01T14:02:55.219040400Z", "votes": 43, "comment_count": 23, "views": 0, "content": "

Given I had the same idea as winners of Dacon molecular to smiles competition I can share the top approach. I only looked a their slides hence my memories may not be very accurate. I welcome corrections from those who were fast enough to clone their code.

\n
    \n
  1. Generate images from train labels. There is a great public notebook by @stainsby about it in this competition.
    2. \n
  2. Generate bounding boxes for atoms and bonds, again from train labels. This requires an analysis of the svg representation of molecules generated by rdkit. The notebook cited above can be a good starting point.
    3. \n
  3. Train an object detection model. I think that they used detectron2.
    4. \n
  4. Infer bboxes on test data.
    5. \n
  5. Create a molecule graph from predicted bboxes. The main trick is to match bboxes for bonds with bboxes for atoms: a bond must have two atoms at each end.
    6. \n
  6. Generate inchi from predicted molecules
    \nI think rdkit can also be used for last step. If not, maybe openbabel?
    7. \n
\n

Edit: you can find a link to clones of top Dacon competition solutions here

", "messages": [ { "id": "1259565", "postDate": "04/01/2021 14:02:55", "content": "

Given I had the same idea as winners of Dacon molecular to smiles competition I can share the top approach. I only looked a their slides hence my memories may not be very accurate. I welcome corrections from those who were fast enough to clone their code.

\n
    \n
  1. Generate images from train labels. There is a great public notebook by @stainsby about it in this competition.
    2. \n
  2. Generate bounding boxes for atoms and bonds, again from train labels. This requires an analysis of the svg representation of molecules generated by rdkit. The notebook cited above can be a good starting point.
    3. \n
  3. Train an object detection model. I think that they used detectron2.
    4. \n
  4. Infer bboxes on test data.
    5. \n
  5. Create a molecule graph from predicted bboxes. The main trick is to match bboxes for bonds with bboxes for atoms: a bond must have two atoms at each end.
    6. \n
  6. Generate inchi from predicted molecules
    \nI think rdkit can also be used for last step. If not, maybe openbabel?
    7. \n
\n

Edit: you can find a link to clones of top Dacon competition solutions here

", "rawMarkdown": "Given I had the same idea as winners of Dacon molecular to smiles competition I can share the top approach. I only looked a their slides hence my memories may not be very accurate. I welcome corrections from those who were fast enough to clone their code.\n\n1. Generate images from train labels. There is a [great public notebook](https://www.kaggle.com/stainsby/improved-synthetic-data-for-bms-competition-v3?scriptVersionId=58294825) by @stainsby about it in this competition.\n2. Generate bounding boxes for atoms and bonds, again from train labels. This requires an analysis of the svg representation of molecules generated by rdkit. The notebook cited above can be a good starting point.\n3. Train an object detection model. I think that they used detectron2.\n4. Infer bboxes on test data. \n5. Create a molecule graph from predicted bboxes. The main trick is to match bboxes for bonds with bboxes for atoms: a bond must have two atoms at each end.\n6. Generate inchi from predicted molecules\nI think rdkit can also be used for last step. If not, maybe openbabel?\n\nEdit: you can find a link to clones of top Dacon competition solutions [here](https://www.kaggle.com/c/bms-molecular-translation/discussion/229978)", "votes": null }, { "id": "1259575", "postDate": "04/01/2021 14:12:57", "content": "

I can add about 2nd place solution:
\n1) Train Densnet
\n2) Use it to extract feature vector -> image embeddings
\n3) use nn.Embedings to embed SMILE - > text embeddings (they use tenserflow)
\n4) concat (image embeddings with text embeddings) - > image_text_embedding
\n5) add cosine positional encodings to `image_text_embedding
\n6) from here feed to standard transformer architecture (6 layer encoder 6 layer decoder)

", "rawMarkdown": "I can add about `2nd place solution:`\n1) Train Densnet \n2) Use it to extract feature vector -> `image` embeddings \n3) use nn.Embedings to embed `SMILE` - > `text` embeddings (they use tenserflow)\n4) concat (`image` embeddings with `text` embeddings) - > `image_text_embedding`\n5) add `cosine` positional encodings to `image_text_embedding\n6) from here feed to standard transformer architecture (6 layer encoder 6 layer decoder)", "votes": null }, { "id": "1259583", "postDate": "04/01/2021 14:18:29", "content": "

Is it what you use?

\n

I don't really expect an answer ;)

\n

Thanks for sharing.

", "rawMarkdown": "Is it what you use? \n\nI don't really expect an answer ;)\n\nThanks for sharing.", "votes": null }, { "id": "1259587", "postDate": "04/01/2021 14:20:40", "content": "

hahah =) I can be honest we don't use this. We also not using bounding box solution=)

", "rawMarkdown": "hahah =) I can be honest we don't use this. We also not using bounding box solution=)", "votes": null }, { "id": "1259611", "postDate": "04/01/2021 14:44:54", "content": "

what do you think about applying graph neural networks to this problem?

", "rawMarkdown": "what do you think about applying graph neural networks to this problem?", "votes": null }, { "id": "1259656", "postDate": "04/01/2021 15:06:49", "content": "

I don't know as I am entering the comp. I guess I'll go with what I shared above first, but who knows what will work?

", "rawMarkdown": "I don't know as I am entering the comp. I guess I'll go with what I shared above first, but who knows what will work?", "votes": null }, { "id": "1259660", "postDate": "04/01/2021 15:09:27", "content": "

I hadn't even considered doing something like this. This is a pretty cool idea.

\n

Thank you for sharing!

", "rawMarkdown": "I hadn't even considered doing something like this. This is a pretty cool idea.\n\nThank you for sharing!", "votes": null }, { "id": "1259661", "postDate": "04/01/2021 15:09:30", "content": "

If I remember correctly, the 1st place solution generated images for the labels that there was no image counterpart for.

", "rawMarkdown": "If I remember correctly, the 1st place solution generated images for the labels that there was no image counterpart for.", "votes": null }, { "id": "1259694", "postDate": "04/01/2021 15:36:42", "content": "

I'm not quite understand about the third step… Where can I get the SMILE->text embeddings.

", "rawMarkdown": "I'm not quite understand about the third step... Where can I get the SMILE->text embeddings.", "votes": null }, { "id": "1259701", "postDate": "04/01/2021 15:42:50", "content": "

Thank me only if it works ;)

", "rawMarkdown": "Thank me only if it works ;)", "votes": null }, { "id": "1259710", "postDate": "04/01/2021 15:47:11", "content": "

In that completion they used SMILES we are using INCHI. SMILES is just different representation.

", "rawMarkdown": "In that completion they used `SMILES` we are using `INCHI`. SMILES is just different representation.", "votes": null }, { "id": "1259725", "postDate": "04/01/2021 16:00:51", "content": "

Yes, I see. But during the inference time we can't access the ground truth text. The 3rd step is just a normal decode step? Like feeding the \"<SOS>\" to the model first?

", "rawMarkdown": "Yes, I see. But during the inference time we can't access the ground truth text. The 3rd step is just a normal decode step? Like feeding the `\"\"` to the model first?", "votes": null }, { "id": "1259734", "postDate": "04/01/2021 16:05:33", "content": "

yes that is my understanding.

", "rawMarkdown": "yes that is my understanding.", "votes": null }, { "id": "1259735", "postDate": "04/01/2021 16:06:52", "content": "

Ha, Thanks for your explanation😁

", "rawMarkdown": "Ha, Thanks for your explanation😁", "votes": null }, { "id": "1260692", "postDate": "04/02/2021 10:52:12", "content": "

scene graph for image caption?

", "rawMarkdown": "scene graph for image caption?", "votes": null }, { "id": "1260743", "postDate": "04/02/2021 11:39:27", "content": "

Tx, I didn't know scene graph was a key word. There are indeed quite a few publications for image captioning using scene graph, like this https://arxiv.org/pdf/2007.11731.pdf

\n

I don't know if this is the same as the molecular graph they used in Dacon competition, but I'll read some papers to check.

", "rawMarkdown": "Tx, I didn't know scene graph was a key word. There are indeed quite a few publications for image captioning using scene graph, like this https://arxiv.org/pdf/2007.11731.pdf\n\nI don't know if this is the same as the molecular graph they used in Dacon competition, but I'll read some papers to check.", "votes": null }, { "id": "1261025", "postDate": "04/02/2021 16:22:49", "content": "

As a noob, I have one question that How can I generate the boundary box for images? Dataset is huge to generate boundary box manually will be so hard.

", "rawMarkdown": "As a noob, I have one question that How can I generate the boundary box for images? Dataset is huge to generate boundary box manually will be so hard.", "votes": null }, { "id": "1261032", "postDate": "04/02/2021 16:32:57", "content": "

Doing it manually is not feasible indeed. You ay want to look at code shared there: https://www.kaggle.com/c/bms-molecular-translation/discussion/229978

", "rawMarkdown": "Doing it manually is not feasible indeed. You ay want to look at code shared there: https://www.kaggle.com/c/bms-molecular-translation/discussion/229978", "votes": null }, { "id": "1261135", "postDate": "04/02/2021 18:09:06", "content": "

@cpmpml check this !!!!!

\n

\"\"

", "rawMarkdown": "cpmpml check this !!!!!\n\n![](https://i.ibb.co/4WXQhXB/Selection-031.png)", "votes": null }, { "id": "1261147", "postDate": "04/02/2021 18:20:50", "content": "

https://openreview.net/pdf?id=XEw5Onu69uu
\nSELF-LABELING OF FULLY MEDIATING REPRESENTATIONS BY GRAPH ALIGNMENT

\n

\"\"

\n

once we have the graph, then we can feed it into InchI generator API. This guarantee valid Inchi

", "rawMarkdown": "https://openreview.net/pdf?id=XEw5Onu69uu\nSELF-LABELING OF FULLY MEDIATING REPRESENTATIONS BY GRAPH ALIGNMENT\n\n![](https://i.ibb.co/VWJktfq/Selection-032.png)\n\n\nonce we have the graph, then we can feed it into InchI generator API. This guarantee valid Inchi", "votes": null }, { "id": "1261156", "postDate": "04/02/2021 18:25:35", "content": "

another related topics: RETROSYNTHESIS

", "rawMarkdown": "another related topics: RETROSYNTHESIS", "votes": null }, { "id": "1261882", "postDate": "04/03/2021 14:41:11", "content": "

I read the chemgrpaher paper but I haven't read the second one.

", "rawMarkdown": "I read the chemgrpaher paper but I haven't read the second one.", "votes": null }, { "id": "1264402", "postDate": "04/06/2021 06:10:24", "content": "

I was thinking along the same lines. The difference is that I thought it would be cool to use a GNN and the recursive prediction schema people were using, maybe even a GAN GNN.

", "rawMarkdown": "I was thinking along the same lines. The difference is that I thought it would be cool to use a GNN and the recursive prediction schema people were using, maybe even a GAN GNN.", "votes": null }, { "id": "1274730", "postDate": "04/15/2021 14:52:29", "content": "

Is anybody tried to experiment with 1place solution code
\nto adapt it to inchi?

\n

I tried this with no effort (just replaced
\nMolToSmiles to MolToInchi and other insignificant stuff).

\n

Got bad score - mean levenstein is ~ 50
\non the validation set and a lot of warnings in rdkit,
\ntrying to investigate further.

", "rawMarkdown": "Is anybody tried to experiment with 1place solution code\nto adapt it to inchi?\n\nI tried this with no effort (just replaced \nMolToSmiles to MolToInchi and other insignificant stuff). \n\nGot bad score - mean levenstein is ~ 50 \non the validation set and a lot of warnings in rdkit,\ntrying to investigate further.", "votes": null } ], "comments": [ { "id": 1259575, "author_name": "drhabib", "author_url": "", "post_date": "04/01/2021 14:12:57", "content": "

I can add about 2nd place solution:
\n1) Train Densnet
\n2) Use it to extract feature vector -> image embeddings
\n3) use nn.Embedings to embed SMILE - > text embeddings (they use tenserflow)
\n4) concat (image embeddings with text embeddings) - > image_text_embedding
\n5) add cosine positional encodings to `image_text_embedding
\n6) from here feed to standard transformer architecture (6 layer encoder 6 layer decoder)

", "votes": null, "replies": [ { "id": 1259583, "author_name": "cpmpml", "author_url": "", "post_date": "04/01/2021 14:18:29", "content": "

Is it what you use?

\n

I don't really expect an answer ;)

\n

Thanks for sharing.

", "votes": null, "replies": [] }, { "id": 1259587, "author_name": "drhabib", "author_url": "", "post_date": "04/01/2021 14:20:40", "content": "

hahah =) I can be honest we don't use this. We also not using bounding box solution=)

", "votes": null, "replies": [] }, { "id": 1259661, "author_name": "nofreewill", "author_url": "", "post_date": "04/01/2021 15:09:30", "content": "

If I remember correctly, the 1st place solution generated images for the labels that there was no image counterpart for.

", "votes": null, "replies": [] }, { "id": 1259694, "author_name": "tonyxu", "author_url": "", "post_date": "04/01/2021 15:36:42", "content": "

I'm not quite understand about the third step… Where can I get the SMILE->text embeddings.

", "votes": null, "replies": [] }, { "id": 1259710, "author_name": "drhabib", "author_url": "", "post_date": "04/01/2021 15:47:11", "content": "

In that completion they used SMILES we are using INCHI. SMILES is just different representation.

", "votes": null, "replies": [] }, { "id": 1259725, "author_name": "tonyxu", "author_url": "", "post_date": "04/01/2021 16:00:51", "content": "

Yes, I see. But during the inference time we can't access the ground truth text. The 3rd step is just a normal decode step? Like feeding the \"<SOS>\" to the model first?

", "votes": null, "replies": [] }, { "id": 1259734, "author_name": "drhabib", "author_url": "", "post_date": "04/01/2021 16:05:33", "content": "

yes that is my understanding.

", "votes": null, "replies": [] }, { "id": 1259735, "author_name": "tonyxu", "author_url": "", "post_date": "04/01/2021 16:06:52", "content": "

Ha, Thanks for your explanation😁

", "votes": null, "replies": [] } ] }, { "id": 1259611, "author_name": "skull8888888", "author_url": "", "post_date": "04/01/2021 14:44:54", "content": "

what do you think about applying graph neural networks to this problem?

", "votes": null, "replies": [ { "id": 1259656, "author_name": "cpmpml", "author_url": "", "post_date": "04/01/2021 15:06:49", "content": "

I don't know as I am entering the comp. I guess I'll go with what I shared above first, but who knows what will work?

", "votes": null, "replies": [] } ] }, { "id": 1259660, "author_name": "dschettler8845", "author_url": "", "post_date": "04/01/2021 15:09:27", "content": "

I hadn't even considered doing something like this. This is a pretty cool idea.

\n

Thank you for sharing!

", "votes": null, "replies": [ { "id": 1259701, "author_name": "cpmpml", "author_url": "", "post_date": "04/01/2021 15:42:50", "content": "

Thank me only if it works ;)

", "votes": null, "replies": [] } ] }, { "id": 1260692, "author_name": "hengck23", "author_url": "", "post_date": "04/02/2021 10:52:12", "content": "

scene graph for image caption?

", "votes": null, "replies": [ { "id": 1260743, "author_name": "cpmpml", "author_url": "", "post_date": "04/02/2021 11:39:27", "content": "

Tx, I didn't know scene graph was a key word. There are indeed quite a few publications for image captioning using scene graph, like this https://arxiv.org/pdf/2007.11731.pdf

\n

I don't know if this is the same as the molecular graph they used in Dacon competition, but I'll read some papers to check.

", "votes": null, "replies": [] } ] }, { "id": 1261025, "author_name": "aifahim", "author_url": "", "post_date": "04/02/2021 16:22:49", "content": "

As a noob, I have one question that How can I generate the boundary box for images? Dataset is huge to generate boundary box manually will be so hard.

", "votes": null, "replies": [ { "id": 1261032, "author_name": "cpmpml", "author_url": "", "post_date": "04/02/2021 16:32:57", "content": "

Doing it manually is not feasible indeed. You ay want to look at code shared there: https://www.kaggle.com/c/bms-molecular-translation/discussion/229978

", "votes": null, "replies": [] } ] }, { "id": 1261135, "author_name": "hengck23", "author_url": "", "post_date": "04/02/2021 18:09:06", "content": "

@cpmpml check this !!!!!

\n

\"\"

", "votes": null, "replies": [ { "id": 1261147, "author_name": "hengck23", "author_url": "", "post_date": "04/02/2021 18:20:50", "content": "

https://openreview.net/pdf?id=XEw5Onu69uu
\nSELF-LABELING OF FULLY MEDIATING REPRESENTATIONS BY GRAPH ALIGNMENT

\n

\"\"

\n

once we have the graph, then we can feed it into InchI generator API. This guarantee valid Inchi

", "votes": null, "replies": [] }, { "id": 1261156, "author_name": "hengck23", "author_url": "", "post_date": "04/02/2021 18:25:35", "content": "

another related topics: RETROSYNTHESIS

", "votes": null, "replies": [] }, { "id": 1261882, "author_name": "cpmpml", "author_url": "", "post_date": "04/03/2021 14:41:11", "content": "

I read the chemgrpaher paper but I haven't read the second one.

", "votes": null, "replies": [] } ] }, { "id": 1264402, "author_name": "eladwar", "author_url": "", "post_date": "04/06/2021 06:10:24", "content": "

I was thinking along the same lines. The difference is that I thought it would be cool to use a GNN and the recursive prediction schema people were using, maybe even a GAN GNN.

", "votes": null, "replies": [] }, { "id": 1274730, "author_name": "kruntuid", "author_url": "", "post_date": "04/15/2021 14:52:29", "content": "

Is anybody tried to experiment with 1place solution code
\nto adapt it to inchi?

\n

I tried this with no effort (just replaced
\nMolToSmiles to MolToInchi and other insignificant stuff).

\n

Got bad score - mean levenstein is ~ 50
\non the validation set and a lot of warnings in rdkit,
\ntrying to investigate further.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1259565": "Given I had the same idea as winners of Dacon molecular to smiles competition I can share the top approach. I only looked a their slides hence my memories may not be very accurate. I welcome corrections from those who were fast enough to clone their code.\n\n1. Generate images from train labels. There is a [great public notebook](https://www.kaggle.com/stainsby/improved-synthetic-data-for-bms-competition-v3?scriptVersionId=58294825) by @stainsby about it in this competition.\n2. Generate bounding boxes for atoms and bonds, again from train labels. This requires an analysis of the svg representation of molecules generated by rdkit. The notebook cited above can be a good starting point.\n3. Train an object detection model. I think that they used detectron2.\n4. Infer bboxes on test data. \n5. Create a molecule graph from predicted bboxes. The main trick is to match bboxes for bonds with bboxes for atoms: a bond must have two atoms at each end.\n6. Generate inchi from predicted molecules\nI think rdkit can also be used for last step. If not, maybe openbabel?\n\nEdit: you can find a link to clones of top Dacon competition solutions [here](https://www.kaggle.com/c/bms-molecular-translation/discussion/229978)", "1259575": "I can add about `2nd place solution:`\n1) Train Densnet \n2) Use it to extract feature vector -> `image` embeddings \n3) use nn.Embedings to embed `SMILE` - > `text` embeddings (they use tenserflow)\n4) concat (`image` embeddings with `text` embeddings) - > `image_text_embedding`\n5) add `cosine` positional encodings to `image_text_embedding\n6) from here feed to standard transformer architecture (6 layer encoder 6 layer decoder)", "1259583": "Is it what you use? \n\nI don't really expect an answer ;)\n\nThanks for sharing.", "1259587": "hahah =) I can be honest we don't use this. We also not using bounding box solution=)", "1259611": "what do you think about applying graph neural networks to this problem?", "1259656": "I don't know as I am entering the comp. I guess I'll go with what I shared above first, but who knows what will work?", "1259660": "I hadn't even considered doing something like this. This is a pretty cool idea.\n\nThank you for sharing!", "1259661": "If I remember correctly, the 1st place solution generated images for the labels that there was no image counterpart for.", "1259694": "I'm not quite understand about the third step... Where can I get the SMILE->text embeddings.", "1259701": "Thank me only if it works ;)", "1259710": "In that completion they used `SMILES` we are using `INCHI`. SMILES is just different representation.", "1259725": "Yes, I see. But during the inference time we can't access the ground truth text. The 3rd step is just a normal decode step? Like feeding the `\"\"` to the model first?", "1259734": "yes that is my understanding.", "1259735": "Ha, Thanks for your explanation😁", "1260692": "scene graph for image caption?", "1260743": "Tx, I didn't know scene graph was a key word. There are indeed quite a few publications for image captioning using scene graph, like this https://arxiv.org/pdf/2007.11731.pdf\n\nI don't know if this is the same as the molecular graph they used in Dacon competition, but I'll read some papers to check.", "1261025": "As a noob, I have one question that How can I generate the boundary box for images? Dataset is huge to generate boundary box manually will be so hard.", "1261032": "Doing it manually is not feasible indeed. You ay want to look at code shared there: https://www.kaggle.com/c/bms-molecular-translation/discussion/229978", "1261135": "cpmpml check this !!!!!\n\n![](https://i.ibb.co/4WXQhXB/Selection-031.png)", "1261147": "https://openreview.net/pdf?id=XEw5Onu69uu\nSELF-LABELING OF FULLY MEDIATING REPRESENTATIONS BY GRAPH ALIGNMENT\n\n![](https://i.ibb.co/VWJktfq/Selection-032.png)\n\n\nonce we have the graph, then we can feed it into InchI generator API. This guarantee valid Inchi", "1261156": "another related topics: RETROSYNTHESIS", "1261882": "I read the chemgrpaher paper but I haven't read the second one.", "1264402": "I was thinking along the same lines. The difference is that I thought it would be cool to use a GNN and the recursive prediction schema people were using, maybe even a GAN GNN.", "1274730": "Is anybody tried to experiment with 1place solution code\nto adapt it to inchi?\n\nI tried this with no effort (just replaced \nMolToSmiles to MolToInchi and other insignificant stuff). \n\nGot bad score - mean levenstein is ~ 50 \non the validation set and a lot of warnings in rdkit,\ntrying to investigate further." }, "source": "meta" }