I found the following paper:
\nAbstract:
\nThe automatic recognition of chemical structure diagrams from the literature is an indispensable component of workfows to re-discover information about chemicals and to make it available in open-access databases. Here we report preliminary fndings in our development of Deep lEarning for Chemical ImagE Recognition (DECIMER), a deep learning method based on existing show-and-tell deep neural networks, which makes very few assumptions about the structure of the underlying problem. It translates a bitmap image of a molecule, as found in publications, into a SMILES. The training state reported here does not yet rival the performance of existing traditional approaches, but we present evidence that our method will reach a comparable detection power with sufcient training time. Training success of DECIMER depends on the input data representation: DeepSMILES are superior over SMILES and we have a preliminary indication that the recently reported SELFIES outperform DeepSMILES. An extrapolation of our results towards larger training data sizes suggests that we might be able to achieve near-accurate prediction with 50 to 100 million training structures. This work is entirely based on open-source software and open data and is available to the general public for any purpose.
Code: https://github.com/Kohulan/DECIMER-Image-to-SMILES
\nCan we repurpose the pipeline of this work to target International Chemical Identifier (InChI) instead of SMILES? I hope to open a passionate discussion here.
\nI want to add that the another work called \"CLIP: Connecting\nText and Images\" could potentiate this.
", "messages": [ { "id": 1224590, "postDate": "2021-03-02T21:57:58.660Z", "content": "I found the following paper:
\nAbstract:
\nThe automatic recognition of chemical structure diagrams from the literature is an indispensable component of workfows to re-discover information about chemicals and to make it available in open-access databases. Here we report preliminary fndings in our development of Deep lEarning for Chemical ImagE Recognition (DECIMER), a deep learning method based on existing show-and-tell deep neural networks, which makes very few assumptions about the structure of the underlying problem. It translates a bitmap image of a molecule, as found in publications, into a SMILES. The training state reported here does not yet rival the performance of existing traditional approaches, but we present evidence that our method will reach a comparable detection power with sufcient training time. Training success of DECIMER depends on the input data representation: DeepSMILES are superior over SMILES and we have a preliminary indication that the recently reported SELFIES outperform DeepSMILES. An extrapolation of our results towards larger training data sizes suggests that we might be able to achieve near-accurate prediction with 50 to 100 million training structures. This work is entirely based on open-source software and open data and is available to the general public for any purpose.
Code: https://github.com/Kohulan/DECIMER-Image-to-SMILES
\nCan we repurpose the pipeline of this work to target International Chemical Identifier (InChI) instead of SMILES? I hope to open a passionate discussion here.
\nI want to add that the another work called \"CLIP: Connecting\nText and Images\" could potentiate this.
", "rawMarkdown": "I found the following [paper](https://jcheminf.biomedcentral.com/track/pdf/10.1186/s13321-020-00469-w.pdf):\n\n**Abstract**:\nThe automatic recognition of chemical structure diagrams from the literature is an indispensable component of workfows to re-discover information about chemicals and to make it available in open-access databases. Here we report preliminary fndings in our development of *Deep lEarning for Chemical ImagE Recognition (DECIMER)*, a deep learning method based on existing show-and-tell deep neural networks, which makes very few assumptions about the structure of the underlying problem. It translates a bitmap image of a molecule, as found in publications, into a *SMILES*. The training state reported here does not yet rival the performance of existing traditional approaches, but we present evidence that our method will reach a comparable detection power with sufcient training time. Training success of *DECIMER* depends on the input data representation: *DeepSMILES* are superior over *SMILES* and we have a preliminary indication that the recently reported *SELFIES* outperform *DeepSMILES*. An extrapolation of our results towards larger training data sizes suggests that we might be able to achieve near-accurate prediction with 50 to 100 million training structures. This work is entirely based on open-source software and open data and is available to the general public for any purpose.\n\nCode: https://github.com/Kohulan/DECIMER-Image-to-SMILES\n\nCan we repurpose the pipeline of this work to target **[International Chemical Identifier (InChI)](https://en.wikipedia.org/wiki/International_Chemical_Identifier)** instead of **SMILES**? I hope to open a passionate discussion here.\n\nI want to add that the another work called \"[CLIP: Connecting\nText and Images](https://openai.com/blog/clip/)\" could potentiate this.", "votes": 26 }, { "id": 1231437, "postDate": "2021-03-09T01:33:13.523Z", "content": "DECIMER-Segmentation: Automated extraction of chemical structure depictions from scientific literature | Journal of Cheminformatics
\nhttps://jcheminf.biomedcentral.com/articles/10.1186/s13321-021-00496-1
DECIMER: towards deep learning for chemical image recognition | Journal of Cheminformatics
\nhttps://jcheminf.biomedcentral.com/articles/10.1186/s13321-020-00469-w
A new work based on DECIMER, that demonstrates the robustness of the original work, thanks for share.
", "rawMarkdown": "A new work based on DECIMER, that demonstrates the robustness of the original work, thanks for share.", "votes": 1 } ] }, { "id": 1232853, "postDate": "2021-03-10T02:57:07.450Z", "content": "@hiramcho The idea of autoencoder based deep learning architecture to solve chemical image recognition sounds interesting . Thanks for sharing
", "rawMarkdown": "@hiramcho The idea of autoencoder based deep learning architecture to solve chemical image recognition sounds interesting . Thanks for sharing ", "votes": 1 }, { "id": 1236532, "postDate": "2021-03-13T08:27:16.570Z", "content": "ChemPix: Automated Recognition of Hand-drawn Hydrocarbon Structures Using Deep Learning
\nhttps://chemrxiv.org/articles/preprint/ChemPix_Automated_Recognition_of_Hand-drawn_Hydrocarbon_Structures_Using_Deep_Learning/14156957/1
Another interesting paper. They developed a phone application with their model, that's impressive.
", "rawMarkdown": "Another interesting paper. They developed a phone application with their model, that's impressive." } ] }, { "id": 1234994, "postDate": "2021-03-11T18:27:49.943Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1225124, "postDate": "2021-03-03T10:40:17.110Z", "content": "thanks 💯
", "rawMarkdown": "thanks 💯", "votes": 3 }, { "id": 1225700, "postDate": "2021-03-03T20:48:41.170Z", "content": "Thanks for sharing😳
", "rawMarkdown": "Thanks for sharing😳", "votes": 1 } ], "comments": [ { "id": 1231437, "author_name": "Neko Jara Shi", "author_url": "", "post_date": "2021-03-09T01:33:13.523000", "content": "DECIMER-Segmentation: Automated extraction of chemical structure depictions from scientific literature | Journal of Cheminformatics
\nhttps://jcheminf.biomedcentral.com/articles/10.1186/s13321-021-00496-1
DECIMER: towards deep learning for chemical image recognition | Journal of Cheminformatics
\nhttps://jcheminf.biomedcentral.com/articles/10.1186/s13321-020-00469-w
A new work based on DECIMER, that demonstrates the robustness of the original work, thanks for share.
", "votes": 1, "replies": [] } ] }, { "id": 1232853, "author_name": "Tensor Girl", "author_url": "", "post_date": "2021-03-10T02:57:07.450000", "content": "@hiramcho The idea of autoencoder based deep learning architecture to solve chemical image recognition sounds interesting . Thanks for sharing
", "votes": 1, "replies": [] }, { "id": 1236532, "author_name": "Neko Jara Shi", "author_url": "", "post_date": "2021-03-13T08:27:16.570000", "content": "ChemPix: Automated Recognition of Hand-drawn Hydrocarbon Structures Using Deep Learning
\nhttps://chemrxiv.org/articles/preprint/ChemPix_Automated_Recognition_of_Hand-drawn_Hydrocarbon_Structures_Using_Deep_Learning/14156957/1
Another interesting paper. They developed a phone application with their model, that's impressive.
", "votes": 0, "replies": [] } ] }, { "id": 1234994, "author_name": "", "author_url": "", "post_date": "2021-03-11T18:27:49.943000", "content": "", "votes": 0, "replies": [] }, { "id": 1225124, "author_name": "maoz ragab", "author_url": "", "post_date": "2021-03-03T10:40:17.110000", "content": "thanks 💯
", "votes": 3, "replies": [] }, { "id": 1225700, "author_name": "Satori", "author_url": "", "post_date": "2021-03-03T20:48:41.170000", "content": "Thanks for sharing😳
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "1224590": "I found the following [paper](https://jcheminf.biomedcentral.com/track/pdf/10.1186/s13321-020-00469-w.pdf):\n\n**Abstract**:\nThe automatic recognition of chemical structure diagrams from the literature is an indispensable component of workfows to re-discover information about chemicals and to make it available in open-access databases. Here we report preliminary fndings in our development of *Deep lEarning for Chemical ImagE Recognition (DECIMER)*, a deep learning method based on existing show-and-tell deep neural networks, which makes very few assumptions about the structure of the underlying problem. It translates a bitmap image of a molecule, as found in publications, into a *SMILES*. The training state reported here does not yet rival the performance of existing traditional approaches, but we present evidence that our method will reach a comparable detection power with sufcient training time. Training success of *DECIMER* depends on the input data representation: *DeepSMILES* are superior over *SMILES* and we have a preliminary indication that the recently reported *SELFIES* outperform *DeepSMILES*. An extrapolation of our results towards larger training data sizes suggests that we might be able to achieve near-accurate prediction with 50 to 100 million training structures. This work is entirely based on open-source software and open data and is available to the general public for any purpose.\n\nCode: https://github.com/Kohulan/DECIMER-Image-to-SMILES\n\nCan we repurpose the pipeline of this work to target **[International Chemical Identifier (InChI)](https://en.wikipedia.org/wiki/International_Chemical_Identifier)** instead of **SMILES**? I hope to open a passionate discussion here.\n\nI want to add that the another work called \"[CLIP: Connecting\nText and Images](https://openai.com/blog/clip/)\" could potentiate this.", "1231437": "DECIMER-Segmentation: Automated extraction of chemical structure depictions from scientific literature | Journal of Cheminformatics\n[https://jcheminf.biomedcentral.com/articles/10.1186/s13321-021-00496-1](https://jcheminf.biomedcentral.com/articles/10.1186/s13321-021-00496-1)\n\nDECIMER: towards deep learning for chemical image recognition | Journal of Cheminformatics\n[https://jcheminf.biomedcentral.com/articles/10.1186/s13321-020-00469-w](https://jcheminf.biomedcentral.com/articles/10.1186/s13321-020-00469-w)\n", "1232853": "@hiramcho The idea of autoencoder based deep learning architecture to solve chemical image recognition sounds interesting . Thanks for sharing ", "1236532": "ChemPix: Automated Recognition of Hand-drawn Hydrocarbon Structures Using Deep Learning\nhttps://chemrxiv.org/articles/preprint/ChemPix_Automated_Recognition_of_Hand-drawn_Hydrocarbon_Structures_Using_Deep_Learning/14156957/1\n", "1234994": "", "1225124": "thanks 💯", "1225700": "Thanks for sharing😳" } }