The first step to the solution is to make efficient feature Extraction. The problem that ImageNet weights will not extract appropriate features. That is why I will publish later weights that should perform better. I will attach also the algorithm used to train the feature extractor (i will not be a part of this competition but I hope I will help as much as I could).
\nThe second step I am thinking of is using GANS and try to solve this problem as an image captioning problem.
\nIf you want also to go in this direction, then these links could be helpful:
\nhttps://github.com/aimagelab/meshed-memory-transformer
\nhttps://github.com/LuoweiZhou/VLP
\nhttps://github.com/rakshithShetty/captionGAN
\nhttps://paperswithcode.com/task/image-captioning
\nThe starter I already published is using Encoder and decoder for the features will another loss function small changes could also give good results but in general, it is not a good solution.
IMPORTANT NOTES:
\nDefining rules for the labels will help a lot and I can see directly a lot of rules that will make the solution less complicated.
\nI see this as a Transformers, GANS competition.
\nHope that will be useful. Best of luck to all of you
UPDATE:
\nThe first step to the solution is to make efficient feature Extraction. The problem that ImageNet weights will not extract appropriate features. That is why I will publish later weights that should perform better. I will attach also the algorithm used to train the feature extractor (i will not be a part of this competition but I hope I will help as much as I could).
\nThe second step I am thinking of is using GANS and try to solve this problem as an image captioning problem.
\nIf you want also to go in this direction, then these links could be helpful:
\nhttps://github.com/aimagelab/meshed-memory-transformer
\nhttps://github.com/LuoweiZhou/VLP
\nhttps://github.com/rakshithShetty/captionGAN
\nhttps://paperswithcode.com/task/image-captioning
\nThe starter I already published is using Encoder and decoder for the features will another loss function small changes could also give good results but in general, it is not a good solution.
IMPORTANT NOTES:
\nDefining rules for the labels will help a lot and I can see directly a lot of rules that will make the solution less complicated.
\nI see this as a Transformers, GANS competition.
\nHope that will be useful. Best of luck to all of you
UPDATE:
\nI understand the role of transformer in this competition. What is the role of GAN?
", "rawMarkdown": "I understand the role of transformer in this competition. What is the role of GAN? ", "votes": 1, "replies": [ { "id": 1244710, "postDate": "2021-03-19T07:11:45.580Z", "content": "I will refer to these papers:
\nhttps://ieeexplore.ieee.org/document/8545049
\nhttps://www.researchgate.net/publication/335699458_Conditional_GANs_for_Image_Captioning_with_Sentiments
\nhttps://vigilworkshop.github.io/static/papers/38.pdf
Wow! Interesting! This competition just got exciting for me.
\nNever really thought generative models can be a major ingredient in a ML competition.
", "rawMarkdown": "Wow! Interesting! This competition just got exciting for me.\n\nNever really thought generative models can be a major ingredient in a ML competition.", "votes": 2 }, { "id": 1244728, "postDate": "2021-03-19T07:30:50.033Z", "content": "Yes, it is a really interesting competition. Unfortunately, 4 Million images need high computational resources. Good luck and keep the great work
", "rawMarkdown": "Yes, it is a really interesting competition. Unfortunately, 4 Million images need high computational resources. Good luck and keep the great work " }, { "id": 1245194, "postDate": "2021-03-19T14:55:59.780Z", "content": "Yea, training a generative model on 4-million images will need quite a bit of compute time.
\nOn the flip side, unlike RANZCR where high res and large models are keys, I don't think you need a particularly large model (large model wouldn't hurt though) for this.
\nI may revisit this after my thesis submission and presentation.
", "rawMarkdown": "Yea, training a generative model on 4-million images will need quite a bit of compute time.\n\nOn the flip side, unlike RANZCR where high res and large models are keys, I don't think you need a particularly large model (large model wouldn't hurt though) for this. \n\nI may revisit this after my thesis submission and presentation." } ] }, { "id": 1235154, "postDate": "2021-03-11T21:54:37.550Z", "content": "https://pypi.org/project/ChemSpiPy/ is limited to 1000 calls per month…
", "rawMarkdown": "https://pypi.org/project/ChemSpiPy/ is limited to 1000 calls per month...", "votes": 1, "replies": [ { "id": 1244708, "postDate": "2021-03-19T07:07:15.347Z", "content": "I am sorry i didn't know that
", "rawMarkdown": "I am sorry i didn't know that " } ] }, { "id": 1231063, "postDate": "2021-03-08T16:34:17.993Z", "content": "@ammarali32 Why will you not be part of this competition ?
", "rawMarkdown": "@ammarali32 Why will you not be part of this competition ?", "votes": 1, "replies": [ { "id": 1231586, "postDate": "2021-03-09T05:00:54.153Z", "content": "Thanks for your interest. Actually, I don't have time or resources for large data I am just using Kaggle resources ))
", "rawMarkdown": "Thanks for your interest. Actually, I don't have time or resources for large data I am just using Kaggle resources ))", "votes": 1 }, { "id": 1246302, "postDate": "2021-03-20T16:19:25.487Z", "content": "Are you still using only Kaggle resources? If yes, then I am really curious to know how did you achieve this score because I don't have resources either other than Kaggle.
\n@ammarali32
Thanks for posting!
\n\n\nDefining rules for the labels will help a lot and I can see directly a lot of rules that will make the solution less complicated.
\n
Sorry it wasn't clear to me, are you referring to writing a fn that would correct the o/p from models to correct the notation generated based on what molecules are actually possible?
", "rawMarkdown": "Thanks for posting! \n\n> Defining rules for the labels will help a lot and I can see directly a lot of rules that will make the solution less complicated.\n\nSorry it wasn't clear to me, are you referring to writing a fn that would correct the o/p from models to correct the notation generated based on what molecules are actually possible?", "votes": 1, "replies": [ { "id": 1231584, "postDate": "2021-03-09T04:59:25.617Z", "content": "Not exactly but for example assume that you are able with high accuracy to predict the number of carbon atoms and the other atoms is already written in this case you can check the number of Hydrogen atoms in \"chemistry\" mathematically. This presentation \"InChi\" describes the distribution and relations for a particular molecular. you need to know how to write InChi representation manually after that a lot of rules for auto-correction will help.
", "rawMarkdown": "Not exactly but for example assume that you are able with high accuracy to predict the number of carbon atoms and the other atoms is already written in this case you can check the number of Hydrogen atoms in \"chemistry\" mathematically. This presentation \"InChi\" describes the distribution and relations for a particular molecular. you need to know how to write InChi representation manually after that a lot of rules for auto-correction will help.", "votes": 1 } ] }, { "id": 1225048, "postDate": "2021-03-03T09:14:37.803Z", "content": "Interesting review from a Data Polymath
", "rawMarkdown": "Interesting review from a Data Polymath", "votes": 2, "replies": [ { "id": 1225054, "postDate": "2021-03-03T09:16:07.050Z", "content": "Thank you))
", "rawMarkdown": "Thank you))", "votes": 2 }, { "id": 1231062, "postDate": "2021-03-08T16:33:26.987Z", "content": "@kingabzpro Looks like you changed the color of your profile pic from red to black .
", "rawMarkdown": "@kingabzpro Looks like you changed the color of your profile pic from red to black .", "votes": 1 }, { "id": 1231070, "postDate": "2021-03-08T16:39:02.313Z", "content": "yes, just like Kaggle changed from color to black, just trying to blend in.
", "rawMarkdown": "yes, just like Kaggle changed from color to black, just trying to blend in. " } ] }, { "id": 1227077, "postDate": "2021-03-05T07:09:04.237Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1244699, "author_name": "JunYong Tong", "author_url": "", "post_date": "2021-03-19T06:55:55.687000", "content": "I understand the role of transformer in this competition. What is the role of GAN?
", "votes": 1, "replies": [ { "id": 1244710, "author_name": "ammarali32", "author_url": "", "post_date": "2021-03-19T07:11:45.580000", "content": "I will refer to these papers:
\nhttps://ieeexplore.ieee.org/document/8545049
\nhttps://www.researchgate.net/publication/335699458_Conditional_GANs_for_Image_Captioning_with_Sentiments
\nhttps://vigilworkshop.github.io/static/papers/38.pdf
Wow! Interesting! This competition just got exciting for me.
\nNever really thought generative models can be a major ingredient in a ML competition.
", "votes": 2, "replies": [] }, { "id": 1244728, "author_name": "ammarali32", "author_url": "", "post_date": "2021-03-19T07:30:50.033000", "content": "Yes, it is a really interesting competition. Unfortunately, 4 Million images need high computational resources. Good luck and keep the great work
", "votes": 0, "replies": [] }, { "id": 1245194, "author_name": "JunYong Tong", "author_url": "", "post_date": "2021-03-19T14:55:59.780000", "content": "Yea, training a generative model on 4-million images will need quite a bit of compute time.
\nOn the flip side, unlike RANZCR where high res and large models are keys, I don't think you need a particularly large model (large model wouldn't hurt though) for this.
\nI may revisit this after my thesis submission and presentation.
", "votes": 0, "replies": [] } ] }, { "id": 1235154, "author_name": "Alexander Scarlat MD", "author_url": "", "post_date": "2021-03-11T21:54:37.550000", "content": "https://pypi.org/project/ChemSpiPy/ is limited to 1000 calls per month…
", "votes": 1, "replies": [ { "id": 1244708, "author_name": "ammarali32", "author_url": "", "post_date": "2021-03-19T07:07:15.347000", "content": "I am sorry i didn't know that
", "votes": 0, "replies": [] } ] }, { "id": 1231063, "author_name": "Tensor Girl", "author_url": "", "post_date": "2021-03-08T16:34:17.993000", "content": "@ammarali32 Why will you not be part of this competition ?
", "votes": 1, "replies": [ { "id": 1231586, "author_name": "ammarali32", "author_url": "", "post_date": "2021-03-09T05:00:54.153000", "content": "Thanks for your interest. Actually, I don't have time or resources for large data I am just using Kaggle resources ))
", "votes": 1, "replies": [] }, { "id": 1246302, "author_name": "Shweta Goyal", "author_url": "", "post_date": "2021-03-20T16:19:25.487000", "content": "Are you still using only Kaggle resources? If yes, then I am really curious to know how did you achieve this score because I don't have resources either other than Kaggle.
\n@ammarali32
Thanks for posting!
\n\n\nDefining rules for the labels will help a lot and I can see directly a lot of rules that will make the solution less complicated.
\n
Sorry it wasn't clear to me, are you referring to writing a fn that would correct the o/p from models to correct the notation generated based on what molecules are actually possible?
", "votes": 1, "replies": [ { "id": 1231584, "author_name": "ammarali32", "author_url": "", "post_date": "2021-03-09T04:59:25.617000", "content": "Not exactly but for example assume that you are able with high accuracy to predict the number of carbon atoms and the other atoms is already written in this case you can check the number of Hydrogen atoms in \"chemistry\" mathematically. This presentation \"InChi\" describes the distribution and relations for a particular molecular. you need to know how to write InChi representation manually after that a lot of rules for auto-correction will help.
", "votes": 1, "replies": [] } ] }, { "id": 1225048, "author_name": "Abid Ali Awan", "author_url": "", "post_date": "2021-03-03T09:14:37.803000", "content": "Interesting review from a Data Polymath
", "votes": 2, "replies": [ { "id": 1225054, "author_name": "ammarali32", "author_url": "", "post_date": "2021-03-03T09:16:07.050000", "content": "Thank you))
", "votes": 2, "replies": [] }, { "id": 1231062, "author_name": "Tensor Girl", "author_url": "", "post_date": "2021-03-08T16:33:26.987000", "content": "@kingabzpro Looks like you changed the color of your profile pic from red to black .
", "votes": 1, "replies": [] }, { "id": 1231070, "author_name": "Abid Ali Awan", "author_url": "", "post_date": "2021-03-08T16:39:02.313000", "content": "yes, just like Kaggle changed from color to black, just trying to blend in.
", "votes": 0, "replies": [] } ] }, { "id": 1227077, "author_name": "", "author_url": "", "post_date": "2021-03-05T07:09:04.237000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1224767": "The first step to the solution is to make efficient feature Extraction. The problem that ImageNet weights will not extract appropriate features. That is why I will publish later weights that should perform better. I will attach also the algorithm used to train the feature extractor (i will not be a part of this competition but I hope I will help as much as I could).\nThe second step I am thinking of is using GANS and try to solve this problem as an image captioning problem. \nIf you want also to go in this direction, then these links could be helpful:\nhttps://github.com/aimagelab/meshed-memory-transformer\nhttps://github.com/LuoweiZhou/VLP\nhttps://github.com/rakshithShetty/captionGAN\nhttps://paperswithcode.com/task/image-captioning\nThe starter I already published is using Encoder and decoder for the features will another loss function small changes could also give good results but in general, it is not a good solution.\n\nIMPORTANT NOTES:\nDefining rules for the labels will help a lot and I can see directly a lot of rules that will make the solution less complicated.\nI see this as a Transformers, GANS competition.\nHope that will be useful. Best of luck to all of you\n\nUPDATE:\n- Trained model to get the molecular smiles Representation\nhttps://github.com/Kohulan/DECIMER-Image-to-SMILES \n- Open source library to convert from SMILES to InChi\nhttps://pypi.org/project/ChemSpiPy/\n", "1244699": "I understand the role of transformer in this competition. What is the role of GAN? ", "1235154": "https://pypi.org/project/ChemSpiPy/ is limited to 1000 calls per month...", "1231063": "@ammarali32 Why will you not be part of this competition ?", "1230684": "Thanks for posting! \n\n> Defining rules for the labels will help a lot and I can see directly a lot of rules that will make the solution less complicated.\n\nSorry it wasn't clear to me, are you referring to writing a fn that would correct the o/p from models to correct the notation generated based on what molecules are actually possible?", "1225048": "Interesting review from a Data Polymath", "1227077": "" } }