\n
![\"images\"](\"https://i.postimg.cc/t4KqLMNC/inchi.jpg\")
First, let me congratulate all the winners and thank the competition hosts - it was a really interesting challenge! It was great to be a part of a competition requiring both CV and NLP skills and having such a large dataset with a very minimal shakeup.
\nThe (not so) funny thing is that I was confident that the competition deadline is actually two days later, 05.06. I mistakingly created a calendar entry for the deadline and never checked if it's actually correct. What a surprise for me was to check out the competition page yesterday and realize that it was just 10 hours before the end! 😅
\nMy solution is an ensemble of seven CNN-LSTM Encoder-Decoder models. All models are implemented in PyTorch and trained on a local machine with Quadro RTX 6000 GPU.
\nextra_approved_InChIs using RDKit (thanks @tuckerarrants)image = cv2.morphologyEx(image, cv2.MORPH_OPEN, np.ones((2, 2)))\nimage = cv2.erode(image, np.ones((2, 2)))\nShiftScaleRotate(0.01, 0.01, 0.10)\nAll models used EfficientNet-based CNN encoder and LSTM-based RNN decoder. The table below from Neptune.ai provides the main architecture and training parameters of my top models:
\n![\"models\"](\"https://i.postimg.cc/cLrTp1Pc/Screen-2021-06-04-at-10-17-02.jpg\")
The final solution achieves 1.32 on the public and 1.31 on the private LB (47th place). I hope this summary was useful for some of you. Happy to answer any questions in the comments and see you in the next competitions! 😊
", "messages": [ { "id": 1335454, "postDate": "2021-06-04T08:26:48.510Z", "content": "First, let me congratulate all the winners and thank the competition hosts - it was a really interesting challenge! It was great to be a part of a competition requiring both CV and NLP skills and having such a large dataset with a very minimal shakeup.
\nThe (not so) funny thing is that I was confident that the competition deadline is actually two days later, 05.06. I mistakingly created a calendar entry for the deadline and never checked if it's actually correct. What a surprise for me was to check out the competition page yesterday and realize that it was just 10 hours before the end! 😅
\nMy solution is an ensemble of seven CNN-LSTM Encoder-Decoder models. All models are implemented in PyTorch and trained on a local machine with Quadro RTX 6000 GPU.
\nextra_approved_InChIs using RDKit (thanks @tuckerarrants)image = cv2.morphologyEx(image, cv2.MORPH_OPEN, np.ones((2, 2)))\nimage = cv2.erode(image, np.ones((2, 2)))\nShiftScaleRotate(0.01, 0.01, 0.10)\nAll models used EfficientNet-based CNN encoder and LSTM-based RNN decoder. The table below from Neptune.ai provides the main architecture and training parameters of my top models:
\n
The final solution achieves 1.32 on the public and 1.31 on the private LB (47th place). I hope this summary was useful for some of you. Happy to answer any questions in the comments and see you in the next competitions! 😊
", "rawMarkdown": "### Summary\n\nFirst, let me congratulate all the winners and thank the competition hosts - it was a really interesting challenge! It was great to be a part of a competition requiring both CV and NLP skills and having such a large dataset with a very minimal shakeup. \n\nThe (not so) funny thing is that I was confident that the competition deadline is actually two days later, 05.06. I mistakingly created a calendar entry for the deadline and never checked if it's actually correct. What a surprise for me was to check out the competition page yesterday and realize that it was just 10 hours before the end! 😅\n\nMy solution is an ensemble of seven CNN-LSTM Encoder-Decoder models. All models are implemented in PyTorch and trained on a local machine with Quadro RTX 6000 GPU.\n\n\n### Code\n- [Kaggle notebook](https://www.kaggle.com/kozodoi/47th-place-solution-bms-ensembling) reproducing my submission and ensembling pipeline\n- [GitHub repo](https://github.com/kozodoi/BMS_Molecular_Translation) with the complete training codes\n\n\n### Data\n- single 80/20 train/test split stratified by the molecule length\n- creating 3M extra images from `extra_approved_InChIs` using RDKit (thanks @tuckerarrants)\n- on each epoch, I was taking all train images + random 1M extra images for training\n\n\n### Tokenizer\n- my tokenizer is very similar to the one proposed by @yasufuminakama in his great pipeline\n- using training + extra data to fit the tokenizer, so it ended up having about 20 more tokens due to more distinct molecules in the data\n\n\n### Image augmentations\n- making lines and letters thicker with cv2 morphological transformations:\n```\nimage = cv2.morphologyEx(image, cv2.MORPH_OPEN, np.ones((2, 2)))\nimage = cv2.erode(image, np.ones((2, 2)))\n```\n- small image rotations during training:\n```\nShiftScaleRotate(0.01, 0.01, 0.10)\n```\n- cropping pictures with large empty borders (thanks @markwijkhuizen)\n- rotating test images with height > width (thanks @hengck23)\n\n\n\n### Base models\nAll models used EfficientNet-based CNN encoder and LSTM-based RNN decoder. The table below from Neptune.ai provides the main architecture and training parameters of my top models:\n\n\n\n### Ensembling & post-processing \n- using beam search with k = 5 on most of the base models (thanks @tugstugi)\n- normalizing each model predictions with RDKit (thanks @nofreewill)\n- ensembling predictions from base models using majority voting\n- setting \"unconfident\" predictions to the ones produced by the lowest-CV model that has a valid prediction according to RDKit (thanks @nofreewill again)\n\n\n### What else I planned to try\n- adding image meta-data such as pixel resolution and height/width ratio as input features\n- iteratively predicting different parts of the InChI (formula, /c and /h parts) with separate models that would take image and predictions of the previous molecule part as input\n- switching to transformers. I only started working on transformer pipeline a few days ago and did not have enough time to train a heavy model due to limited resources\n\nThe final solution achieves **1.32** on the public and **1.31** on the private LB (47th place). I hope this summary was useful for some of you. Happy to answer any questions in the comments and see you in the next competitions! 😊", "votes": 31 }, { "id": 1345865, "postDate": "2021-06-11T22:34:52.537Z", "content": "Good work, its helpful
", "rawMarkdown": "Good work, its helpful", "votes": 1 }, { "id": 1344286, "postDate": "2021-06-10T19:16:08.737Z", "content": "Nice! Good job
", "rawMarkdown": "Nice! Good job" }, { "id": 1339101, "postDate": "2021-06-07T03:28:09.053Z", "content": "Nice keep up
", "rawMarkdown": "Nice keep up" }, { "id": 1338984, "postDate": "2021-06-06T23:24:46.457Z", "content": "Thanks for sharing your write-up. I have also read your blog and you have inspired me to try using neptune.ai on my next project.
", "rawMarkdown": "Thanks for sharing your write-up. I have also read your blog and you have inspired me to try using neptune.ai on my next project. ", "replies": [ { "id": 1339404, "postDate": "2021-06-07T08:16:31.300Z", "content": "@talktocharles great to hear that, hope it will be useful for you future projects. Thanks!
", "rawMarkdown": "@talktocharles great to hear that, hope it will be useful for you future projects. Thanks!", "votes": 1 } ] }, { "id": 1338477, "postDate": "2021-06-06T13:40:32.973Z", "content": "Congrats and thanks for the great write-up and your released code! 👍
", "rawMarkdown": "Congrats and thanks for the great write-up and your released code! 👍", "replies": [ { "id": 1338871, "postDate": "2021-06-06T19:52:55.860Z", "content": "Thanks @alexlwh! I am glad you found it helpful.
", "rawMarkdown": "Thanks @alexlwh! I am glad you found it helpful.", "votes": 1 } ] }, { "id": 1344701, "postDate": "2021-06-11T04:53:43.433Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1345865, "author_name": "Paramatma Pulivarthi", "author_url": "", "post_date": "2021-06-11T22:34:52.537000", "content": "Good work, its helpful
", "votes": 1, "replies": [] }, { "id": 1344286, "author_name": "Razvan Milicin", "author_url": "", "post_date": "2021-06-10T19:16:08.737000", "content": "Nice! Good job
", "votes": 0, "replies": [] }, { "id": 1339101, "author_name": "Mohamed Essam", "author_url": "", "post_date": "2021-06-07T03:28:09.053000", "content": "Nice keep up
", "votes": 0, "replies": [] }, { "id": 1338984, "author_name": "Charles", "author_url": "", "post_date": "2021-06-06T23:24:46.457000", "content": "Thanks for sharing your write-up. I have also read your blog and you have inspired me to try using neptune.ai on my next project.
", "votes": 0, "replies": [ { "id": 1339404, "author_name": "Nikita Kozodoi", "author_url": "", "post_date": "2021-06-07T08:16:31.300000", "content": "@talktocharles great to hear that, hope it will be useful for you future projects. Thanks!
", "votes": 1, "replies": [] } ] }, { "id": 1338477, "author_name": "Alex Lau", "author_url": "", "post_date": "2021-06-06T13:40:32.973000", "content": "Congrats and thanks for the great write-up and your released code! 👍
", "votes": 0, "replies": [ { "id": 1338871, "author_name": "Nikita Kozodoi", "author_url": "", "post_date": "2021-06-06T19:52:55.860000", "content": "Thanks @alexlwh! I am glad you found it helpful.
", "votes": 1, "replies": [] } ] }, { "id": 1344701, "author_name": "", "author_url": "", "post_date": "2021-06-11T04:53:43.433000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1335454": "### Summary\n\nFirst, let me congratulate all the winners and thank the competition hosts - it was a really interesting challenge! It was great to be a part of a competition requiring both CV and NLP skills and having such a large dataset with a very minimal shakeup. \n\nThe (not so) funny thing is that I was confident that the competition deadline is actually two days later, 05.06. I mistakingly created a calendar entry for the deadline and never checked if it's actually correct. What a surprise for me was to check out the competition page yesterday and realize that it was just 10 hours before the end! 😅\n\nMy solution is an ensemble of seven CNN-LSTM Encoder-Decoder models. All models are implemented in PyTorch and trained on a local machine with Quadro RTX 6000 GPU.\n\n\n### Code\n- [Kaggle notebook](https://www.kaggle.com/kozodoi/47th-place-solution-bms-ensembling) reproducing my submission and ensembling pipeline\n- [GitHub repo](https://github.com/kozodoi/BMS_Molecular_Translation) with the complete training codes\n\n\n### Data\n- single 80/20 train/test split stratified by the molecule length\n- creating 3M extra images from `extra_approved_InChIs` using RDKit (thanks @tuckerarrants)\n- on each epoch, I was taking all train images + random 1M extra images for training\n\n\n### Tokenizer\n- my tokenizer is very similar to the one proposed by @yasufuminakama in his great pipeline\n- using training + extra data to fit the tokenizer, so it ended up having about 20 more tokens due to more distinct molecules in the data\n\n\n### Image augmentations\n- making lines and letters thicker with cv2 morphological transformations:\n```\nimage = cv2.morphologyEx(image, cv2.MORPH_OPEN, np.ones((2, 2)))\nimage = cv2.erode(image, np.ones((2, 2)))\n```\n- small image rotations during training:\n```\nShiftScaleRotate(0.01, 0.01, 0.10)\n```\n- cropping pictures with large empty borders (thanks @markwijkhuizen)\n- rotating test images with height > width (thanks @hengck23)\n\n\n\n### Base models\nAll models used EfficientNet-based CNN encoder and LSTM-based RNN decoder. The table below from Neptune.ai provides the main architecture and training parameters of my top models:\n\n\n\n### Ensembling & post-processing \n- using beam search with k = 5 on most of the base models (thanks @tugstugi)\n- normalizing each model predictions with RDKit (thanks @nofreewill)\n- ensembling predictions from base models using majority voting\n- setting \"unconfident\" predictions to the ones produced by the lowest-CV model that has a valid prediction according to RDKit (thanks @nofreewill again)\n\n\n### What else I planned to try\n- adding image meta-data such as pixel resolution and height/width ratio as input features\n- iteratively predicting different parts of the InChI (formula, /c and /h parts) with separate models that would take image and predictions of the previous molecule part as input\n- switching to transformers. I only started working on transformer pipeline a few days ago and did not have enough time to train a heavy model due to limited resources\n\nThe final solution achieves **1.32** on the public and **1.31** on the private LB (47th place). I hope this summary was useful for some of you. Happy to answer any questions in the comments and see you in the next competitions! 😊", "1345865": "Good work, its helpful", "1344286": "Nice! Good job", "1339101": "Nice keep up", "1338984": "Thanks for sharing your write-up. I have also read your blog and you have inspired me to try using neptune.ai on my next project. ", "1338477": "Congrats and thanks for the great write-up and your released code! 👍", "1344701": "" } }