{ "id": 229992, "title": "Help With TF2 Image Captioning Pipeline", "url": "/competitions/bms-molecular-translation/discussion/229992", "author_name": "Darien Schettler", "post_date": "2021-04-01T15:02:09.450000", "votes": 4, "comment_count": 0, "views": 0, "content": "

Hi there, I am struggling with my implementation (TF 2+) of an image captioning pipeline. I wanted to reach out for help from the Kaggle community.

\n

I have gotten it to 'work', but the scores are less than stellar.

\n


\n

Here is a link to the notebook: NOTEBOOK

\n

Here is a link to the tutorials that inspired the notebook: TF TUTORIAL 1, TF TUTORIAL 2

\n

Here is a link to the paper the decoder architecture is based off of: Show, Attend and Tell

\n


\n

Any feedback would be appreciated… I want to make a simple, easy-to-understand notebook to teach people while learning as I go (similar to this notebook by YNakuma in Pytorch). I believe I should be able to achieve a distance of ~20 or less with the method I am attempting. I was on the fence about posting this but I figured I should reach out for help while in parallel also troubleshooting it on my own.

\n


\n

I've compiled my old updates below.

\n



\n
\n

UPDATE APRIL 1, 2021

\n
\n

I have attempted a revamp using some of my learnings from other notebooks to no avail. The loss decreases and everything looks good, but there is still way too much error present. I will fork @markwijkhuizen wonderful notebook and try and replicate it (form it) to be even more similar to what I am doing. Ideally, this will allow me to identify what I am doing wrong.

\n

I'll post more updates here as I go

\n
\n


\n

UPDATE MARCH 29, 2021

\n
\n

Seeing the success of other similar image captioning architectures in this competition I'm fairly certain I'm doing something wrong. I will be revamping and re-releasing this notebook. I will comment here when that is complete.

\n
\n


\n

Thanks in advance!! If there's anything I can do to provide more information or context please let me know. Also, if this is uncouth or violates any of the written (or unwritten) rules of Kaggle, please let me know and I will delete it.

", "messages": [ { "id": 1259647, "postDate": "2021-04-01T15:02:09.450Z", "content": "

Hi there, I am struggling with my implementation (TF 2+) of an image captioning pipeline. I wanted to reach out for help from the Kaggle community.

\n

I have gotten it to 'work', but the scores are less than stellar.

\n


\n

Here is a link to the notebook: NOTEBOOK

\n

Here is a link to the tutorials that inspired the notebook: TF TUTORIAL 1, TF TUTORIAL 2

\n

Here is a link to the paper the decoder architecture is based off of: Show, Attend and Tell

\n


\n

Any feedback would be appreciated… I want to make a simple, easy-to-understand notebook to teach people while learning as I go (similar to this notebook by YNakuma in Pytorch). I believe I should be able to achieve a distance of ~20 or less with the method I am attempting. I was on the fence about posting this but I figured I should reach out for help while in parallel also troubleshooting it on my own.

\n


\n

I've compiled my old updates below.

\n



\n
\n

UPDATE APRIL 1, 2021

\n
\n

I have attempted a revamp using some of my learnings from other notebooks to no avail. The loss decreases and everything looks good, but there is still way too much error present. I will fork @markwijkhuizen wonderful notebook and try and replicate it (form it) to be even more similar to what I am doing. Ideally, this will allow me to identify what I am doing wrong.

\n

I'll post more updates here as I go

\n
\n


\n

UPDATE MARCH 29, 2021

\n
\n

Seeing the success of other similar image captioning architectures in this competition I'm fairly certain I'm doing something wrong. I will be revamping and re-releasing this notebook. I will comment here when that is complete.

\n
\n


\n

Thanks in advance!! If there's anything I can do to provide more information or context please let me know. Also, if this is uncouth or violates any of the written (or unwritten) rules of Kaggle, please let me know and I will delete it.

", "rawMarkdown": "Hi there, I am struggling with my implementation (TF 2+) of an image captioning pipeline. I wanted to reach out for help from the Kaggle community.\n\nI have gotten it to 'work', but the scores are less than stellar.\n\n
\n\nHere is a link to the notebook: [**NOTEBOOK**](https://www.kaggle.com/dschettler8845/bms-image-captioning-w-attention-train/edit/run/58303170)\n\nHere is a link to the tutorials that inspired the notebook: [**TF TUTORIAL 1**](https://www.tensorflow.org/tutorials/text/image_captioning), [**TF TUTORIAL 2**](https://www.tensorflow.org/tutorials/text/transformer)\n\nHere is a link to the paper the decoder architecture is based off of: [**Show, Attend and Tell**](https://arxiv.org/pdf/1502.03044.pdf)\n\n
\n\nAny feedback would be appreciated... I want to make a simple, easy-to-understand notebook to teach people while learning as I go (similar to [**this notebook**](https://www.kaggle.com/yasufuminakama/inchi-resnet-lstm-with-attention-starter) by [**YNakuma**](https://www.kaggle.com/yasufuminakama) in Pytorch). I believe I should be able to achieve a distance of ~20 or less with the method I am attempting. I was on the fence about posting this but I figured I should reach out for help while in parallel also troubleshooting it on my own. \n\n
\n\nI've compiled my old updates below.\n\n

\n\n---\n\n**UPDATE APRIL 1, 2021** \n\n---\n\nI have attempted a revamp using some of my learnings from other notebooks to no avail. The loss decreases and everything looks good, but there is still way too much error present. I will fork @markwijkhuizen wonderful notebook and try and replicate it (form it) to be even more similar to what I am doing. Ideally, this will allow me to identify what I am doing wrong.\n\nI'll post more updates here as I go\n\n---\n\n
\n\n**UPDATE MARCH 29, 2021** \n\n---\n\nSeeing the success of other similar image captioning architectures in this competition I'm fairly certain I'm doing something wrong. I will be revamping and re-releasing this notebook. I will comment here when that is complete.\n\n---\n\n
\n\nThanks in advance!! If there's anything I can do to provide more information or context please let me know. Also, if this is uncouth or violates any of the written (or unwritten) rules of Kaggle, please let me know and I will delete it.", "votes": 4 } ], "comments": [], "raw_markdown_by_id": { "1259647": "Hi there, I am struggling with my implementation (TF 2+) of an image captioning pipeline. I wanted to reach out for help from the Kaggle community.\n\nI have gotten it to 'work', but the scores are less than stellar.\n\n
\n\nHere is a link to the notebook: [**NOTEBOOK**](https://www.kaggle.com/dschettler8845/bms-image-captioning-w-attention-train/edit/run/58303170)\n\nHere is a link to the tutorials that inspired the notebook: [**TF TUTORIAL 1**](https://www.tensorflow.org/tutorials/text/image_captioning), [**TF TUTORIAL 2**](https://www.tensorflow.org/tutorials/text/transformer)\n\nHere is a link to the paper the decoder architecture is based off of: [**Show, Attend and Tell**](https://arxiv.org/pdf/1502.03044.pdf)\n\n
\n\nAny feedback would be appreciated... I want to make a simple, easy-to-understand notebook to teach people while learning as I go (similar to [**this notebook**](https://www.kaggle.com/yasufuminakama/inchi-resnet-lstm-with-attention-starter) by [**YNakuma**](https://www.kaggle.com/yasufuminakama) in Pytorch). I believe I should be able to achieve a distance of ~20 or less with the method I am attempting. I was on the fence about posting this but I figured I should reach out for help while in parallel also troubleshooting it on my own. \n\n
\n\nI've compiled my old updates below.\n\n

\n\n---\n\n**UPDATE APRIL 1, 2021** \n\n---\n\nI have attempted a revamp using some of my learnings from other notebooks to no avail. The loss decreases and everything looks good, but there is still way too much error present. I will fork @markwijkhuizen wonderful notebook and try and replicate it (form it) to be even more similar to what I am doing. Ideally, this will allow me to identify what I am doing wrong.\n\nI'll post more updates here as I go\n\n---\n\n
\n\n**UPDATE MARCH 29, 2021** \n\n---\n\nSeeing the success of other similar image captioning architectures in this competition I'm fairly certain I'm doing something wrong. I will be revamping and re-releasing this notebook. I will comment here when that is complete.\n\n---\n\n
\n\nThanks in advance!! If there's anything I can do to provide more information or context please let me know. Also, if this is uncouth or violates any of the written (or unwritten) rules of Kaggle, please let me know and I will delete it." } }