![\"\"](\"https://pbs.twimg.com/media/E_SGbgMXsAguHJd?format=jpg&name=medium\")
Using a heuristic of converting page_url to caption, we get pretty promising outcomes on train data:
\nWhy does this score 0 on our metric? I probably don't fully understand the task and/or metric here…
", "messages": [ { "id": "1513182", "postDate": "09/14/2021 23:33:05", "content": "Using a heuristic of converting page_url to caption, we get pretty promising outcomes on train data:
\nWhy does this score 0 on our metric? I probably don't fully understand the task and/or metric here…
", "rawMarkdown": "Using a heuristic of converting page_url to caption, we get pretty promising outcomes on train data: \n\n\n\nWhy does this score 0 on our metric? I probably don't fully understand the task and/or metric here...", "votes": null }, { "id": "1513470", "postDate": "09/15/2021 07:35:45", "content": "I'm also not very confident about how the evaluation actually works. But as the model should give the images as results for search queries, I suspect that it requires less specific predictions. If you search for \"Pariser Kanonen\" you need no advanced technique to give \"Pariser Kanonen\" as a result. So, a good search model should rather give the \"Pariser Kanonen\" image as an result for searches like \"Paris\",\"cannons\", \"weapons\",\"military France\" or \"paris history\". Maybe you try to simply split your predictions into single results Pariser,Kanonen to see if it scores better than 0.0.
Update: Me and everybody else scored 0.0 too (so far), so I guess its a problem with the evaluation process.
", "rawMarkdown": "Update: Me and everybody else scored 0.0 too (so far), so I guess its a problem with the evaluation process.", "votes": null }, { "id": "1514028", "postDate": "09/15/2021 15:56:39", "content": "I think we're missing some data. The host clarified in a separate thread that we should match images with a subset of a predefined set of test captions, but I haven't been able to find those captions in the test data.
", "rawMarkdown": "I think we're missing some data. The host clarified in [a separate thread](https://www.kaggle.com/c/wikipedia-image-caption/discussion/272023#1513617) that we should match images with a subset of a predefined set of test captions, but I haven't been able to find those captions in the test data.", "votes": null }, { "id": "1514040", "postDate": "09/15/2021 16:17:28", "content": "HI @alexanderbader and @thedrcat we are working on solving this issue! Please give us a few hours. Thanks again for working on this competition!
", "rawMarkdown": "HI @alexanderbader and @thedrcat we are working on solving this issue! Please give us a few hours. Thanks again for working on this competition!", "votes": null } ], "comments": [ { "id": 1513470, "author_name": "alexanderbader", "author_url": "", "post_date": "09/15/2021 07:35:45", "content": "I'm also not very confident about how the evaluation actually works. But as the model should give the images as results for search queries, I suspect that it requires less specific predictions. If you search for \"Pariser Kanonen\" you need no advanced technique to give \"Pariser Kanonen\" as a result. So, a good search model should rather give the \"Pariser Kanonen\" image as an result for searches like \"Paris\",\"cannons\", \"weapons\",\"military France\" or \"paris history\". Maybe you try to simply split your predictions into single results Pariser,Kanonen to see if it scores better than 0.0.
Update: Me and everybody else scored 0.0 too (so far), so I guess its a problem with the evaluation process.
", "votes": null, "replies": [ { "id": 1514028, "author_name": "thedrcat", "author_url": "", "post_date": "09/15/2021 15:56:39", "content": "I think we're missing some data. The host clarified in a separate thread that we should match images with a subset of a predefined set of test captions, but I haven't been able to find those captions in the test data.
", "votes": null, "replies": [] }, { "id": 1514040, "author_name": "miriamredi", "author_url": "", "post_date": "09/15/2021 16:17:28", "content": "HI @alexanderbader and @thedrcat we are working on solving this issue! Please give us a few hours. Thanks again for working on this competition!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1513182": "Using a heuristic of converting page_url to caption, we get pretty promising outcomes on train data: \n\n\n\nWhy does this score 0 on our metric? I probably don't fully understand the task and/or metric here...", "1513470": "I'm also not very confident about how the evaluation actually works. But as the model should give the images as results for search queries, I suspect that it requires less specific predictions. If you search for \"Pariser Kanonen\" you need no advanced technique to give \"Pariser Kanonen\" as a result. So, a good search model should rather give the \"Pariser Kanonen\" image as an result for searches like \"Paris\",\"cannons\", \"weapons\",\"military France\" or \"paris history\". Maybe you try to simply split your predictions into single results `Pariser,Kanonen` to see if it scores better than 0.0.", "1513997": "Update: Me and everybody else scored 0.0 too (so far), so I guess its a problem with the evaluation process.", "1514028": "I think we're missing some data. The host clarified in [a separate thread](https://www.kaggle.com/c/wikipedia-image-caption/discussion/272023#1513617) that we should match images with a subset of a predefined set of test captions, but I haven't been able to find those captions in the test data.", "1514040": "HI @alexanderbader and @thedrcat we are working on solving this issue! Please give us a few hours. Thanks again for working on this competition!" }, "source": "meta" }