So I'm trying to better understand how the competition metric works because this metric and the nature of this competition is so different. I've split how I understand the evaluation process into 4 steps.
\n\nHere's how I'm guessing step 3 works.
\n\nFor a set of images in all sets of images: \n Get the embeddings for those images in the set\n For each image embedding, create a list of most similar embeddings\n If top n similar embeddings for this image contains another image with the same landmark\n more points?\n maybe even more points if you get all of the same landmark in the top n?\n else:\n less points?
\n\nI'm probably wrong here so I'm hoping by posting the incorrect understanding someone will be angry enough to correct me here.
\n\nMy intuition from reading the evaluation metric formula is that models that output similar embeddings for the same landmark and dissimilar embeddings for different landmarks are supposed to perform well.
", "messages": [ { "id": "948351", "postDate": "07/27/2020 20:53:29", "content": "So I'm trying to better understand how the competition metric works because this metric and the nature of this competition is so different. I've split how I understand the evaluation process into 4 steps.
\n\nHere's how I'm guessing step 3 works.
\n\nFor a set of images in all sets of images: \n Get the embeddings for those images in the set\n For each image embedding, create a list of most similar embeddings\n If top n similar embeddings for this image contains another image with the same landmark\n more points?\n maybe even more points if you get all of the same landmark in the top n?\n else:\n less points?
\n\nI'm probably wrong here so I'm hoping by posting the incorrect understanding someone will be angry enough to correct me here.
\n\nMy intuition from reading the evaluation metric formula is that models that output similar embeddings for the same landmark and dissimilar embeddings for different landmarks are supposed to perform well.
", "rawMarkdown": "So I'm trying to better understand how the competition metric works because this metric and the nature of this competition is so different. I've split how I understand the evaluation process into 4 steps.\n\n1. We submit a network that takes an image and outputs an embedding \n2. Kaggle uses some code to generate a bunch of embeddings with the network\n3. ???\n4. You get an LB score \n\nHere's how I'm guessing step 3 works. \n\n\nFor a set of images in all sets of images: \n Get the embeddings for those images in the set\n For each image embedding, create a list of most similar embeddings\n If top n similar embeddings for this image contains another image with the same landmark\n more points?\n maybe even more points if you get all of the same landmark in the top n?\n else:\n less points?\n\nI'm probably wrong here so I'm hoping by posting the incorrect understanding someone will be angry enough to correct me here. \n\nMy intuition from reading the evaluation metric formula is that models that output similar embeddings for the same landmark and dissimilar embeddings for different landmarks are supposed to perform well.", "votes": null }, { "id": "962723", "postDate": "08/08/2020 11:30:37", "content": "I believe step 3 is explained here : https://www.kaggle.com/philculliton/landmark-retrieval-2020-shared-scoring-script
", "rawMarkdown": "I believe step 3 is explained here : https://www.kaggle.com/philculliton/landmark-retrieval-2020-shared-scoring-script", "votes": null } ], "comments": [ { "id": 962723, "author_name": "ogrellier", "author_url": "", "post_date": "08/08/2020 11:30:37", "content": "I believe step 3 is explained here : https://www.kaggle.com/philculliton/landmark-retrieval-2020-shared-scoring-script
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "948351": "So I'm trying to better understand how the competition metric works because this metric and the nature of this competition is so different. I've split how I understand the evaluation process into 4 steps.\n\n1. We submit a network that takes an image and outputs an embedding \n2. Kaggle uses some code to generate a bunch of embeddings with the network\n3. ???\n4. You get an LB score \n\nHere's how I'm guessing step 3 works. \n\n\nFor a set of images in all sets of images: \n Get the embeddings for those images in the set\n For each image embedding, create a list of most similar embeddings\n If top n similar embeddings for this image contains another image with the same landmark\n more points?\n maybe even more points if you get all of the same landmark in the top n?\n else:\n less points?\n\nI'm probably wrong here so I'm hoping by posting the incorrect understanding someone will be angry enough to correct me here. \n\nMy intuition from reading the evaluation metric formula is that models that output similar embeddings for the same landmark and dissimilar embeddings for different landmarks are supposed to perform well.", "962723": "I believe step 3 is explained here : https://www.kaggle.com/philculliton/landmark-retrieval-2020-shared-scoring-script" }, "source": "meta" }