{
  "id": 493626,
  "title": "Question about the metric",
  "url": "/competitions/image-matching-challenge-2024/discussion/493626",
  "author_name": "",
  "post_date": "2024-04-14T07:35:59.299191400Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I have a question regarding how is the prediction is evaluated. According to the <a href=\"https://www.kaggle.com/code/fabiobellavia/imc2024-3d-metric-evaluation-example/notebook\" target=\"_blank\">metric</a>, approximately only 55 images will be kept in every scene and then evaluated and others are dropped. Will random dropping be applied to our submission as well? Also, the metric seems to be applied to <code>train_label.csv</code> rather than <code>sample_submission.csv</code>, so  I wonder if there  is another version of metric that can be made public</p>",
  "messages": [
    {
      "id": "2751383",
      "postDate": "04/14/2024 07:35:59",
      "content": "<p>I have a question regarding how is the prediction is evaluated. According to the <a href=\"https://www.kaggle.com/code/fabiobellavia/imc2024-3d-metric-evaluation-example/notebook\" target=\"_blank\">metric</a>, approximately only 55 images will be kept in every scene and then evaluated and others are dropped. Will random dropping be applied to our submission as well? Also, the metric seems to be applied to <code>train_label.csv</code> rather than <code>sample_submission.csv</code>, so  I wonder if there  is another version of metric that can be made public</p>",
      "rawMarkdown": "I have a question regarding how is the prediction is evaluated. According to the [metric](https://www.kaggle.com/code/fabiobellavia/imc2024-3d-metric-evaluation-example/notebook), approximately only 55 images will be kept in every scene and then evaluated and others are dropped. Will random dropping be applied to our submission as well? Also, the metric seems to be applied to `train_label.csv` rather than `sample_submission.csv`, so  I wonder if there  is another version of metric that can be made public",
      "votes": null
    },
    {
      "id": "2751547",
      "postDate": "04/14/2024 10:27:01",
      "content": "<p>This is just an example script and num=55 used to avoid running on other sequences. </p>\n<p>There is no such limitation in metric script. I also don’t see a reason to share “real metric” script, as the change is trivial and participants can do it on their own. </p>",
      "rawMarkdown": "This is just an example script and num=55 used to avoid running on other sequences. \n\nThere is no such limitation in metric script. I also don’t see a reason to share “real metric” script, as the change is trivial and participants can do it on their own.",
      "votes": null
    },
    {
      "id": "2751556",
      "postDate": "04/14/2024 10:34:06",
      "content": "<p><code>as the change is trivial and participants can do it on their own.</code></p>\n<p>You are right.</p>\n<p><code>running on other sequences.</code></p>\n<p>Do you mind clarify it a little, thanks</p>",
      "rawMarkdown": "`as the change is trivial and participants can do it on their own.`\n\nYou are right.\n\n`running on other sequences.`\n\nDo you mind clarify it a little, thanks",
      "votes": null
    },
    {
      "id": "2752124",
      "postDate": "04/14/2024 18:16:05",
      "content": "<p>The metric notebook is run on 1 scene of the training set instead of all of them, for the speed reasons.</p>",
      "rawMarkdown": "The metric notebook is run on 1 scene of the training set instead of all of them, for the speed reasons.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2751547,
      "author_name": "oldufo",
      "author_url": "",
      "post_date": "04/14/2024 10:27:01",
      "content": "<p>This is just an example script and num=55 used to avoid running on other sequences. </p>\n<p>There is no such limitation in metric script. I also don’t see a reason to share “real metric” script, as the change is trivial and participants can do it on their own. </p>",
      "votes": null,
      "replies": [
        {
          "id": 2751556,
          "author_name": "renyiwei",
          "author_url": "",
          "post_date": "04/14/2024 10:34:06",
          "content": "<p><code>as the change is trivial and participants can do it on their own.</code></p>\n<p>You are right.</p>\n<p><code>running on other sequences.</code></p>\n<p>Do you mind clarify it a little, thanks</p>",
          "votes": null,
          "replies": [
            {
              "id": 2752124,
              "author_name": "oldufo",
              "author_url": "",
              "post_date": "04/14/2024 18:16:05",
              "content": "<p>The metric notebook is run on 1 scene of the training set instead of all of them, for the speed reasons.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2751383": "I have a question regarding how is the prediction is evaluated. According to the [metric](https://www.kaggle.com/code/fabiobellavia/imc2024-3d-metric-evaluation-example/notebook), approximately only 55 images will be kept in every scene and then evaluated and others are dropped. Will random dropping be applied to our submission as well? Also, the metric seems to be applied to `train_label.csv` rather than `sample_submission.csv`, so  I wonder if there  is another version of metric that can be made public",
    "2751547": "This is just an example script and num=55 used to avoid running on other sequences. \n\nThere is no such limitation in metric script. I also don’t see a reason to share “real metric” script, as the change is trivial and participants can do it on their own.",
    "2751556": "`as the change is trivial and participants can do it on their own.`\n\nYou are right.\n\n`running on other sequences.`\n\nDo you mind clarify it a little, thanks",
    "2752124": "The metric notebook is run on 1 scene of the training set instead of all of them, for the speed reasons."
  },
  "source": "meta"
}