{
  "id": 580378,
  "title": "Regarding the Metric Bug",
  "url": "/competitions/image-matching-challenge-2025/discussion/580378",
  "author_name": "",
  "post_date": "2025-05-23T20:23:28.670257600Z",
  "votes": 8,
  "comment_count": 1,
  "views": 0,
  "content": "<p>It seems that, except for the previously calculated public scores, the correct scores can now be obtained. (The submission results checked 10 minutes ago reflected this.)</p>\n<p>Therefore, this is solely for CV calculation purposes, but I will explain how to overcome the metric bug.</p>\n<p>It's simple—just include this code before submitting:</p>\n<p><code>predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else 100)</code></p>\n<p>This bug occurs when \"outlier\" rows appear before non-outlier rows in the submission file. By avoiding this condition, you can retrieve the intended score.</p>\n<p>By the way, hacking the metric required the following code (which landed me in third place by submitting this code and only one cluster, but it doesn't really mean anything—haha):</p>\n<p><code>predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else -1)</code></p>\n<pre><code>submission_path = os.path.join(feature_dir, )\n (submission_path, )  f:\n     CONFIG.is_train:\n        f.write()\n        predictions = (predictions, key= t: t.cluster_index  t.cluster_index !=   )\n        \n         prediction  predictions:\n            cluster_name =   prediction.cluster_index    \n            rotation = none_to_str()  prediction.rotation    array_to_str(prediction.rotation.flatten())\n            translation = none_to_str()  prediction.translation    array_to_str(prediction.translation)\n            f.write()\n    :\n        f.write()\n        predictions = (predictions, key= t: t.cluster_index  t.cluster_index !=   )\n        \n         prediction  predictions:\n            cluster_name =   prediction.cluster_index    \n            rotation = none_to_str()  prediction.rotation    array_to_str(prediction.rotation.flatten())\n            translation = none_to_str()  prediction.translation    array_to_str(prediction.translation)\n            f.write()\n</code></pre>\n<p>PS:<br>\nThe administration has updated the metric file, so replacing it would be the preferable course of action.<br>\n<a href=\"https://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py\" target=\"_blank\">https://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py</a></p>\n<p>PS2:<br>\nThe code above is a slight change from the baseline. In the baseline, add the following code:<br>\n<code>samples[dataset] = sorted(samples[dataset], key=lambda t: t.cluster_index if t.cluster_index != None else 100)</code></p>",
  "messages": [
    {
      "id": "3208238",
      "postDate": "05/23/2025 20:23:28",
      "content": "<p>It seems that, except for the previously calculated public scores, the correct scores can now be obtained. (The submission results checked 10 minutes ago reflected this.)</p>\n<p>Therefore, this is solely for CV calculation purposes, but I will explain how to overcome the metric bug.</p>\n<p>It's simple—just include this code before submitting:</p>\n<p><code>predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else 100)</code></p>\n<p>This bug occurs when \"outlier\" rows appear before non-outlier rows in the submission file. By avoiding this condition, you can retrieve the intended score.</p>\n<p>By the way, hacking the metric required the following code (which landed me in third place by submitting this code and only one cluster, but it doesn't really mean anything—haha):</p>\n<p><code>predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else -1)</code></p>\n<pre><code>submission_path = os.path.join(feature_dir, )\n (submission_path, )  f:\n     CONFIG.is_train:\n        f.write()\n        predictions = (predictions, key= t: t.cluster_index  t.cluster_index !=   )\n        \n         prediction  predictions:\n            cluster_name =   prediction.cluster_index    \n            rotation = none_to_str()  prediction.rotation    array_to_str(prediction.rotation.flatten())\n            translation = none_to_str()  prediction.translation    array_to_str(prediction.translation)\n            f.write()\n    :\n        f.write()\n        predictions = (predictions, key= t: t.cluster_index  t.cluster_index !=   )\n        \n         prediction  predictions:\n            cluster_name =   prediction.cluster_index    \n            rotation = none_to_str()  prediction.rotation    array_to_str(prediction.rotation.flatten())\n            translation = none_to_str()  prediction.translation    array_to_str(prediction.translation)\n            f.write()\n</code></pre>\n<p>PS:<br>\nThe administration has updated the metric file, so replacing it would be the preferable course of action.<br>\n<a href=\"https://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py\" target=\"_blank\">https://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py</a></p>\n<p>PS2:<br>\nThe code above is a slight change from the baseline. In the baseline, add the following code:<br>\n<code>samples[dataset] = sorted(samples[dataset], key=lambda t: t.cluster_index if t.cluster_index != None else 100)</code></p>",
      "rawMarkdown": "It seems that, except for the previously calculated public scores, the correct scores can now be obtained. (The submission results checked 10 minutes ago reflected this.)\n\nTherefore, this is solely for CV calculation purposes, but I will explain how to overcome the metric bug.\n\nIt's simple—just include this code before submitting:\n\n`predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else 100)`\n\nThis bug occurs when \"outlier\" rows appear before non-outlier rows in the submission file. By avoiding this condition, you can retrieve the intended score.\n\nBy the way, hacking the metric required the following code (which landed me in third place by submitting this code and only one cluster, but it doesn't really mean anything—haha):\n\n`predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else -1)`\n\n```python\nsubmission_path = os.path.join(feature_dir, 'submission.csv')\nwith open(submission_path, 'w') as f:\n    if CONFIG.is_train:\n        f.write('dataset,scene,image,rotation_matrix,translation_vector\\n')\n        predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else 100)\n        #predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else -1)\n        for prediction in predictions:\n            cluster_name = 'outliers' if prediction.cluster_index is None else f'cluster{prediction.cluster_index}'\n            rotation = none_to_str(9) if prediction.rotation is None else array_to_str(prediction.rotation.flatten())\n            translation = none_to_str(3) if prediction.translation is None else array_to_str(prediction.translation)\n            f.write(f'{prediction.dataset},{cluster_name},{prediction.filename},{rotation},{translation}\\n')\n    else:\n        f.write('image_id,dataset,scene,image,rotation_matrix,translation_vector\\n')\n        predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else 100)\n        #predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else -1)\n        for prediction in predictions:\n            cluster_name = 'outliers' if prediction.cluster_index is None else f'cluster{prediction.cluster_index}'\n            rotation = none_to_str(9) if prediction.rotation is None else array_to_str(prediction.rotation.flatten())\n            translation = none_to_str(3) if prediction.translation is None else array_to_str(prediction.translation)\n            f.write(f'{prediction.image_id},{prediction.dataset},{cluster_name},{prediction.filename},{rotation},{translation}\\n')\n```\nPS:\nThe administration has updated the metric file, so replacing it would be the preferable course of action.\nhttps://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py\n\nPS2:\nThe code above is a slight change from the baseline. In the baseline, add the following code:\n`samples[dataset] = sorted(samples[dataset], key=lambda t: t.cluster_index if t.cluster_index != None else 100)`",
      "votes": null
    },
    {
      "id": "3211702",
      "postDate": "05/28/2025 18:22:58",
      "content": "<p>This was helpful thanks! I was very confused due this bug</p>",
      "rawMarkdown": "This was helpful thanks! I was very confused due this bug",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3211702,
      "author_name": "icebearogo",
      "author_url": "",
      "post_date": "05/28/2025 18:22:58",
      "content": "<p>This was helpful thanks! I was very confused due this bug</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3208238": "It seems that, except for the previously calculated public scores, the correct scores can now be obtained. (The submission results checked 10 minutes ago reflected this.)\n\nTherefore, this is solely for CV calculation purposes, but I will explain how to overcome the metric bug.\n\nIt's simple—just include this code before submitting:\n\n`predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else 100)`\n\nThis bug occurs when \"outlier\" rows appear before non-outlier rows in the submission file. By avoiding this condition, you can retrieve the intended score.\n\nBy the way, hacking the metric required the following code (which landed me in third place by submitting this code and only one cluster, but it doesn't really mean anything—haha):\n\n`predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else -1)`\n\n```python\nsubmission_path = os.path.join(feature_dir, 'submission.csv')\nwith open(submission_path, 'w') as f:\n    if CONFIG.is_train:\n        f.write('dataset,scene,image,rotation_matrix,translation_vector\\n')\n        predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else 100)\n        #predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else -1)\n        for prediction in predictions:\n            cluster_name = 'outliers' if prediction.cluster_index is None else f'cluster{prediction.cluster_index}'\n            rotation = none_to_str(9) if prediction.rotation is None else array_to_str(prediction.rotation.flatten())\n            translation = none_to_str(3) if prediction.translation is None else array_to_str(prediction.translation)\n            f.write(f'{prediction.dataset},{cluster_name},{prediction.filename},{rotation},{translation}\\n')\n    else:\n        f.write('image_id,dataset,scene,image,rotation_matrix,translation_vector\\n')\n        predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else 100)\n        #predictions = sorted(predictions, key=lambda t: t.cluster_index if t.cluster_index != None else -1)\n        for prediction in predictions:\n            cluster_name = 'outliers' if prediction.cluster_index is None else f'cluster{prediction.cluster_index}'\n            rotation = none_to_str(9) if prediction.rotation is None else array_to_str(prediction.rotation.flatten())\n            translation = none_to_str(3) if prediction.translation is None else array_to_str(prediction.translation)\n            f.write(f'{prediction.image_id},{prediction.dataset},{cluster_name},{prediction.filename},{rotation},{translation}\\n')\n```\nPS:\nThe administration has updated the metric file, so replacing it would be the preferable course of action.\nhttps://www.kaggle.com/datasets/eduardtrulls/imc25-utils/data?select=metric.py\n\nPS2:\nThe code above is a slight change from the baseline. In the baseline, add the following code:\n`samples[dataset] = sorted(samples[dataset], key=lambda t: t.cluster_index if t.cluster_index != None else 100)`",
    "3211702": "This was helpful thanks! I was very confused due this bug"
  },
  "source": "meta"
}