{
  "id": 68066,
  "title": "Score Calculation Issue",
  "url": "/competitions/imagenet-object-localization-challenge/discussion/68066",
  "author_name": "",
  "post_date": "2018-10-09T00:57:05.724302700Z",
  "votes": 21,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I found a bug in score calculation. <br>\nSubmitting empty \"PredictionString\" for all test data (as below) results perfect score(0.00000) on public leaderboard. </p>\n\n<pre><code>ImageId,PredictionString\nILSVRC2012_test_00000001,\nILSVRC2012_test_00000002,\nILSVRC2012_test_00000003,\n......\n......\nILSVRC2012_test_00100000,\n</code></pre>",
  "messages": [
    {
      "id": "400827",
      "postDate": "10/09/2018 00:57:05",
      "content": "<p>I found a bug in score calculation. <br>\nSubmitting empty \"PredictionString\" for all test data (as below) results perfect score(0.00000) on public leaderboard. </p>\n\n<pre><code>ImageId,PredictionString\nILSVRC2012_test_00000001,\nILSVRC2012_test_00000002,\nILSVRC2012_test_00000003,\n......\n......\nILSVRC2012_test_00100000,\n</code></pre>",
      "rawMarkdown": "I found a bug in score calculation.  \nSubmitting empty \"PredictionString\" for all test data (as below) results perfect score(0.00000) on public leaderboard. \n\n    ImageId,PredictionString\n    ILSVRC2012_test_00000001,\n    ILSVRC2012_test_00000002,\n    ILSVRC2012_test_00000003,\n    ......\n    ......\n    ILSVRC2012_test_00100000,",
      "votes": null
    },
    {
      "id": "556342",
      "postDate": "06/20/2019 06:09:51",
      "content": "<p>The evaluation metric is a complete nonsense, seems like somebody did not pay much attention to it. The more severe issue is that you can achieve perfect score by generating all possible detection boxes with all possible classifications.</p>",
      "rawMarkdown": "The evaluation metric is a complete nonsense, seems like somebody did not pay much attention to it. The more severe issue is that you can achieve perfect score by generating all possible detection boxes with all possible classifications.",
      "votes": null
    },
    {
      "id": "596920",
      "postDate": "08/11/2019 14:05:22",
      "content": "<p>There is a max of 5 bounding boxes per image. But what is strange is that if you find at least one object in the image, the error is 0 for this image even if you miss many other objects. </p>",
      "rawMarkdown": "There is a max of 5 bounding boxes per image. But what is strange is that if you find at least one object in the image, the error is 0 for this image even if you miss many other objects.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 556342,
      "author_name": "svobora",
      "author_url": "",
      "post_date": "06/20/2019 06:09:51",
      "content": "<p>The evaluation metric is a complete nonsense, seems like somebody did not pay much attention to it. The more severe issue is that you can achieve perfect score by generating all possible detection boxes with all possible classifications.</p>",
      "votes": null,
      "replies": [
        {
          "id": 596920,
          "author_name": "ami248",
          "author_url": "",
          "post_date": "08/11/2019 14:05:22",
          "content": "<p>There is a max of 5 bounding boxes per image. But what is strange is that if you find at least one object in the image, the error is 0 for this image even if you miss many other objects. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "400827": "I found a bug in score calculation.  \nSubmitting empty \"PredictionString\" for all test data (as below) results perfect score(0.00000) on public leaderboard. \n\n    ImageId,PredictionString\n    ILSVRC2012_test_00000001,\n    ILSVRC2012_test_00000002,\n    ILSVRC2012_test_00000003,\n    ......\n    ......\n    ILSVRC2012_test_00100000,",
    "556342": "The evaluation metric is a complete nonsense, seems like somebody did not pay much attention to it. The more severe issue is that you can achieve perfect score by generating all possible detection boxes with all possible classifications.",
    "596920": "There is a max of 5 bounding boxes per image. But what is strange is that if you find at least one object in the image, the error is 0 for this image even if you miss many other objects."
  },
  "source": "meta"
}