{
  "id": 106596,
  "title": "Problem with the official evaluation scripts",
  "url": "/competitions/open-images-2019-visual-relationship/discussion/106596",
  "author_name": "Neoutlier",
  "post_date": "2019-08-30T07:27:56.729000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I am trying to use the official evaluation script. As a tryout I first use the ground truth file as prediction and tested it. Yet I got the following results. As I understand both mAPs should be 1.0 and also mAP(rel) should be smaller than mAP(phrase). Therefore I suspect there's a bug in the evaluation script.<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1596529%2Fdf238932ecb8f95ada0c5f57787d3e05%2FScreenshot%20from%202019-08-30%2015-12-16.png?generation=1567150046371942&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": 613048,
      "postDate": "2019-08-30T07:27:56.730Z",
      "content": "<p>I am trying to use the official evaluation script. As a tryout I first use the ground truth file as prediction and tested it. Yet I got the following results. As I understand both mAPs should be 1.0 and also mAP(rel) should be smaller than mAP(phrase). Therefore I suspect there's a bug in the evaluation script.<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1596529%2Fdf238932ecb8f95ada0c5f57787d3e05%2FScreenshot%20from%202019-08-30%2015-12-16.png?generation=1567150046371942&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "I am trying to use the official evaluation script. As a tryout I first use the ground truth file as prediction and tested it. Yet I got the following results. As I understand both mAPs should be 1.0 and also mAP(rel) should be smaller than mAP(phrase). Therefore I suspect there's a bug in the evaluation script.![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1596529%2Fdf238932ecb8f95ada0c5f57787d3e05%2FScreenshot%20from%202019-08-30%2015-12-16.png?generation=1567150046371942&amp;alt=media)\n"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "613048": "I am trying to use the official evaluation script. As a tryout I first use the ground truth file as prediction and tested it. Yet I got the following results. As I understand both mAPs should be 1.0 and also mAP(rel) should be smaller than mAP(phrase). Therefore I suspect there's a bug in the evaluation script.![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1596529%2Fdf238932ecb8f95ada0c5f57787d3e05%2FScreenshot%20from%202019-08-30%2015-12-16.png?generation=1567150046371942&amp;alt=media)\n"
  }
}