{
  "id": 103493,
  "title": "Important information about the challenge data and evaluation",
  "url": "/competitions/open-images-2019-instance-segmentation/discussion/103493",
  "author_name": "Alina",
  "post_date": "2019-08-09T14:54:30.370000",
  "votes": 5,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Based on the results of the previous year's challenge and the questions we received, we would like to stress the following points:</p>\n\n<p>The images are annotated with positive image-level labels, indicating certain object classes are present, and with negative image-level labels, indicating certain classes are absent. These negative labels are all reliable and can be used during training, e.g. for hard-negative mining which is important when training object detectors.</p>\n\n<p>We have annotated bounding boxes for human body parts only for 95,335 images in the training set, due to the overwhelming number of instances (see also the full description). You can use this list of images to make sure you use the data correctly during training (as there might be a positive image label for a human body part, and yet no boxes). Instead, on the validation and Challenge sets, we annotated human body parts on all images for which we have a positive label.</p>\n\n<p>Segmentations are annotated on the subset of the set of bounding boxes. For the specific criteria as to which bounding boxes were selected for annotation check out <a href=\"https://storage.googleapis.com/openimages/web/factsfigures.html\">Open Images V5 website</a></p>\n\n<p>The public leaderboard on the Kaggle website is the most reliable indicator of your performance, as it is based on images and annotations distributed identically to the hidden test set on which participants will be ranked for final evaluation. The validation set is also indicative of performance on the Challenge set, but is not exactly identically distributed. The best reference is the public leaderboard.</p>",
  "messages": [
    {
      "id": 595695,
      "postDate": "2019-08-09T14:54:30.370Z",
      "content": "<p>Based on the results of the previous year's challenge and the questions we received, we would like to stress the following points:</p>\n\n<p>The images are annotated with positive image-level labels, indicating certain object classes are present, and with negative image-level labels, indicating certain classes are absent. These negative labels are all reliable and can be used during training, e.g. for hard-negative mining which is important when training object detectors.</p>\n\n<p>We have annotated bounding boxes for human body parts only for 95,335 images in the training set, due to the overwhelming number of instances (see also the full description). You can use this list of images to make sure you use the data correctly during training (as there might be a positive image label for a human body part, and yet no boxes). Instead, on the validation and Challenge sets, we annotated human body parts on all images for which we have a positive label.</p>\n\n<p>Segmentations are annotated on the subset of the set of bounding boxes. For the specific criteria as to which bounding boxes were selected for annotation check out <a href=\"https://storage.googleapis.com/openimages/web/factsfigures.html\">Open Images V5 website</a></p>\n\n<p>The public leaderboard on the Kaggle website is the most reliable indicator of your performance, as it is based on images and annotations distributed identically to the hidden test set on which participants will be ranked for final evaluation. The validation set is also indicative of performance on the Challenge set, but is not exactly identically distributed. The best reference is the public leaderboard.</p>",
      "rawMarkdown": "Based on the results of the previous year's challenge and the questions we received, we would like to stress the following points:\n\nThe images are annotated with positive image-level labels, indicating certain object classes are present, and with negative image-level labels, indicating certain classes are absent. These negative labels are all reliable and can be used during training, e.g. for hard-negative mining which is important when training object detectors.\n\nWe have annotated bounding boxes for human body parts only for 95,335 images in the training set, due to the overwhelming number of instances (see also the full description). You can use this list of images to make sure you use the data correctly during training (as there might be a positive image label for a human body part, and yet no boxes). Instead, on the validation and Challenge sets, we annotated human body parts on all images for which we have a positive label.\n\nSegmentations are annotated on the subset of the set of bounding boxes. For the specific criteria as to which bounding boxes were selected for annotation check out [Open Images V5 website](https://storage.googleapis.com/openimages/web/factsfigures.html)\n\nThe public leaderboard on the Kaggle website is the most reliable indicator of your performance, as it is based on images and annotations distributed identically to the hidden test set on which participants will be ranked for final evaluation. The validation set is also indicative of performance on the Challenge set, but is not exactly identically distributed. The best reference is the public leaderboard.",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "595695": "Based on the results of the previous year's challenge and the questions we received, we would like to stress the following points:\n\nThe images are annotated with positive image-level labels, indicating certain object classes are present, and with negative image-level labels, indicating certain classes are absent. These negative labels are all reliable and can be used during training, e.g. for hard-negative mining which is important when training object detectors.\n\nWe have annotated bounding boxes for human body parts only for 95,335 images in the training set, due to the overwhelming number of instances (see also the full description). You can use this list of images to make sure you use the data correctly during training (as there might be a positive image label for a human body part, and yet no boxes). Instead, on the validation and Challenge sets, we annotated human body parts on all images for which we have a positive label.\n\nSegmentations are annotated on the subset of the set of bounding boxes. For the specific criteria as to which bounding boxes were selected for annotation check out [Open Images V5 website](https://storage.googleapis.com/openimages/web/factsfigures.html)\n\nThe public leaderboard on the Kaggle website is the most reliable indicator of your performance, as it is based on images and annotations distributed identically to the hidden test set on which participants will be ranked for final evaluation. The validation set is also indicative of performance on the Challenge set, but is not exactly identically distributed. The best reference is the public leaderboard."
  }
}