{
  "id": 274997,
  "title": "Validation metrics",
  "url": "/competitions/landmark-recognition-2021/discussion/274997",
  "author_name": "Mark Babayev",
  "post_date": "2021-09-28T11:38:02.150000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>How to validate the model when training?<br>\nShould the metric be <code>precision</code>, <code>accuracy</code>, <code>sklearn.metrics.average_precision_score</code>?<br>\nMaybe mAP - average of AUC?<br>\n<a href=\"https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173\" target=\"_blank\">https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173</a><br>\nMaybe <code>tf.compat.v1.metrics.average_precision_at_k</code> (with k=1)?<br>\n<a href=\"https://github.com/broadinstitute/keras-rcnn/issues/6\" target=\"_blank\">https://github.com/broadinstitute/keras-rcnn/issues/6</a></p>\n<p>What will happen if I put in the submission file all confidence scores to be \"1\"? The final GAP will grow up?<br>\nIf I delete all confidence scores, less than 0.5? How the final GAP will be affected?</p>",
  "messages": [
    {
      "id": 1526858,
      "postDate": "2021-09-28T11:38:02.150Z",
      "content": "<p>How to validate the model when training?<br>\nShould the metric be <code>precision</code>, <code>accuracy</code>, <code>sklearn.metrics.average_precision_score</code>?<br>\nMaybe mAP - average of AUC?<br>\n<a href=\"https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173\" target=\"_blank\">https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173</a><br>\nMaybe <code>tf.compat.v1.metrics.average_precision_at_k</code> (with k=1)?<br>\n<a href=\"https://github.com/broadinstitute/keras-rcnn/issues/6\" target=\"_blank\">https://github.com/broadinstitute/keras-rcnn/issues/6</a></p>\n<p>What will happen if I put in the submission file all confidence scores to be \"1\"? The final GAP will grow up?<br>\nIf I delete all confidence scores, less than 0.5? How the final GAP will be affected?</p>",
      "rawMarkdown": "How to validate the model when training?\nShould the metric be `precision`, `accuracy`, `sklearn.metrics.average_precision_score`?\nMaybe mAP - average of AUC?\nhttps://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173\nMaybe `tf.compat.v1.metrics.average_precision_at_k` (with k=1)?\nhttps://github.com/broadinstitute/keras-rcnn/issues/6\n\nWhat will happen if I put in the submission file all confidence scores to be \"1\"? The final GAP will grow up?\nIf I delete all confidence scores, less than 0.5? How the final GAP will be affected?",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1526858": "How to validate the model when training?\nShould the metric be `precision`, `accuracy`, `sklearn.metrics.average_precision_score`?\nMaybe mAP - average of AUC?\nhttps://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173\nMaybe `tf.compat.v1.metrics.average_precision_at_k` (with k=1)?\nhttps://github.com/broadinstitute/keras-rcnn/issues/6\n\nWhat will happen if I put in the submission file all confidence scores to be \"1\"? The final GAP will grow up?\nIf I delete all confidence scores, less than 0.5? How the final GAP will be affected?"
  }
}