{
  "id": 141891,
  "title": "Evaluation Metric Clarification",
  "url": "/competitions/imaterialist-fashion-2020-fgvc7/discussion/141891",
  "author_name": "",
  "post_date": "2020-04-07T22:38:51.776247400Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>In the \"Evaluation\" section you wrote the following:</p>\n\n<blockquote>\n  <p>At each threshold pair, t=(ti, tf), a precision value is calculated based on the number of true positives (TP), false negatives (FN), and false positives (FP) resulting from comparing the predicted object to all ground truth objects:\n      <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F853978%2F3f1aec3c5b6fcbba10cc35a53d338810%2Fformula2020.png?generation=1586298781910856&amp;alt=media\" alt=\"\"></p>\n</blockquote>\n\n<p>You mentioned TP, FN and FP, but the formula doesn't actually contain FN.</p>\n\n<p>I believe it should be the same as in the previous year competition:\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F853978%2F1e5e6eb0408b61f0ae68632a13f6b059%2Fformula2019.png?generation=1586298802133924&amp;alt=media\" alt=\"\"></p>\n\n<p>Is that correct?</p>",
  "messages": [
    {
      "id": "800938",
      "postDate": "04/07/2020 22:38:51",
      "content": "<p>In the \"Evaluation\" section you wrote the following:</p>\n\n<blockquote>\n  <p>At each threshold pair, t=(ti, tf), a precision value is calculated based on the number of true positives (TP), false negatives (FN), and false positives (FP) resulting from comparing the predicted object to all ground truth objects:\n      <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F853978%2F3f1aec3c5b6fcbba10cc35a53d338810%2Fformula2020.png?generation=1586298781910856&amp;alt=media\" alt=\"\"></p>\n</blockquote>\n\n<p>You mentioned TP, FN and FP, but the formula doesn't actually contain FN.</p>\n\n<p>I believe it should be the same as in the previous year competition:\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F853978%2F1e5e6eb0408b61f0ae68632a13f6b059%2Fformula2019.png?generation=1586298802133924&amp;alt=media\" alt=\"\"></p>\n\n<p>Is that correct?</p>",
      "rawMarkdown": "In the \"Evaluation\" section you wrote the following:\n&gt; At each threshold pair, t=(ti, tf), a precision value is calculated based on the number of true positives (TP), false negatives (FN), and false positives (FP) resulting from comparing the predicted object to all ground truth objects:\n    ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F853978%2F3f1aec3c5b6fcbba10cc35a53d338810%2Fformula2020.png?generation=1586298781910856&amp;alt=media)\n\nYou mentioned TP, FN and FP, but the formula doesn't actually contain FN.\n\nI believe it should be the same as in the previous year competition:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F853978%2F1e5e6eb0408b61f0ae68632a13f6b059%2Fformula2019.png?generation=1586298802133924&amp;alt=media)\n\nIs that correct?",
      "votes": null
    },
    {
      "id": "806586",
      "postDate": "04/13/2020 21:05:07",
      "content": "<p>hey, thanks for noticing the difference. </p>\n\n<p>This year we adopted the metric from <a href=\"http://cocodataset.org/#detection-eval\">coco challenge</a>, which used TP / (TP + FP). </p>",
      "rawMarkdown": "hey, thanks for noticing the difference. \n\nThis year we adopted the metric from [coco challenge](http://cocodataset.org/#detection-eval), which used TP / (TP + FP).",
      "votes": null
    },
    {
      "id": "807269",
      "postDate": "04/14/2020 14:28:49",
      "content": "<p>Thank you, Menglin!</p>",
      "rawMarkdown": "Thank you, Menglin!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 806586,
      "author_name": "makeitworkjml",
      "author_url": "",
      "post_date": "04/13/2020 21:05:07",
      "content": "<p>hey, thanks for noticing the difference. </p>\n\n<p>This year we adopted the metric from <a href=\"http://cocodataset.org/#detection-eval\">coco challenge</a>, which used TP / (TP + FP). </p>",
      "votes": null,
      "replies": [
        {
          "id": 807269,
          "author_name": "polosin",
          "author_url": "",
          "post_date": "04/14/2020 14:28:49",
          "content": "<p>Thank you, Menglin!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "800938": "In the \"Evaluation\" section you wrote the following:\n&gt; At each threshold pair, t=(ti, tf), a precision value is calculated based on the number of true positives (TP), false negatives (FN), and false positives (FP) resulting from comparing the predicted object to all ground truth objects:\n    ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F853978%2F3f1aec3c5b6fcbba10cc35a53d338810%2Fformula2020.png?generation=1586298781910856&amp;alt=media)\n\nYou mentioned TP, FN and FP, but the formula doesn't actually contain FN.\n\nI believe it should be the same as in the previous year competition:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F853978%2F1e5e6eb0408b61f0ae68632a13f6b059%2Fformula2019.png?generation=1586298802133924&amp;alt=media)\n\nIs that correct?",
    "806586": "hey, thanks for noticing the difference. \n\nThis year we adopted the metric from [coco challenge](http://cocodataset.org/#detection-eval), which used TP / (TP + FP).",
    "807269": "Thank you, Menglin!"
  },
  "source": "meta"
}