{
  "id": 497412,
  "title": "Submission evaluation metric question",
  "url": "/competitions/leash-BELKA/discussion/497412",
  "author_name": "",
  "post_date": "2024-04-24T15:15:07.285033400Z",
  "votes": 8,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Evaluation is described as \"Mean Average Precision (micro) between the predicted probability and the observed target\".  Given there are 3 targets, is MAP calculated for each target, and then averaged?  Or is there a different way to aggregate across targets?</p>",
  "messages": [
    {
      "id": "2772170",
      "postDate": "04/24/2024 15:15:07",
      "content": "<p>Evaluation is described as \"Mean Average Precision (micro) between the predicted probability and the observed target\".  Given there are 3 targets, is MAP calculated for each target, and then averaged?  Or is there a different way to aggregate across targets?</p>",
      "rawMarkdown": "Evaluation is described as \"Mean Average Precision (micro) between the predicted probability and the observed target\".  Given there are 3 targets, is MAP calculated for each target, and then averaged?  Or is there a different way to aggregate across targets?",
      "votes": null
    },
    {
      "id": "2775386",
      "postDate": "04/25/2024 16:32:33",
      "content": "<p>No, all your predictions are treated as a single set of binary predictions, and MAP is applied on all predictions against all ground truth at the same time. </p>\n<p>To restate differently, you are NOT predicting a multi-label problem, like \"which of these 3 proteins did it bind to?\" You are just predicting \"did they bind?\" and which protein is actually just an input feature, really. </p>",
      "rawMarkdown": "No, all your predictions are treated as a single set of binary predictions, and MAP is applied on all predictions against all ground truth at the same time. \n\nTo restate differently, you are NOT predicting a multi-label problem, like \"which of these 3 proteins did it bind to?\" You are just predicting \"did they bind?\" and which protein is actually just an input feature, really.",
      "votes": null
    },
    {
      "id": "2775909",
      "postDate": "04/25/2024 21:52:29",
      "content": "<p>Thanks for the clarification.</p>",
      "rawMarkdown": "Thanks for the clarification.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2775386,
      "author_name": "roberthatch",
      "author_url": "",
      "post_date": "04/25/2024 16:32:33",
      "content": "<p>No, all your predictions are treated as a single set of binary predictions, and MAP is applied on all predictions against all ground truth at the same time. </p>\n<p>To restate differently, you are NOT predicting a multi-label problem, like \"which of these 3 proteins did it bind to?\" You are just predicting \"did they bind?\" and which protein is actually just an input feature, really. </p>",
      "votes": null,
      "replies": [
        {
          "id": 2775909,
          "author_name": "kirkdco",
          "author_url": "",
          "post_date": "04/25/2024 21:52:29",
          "content": "<p>Thanks for the clarification.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2772170": "Evaluation is described as \"Mean Average Precision (micro) between the predicted probability and the observed target\".  Given there are 3 targets, is MAP calculated for each target, and then averaged?  Or is there a different way to aggregate across targets?",
    "2775386": "No, all your predictions are treated as a single set of binary predictions, and MAP is applied on all predictions against all ground truth at the same time. \n\nTo restate differently, you are NOT predicting a multi-label problem, like \"which of these 3 proteins did it bind to?\" You are just predicting \"did they bind?\" and which protein is actually just an input feature, really.",
    "2775909": "Thanks for the clarification."
  },
  "source": "meta"
}