{
  "id": 20209,
  "title": "MAP@5 Scoring?",
  "url": "/competitions/expedia-hotel-recommendations/discussion/20209",
  "author_name": "",
  "post_date": "2016-04-17T22:19:41.030Z",
  "votes": 1,
  "comment_count": 2,
  "views": 1023,
  "content": "<p>Hi all,\nThis is my first competition with the MAP evaluation metric. Can someone help me out a bit with understanding it?</p>\n\n<p>As I understand, each record in the test set has a true single hotel_cluster (of the 100). We are trying to predict that hotel cluster. Obviously, I score better if I pick it, and worse if I do not. But the details elude me. For example:</p>\n\n<p>If the true hotel_cluster is 91, then:</p>\n\n<p>Would submitting 91 alone give a better result than 91 in 5 answers (62,57,89,12,91)?\nDoes order matter (91,87 vs 87,91)?\nWould the ideal strategy be to get the probabilities for all 100 clusters for each observation and just submit the top 5?</p>\n\n<p>Surely some newbie questions, but thanks for the help!</p>",
  "messages": [
    {
      "id": "115307",
      "postDate": "04/17/2016 22:19:41",
      "content": "<p>Hi all,\nThis is my first competition with the MAP evaluation metric. Can someone help me out a bit with understanding it?</p>\n\n<p>As I understand, each record in the test set has a true single hotel_cluster (of the 100). We are trying to predict that hotel cluster. Obviously, I score better if I pick it, and worse if I do not. But the details elude me. For example:</p>\n\n<p>If the true hotel_cluster is 91, then:</p>\n\n<p>Would submitting 91 alone give a better result than 91 in 5 answers (62,57,89,12,91)?\nDoes order matter (91,87 vs 87,91)?\nWould the ideal strategy be to get the probabilities for all 100 clusters for each observation and just submit the top 5?</p>\n\n<p>Surely some newbie questions, but thanks for the help!</p>",
      "rawMarkdown": "Hi all,\r\nThis is my first competition with the MAP evaluation metric. Can someone help me out a bit with understanding it?\r\n\r\nAs I understand, each record in the test set has a true single hotel_cluster (of the 100). We are trying to predict that hotel cluster. Obviously, I score better if I pick it, and worse if I do not. But the details elude me. For example:\r\n\r\nIf the true hotel_cluster is 91, then:\r\n\r\nWould submitting 91 alone give a better result than 91 in 5 answers (62,57,89,12,91)?\r\nDoes order matter (91,87 vs 87,91)?\r\nWould the ideal strategy be to get the probabilities for all 100 clusters for each observation and just submit the top 5?\r\n\r\nSurely some newbie questions, but thanks for the help!",
      "votes": null
    },
    {
      "id": "115310",
      "postDate": "04/17/2016 23:10:22",
      "content": "<p>[quote=Steven Watson;115307]</p>\n\n<p>Hi all,\nThis is my first competition with the MAP evaluation metric. Can someone help me out a bit with understanding it?</p>\n\n<p>As I understand, each record in the test set has a true single hotel_cluster (of the 100). We are trying to predict that hotel cluster. Obviously, I score better if I pick it, and worse if I do not. But the details elude me. For example:</p>\n\n<p>If the true hotel_cluster is 91, then:</p>\n\n<p>Would submitting 91 alone give a better result than 91 in 5 answers (62,57,89,12,91)?\nDoes order matter (91,87 vs 87,91)?\nWould the ideal strategy be to get the probabilities for all 100 clusters for each observation and just submit the top 5?</p>\n\n<p>Surely some newbie questions, but thanks for the help!</p>\n\n<p>[/quote]</p>\n\n<p>Check out this <a href=\"http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/\">http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/</a></p>",
      "rawMarkdown": "[quote=Steven Watson;115307]\r\n\r\nHi all,\r\nThis is my first competition with the MAP evaluation metric. Can someone help me out a bit with understanding it?\r\n\r\nAs I understand, each record in the test set has a true single hotel_cluster (of the 100). We are trying to predict that hotel cluster. Obviously, I score better if I pick it, and worse if I do not. But the details elude me. For example:\r\n\r\nIf the true hotel_cluster is 91, then:\r\n\r\nWould submitting 91 alone give a better result than 91 in 5 answers (62,57,89,12,91)?\r\nDoes order matter (91,87 vs 87,91)?\r\nWould the ideal strategy be to get the probabilities for all 100 clusters for each observation and just submit the top 5?\r\n\r\nSurely some newbie questions, but thanks for the help!\r\n\r\n[/quote]\r\n\r\nCheck out this http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/",
      "votes": null
    },
    {
      "id": "116249",
      "postDate": "04/22/2016 17:45:20",
      "content": "<p>I found this alternative description easy to understand: \n<a href=\"https://www.kaggle.com/c/FacebookRecruiting/forums/t/2002/alternate-explanation-of-mean-average-precision\">https://www.kaggle.com/c/FacebookRecruiting/forums/t/2002/alternate-explanation-of-mean-average-precision</a></p>",
      "rawMarkdown": "I found this alternative description easy to understand: \r\nhttps://www.kaggle.com/c/FacebookRecruiting/forums/t/2002/alternate-explanation-of-mean-average-precision",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 115310,
      "author_name": "thakurrajanand",
      "author_url": "",
      "post_date": "04/17/2016 23:10:22",
      "content": "<p>[quote=Steven Watson;115307]</p>\n\n<p>Hi all,\nThis is my first competition with the MAP evaluation metric. Can someone help me out a bit with understanding it?</p>\n\n<p>As I understand, each record in the test set has a true single hotel_cluster (of the 100). We are trying to predict that hotel cluster. Obviously, I score better if I pick it, and worse if I do not. But the details elude me. For example:</p>\n\n<p>If the true hotel_cluster is 91, then:</p>\n\n<p>Would submitting 91 alone give a better result than 91 in 5 answers (62,57,89,12,91)?\nDoes order matter (91,87 vs 87,91)?\nWould the ideal strategy be to get the probabilities for all 100 clusters for each observation and just submit the top 5?</p>\n\n<p>Surely some newbie questions, but thanks for the help!</p>\n\n<p>[/quote]</p>\n\n<p>Check out this <a href=\"http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/\">http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 116249,
      "author_name": "magevadi",
      "author_url": "",
      "post_date": "04/22/2016 17:45:20",
      "content": "<p>I found this alternative description easy to understand: \n<a href=\"https://www.kaggle.com/c/FacebookRecruiting/forums/t/2002/alternate-explanation-of-mean-average-precision\">https://www.kaggle.com/c/FacebookRecruiting/forums/t/2002/alternate-explanation-of-mean-average-precision</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "115307": "Hi all,\r\nThis is my first competition with the MAP evaluation metric. Can someone help me out a bit with understanding it?\r\n\r\nAs I understand, each record in the test set has a true single hotel_cluster (of the 100). We are trying to predict that hotel cluster. Obviously, I score better if I pick it, and worse if I do not. But the details elude me. For example:\r\n\r\nIf the true hotel_cluster is 91, then:\r\n\r\nWould submitting 91 alone give a better result than 91 in 5 answers (62,57,89,12,91)?\r\nDoes order matter (91,87 vs 87,91)?\r\nWould the ideal strategy be to get the probabilities for all 100 clusters for each observation and just submit the top 5?\r\n\r\nSurely some newbie questions, but thanks for the help!",
    "115310": "[quote=Steven Watson;115307]\r\n\r\nHi all,\r\nThis is my first competition with the MAP evaluation metric. Can someone help me out a bit with understanding it?\r\n\r\nAs I understand, each record in the test set has a true single hotel_cluster (of the 100). We are trying to predict that hotel cluster. Obviously, I score better if I pick it, and worse if I do not. But the details elude me. For example:\r\n\r\nIf the true hotel_cluster is 91, then:\r\n\r\nWould submitting 91 alone give a better result than 91 in 5 answers (62,57,89,12,91)?\r\nDoes order matter (91,87 vs 87,91)?\r\nWould the ideal strategy be to get the probabilities for all 100 clusters for each observation and just submit the top 5?\r\n\r\nSurely some newbie questions, but thanks for the help!\r\n\r\n[/quote]\r\n\r\nCheck out this http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/",
    "116249": "I found this alternative description easy to understand: \r\nhttps://www.kaggle.com/c/FacebookRecruiting/forums/t/2002/alternate-explanation-of-mean-average-precision"
  },
  "source": "meta"
}