{
  "id": 320194,
  "title": "Does the order of the articles per customer in the test set matter?",
  "url": "/competitions/h-and-m-personalized-fashion-recommendations/discussion/320194",
  "author_name": "",
  "post_date": "2022-04-20T13:11:42.780766400Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Forgive me if this topic was already covered somewhere else. I just joined the competition today and I am trying to get my head around the evaluation metric: the MAP@12. This is a good post by the way: <a href=\"https://sdsawtelle.github.io/blog/output/mean-average-precision-MAP-for-recommender-systems.html\" target=\"_blank\">here</a>.<br>\nFrom my understanding, the order of the articles per customer in the submission file matters. No major issues until there. However, I wonder if this order in the set that the organiser uses for the MAP@12 calculation matters - let's call it the true set. Is the MAP@12 only calculated based on if YES or NO the items were purchased within 7 days, or does the order in the true set also matters. And then, if the order matters in the true set matters, what is this order based on?</p>\n<p><strong>Example for AP@3</strong></p>\n<p>Scenario 1: The order in the true set doesn't matter</p>\n<p>prediction for one customer: 1 2 3 <br>\nitems bought within 7 days with no order: 1 and 2<br>\nrecommendations: [1, 1, 0]<br>\nmetric: (1/1 + 1/2)/3 = 0.5</p>\n<p>Scenario 2: The order in the true set matters</p>\n<p>prediction for one customer: 1 2 3 <br>\nitems bought within 7 days with order: 1 4 2<br>\nrecommendations: [1, 0, 1]<br>\nmetric: (1/1 + 1/3)/3 = 0.44</p>",
  "messages": [
    {
      "id": "1762140",
      "postDate": "04/20/2022 13:11:42",
      "content": "<p>Forgive me if this topic was already covered somewhere else. I just joined the competition today and I am trying to get my head around the evaluation metric: the MAP@12. This is a good post by the way: <a href=\"https://sdsawtelle.github.io/blog/output/mean-average-precision-MAP-for-recommender-systems.html\" target=\"_blank\">here</a>.<br>\nFrom my understanding, the order of the articles per customer in the submission file matters. No major issues until there. However, I wonder if this order in the set that the organiser uses for the MAP@12 calculation matters - let's call it the true set. Is the MAP@12 only calculated based on if YES or NO the items were purchased within 7 days, or does the order in the true set also matters. And then, if the order matters in the true set matters, what is this order based on?</p>\n<p><strong>Example for AP@3</strong></p>\n<p>Scenario 1: The order in the true set doesn't matter</p>\n<p>prediction for one customer: 1 2 3 <br>\nitems bought within 7 days with no order: 1 and 2<br>\nrecommendations: [1, 1, 0]<br>\nmetric: (1/1 + 1/2)/3 = 0.5</p>\n<p>Scenario 2: The order in the true set matters</p>\n<p>prediction for one customer: 1 2 3 <br>\nitems bought within 7 days with order: 1 4 2<br>\nrecommendations: [1, 0, 1]<br>\nmetric: (1/1 + 1/3)/3 = 0.44</p>",
      "rawMarkdown": "Forgive me if this topic was already covered somewhere else. I just joined the competition today and I am trying to get my head around the evaluation metric: the MAP@12. This is a good post by the way: [here](https://sdsawtelle.github.io/blog/output/mean-average-precision-MAP-for-recommender-systems.html).\nFrom my understanding, the order of the articles per customer in the submission file matters. No major issues until there. However, I wonder if this order in the set that the organiser uses for the MAP@12 calculation matters - let's call it the true set. Is the MAP@12 only calculated based on if YES or NO the items were purchased within 7 days, or does the order in the true set also matters. And then, if the order matters in the true set matters, what is this order based on?\n\n**Example for AP@3**\n\nScenario 1: The order in the true set doesn't matter\n\nprediction for one customer: 1 2 3 \nitems bought within 7 days with no order: 1 and 2\nrecommendations: [1, 1, 0]\nmetric: (1/1 + 1/2)/3 = 0.5\n\nScenario 2: The order in the true set matters\n\nprediction for one customer: 1 2 3 \nitems bought within 7 days with order: 1 4 2\nrecommendations: [1, 0, 1]\nmetric: (1/1 + 1/3)/3 = 0.44",
      "votes": null
    },
    {
      "id": "1762354",
      "postDate": "04/20/2022 16:12:14",
      "content": "<p>The order in the true set does <strong>not</strong> matter.</p>",
      "rawMarkdown": "The order in the true set does **not** matter.",
      "votes": null
    },
    {
      "id": "1762651",
      "postDate": "04/20/2022 21:43:28",
      "content": "<p>Thanks! Makes all sense now.</p>",
      "rawMarkdown": "Thanks! Makes all sense now.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1762354,
      "author_name": "jacob34",
      "author_url": "",
      "post_date": "04/20/2022 16:12:14",
      "content": "<p>The order in the true set does <strong>not</strong> matter.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1762651,
          "author_name": "loicge",
          "author_url": "",
          "post_date": "04/20/2022 21:43:28",
          "content": "<p>Thanks! Makes all sense now.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1762140": "Forgive me if this topic was already covered somewhere else. I just joined the competition today and I am trying to get my head around the evaluation metric: the MAP@12. This is a good post by the way: [here](https://sdsawtelle.github.io/blog/output/mean-average-precision-MAP-for-recommender-systems.html).\nFrom my understanding, the order of the articles per customer in the submission file matters. No major issues until there. However, I wonder if this order in the set that the organiser uses for the MAP@12 calculation matters - let's call it the true set. Is the MAP@12 only calculated based on if YES or NO the items were purchased within 7 days, or does the order in the true set also matters. And then, if the order matters in the true set matters, what is this order based on?\n\n**Example for AP@3**\n\nScenario 1: The order in the true set doesn't matter\n\nprediction for one customer: 1 2 3 \nitems bought within 7 days with no order: 1 and 2\nrecommendations: [1, 1, 0]\nmetric: (1/1 + 1/2)/3 = 0.5\n\nScenario 2: The order in the true set matters\n\nprediction for one customer: 1 2 3 \nitems bought within 7 days with order: 1 4 2\nrecommendations: [1, 0, 1]\nmetric: (1/1 + 1/3)/3 = 0.44",
    "1762354": "The order in the true set does **not** matter.",
    "1762651": "Thanks! Makes all sense now."
  },
  "source": "meta"
}