{
  "id": 322370,
  "title": "LGBMRanker - competition eval_metric",
  "url": "/competitions/h-and-m-personalized-fashion-recommendations/discussion/322370",
  "author_name": "",
  "post_date": "2022-05-01T18:41:34.876365700Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Has anyone implemented a custom eval_metric for LGBMRanker to match this competition's metric?</p>",
  "messages": [
    {
      "id": "1774052",
      "postDate": "05/01/2022 18:41:34",
      "content": "<p>Has anyone implemented a custom eval_metric for LGBMRanker to match this competition's metric?</p>",
      "rawMarkdown": "Has anyone implemented a custom eval_metric for LGBMRanker to match this competition's metric?",
      "votes": null
    },
    {
      "id": "1774392",
      "postDate": "05/02/2022 06:23:07",
      "content": "<p>This is a great question! I haven't implemented a custom eval_metric for LGBMRanker, but I'm sure it's possible. You could take a look at the source code for the LGBMRanker implementation and see how the metric is calculated.</p>",
      "rawMarkdown": "This is a great question! I haven't implemented a custom eval_metric for LGBMRanker, but I'm sure it's possible. You could take a look at the source code for the LGBMRanker implementation and see how the metric is calculated.",
      "votes": null
    },
    {
      "id": "1776213",
      "postDate": "05/03/2022 19:21:45",
      "content": "<p>Figured out how to do it.</p>\n<p>LGBMRanker only passes the y_true and y_pred labels for the candidates to the metric function, but for the competition metric you also need to know all the other true purchases.</p>\n<pre><code>def comp_metric(y_true, y_pred):\n    if len(y_pred) == len(train_length):\n        score = calculate_model_score(train_truth_df, y_pred)\n    else:\n        score = calculate_model_score(eval_truth_df, y_pred)\n\n    return \"comp_metric\", score, True\n</code></pre>",
      "rawMarkdown": "Figured out how to do it.\n\nLGBMRanker only passes the y_true and y_pred labels for the candidates to the metric function, but for the competition metric you also need to know all the other true purchases.\n\n```\ndef comp_metric(y_true, y_pred):\n    if len(y_pred) == len(train_length):\n        score = calculate_model_score(train_truth_df, y_pred)\n    else:\n        score = calculate_model_score(eval_truth_df, y_pred)\n        \n    return \"comp_metric\", score, True\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1774392,
      "author_name": "",
      "author_url": "",
      "post_date": "05/02/2022 06:23:07",
      "content": "<p>This is a great question! I haven't implemented a custom eval_metric for LGBMRanker, but I'm sure it's possible. You could take a look at the source code for the LGBMRanker implementation and see how the metric is calculated.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1776213,
      "author_name": "jacob34",
      "author_url": "",
      "post_date": "05/03/2022 19:21:45",
      "content": "<p>Figured out how to do it.</p>\n<p>LGBMRanker only passes the y_true and y_pred labels for the candidates to the metric function, but for the competition metric you also need to know all the other true purchases.</p>\n<pre><code>def comp_metric(y_true, y_pred):\n    if len(y_pred) == len(train_length):\n        score = calculate_model_score(train_truth_df, y_pred)\n    else:\n        score = calculate_model_score(eval_truth_df, y_pred)\n\n    return \"comp_metric\", score, True\n</code></pre>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1774052": "Has anyone implemented a custom eval_metric for LGBMRanker to match this competition's metric?",
    "1774392": "This is a great question! I haven't implemented a custom eval_metric for LGBMRanker, but I'm sure it's possible. You could take a look at the source code for the LGBMRanker implementation and see how the metric is calculated.",
    "1776213": "Figured out how to do it.\n\nLGBMRanker only passes the y_true and y_pred labels for the candidates to the metric function, but for the competition metric you also need to know all the other true purchases.\n\n```\ndef comp_metric(y_true, y_pred):\n    if len(y_pred) == len(train_length):\n        score = calculate_model_score(train_truth_df, y_pred)\n    else:\n        score = calculate_model_score(eval_truth_df, y_pred)\n        \n    return \"comp_metric\", score, True\n```"
  },
  "source": "meta"
}