{
  "id": 541228,
  "title": "Use scikit-learn for the custom metric with ease",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/541228",
  "author_name": "Ravi Ramakrishnan",
  "post_date": "2024-10-18T08:42:33.128000",
  "votes": 25,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hello all,</p>\n<p>A lot of public materials in the competition use numpy based implementation of R-squared measure with weights. This is great, but we have an easier way with scikit-learn that could be used to same effect. </p>\n<p>Just use the below for convenience - </p>\n<pre><code> sklearn.metrics  r2_score\nr2_score(ytrue, ypred, sample_weight = sample_weight)\n</code></pre>\n<p>More details are present <a href=\"https://www.kaggle.com/code/ravi20076/janestreet2024-baseline-train-v1\" target=\"_blank\">here</a> for perusal. </p>\n<p>All the best!</p>",
  "messages": [
    {
      "id": 3021184,
      "postDate": "2024-10-18T08:42:33.127Z",
      "content": "<p>Hello all,</p>\n<p>A lot of public materials in the competition use numpy based implementation of R-squared measure with weights. This is great, but we have an easier way with scikit-learn that could be used to same effect. </p>\n<p>Just use the below for convenience - </p>\n<pre><code> sklearn.metrics  r2_score\nr2_score(ytrue, ypred, sample_weight = sample_weight)\n</code></pre>\n<p>More details are present <a href=\"https://www.kaggle.com/code/ravi20076/janestreet2024-baseline-train-v1\" target=\"_blank\">here</a> for perusal. </p>\n<p>All the best!</p>",
      "rawMarkdown": "Hello all,\n\nA lot of public materials in the competition use numpy based implementation of R-squared measure with weights. This is great, but we have an easier way with scikit-learn that could be used to same effect. \n\nJust use the below for convenience - \n\n```python\nfrom sklearn.metrics import r2_score\nr2_score(ytrue, ypred, sample_weight = sample_weight)\n```\n\nMore details are present [here](https://www.kaggle.com/code/ravi20076/janestreet2024-baseline-train-v1) for perusal. \n\nAll the best!",
      "votes": 26
    },
    {
      "id": 3021459,
      "postDate": "2024-10-18T14:23:02.833Z",
      "content": "<p>Thanks for sharing this! However, it’s worth noting that if we refer to the <a href=\"https://github.com/scikit-learn/scikit-learn/blob/6e9039160/sklearn/metrics/_regression.py#L1222\" target=\"_blank\">scikit-learn source</a>, the weighted R-squared score only equals the competition metric when <code>average(y_true, weights=sample_weight) = 0</code>, which isn’t always the case. Given the fine margins in leaderboard, I’ll continue using the competition metric for accuracy.</p>",
      "rawMarkdown": "Thanks for sharing this! However, it’s worth noting that if we refer to the [scikit-learn source](https://github.com/scikit-learn/scikit-learn/blob/6e9039160/sklearn/metrics/_regression.py#L1222), the weighted R-squared score only equals the competition metric when `average(y_true, weights=sample_weight) = 0`, which isn’t always the case. Given the fine margins in leaderboard, I’ll continue using the competition metric for accuracy.",
      "votes": 5
    },
    {
      "id": 3077945,
      "postDate": "2024-12-21T15:29:31.780Z",
      "content": "<p>It‘s really helpful! Thanks for sharing this!</p>",
      "rawMarkdown": "It‘s really helpful! Thanks for sharing this!\n\n\n",
      "votes": -1
    },
    {
      "id": 3077893,
      "postDate": "2024-12-21T13:42:38.450Z",
      "content": "<p>It‘s really helpful! Thanks for sharing this!</p>",
      "rawMarkdown": "It‘s really helpful! Thanks for sharing this!"
    },
    {
      "id": 3021801,
      "postDate": "2024-10-18T21:18:22.740Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 3021459,
      "author_name": "Maaax",
      "author_url": "",
      "post_date": "2024-10-18T14:23:02.833000",
      "content": "<p>Thanks for sharing this! However, it’s worth noting that if we refer to the <a href=\"https://github.com/scikit-learn/scikit-learn/blob/6e9039160/sklearn/metrics/_regression.py#L1222\" target=\"_blank\">scikit-learn source</a>, the weighted R-squared score only equals the competition metric when <code>average(y_true, weights=sample_weight) = 0</code>, which isn’t always the case. Given the fine margins in leaderboard, I’ll continue using the competition metric for accuracy.</p>",
      "votes": 5,
      "replies": []
    },
    {
      "id": 3077945,
      "author_name": "MrSimple",
      "author_url": "",
      "post_date": "2024-12-21T15:29:31.780000",
      "content": "<p>It‘s really helpful! Thanks for sharing this!</p>",
      "votes": -1,
      "replies": []
    },
    {
      "id": 3077893,
      "author_name": "helium lu",
      "author_url": "",
      "post_date": "2024-12-21T13:42:38.450000",
      "content": "<p>It‘s really helpful! Thanks for sharing this!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3021801,
      "author_name": "",
      "author_url": "",
      "post_date": "2024-10-18T21:18:22.740000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3021184": "Hello all,\n\nA lot of public materials in the competition use numpy based implementation of R-squared measure with weights. This is great, but we have an easier way with scikit-learn that could be used to same effect. \n\nJust use the below for convenience - \n\n```python\nfrom sklearn.metrics import r2_score\nr2_score(ytrue, ypred, sample_weight = sample_weight)\n```\n\nMore details are present [here](https://www.kaggle.com/code/ravi20076/janestreet2024-baseline-train-v1) for perusal. \n\nAll the best!",
    "3021459": "Thanks for sharing this! However, it’s worth noting that if we refer to the [scikit-learn source](https://github.com/scikit-learn/scikit-learn/blob/6e9039160/sklearn/metrics/_regression.py#L1222), the weighted R-squared score only equals the competition metric when `average(y_true, weights=sample_weight) = 0`, which isn’t always the case. Given the fine margins in leaderboard, I’ll continue using the competition metric for accuracy.",
    "3077945": "It‘s really helpful! Thanks for sharing this!\n\n\n",
    "3077893": "It‘s really helpful! Thanks for sharing this!",
    "3021801": ""
  }
}