{
  "id": 549209,
  "title": "Does KNNImputer cause data leakage ?",
  "url": "/competitions/child-mind-institute-problematic-internet-use/discussion/549209",
  "author_name": "",
  "post_date": "2024-12-01T07:41:30.800414700Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>In my offline experiments, when using KNNImputer, my CV score is higher than my LB score. However, if I do not use KNNImputer, my CV score is lower than the LB score, and my LB score increases. Does anyone have any insights?😀</p>",
  "messages": [
    {
      "id": "3059934",
      "postDate": "12/01/2024 07:41:30",
      "content": "<p>In my offline experiments, when using KNNImputer, my CV score is higher than my LB score. However, if I do not use KNNImputer, my CV score is lower than the LB score, and my LB score increases. Does anyone have any insights?😀</p>",
      "rawMarkdown": "In my offline experiments, when using KNNImputer, my CV score is higher than my LB score. However, if I do not use KNNImputer, my CV score is lower than the LB score, and my LB score increases. Does anyone have any insights?😀",
      "votes": null
    },
    {
      "id": "3059977",
      "postDate": "12/01/2024 09:09:24",
      "content": "<p>Yes, if you used the KNNImputer, like alot of public notebooks, on the whole dataframe including the target values, it is causing data leak. Since your model can pick up the signal the knn_imputer used, which will be true for the test data in your cvs, since they were all imputed aswell, but not true for the leaderboard.</p>",
      "rawMarkdown": "Yes, if you used the KNNImputer, like alot of public notebooks, on the whole dataframe including the target values, it is causing data leak. Since your model can pick up the signal the knn_imputer used, which will be true for the test data in your cvs, since they were all imputed aswell, but not true for the leaderboard.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3059977,
      "author_name": "mariusheuser",
      "author_url": "",
      "post_date": "12/01/2024 09:09:24",
      "content": "<p>Yes, if you used the KNNImputer, like alot of public notebooks, on the whole dataframe including the target values, it is causing data leak. Since your model can pick up the signal the knn_imputer used, which will be true for the test data in your cvs, since they were all imputed aswell, but not true for the leaderboard.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3059934": "In my offline experiments, when using KNNImputer, my CV score is higher than my LB score. However, if I do not use KNNImputer, my CV score is lower than the LB score, and my LB score increases. Does anyone have any insights?😀",
    "3059977": "Yes, if you used the KNNImputer, like alot of public notebooks, on the whole dataframe including the target values, it is causing data leak. Since your model can pick up the signal the knn_imputer used, which will be true for the test data in your cvs, since they were all imputed aswell, but not true for the leaderboard."
  },
  "source": "meta"
}