{
  "id": 343972,
  "title": "CV score 0.8016 but LB score 0.795, where is my problem?",
  "url": "/competitions/amex-default-prediction/discussion/343972",
  "author_name": "bjjiang",
  "post_date": "2022-08-13T11:20:24.271000",
  "votes": 3,
  "comment_count": 17,
  "views": 0,
  "content": "<p>CV score 0.8016 but LB score 0.795, where is my problem?</p>",
  "messages": [
    {
      "id": 1897145,
      "postDate": "2022-08-13T13:28:23.153Z",
      "content": "<p>You have information leakage in your validation folds. probably you did some target encoding improperly</p>",
      "rawMarkdown": "You have information leakage in your validation folds. probably you did some target encoding improperly",
      "votes": 8,
      "replies": [
        {
          "id": 1897160,
          "postDate": "2022-08-13T13:43:53.400Z",
          "content": "<p>yes，I did some encoding，but I did the same in test folds. Is that affact the result?</p>",
          "rawMarkdown": "yes，I did some encoding，but I did the same in test folds. Is that affact the result?"
        },
        {
          "id": 1897198,
          "postDate": "2022-08-13T14:31:54.570Z",
          "content": "<p>yes. you made a mistake. you have to learn how to do it properly :)</p>",
          "rawMarkdown": "yes. you made a mistake. you have to learn how to do it properly :)",
          "votes": 4
        },
        {
          "id": 1897258,
          "postDate": "2022-08-13T15:30:49.133Z",
          "content": "<p>Thank you very much!</p>",
          "rawMarkdown": "Thank you very much!",
          "votes": 1
        },
        {
          "id": 1898200,
          "postDate": "2022-08-14T11:51:42.127Z",
          "rawMarkdown": "",
          "votes": 2,
          "isDeleted": true
        },
        {
          "id": 1898219,
          "postDate": "2022-08-14T12:10:31.287Z",
          "content": "<p><a href=\"https://www.kaggle.com/yiwangnz\" target=\"_blank\">@yiwangnz</a> sorry, but I would not recommend this as a good way to do target encoding.</p>",
          "rawMarkdown": "@yiwangnz sorry, but I would not recommend this as a good way to do target encoding.",
          "votes": 1
        },
        {
          "id": 1898300,
          "postDate": "2022-08-14T13:39:20.057Z",
          "content": "<p>It is not really about how you encode but doing it fold wise. If you do it on the original data set you will have leakage across folds. </p>",
          "rawMarkdown": "It is not really about how you encode but doing it fold wise. If you do it on the original data set you will have leakage across folds. ",
          "votes": 1
        }
      ]
    },
    {
      "id": 1897005,
      "postDate": "2022-08-13T11:20:24.270Z",
      "content": "<p>CV score 0.8016 but LB score 0.795, where is my problem?</p>",
      "rawMarkdown": "CV score 0.8016 but LB score 0.795, where is my problem?",
      "votes": 3
    },
    {
      "id": 1897015,
      "postDate": "2022-08-13T11:30:44.767Z",
      "content": "<p>CV of 0.8016 seems very high and I would think even the current #1 hasn't got such a cv score. Have you used any validation data for training? You should carefully check your pipeline for any leaks.</p>",
      "rawMarkdown": "CV of 0.8016 seems very high and I would think even the current #1 hasn't got such a cv score. Have you used any validation data for training? You should carefully check your pipeline for any leaks.",
      "votes": 3,
      "replies": [
        {
          "id": 1897163,
          "postDate": "2022-08-13T13:46:57.033Z",
          "content": "<p>Do you mean that all the top ten on the public leaderboard have used blending or Bagging methods to get their score???</p>",
          "rawMarkdown": "Do you mean that all the top ten on the public leaderboard have used blending or Bagging methods to get their score???"
        },
        {
          "id": 1898291,
          "postDate": "2022-08-14T13:26:45.227Z",
          "content": "<p>Not sure, but I believe so.  </p>\n<p>(That and nobody has .8016 on the public LB yet, since it would round up to .802 ;) )</p>",
          "rawMarkdown": "Not sure, but I believe so.  \n\n(That and nobody has .8016 on the public LB yet, since it would round up to .802 ;) )"
        },
        {
          "id": 1898558,
          "postDate": "2022-08-14T16:26:58.817Z",
          "content": "<p>LB doesnt round up. it uses floor so 0.801999999(9) is 0.801</p>",
          "rawMarkdown": "LB doesnt round up. it uses floor so 0.801999999(9) is 0.801",
          "votes": 4
        },
        {
          "id": 1898567,
          "postDate": "2022-08-14T16:43:43.703Z",
          "content": "<p>I am confused about this competition, why can't anyone raise their score above 0.802?<br>\nis there the same method of working in the top 10 on Leaderboard</p>",
          "rawMarkdown": "I am confused about this competition, why can't anyone raise their score above 0.802?\nis there the same method of working in the top 10 on Leaderboard"
        },
        {
          "id": 1898770,
          "postDate": "2022-08-14T19:48:29.030Z",
          "content": "<p>what about if someone reaches 0.802? will you ask the very same question why noone is 0.803?</p>",
          "rawMarkdown": "what about if someone reaches 0.802? will you ask the very same question why noone is 0.803?",
          "votes": 6
        },
        {
          "id": 1898786,
          "postDate": "2022-08-14T20:18:03.800Z",
          "content": "<p>No, I asked because the public leaderboard hasn't changed in a long time, Is this the maximum level of the score must possible for us or not, </p>",
          "rawMarkdown": "No, I asked because the public leaderboard hasn't changed in a long time, Is this the maximum level of the score must possible for us or not, "
        }
      ]
    },
    {
      "id": 1906463,
      "postDate": "2022-08-19T22:51:46.043Z",
      "content": "<p>Check what features you calculated before you split your data into folds. <br>\nMaybe some feature is leaking across the different folds. </p>\n<p>The Devastator.</p>",
      "rawMarkdown": "Check what features you calculated before you split your data into folds. \nMaybe some feature is leaking across the different folds. \n\nThe Devastator.\n",
      "votes": 1
    },
    {
      "id": 1897157,
      "postDate": "2022-08-13T13:41:20.090Z",
      "content": "<p>This is picture</p>",
      "rawMarkdown": "This is picture"
    },
    {
      "id": 1899105,
      "postDate": "2022-08-15T04:18:57.147Z",
      "content": "<p>useful topic,thanks</p>",
      "rawMarkdown": "useful topic,thanks"
    }
  ],
  "comments": [
    {
      "id": 1897145,
      "author_name": "raddar",
      "author_url": "",
      "post_date": "2022-08-13T13:28:23.153000",
      "content": "<p>You have information leakage in your validation folds. probably you did some target encoding improperly</p>",
      "votes": 8,
      "replies": [
        {
          "id": 1897160,
          "author_name": "bjjiang",
          "author_url": "",
          "post_date": "2022-08-13T13:43:53.400000",
          "content": "<p>yes，I did some encoding，but I did the same in test folds. Is that affact the result?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1897198,
          "author_name": "raddar",
          "author_url": "",
          "post_date": "2022-08-13T14:31:54.570000",
          "content": "<p>yes. you made a mistake. you have to learn how to do it properly :)</p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 1897258,
          "author_name": "bjjiang",
          "author_url": "",
          "post_date": "2022-08-13T15:30:49.133000",
          "content": "<p>Thank you very much!</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1898200,
          "author_name": "",
          "author_url": "",
          "post_date": "2022-08-14T11:51:42.127000",
          "content": "",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1898219,
          "author_name": "raddar",
          "author_url": "",
          "post_date": "2022-08-14T12:10:31.287000",
          "content": "<p><a href=\"https://www.kaggle.com/yiwangnz\" target=\"_blank\">@yiwangnz</a> sorry, but I would not recommend this as a good way to do target encoding.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1898300,
          "author_name": "Lucas Morin",
          "author_url": "",
          "post_date": "2022-08-14T13:39:20.057000",
          "content": "<p>It is not really about how you encode but doing it fold wise. If you do it on the original data set you will have leakage across folds. </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1897015,
      "author_name": "Fritz Cremer",
      "author_url": "",
      "post_date": "2022-08-13T11:30:44.767000",
      "content": "<p>CV of 0.8016 seems very high and I would think even the current #1 hasn't got such a cv score. Have you used any validation data for training? You should carefully check your pipeline for any leaks.</p>",
      "votes": 3,
      "replies": [
        {
          "id": 1897163,
          "author_name": "EL Younes",
          "author_url": "",
          "post_date": "2022-08-13T13:46:57.033000",
          "content": "<p>Do you mean that all the top ten on the public leaderboard have used blending or Bagging methods to get their score???</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1898291,
          "author_name": "happycube",
          "author_url": "",
          "post_date": "2022-08-14T13:26:45.227000",
          "content": "<p>Not sure, but I believe so.  </p>\n<p>(That and nobody has .8016 on the public LB yet, since it would round up to .802 ;) )</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1898558,
          "author_name": "raddar",
          "author_url": "",
          "post_date": "2022-08-14T16:26:58.817000",
          "content": "<p>LB doesnt round up. it uses floor so 0.801999999(9) is 0.801</p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 1898567,
          "author_name": "EL Younes",
          "author_url": "",
          "post_date": "2022-08-14T16:43:43.703000",
          "content": "<p>I am confused about this competition, why can't anyone raise their score above 0.802?<br>\nis there the same method of working in the top 10 on Leaderboard</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1898770,
          "author_name": "raddar",
          "author_url": "",
          "post_date": "2022-08-14T19:48:29.030000",
          "content": "<p>what about if someone reaches 0.802? will you ask the very same question why noone is 0.803?</p>",
          "votes": 6,
          "replies": []
        },
        {
          "id": 1898786,
          "author_name": "EL Younes",
          "author_url": "",
          "post_date": "2022-08-14T20:18:03.800000",
          "content": "<p>No, I asked because the public leaderboard hasn't changed in a long time, Is this the maximum level of the score must possible for us or not, </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1906463,
      "author_name": "The Devastator",
      "author_url": "",
      "post_date": "2022-08-19T22:51:46.043000",
      "content": "<p>Check what features you calculated before you split your data into folds. <br>\nMaybe some feature is leaking across the different folds. </p>\n<p>The Devastator.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1897157,
      "author_name": "bjjiang",
      "author_url": "",
      "post_date": "2022-08-13T13:41:20.090000",
      "content": "<p>This is picture</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1899105,
      "author_name": "Ross",
      "author_url": "",
      "post_date": "2022-08-15T04:18:57.147000",
      "content": "<p>useful topic,thanks</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1897145": "You have information leakage in your validation folds. probably you did some target encoding improperly",
    "1897005": "CV score 0.8016 but LB score 0.795, where is my problem?",
    "1897015": "CV of 0.8016 seems very high and I would think even the current #1 hasn't got such a cv score. Have you used any validation data for training? You should carefully check your pipeline for any leaks.",
    "1906463": "Check what features you calculated before you split your data into folds. \nMaybe some feature is leaking across the different folds. \n\nThe Devastator.\n",
    "1897157": "This is picture",
    "1899105": "useful topic,thanks"
  }
}