{
  "id": 497051,
  "title": "Why are the scores obtained by direct training and pre-trained model + inference different?",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/497051",
  "author_name": "Luck is what I need",
  "post_date": "2024-04-23T12:11:57.265000",
  "votes": 2,
  "comment_count": 5,
  "views": 0,
  "content": "<p>After I tried the pre-trained model, I found that the effect was 0.0x worse, and the score was not as good as the direct training. Why?</p>",
  "messages": [
    {
      "id": 2769810,
      "postDate": "2024-04-23T15:00:28.180Z",
      "content": "<p>Kindly check the usage of the random seed and use <code>gpu_use_dp = True</code> in LGBM to try and replicate the results <a href=\"https://www.kaggle.com/majiaqi111\" target=\"_blank\">@majiaqi111</a> </p>",
      "rawMarkdown": "Kindly check the usage of the random seed and use `gpu_use_dp = True` in LGBM to try and replicate the results @majiaqi111 ",
      "votes": 3
    },
    {
      "id": 2769873,
      "postDate": "2024-04-23T15:27:25.200Z",
      "content": "<p>0.0x is abnormal. Possible situations:<br>\n1) the predicted columns are shuffled;<br>\n2) different process of train and test data, possible missing the catergory process;<br>\n3) model save bug, only the initialization defined model has been saved.</p>",
      "rawMarkdown": "0.0x is abnormal. Possible situations:\n1) the predicted columns are shuffled;\n2) different process of train and test data, possible missing the catergory process;\n3) model save bug, only the initialization defined model has been saved.",
      "votes": 1
    },
    {
      "id": 2769548,
      "postDate": "2024-04-23T12:11:57.267Z",
      "content": "<p>After I tried the pre-trained model, I found that the effect was 0.0x worse, and the score was not as good as the direct training. Why?</p>",
      "rawMarkdown": "After I tried the pre-trained model, I found that the effect was 0.0x worse, and the score was not as good as the direct training. Why?",
      "votes": 2
    },
    {
      "id": 2807955,
      "postDate": "2024-05-12T02:15:13.143Z",
      "content": "<p>Did you find the answer? What was the cause?</p>",
      "rawMarkdown": "Did you find the answer? What was the cause?"
    },
    {
      "id": 2769591,
      "postDate": "2024-04-23T12:52:26.733Z",
      "content": "<p>Is it possible that it's just a coincidence? I remember a post saying that the program runs differently every time.</p>",
      "rawMarkdown": "Is it possible that it's just a coincidence? I remember a post saying that the program runs differently every time."
    },
    {
      "id": 3155680,
      "postDate": "2025-03-21T09:47:59.333Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2769810,
      "author_name": "Ravi Ramakrishnan",
      "author_url": "",
      "post_date": "2024-04-23T15:00:28.180000",
      "content": "<p>Kindly check the usage of the random seed and use <code>gpu_use_dp = True</code> in LGBM to try and replicate the results <a href=\"https://www.kaggle.com/majiaqi111\" target=\"_blank\">@majiaqi111</a> </p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 2769873,
      "author_name": "KIT",
      "author_url": "",
      "post_date": "2024-04-23T15:27:25.200000",
      "content": "<p>0.0x is abnormal. Possible situations:<br>\n1) the predicted columns are shuffled;<br>\n2) different process of train and test data, possible missing the catergory process;<br>\n3) model save bug, only the initialization defined model has been saved.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2807955,
      "author_name": "ano",
      "author_url": "",
      "post_date": "2024-05-12T02:15:13.143000",
      "content": "<p>Did you find the answer? What was the cause?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2769591,
      "author_name": "yunsuxiaozi",
      "author_url": "",
      "post_date": "2024-04-23T12:52:26.733000",
      "content": "<p>Is it possible that it's just a coincidence? I remember a post saying that the program runs differently every time.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3155680,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-03-21T09:47:59.333000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2769810": "Kindly check the usage of the random seed and use `gpu_use_dp = True` in LGBM to try and replicate the results @majiaqi111 ",
    "2769873": "0.0x is abnormal. Possible situations:\n1) the predicted columns are shuffled;\n2) different process of train and test data, possible missing the catergory process;\n3) model save bug, only the initialization defined model has been saved.",
    "2769548": "After I tried the pre-trained model, I found that the effect was 0.0x worse, and the score was not as good as the direct training. Why?",
    "2807955": "Did you find the answer? What was the cause?",
    "2769591": "Is it possible that it's just a coincidence? I remember a post saying that the program runs differently every time.",
    "3155680": ""
  }
}