{
  "id": 347705,
  "title": "Saving the best model doesn't seem to work",
  "url": "/competitions/amex-default-prediction/discussion/347705",
  "author_name": "no-magic",
  "post_date": "2022-08-25T07:36:05.669000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Thank you competitors and host! During the competition, I find one thing works well both on the cv and public lb, but it fails on the private lb: using the best model during the training of the dart lgbm. I compared one feature sets: (another feature set has the similar perf.)<br>\n1809 features, using last - cv: 0.79729,  public lb: 0.79942, private lb: 0.80679; using best - cv: 0.79792, public lb: 0.79967, private lb: 0.80660. <br>\nSo does it mean that using the best doesn't work in the dart, or it is just caused by prediction variance. </p>",
  "messages": [
    {
      "id": 1913168,
      "postDate": "2022-08-25T07:36:05.670Z",
      "content": "<p>Thank you competitors and host! During the competition, I find one thing works well both on the cv and public lb, but it fails on the private lb: using the best model during the training of the dart lgbm. I compared one feature sets: (another feature set has the similar perf.)<br>\n1809 features, using last - cv: 0.79729,  public lb: 0.79942, private lb: 0.80679; using best - cv: 0.79792, public lb: 0.79967, private lb: 0.80660. <br>\nSo does it mean that using the best doesn't work in the dart, or it is just caused by prediction variance. </p>",
      "rawMarkdown": "Thank you competitors and host! During the competition, I find one thing works well both on the cv and public lb, but it fails on the private lb: using the best model during the training of the dart lgbm. I compared one feature sets: (another feature set has the similar perf.)\n1809 features, using last - cv: 0.79729,  public lb: 0.79942, private lb: 0.80679; using best - cv: 0.79792, public lb: 0.79967, private lb: 0.80660. \nSo does it mean that using the best doesn't work in the dart, or it is just caused by prediction variance. \n",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1913168": "Thank you competitors and host! During the competition, I find one thing works well both on the cv and public lb, but it fails on the private lb: using the best model during the training of the dart lgbm. I compared one feature sets: (another feature set has the similar perf.)\n1809 features, using last - cv: 0.79729,  public lb: 0.79942, private lb: 0.80679; using best - cv: 0.79792, public lb: 0.79967, private lb: 0.80660. \nSo does it mean that using the best doesn't work in the dart, or it is just caused by prediction variance. \n"
  }
}