{
  "id": 77356,
  "title": "CV vs. LB Scores",
  "url": "/competitions/LANL-Earthquake-Prediction/discussion/77356",
  "author_name": "Abhishek Thakur",
  "post_date": "2019-01-11T20:00:54.057000",
  "votes": 6,
  "comment_count": 7,
  "views": 0,
  "content": "<p>I’m not doing a very good validation as of now but my holdout vs lb looks quite off.\nGiven that LB is only on 13% data, one should be very careful </p>\n\n<p>I’ll update my CV/LB scores once i have a proper validation :) Whats your CV/LB score?</p>",
  "messages": [
    {
      "id": 454539,
      "postDate": "2019-01-11T20:00:54.057Z",
      "content": "<p>I’m not doing a very good validation as of now but my holdout vs lb looks quite off.\nGiven that LB is only on 13% data, one should be very careful </p>\n\n<p>I’ll update my CV/LB scores once i have a proper validation :) Whats your CV/LB score?</p>",
      "rawMarkdown": "I’m not doing a very good validation as of now but my holdout vs lb looks quite off.\nGiven that LB is only on 13% data, one should be very careful \n\nI’ll update my CV/LB scores once i have a proper validation :) Whats your CV/LB score?\n",
      "votes": 5
    },
    {
      "id": 455822,
      "postDate": "2019-01-14T16:51:47.347Z",
      "content": "<p>For all my previous submissions prior to yesterday, \nthe ratios had been around 1.30~1.33, same as rest of the folks.</p>\n\n<p>Yesterday, I revamped my model ground up. CV seed has been kept the same. \nThat ratio dropped to 1.13 (the model didn't yield better result on LB though).\nI am still examining the cause. \nThe comment will be deleted if the ratio change is due to my stupid coding glitch :)\n————————————————————————————————————————————\nupdate:\nThat seems to be some memory issue, i resubmitted the result, the CV/LB ratio was 1.29\n————————————————————————————————————————————\nupdate:\nTried lgbm just now, ceteris paribus. The ratio dropped to 1.22, with terrible LB score.\nI purposely overfitted it a little bit, just wanted to confirm how unreliable things could get.</p>",
      "rawMarkdown": "For all my previous submissions prior to yesterday, \nthe ratios had been around 1.30~1.33, same as rest of the folks.\n\nYesterday, I revamped my model ground up. CV seed has been kept the same. \nThat ratio dropped to 1.13 (the model didn't yield better result on LB though).\nI am still examining the cause. \nThe comment will be deleted if the ratio change is due to my stupid coding glitch :)\n————————————————————————————————————————————\nupdate:\nThat seems to be some memory issue, i resubmitted the result, the CV/LB ratio was 1.29\n————————————————————————————————————————————\nupdate:\nTried lgbm just now, ceteris paribus. The ratio dropped to 1.22, with terrible LB score.\nI purposely overfitted it a little bit, just wanted to confirm how unreliable things could get.",
      "votes": 1
    },
    {
      "id": 455655,
      "postDate": "2019-01-14T10:03:11.310Z",
      "content": "<p>CV: 2.036 / LB: 1.531</p>",
      "rawMarkdown": "CV: 2.036 / LB: 1.531"
    },
    {
      "id": 455451,
      "postDate": "2019-01-13T23:48:58.307Z",
      "content": "<p>that's why need choose two submissions carefully for final..</p>",
      "rawMarkdown": "that's why need choose two submissions carefully for final.."
    },
    {
      "id": 455369,
      "postDate": "2019-01-13T17:15:07.413Z",
      "content": "<p>CV: 2.07 / LB: 1.505</p>",
      "rawMarkdown": "CV: 2.07 / LB: 1.505"
    },
    {
      "id": 454994,
      "postDate": "2019-01-12T17:54:36.413Z",
      "content": "<p>CV (7 folds) 2.03 / LB 1.552.</p>",
      "rawMarkdown": "CV (7 folds) 2.03 / LB 1.552."
    },
    {
      "id": 454810,
      "postDate": "2019-01-12T09:00:26.713Z",
      "content": "<p>CV 2.199 / LB 1.700</p>",
      "rawMarkdown": "CV 2.199 / LB 1.700"
    },
    {
      "id": 454935,
      "postDate": "2019-01-12T14:46:02.137Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 455822,
      "author_name": "Elliot",
      "author_url": "",
      "post_date": "2019-01-14T16:51:47.347000",
      "content": "<p>For all my previous submissions prior to yesterday, \nthe ratios had been around 1.30~1.33, same as rest of the folks.</p>\n\n<p>Yesterday, I revamped my model ground up. CV seed has been kept the same. \nThat ratio dropped to 1.13 (the model didn't yield better result on LB though).\nI am still examining the cause. \nThe comment will be deleted if the ratio change is due to my stupid coding glitch :)\n————————————————————————————————————————————\nupdate:\nThat seems to be some memory issue, i resubmitted the result, the CV/LB ratio was 1.29\n————————————————————————————————————————————\nupdate:\nTried lgbm just now, ceteris paribus. The ratio dropped to 1.22, with terrible LB score.\nI purposely overfitted it a little bit, just wanted to confirm how unreliable things could get.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 455655,
      "author_name": "Patrick Yam",
      "author_url": "",
      "post_date": "2019-01-14T10:03:11.310000",
      "content": "<p>CV: 2.036 / LB: 1.531</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 455451,
      "author_name": "Jon",
      "author_url": "",
      "post_date": "2019-01-13T23:48:58.307000",
      "content": "<p>that's why need choose two submissions carefully for final..</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 455369,
      "author_name": "Abhishek Thakur",
      "author_url": "",
      "post_date": "2019-01-13T17:15:07.413000",
      "content": "<p>CV: 2.07 / LB: 1.505</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 454994,
      "author_name": "DmitryS",
      "author_url": "",
      "post_date": "2019-01-12T17:54:36.413000",
      "content": "<p>CV (7 folds) 2.03 / LB 1.552.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 454810,
      "author_name": "Corey Carlson",
      "author_url": "",
      "post_date": "2019-01-12T09:00:26.713000",
      "content": "<p>CV 2.199 / LB 1.700</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 454935,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-01-12T14:46:02.137000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "454539": "I’m not doing a very good validation as of now but my holdout vs lb looks quite off.\nGiven that LB is only on 13% data, one should be very careful \n\nI’ll update my CV/LB scores once i have a proper validation :) Whats your CV/LB score?\n",
    "455822": "For all my previous submissions prior to yesterday, \nthe ratios had been around 1.30~1.33, same as rest of the folks.\n\nYesterday, I revamped my model ground up. CV seed has been kept the same. \nThat ratio dropped to 1.13 (the model didn't yield better result on LB though).\nI am still examining the cause. \nThe comment will be deleted if the ratio change is due to my stupid coding glitch :)\n————————————————————————————————————————————\nupdate:\nThat seems to be some memory issue, i resubmitted the result, the CV/LB ratio was 1.29\n————————————————————————————————————————————\nupdate:\nTried lgbm just now, ceteris paribus. The ratio dropped to 1.22, with terrible LB score.\nI purposely overfitted it a little bit, just wanted to confirm how unreliable things could get.",
    "455655": "CV: 2.036 / LB: 1.531",
    "455451": "that's why need choose two submissions carefully for final..",
    "455369": "CV: 2.07 / LB: 1.505",
    "454994": "CV (7 folds) 2.03 / LB 1.552.",
    "454810": "CV 2.199 / LB 1.700",
    "454935": ""
  }
}