{
  "id": 323650,
  "title": "CV score-vs-LB score",
  "url": "/competitions/smartphone-decimeter-2022/discussion/323650",
  "author_name": "",
  "post_date": "2022-05-07T15:02:56.726594800Z",
  "votes": 7,
  "comment_count": 2,
  "views": 0,
  "content": "<h1>What is the relationship between your score on the training data and your score on LB?</h1>\n<p>In the last competition, I missed the solo gold by one step. <br>\nI would like to do my best.</p>\n<p>My results are as follows.</p>\n<ul>\n<li>CV: 3.963, LB: 4.308</li>\n<li>CV: 3.788, LB: 4.220</li>\n</ul>\n<p>I will add the score if I can update it.</p>\n<h1>Memo</h1>\n<p>Previous 'CV score-vs-LB score' Discussion.<br>\n<a href=\"https://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/discussion/254574\" target=\"_blank\">https://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/discussion/254574</a></p>\n<p>The last competition shook up the ranking somewhat.<br>\n<a href=\"https://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/leaderboard\" target=\"_blank\">https://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/leaderboard</a></p>",
  "messages": [
    {
      "id": "1780552",
      "postDate": "05/07/2022 15:02:56",
      "content": "<h1>What is the relationship between your score on the training data and your score on LB?</h1>\n<p>In the last competition, I missed the solo gold by one step. <br>\nI would like to do my best.</p>\n<p>My results are as follows.</p>\n<ul>\n<li>CV: 3.963, LB: 4.308</li>\n<li>CV: 3.788, LB: 4.220</li>\n</ul>\n<p>I will add the score if I can update it.</p>\n<h1>Memo</h1>\n<p>Previous 'CV score-vs-LB score' Discussion.<br>\n<a href=\"https://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/discussion/254574\" target=\"_blank\">https://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/discussion/254574</a></p>\n<p>The last competition shook up the ranking somewhat.<br>\n<a href=\"https://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/leaderboard\" target=\"_blank\">https://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/leaderboard</a></p>",
      "rawMarkdown": "# What is the relationship between your score on the training data and your score on LB?\n\nIn the last competition, I missed the solo gold by one step. \nI would like to do my best.\n\nMy results are as follows.\n- CV: 3.963, LB: 4.308\n- CV: 3.788, LB: 4.220\n\nI will add the score if I can update it.\n\n# Memo\nPrevious 'CV score-vs-LB score' Discussion.\nhttps://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/discussion/254574\n\nThe last competition shook up the ranking somewhat.\nhttps://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/leaderboard",
      "votes": null
    },
    {
      "id": "1780580",
      "postDate": "05/07/2022 15:37:04",
      "content": "<p>Even the provided baseline solution has very different scores on the public LB.  On the training set, it scores 4.357m whereas the public LB score is 4.870m.  That's 51.3cm or 11.8% worse on public LB ☹️.</p>\n<p>My own pairs so far (with diffs from the previous in brackets) are…</p>\n<ul>\n<li>Train 4.357m (base), Public LB 4.870, (base)</li>\n<li>Train 3.996 (-36cm), Public LB 4.463 (-41cm)</li>\n<li>Train 3.957 (-4cm), Public LB 4.405 (-6cm)</li>\n<li>Train 3.925 (-3cm), Public LB 4.322 (-8cm)</li>\n<li>Train 3.808 (-12cm), Public LB 4.204 (-12cm)</li>\n</ul>\n<p>…so at least the improvement from one submission to the next has always been at least as large on the public LB as on the training set.  (I should really make myself a locally held-out validation set, but haven't got round to it yet.)</p>",
      "rawMarkdown": "Even the provided baseline solution has very different scores on the public LB.  On the training set, it scores 4.357m whereas the public LB score is 4.870m.  That's 51.3cm or 11.8% worse on public LB ☹️.\n\nMy own pairs so far (with diffs from the previous in brackets) are...\n\n- Train 4.357m (base), Public LB 4.870, (base)\n- Train 3.996 (-36cm), Public LB 4.463 (-41cm)\n- Train 3.957 (-4cm), Public LB 4.405 (-6cm)\n- Train 3.925 (-3cm), Public LB 4.322 (-8cm)\n- Train 3.808 (-12cm), Public LB 4.204 (-12cm)\n\n...so at least the improvement from one submission to the next has always been at least as large on the public LB as on the training set.  (I should really make myself a locally held-out validation set, but haven't got round to it yet.)",
      "votes": null
    },
    {
      "id": "1780976",
      "postDate": "05/08/2022 05:09:43",
      "content": "<p>Thank you for setting up the CV-vs-LB!<br>\nI'll share the results as much as possible too.</p>\n<ul>\n<li>exp005: CV 4.1078, LB 4.135</li>\n</ul>\n<p>In exp005, I used machine learning with GroupKFold(group=\"collectionName\") to split the validation data.<br>\nThe collectionName means \"The name of the \"grand\" parent folder\" in the previous competition.</p>",
      "rawMarkdown": "Thank you for setting up the CV-vs-LB!\nI'll share the results as much as possible too.\n\n* exp005: CV 4.1078, LB 4.135\n\nIn exp005, I used machine learning with GroupKFold(group=\"collectionName\") to split the validation data.\nThe collectionName means \"The name of the \"grand\" parent folder\" in the previous competition.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1780580,
      "author_name": "andrewrrose",
      "author_url": "",
      "post_date": "05/07/2022 15:37:04",
      "content": "<p>Even the provided baseline solution has very different scores on the public LB.  On the training set, it scores 4.357m whereas the public LB score is 4.870m.  That's 51.3cm or 11.8% worse on public LB ☹️.</p>\n<p>My own pairs so far (with diffs from the previous in brackets) are…</p>\n<ul>\n<li>Train 4.357m (base), Public LB 4.870, (base)</li>\n<li>Train 3.996 (-36cm), Public LB 4.463 (-41cm)</li>\n<li>Train 3.957 (-4cm), Public LB 4.405 (-6cm)</li>\n<li>Train 3.925 (-3cm), Public LB 4.322 (-8cm)</li>\n<li>Train 3.808 (-12cm), Public LB 4.204 (-12cm)</li>\n</ul>\n<p>…so at least the improvement from one submission to the next has always been at least as large on the public LB as on the training set.  (I should really make myself a locally held-out validation set, but haven't got round to it yet.)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1780976,
      "author_name": "columbia2131",
      "author_url": "",
      "post_date": "05/08/2022 05:09:43",
      "content": "<p>Thank you for setting up the CV-vs-LB!<br>\nI'll share the results as much as possible too.</p>\n<ul>\n<li>exp005: CV 4.1078, LB 4.135</li>\n</ul>\n<p>In exp005, I used machine learning with GroupKFold(group=\"collectionName\") to split the validation data.<br>\nThe collectionName means \"The name of the \"grand\" parent folder\" in the previous competition.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1780552": "# What is the relationship between your score on the training data and your score on LB?\n\nIn the last competition, I missed the solo gold by one step. \nI would like to do my best.\n\nMy results are as follows.\n- CV: 3.963, LB: 4.308\n- CV: 3.788, LB: 4.220\n\nI will add the score if I can update it.\n\n# Memo\nPrevious 'CV score-vs-LB score' Discussion.\nhttps://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/discussion/254574\n\nThe last competition shook up the ranking somewhat.\nhttps://www.kaggle.com/competitions/google-smartphone-decimeter-challenge/leaderboard",
    "1780580": "Even the provided baseline solution has very different scores on the public LB.  On the training set, it scores 4.357m whereas the public LB score is 4.870m.  That's 51.3cm or 11.8% worse on public LB ☹️.\n\nMy own pairs so far (with diffs from the previous in brackets) are...\n\n- Train 4.357m (base), Public LB 4.870, (base)\n- Train 3.996 (-36cm), Public LB 4.463 (-41cm)\n- Train 3.957 (-4cm), Public LB 4.405 (-6cm)\n- Train 3.925 (-3cm), Public LB 4.322 (-8cm)\n- Train 3.808 (-12cm), Public LB 4.204 (-12cm)\n\n...so at least the improvement from one submission to the next has always been at least as large on the public LB as on the training set.  (I should really make myself a locally held-out validation set, but haven't got round to it yet.)",
    "1780976": "Thank you for setting up the CV-vs-LB!\nI'll share the results as much as possible too.\n\n* exp005: CV 4.1078, LB 4.135\n\nIn exp005, I used machine learning with GroupKFold(group=\"collectionName\") to split the validation data.\nThe collectionName means \"The name of the \"grand\" parent folder\" in the previous competition."
  },
  "source": "meta"
}