{
  "id": 310365,
  "title": "▲CV vs LB ▼",
  "url": "/competitions/h-and-m-personalized-fashion-recommendations/discussion/310365",
  "author_name": "senkin13",
  "post_date": "2022-03-01T01:52:42.658000",
  "votes": 67,
  "comment_count": 24,
  "views": 0,
  "content": "<p>Hi everyone. Since there is no topic discussing CV and LB.I found CV is much better than LB,I'm a bit worried about my CV vs LB and would love to see how everyone else is doing so we might have some benchmarks.</p>\n<p>I created a rule base model with last week as validation dataset(2020-09-16 ~ 2020-09-22).CV is 0.0284 and LB is 0.0232</p>",
  "messages": [
    {
      "id": 1707978,
      "postDate": "2022-03-01T01:52:42.660Z",
      "content": "<p>Hi everyone. Since there is no topic discussing CV and LB.I found CV is much better than LB,I'm a bit worried about my CV vs LB and would love to see how everyone else is doing so we might have some benchmarks.</p>\n<p>I created a rule base model with last week as validation dataset(2020-09-16 ~ 2020-09-22).CV is 0.0284 and LB is 0.0232</p>",
      "rawMarkdown": "Hi everyone. Since there is no topic discussing CV and LB.I found CV is much better than LB,I'm a bit worried about my CV vs LB and would love to see how everyone else is doing so we might have some benchmarks.\n\nI created a rule base model with last week as validation dataset(2020-09-16 ~ 2020-09-22).CV is 0.0284 and LB is 0.0232",
      "votes": 66
    },
    {
      "id": 1712231,
      "postDate": "2022-03-04T17:34:43.867Z",
      "content": "<p>One fold validation: 0.0392 LB 0.0317</p>\n<p>The difference is higher than other competitors but it is stable.</p>\n<table>\n<thead>\n<tr>\n<th>Full validation</th>\n<th>Leaderboard</th>\n<th>Diff</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0,0256</td>\n<td>0,017</td>\n<td>0,0086</td>\n</tr>\n<tr>\n<td>0,0287</td>\n<td>0,022</td>\n<td>0,0067</td>\n</tr>\n<tr>\n<td>0,029</td>\n<td>0,023</td>\n<td>0,006</td>\n</tr>\n<tr>\n<td>0,0314</td>\n<td>0,025</td>\n<td>0,0064</td>\n</tr>\n<tr>\n<td>0,032</td>\n<td>0,026</td>\n<td>0,006</td>\n</tr>\n<tr>\n<td>0,0341</td>\n<td>0,028</td>\n<td>0,0061</td>\n</tr>\n<tr>\n<td>0,0351</td>\n<td>0,029</td>\n<td>0,0061</td>\n</tr>\n<tr>\n<td>0,0379</td>\n<td>0,03</td>\n<td>0,0079</td>\n</tr>\n<tr>\n<td>0,0373</td>\n<td>0,03</td>\n<td>0,0073</td>\n</tr>\n<tr>\n<td>0,0392</td>\n<td>0,0317</td>\n<td>0,0075</td>\n</tr>\n</tbody>\n</table>",
      "rawMarkdown": "One fold validation: 0.0392 LB 0.0317\n\nThe difference is higher than other competitors but it is stable.\n\n| Full validation | Leaderboard | Diff   |\n|-----------------|-------------|--------|\n| 0,0256          | 0,017       | 0,0086 |\n| 0,0287          | 0,022       | 0,0067 |\n| 0,029           | 0,023       | 0,006  |\n| 0,0314          | 0,025       | 0,0064 |\n| 0,032           | 0,026       | 0,006  |\n| 0,0341          | 0,028       | 0,0061 |\n| 0,0351          | 0,029       | 0,0061 |\n| 0,0379          | 0,03        | 0,0079 |\n| 0,0373          | 0,03        | 0,0073 |\n| 0,0392          | 0,0317      | 0,0075  |\n",
      "votes": 19,
      "replies": [
        {
          "id": 1746826,
          "postDate": "2022-04-06T06:09:33.677Z",
          "content": "<p>This post helps me a lot during my score climbing progress with your another amazing two-step modeling post. Thanks again Pawel! In my case, the difference is always higher than yours. And for my current status, CV: 0.0406, LB: 0.0323, the difference of which is 0.0083. </p>",
          "rawMarkdown": "This post helps me a lot during my score climbing progress with your another amazing two-step modeling post. Thanks again Pawel! In my case, the difference is always higher than yours. And for my current status, CV: 0.0406, LB: 0.0323, the difference of which is 0.0083. ",
          "votes": 1
        }
      ]
    },
    {
      "id": 1708053,
      "postDate": "2022-03-01T04:44:12.427Z",
      "content": "<p>My current model is a more elaborate version of my notebook <a href=\"https://www.kaggle.com/cdeotte/recommend-items-purchased-together-0-021\" target=\"_blank\">here</a>. I have been checking 3 folds of validation scores to prevent overfitting to the last week of train. I have noticed that some changes improve last week val score but do not improve the other 2 folds. (So i don't keep those changes).</p>\n<p>Local validation scores are <strong>CV 0.0274, 0.0259, and 0.0251</strong> on folds 0, 1, 2 where fold 0 validation is last week of train (and train on data before). Fold 1 is second to last week (and train on data before). Fold 2 is third to last week (and train on data before). My CV scheme is described <a href=\"https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308919\" target=\"_blank\">here</a>. LB score is <strong>LB 0.0239</strong> . </p>",
      "rawMarkdown": "My current model is a more elaborate version of my notebook [here][2]. I have been checking 3 folds of validation scores to prevent overfitting to the last week of train. I have noticed that some changes improve last week val score but do not improve the other 2 folds. (So i don't keep those changes).\n\nLocal validation scores are **CV 0.0274, 0.0259, and 0.0251** on folds 0, 1, 2 where fold 0 validation is last week of train (and train on data before). Fold 1 is second to last week (and train on data before). Fold 2 is third to last week (and train on data before). My CV scheme is described [here][1]. LB score is **LB 0.0239** . \n\n[1]: https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308919\n[2]: https://www.kaggle.com/cdeotte/recommend-items-purchased-together-0-021",
      "votes": 13,
      "replies": [
        {
          "id": 1708140,
          "postDate": "2022-03-01T06:23:24Z",
          "content": "<p>thanks,3 folds CV scheme is reasonbale</p>",
          "rawMarkdown": "thanks,3 folds CV scheme is reasonbale",
          "votes": 2
        },
        {
          "id": 1713283,
          "postDate": "2022-03-05T20:24:57.400Z",
          "content": "<p>UPDATE: <strong>CV 0.281, 0.269, 0.259</strong> for folds 0,1,2 and  <strong>LB 0.257</strong></p>\n<p>This was a nice surprise. The CV boosted +0.007, +0.010, +0.008 on folds 0,1,2, and LB boosted +0.018!</p>",
          "rawMarkdown": "UPDATE: **CV 0.281, 0.269, 0.259** for folds 0,1,2 and  **LB 0.257**\n\nThis was a nice surprise. The CV boosted +0.007, +0.010, +0.008 on folds 0,1,2, and LB boosted +0.018!",
          "votes": 2
        }
      ]
    },
    {
      "id": 1708453,
      "postDate": "2022-03-01T13:11:38.520Z",
      "content": "<p>I use 3 folds CV.</p>\n<p>CV<br>\n2020-09-16 - 2020-09-22 : 0.0304<br>\n2020-09-09 - 2020-09-15 : 0.0305<br>\n2020-09-02 - 2020-09-08 : 0.0293</p>\n<p>LB<br>\n0.0264</p>",
      "rawMarkdown": "I use 3 folds CV.\n\nCV\n2020-09-16 - 2020-09-22 : 0.0304\n2020-09-09 - 2020-09-15 : 0.0305\n2020-09-02 - 2020-09-08 : 0.0293\n\nLB\n0.0264",
      "votes": 11,
      "replies": [
        {
          "id": 1708542,
          "postDate": "2022-03-01T14:13:13.230Z",
          "content": "<p>seems stable, nice start dash!</p>",
          "rawMarkdown": "seems stable, nice start dash!"
        },
        {
          "id": 1708594,
          "postDate": "2022-03-01T14:54:32.720Z",
          "content": "<p>thank's for sharing, what about train, do you change the train period for each fold ? </p>",
          "rawMarkdown": "thank's for sharing, what about train, do you change the train period for each fold ? ",
          "votes": 1
        },
        {
          "id": 1708833,
          "postDate": "2022-03-01T18:26:49.537Z",
          "content": "<p><a href=\"https://www.kaggle.com/senkin13\" target=\"_blank\">@senkin13</a> <br>\nThank you! I'll keep on trying.</p>",
          "rawMarkdown": "@senkin13 \nThank you! I'll keep on trying."
        },
        {
          "id": 1708839,
          "postDate": "2022-03-01T18:29:22.363Z",
          "content": "<p><a href=\"https://www.kaggle.com/souamesannis\" target=\"_blank\">@souamesannis</a> <br>\nYes, I'm using data from before each evaluation period as training data.</p>",
          "rawMarkdown": "@souamesannis \nYes, I'm using data from before each evaluation period as training data.",
          "votes": 3
        }
      ]
    },
    {
      "id": 1710535,
      "postDate": "2022-03-03T06:13:21.760Z",
      "content": "<p>validation<br>\n2020-09-16 - 2020-09-22 : 0.02850</p>\n<p>LB<br>\n0.0235</p>",
      "rawMarkdown": "validation\n2020-09-16 - 2020-09-22 : 0.02850\n\nLB\n0.0235",
      "votes": 7,
      "replies": [
        {
          "id": 1715128,
          "postDate": "2022-03-07T16:51:47.960Z",
          "content": "<p>validation<br>\n2020-09-16 - 2020-09-22 : 0.03071</p>\n<p>LB<br>\n0.0256</p>",
          "rawMarkdown": "validation\n2020-09-16 - 2020-09-22 : 0.03071\n\nLB\n0.0256",
          "votes": 4
        }
      ]
    },
    {
      "id": 1708028,
      "postDate": "2022-03-01T03:48:21.187Z",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/senkin13\" target=\"_blank\">@senkin13</a> <br>\nI kinda agree with you on <code>CV is much better than LB</code> </p>\n<p>I compare several public notebook CV and LB <br>\nand the result say the same with yours </p>\n<p>Here is the notebook <br>\n<a href=\"https://www.kaggle.com/hervind/h-m-calculate-map-12-from-public-notebooks\" target=\"_blank\">https://www.kaggle.com/hervind/h-m-calculate-map-12-from-public-notebooks</a></p>",
      "rawMarkdown": "Hi @senkin13 \nI kinda agree with you on `CV is much better than LB` \n\nI compare several public notebook CV and LB \nand the result say the same with yours \n\nHere is the notebook \nhttps://www.kaggle.com/hervind/h-m-calculate-map-12-from-public-notebooks",
      "votes": 8,
      "replies": [
        {
          "id": 1708046,
          "postDate": "2022-03-01T04:29:50.377Z",
          "content": "<p>Excellent work Hervind. Computing CV scores is very important.</p>",
          "rawMarkdown": "Excellent work Hervind. Computing CV scores is very important.",
          "votes": 5
        },
        {
          "id": 1708141,
          "postDate": "2022-03-01T06:24:29.250Z",
          "content": "<p>nice work,upvoted</p>",
          "rawMarkdown": "nice work,upvoted",
          "votes": 4
        }
      ]
    },
    {
      "id": 1775490,
      "postDate": "2022-05-03T05:04:54.497Z",
      "content": "<p>CV: 0.0385<br>\nLB: 0.0326</p>\n<p>Like other competitors, there is a big difference between CV and LB.</p>",
      "rawMarkdown": "CV: 0.0385\nLB: 0.0326\n\nLike other competitors, there is a big difference between CV and LB.\n",
      "votes": 1,
      "replies": [
        {
          "id": 1777058,
          "postDate": "2022-05-04T11:20:18.570Z",
          "content": "<p>better than me 👍<br>\nCV: 0.03898<br>\nLB: 0.0320</p>",
          "rawMarkdown": "better than me 👍\nCV: 0.03898\nLB: 0.0320",
          "votes": 1
        },
        {
          "id": 1777422,
          "postDate": "2022-05-04T15:33:24.100Z",
          "content": "<p>cv 0.0412<br>\nlb 0.0347<br>\nThe difference between cv and lb is similar to yours</p>",
          "rawMarkdown": "cv 0.0412\nlb 0.0347\nThe difference between cv and lb is similar to yours",
          "votes": 2
        },
        {
          "id": 1777450,
          "postDate": "2022-05-04T15:47:02.633Z",
          "content": "<p>Wow, I wish I could get my CV scores so high. I'm at:<br>\ncv 0.0353<br>\nlb 0.0323</p>",
          "rawMarkdown": "Wow, I wish I could get my CV scores so high. I'm at:\ncv 0.0353\nlb 0.0323"
        },
        {
          "id": 1777928,
          "postDate": "2022-05-04T23:18:14.937Z",
          "content": "<p>In my case, the difference is smaller than yours.<br>\nI'm afraid of shake down.<br>\nCV : 0.0330<br>\nLB  : 0.0313</p>",
          "rawMarkdown": "In my case, the difference is smaller than yours.\nI'm afraid of shake down.\nCV : 0.0330\nLB  : 0.0313"
        },
        {
          "id": 1778883,
          "postDate": "2022-05-05T20:08:48.917Z",
          "content": "<p>CV LB gap comes from the difference of how to enumerate the candidates between train and test.<br>\nWe don't need to worry about it if the conditions are same</p>",
          "rawMarkdown": "CV LB gap comes from the difference of how to enumerate the candidates between train and test.\nWe don't need to worry about it if the conditions are same",
          "votes": 2
        },
        {
          "id": 1778913,
          "postDate": "2022-05-05T21:30:36.910Z",
          "content": "<p>Thanks. I'm relieved.<br>\nI look forward to hearing your candidate strategies.</p>",
          "rawMarkdown": "Thanks. I'm relieved.\nI look forward to hearing your candidate strategies."
        }
      ]
    },
    {
      "id": 1729863,
      "postDate": "2022-03-20T16:15:49.230Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true,
      "replies": [
        {
          "id": 1730028,
          "postDate": "2022-03-20T20:34:49.877Z",
          "content": "<p>CV: cross validation (the score you achieve locally in your experiments)    <br>\nLB: Leaderboard (the score that appears when you submit to Kaggle)</p>",
          "rawMarkdown": "CV: cross validation (the score you achieve locally in your experiments)    \nLB: Leaderboard (the score that appears when you submit to Kaggle)",
          "votes": 2
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1712231,
      "author_name": "Paweł Jankiewicz",
      "author_url": "",
      "post_date": "2022-03-04T17:34:43.867000",
      "content": "<p>One fold validation: 0.0392 LB 0.0317</p>\n<p>The difference is higher than other competitors but it is stable.</p>\n<table>\n<thead>\n<tr>\n<th>Full validation</th>\n<th>Leaderboard</th>\n<th>Diff</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0,0256</td>\n<td>0,017</td>\n<td>0,0086</td>\n</tr>\n<tr>\n<td>0,0287</td>\n<td>0,022</td>\n<td>0,0067</td>\n</tr>\n<tr>\n<td>0,029</td>\n<td>0,023</td>\n<td>0,006</td>\n</tr>\n<tr>\n<td>0,0314</td>\n<td>0,025</td>\n<td>0,0064</td>\n</tr>\n<tr>\n<td>0,032</td>\n<td>0,026</td>\n<td>0,006</td>\n</tr>\n<tr>\n<td>0,0341</td>\n<td>0,028</td>\n<td>0,0061</td>\n</tr>\n<tr>\n<td>0,0351</td>\n<td>0,029</td>\n<td>0,0061</td>\n</tr>\n<tr>\n<td>0,0379</td>\n<td>0,03</td>\n<td>0,0079</td>\n</tr>\n<tr>\n<td>0,0373</td>\n<td>0,03</td>\n<td>0,0073</td>\n</tr>\n<tr>\n<td>0,0392</td>\n<td>0,0317</td>\n<td>0,0075</td>\n</tr>\n</tbody>\n</table>",
      "votes": 19,
      "replies": [
        {
          "id": 1746826,
          "author_name": "HAO",
          "author_url": "",
          "post_date": "2022-04-06T06:09:33.677000",
          "content": "<p>This post helps me a lot during my score climbing progress with your another amazing two-step modeling post. Thanks again Pawel! In my case, the difference is always higher than yours. And for my current status, CV: 0.0406, LB: 0.0323, the difference of which is 0.0083. </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1708053,
      "author_name": "Chris Deotte",
      "author_url": "",
      "post_date": "2022-03-01T04:44:12.427000",
      "content": "<p>My current model is a more elaborate version of my notebook <a href=\"https://www.kaggle.com/cdeotte/recommend-items-purchased-together-0-021\" target=\"_blank\">here</a>. I have been checking 3 folds of validation scores to prevent overfitting to the last week of train. I have noticed that some changes improve last week val score but do not improve the other 2 folds. (So i don't keep those changes).</p>\n<p>Local validation scores are <strong>CV 0.0274, 0.0259, and 0.0251</strong> on folds 0, 1, 2 where fold 0 validation is last week of train (and train on data before). Fold 1 is second to last week (and train on data before). Fold 2 is third to last week (and train on data before). My CV scheme is described <a href=\"https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308919\" target=\"_blank\">here</a>. LB score is <strong>LB 0.0239</strong> . </p>",
      "votes": 13,
      "replies": [
        {
          "id": 1708140,
          "author_name": "senkin13",
          "author_url": "",
          "post_date": "2022-03-01T06:23:24",
          "content": "<p>thanks,3 folds CV scheme is reasonbale</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1713283,
          "author_name": "Chris Deotte",
          "author_url": "",
          "post_date": "2022-03-05T20:24:57.400000",
          "content": "<p>UPDATE: <strong>CV 0.281, 0.269, 0.259</strong> for folds 0,1,2 and  <strong>LB 0.257</strong></p>\n<p>This was a nice surprise. The CV boosted +0.007, +0.010, +0.008 on folds 0,1,2, and LB boosted +0.018!</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 1708453,
      "author_name": "T88",
      "author_url": "",
      "post_date": "2022-03-01T13:11:38.520000",
      "content": "<p>I use 3 folds CV.</p>\n<p>CV<br>\n2020-09-16 - 2020-09-22 : 0.0304<br>\n2020-09-09 - 2020-09-15 : 0.0305<br>\n2020-09-02 - 2020-09-08 : 0.0293</p>\n<p>LB<br>\n0.0264</p>",
      "votes": 11,
      "replies": [
        {
          "id": 1708542,
          "author_name": "senkin13",
          "author_url": "",
          "post_date": "2022-03-01T14:13:13.230000",
          "content": "<p>seems stable, nice start dash!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1708594,
          "author_name": "Mohamed Annis SOUAMES",
          "author_url": "",
          "post_date": "2022-03-01T14:54:32.720000",
          "content": "<p>thank's for sharing, what about train, do you change the train period for each fold ? </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1708833,
          "author_name": "T88",
          "author_url": "",
          "post_date": "2022-03-01T18:26:49.537000",
          "content": "<p><a href=\"https://www.kaggle.com/senkin13\" target=\"_blank\">@senkin13</a> <br>\nThank you! I'll keep on trying.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1708839,
          "author_name": "T88",
          "author_url": "",
          "post_date": "2022-03-01T18:29:22.363000",
          "content": "<p><a href=\"https://www.kaggle.com/souamesannis\" target=\"_blank\">@souamesannis</a> <br>\nYes, I'm using data from before each evaluation period as training data.</p>",
          "votes": 3,
          "replies": []
        }
      ]
    },
    {
      "id": 1710535,
      "author_name": "ONODERA",
      "author_url": "",
      "post_date": "2022-03-03T06:13:21.760000",
      "content": "<p>validation<br>\n2020-09-16 - 2020-09-22 : 0.02850</p>\n<p>LB<br>\n0.0235</p>",
      "votes": 7,
      "replies": [
        {
          "id": 1715128,
          "author_name": "ONODERA",
          "author_url": "",
          "post_date": "2022-03-07T16:51:47.960000",
          "content": "<p>validation<br>\n2020-09-16 - 2020-09-22 : 0.03071</p>\n<p>LB<br>\n0.0256</p>",
          "votes": 4,
          "replies": []
        }
      ]
    },
    {
      "id": 1708028,
      "author_name": "Hervind Philipe",
      "author_url": "",
      "post_date": "2022-03-01T03:48:21.187000",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/senkin13\" target=\"_blank\">@senkin13</a> <br>\nI kinda agree with you on <code>CV is much better than LB</code> </p>\n<p>I compare several public notebook CV and LB <br>\nand the result say the same with yours </p>\n<p>Here is the notebook <br>\n<a href=\"https://www.kaggle.com/hervind/h-m-calculate-map-12-from-public-notebooks\" target=\"_blank\">https://www.kaggle.com/hervind/h-m-calculate-map-12-from-public-notebooks</a></p>",
      "votes": 8,
      "replies": [
        {
          "id": 1708046,
          "author_name": "Chris Deotte",
          "author_url": "",
          "post_date": "2022-03-01T04:29:50.377000",
          "content": "<p>Excellent work Hervind. Computing CV scores is very important.</p>",
          "votes": 5,
          "replies": []
        },
        {
          "id": 1708141,
          "author_name": "senkin13",
          "author_url": "",
          "post_date": "2022-03-01T06:24:29.250000",
          "content": "<p>nice work,upvoted</p>",
          "votes": 4,
          "replies": []
        }
      ]
    },
    {
      "id": 1775490,
      "author_name": "Apolo",
      "author_url": "",
      "post_date": "2022-05-03T05:04:54.497000",
      "content": "<p>CV: 0.0385<br>\nLB: 0.0326</p>\n<p>Like other competitors, there is a big difference between CV and LB.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1777058,
          "author_name": "ONODERA",
          "author_url": "",
          "post_date": "2022-05-04T11:20:18.570000",
          "content": "<p>better than me 👍<br>\nCV: 0.03898<br>\nLB: 0.0320</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1777422,
          "author_name": "sirius",
          "author_url": "",
          "post_date": "2022-05-04T15:33:24.100000",
          "content": "<p>cv 0.0412<br>\nlb 0.0347<br>\nThe difference between cv and lb is similar to yours</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1777450,
          "author_name": "Silogram",
          "author_url": "",
          "post_date": "2022-05-04T15:47:02.633000",
          "content": "<p>Wow, I wish I could get my CV scores so high. I'm at:<br>\ncv 0.0353<br>\nlb 0.0323</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1777928,
          "author_name": "dehokanta",
          "author_url": "",
          "post_date": "2022-05-04T23:18:14.937000",
          "content": "<p>In my case, the difference is smaller than yours.<br>\nI'm afraid of shake down.<br>\nCV : 0.0330<br>\nLB  : 0.0313</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1778883,
          "author_name": "ONODERA",
          "author_url": "",
          "post_date": "2022-05-05T20:08:48.917000",
          "content": "<p>CV LB gap comes from the difference of how to enumerate the candidates between train and test.<br>\nWe don't need to worry about it if the conditions are same</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1778913,
          "author_name": "dehokanta",
          "author_url": "",
          "post_date": "2022-05-05T21:30:36.910000",
          "content": "<p>Thanks. I'm relieved.<br>\nI look forward to hearing your candidate strategies.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1729863,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-03-20T16:15:49.230000",
      "content": "",
      "votes": 1,
      "replies": [
        {
          "id": 1730028,
          "author_name": "Igor Kuivjogi Fernandes",
          "author_url": "",
          "post_date": "2022-03-20T20:34:49.877000",
          "content": "<p>CV: cross validation (the score you achieve locally in your experiments)    <br>\nLB: Leaderboard (the score that appears when you submit to Kaggle)</p>",
          "votes": 2,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1707978": "Hi everyone. Since there is no topic discussing CV and LB.I found CV is much better than LB,I'm a bit worried about my CV vs LB and would love to see how everyone else is doing so we might have some benchmarks.\n\nI created a rule base model with last week as validation dataset(2020-09-16 ~ 2020-09-22).CV is 0.0284 and LB is 0.0232",
    "1712231": "One fold validation: 0.0392 LB 0.0317\n\nThe difference is higher than other competitors but it is stable.\n\n| Full validation | Leaderboard | Diff   |\n|-----------------|-------------|--------|\n| 0,0256          | 0,017       | 0,0086 |\n| 0,0287          | 0,022       | 0,0067 |\n| 0,029           | 0,023       | 0,006  |\n| 0,0314          | 0,025       | 0,0064 |\n| 0,032           | 0,026       | 0,006  |\n| 0,0341          | 0,028       | 0,0061 |\n| 0,0351          | 0,029       | 0,0061 |\n| 0,0379          | 0,03        | 0,0079 |\n| 0,0373          | 0,03        | 0,0073 |\n| 0,0392          | 0,0317      | 0,0075  |\n",
    "1708053": "My current model is a more elaborate version of my notebook [here][2]. I have been checking 3 folds of validation scores to prevent overfitting to the last week of train. I have noticed that some changes improve last week val score but do not improve the other 2 folds. (So i don't keep those changes).\n\nLocal validation scores are **CV 0.0274, 0.0259, and 0.0251** on folds 0, 1, 2 where fold 0 validation is last week of train (and train on data before). Fold 1 is second to last week (and train on data before). Fold 2 is third to last week (and train on data before). My CV scheme is described [here][1]. LB score is **LB 0.0239** . \n\n[1]: https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/308919\n[2]: https://www.kaggle.com/cdeotte/recommend-items-purchased-together-0-021",
    "1708453": "I use 3 folds CV.\n\nCV\n2020-09-16 - 2020-09-22 : 0.0304\n2020-09-09 - 2020-09-15 : 0.0305\n2020-09-02 - 2020-09-08 : 0.0293\n\nLB\n0.0264",
    "1710535": "validation\n2020-09-16 - 2020-09-22 : 0.02850\n\nLB\n0.0235",
    "1708028": "Hi @senkin13 \nI kinda agree with you on `CV is much better than LB` \n\nI compare several public notebook CV and LB \nand the result say the same with yours \n\nHere is the notebook \nhttps://www.kaggle.com/hervind/h-m-calculate-map-12-from-public-notebooks",
    "1775490": "CV: 0.0385\nLB: 0.0326\n\nLike other competitors, there is a big difference between CV and LB.\n",
    "1729863": ""
  }
}