{
  "id": 379082,
  "title": "CV vs public LB",
  "url": "/competitions/nfl-player-contact-detection/discussion/379082",
  "author_name": "",
  "post_date": "2023-01-18T01:21:35.737601200Z",
  "votes": 30,
  "comment_count": 22,
  "views": 0,
  "content": "<p>Our team has lower LB than CV. How about you?<br>\nCross-validation is a 5-fold GroupKFold by \"game_key\".</p>\n<p>(Edit: There was a bug in the LB 0.773 sub, so the scores in the table have been corrected.)</p>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.76933</td>\n<td>0.768</td>\n</tr>\n<tr>\n<td>0.77354</td>\n<td>0.771</td>\n</tr>\n<tr>\n<td></td>\n<td></td>\n</tr>\n<tr>\n<td>0.77949</td>\n<td>0.776</td>\n</tr>\n</tbody>\n</table>",
  "messages": [
    {
      "id": "2104697",
      "postDate": "01/18/2023 01:21:35",
      "content": "<p>Our team has lower LB than CV. How about you?<br>\nCross-validation is a 5-fold GroupKFold by \"game_key\".</p>\n<p>(Edit: There was a bug in the LB 0.773 sub, so the scores in the table have been corrected.)</p>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.76933</td>\n<td>0.768</td>\n</tr>\n<tr>\n<td>0.77354</td>\n<td>0.771</td>\n</tr>\n<tr>\n<td></td>\n<td></td>\n</tr>\n<tr>\n<td>0.77949</td>\n<td>0.776</td>\n</tr>\n</tbody>\n</table>",
      "rawMarkdown": "Our team has lower LB than CV. How about you?\nCross-validation is a 5-fold GroupKFold by \"game_key\".\n\n(Edit: There was a bug in the LB 0.773 sub, so the scores in the table have been corrected.)\n\n| CV  | LB |\n| --- | --- |\n| 0.76933  | 0.768 |\n| 0.77354  | 0.771 |\n| ~~0.77794  ~~| ~~0.773 ~~|\n| 0.77949| 0.776 |",
      "votes": null
    },
    {
      "id": "2104758",
      "postDate": "01/18/2023 02:39:55",
      "content": "<p>I have the opposite, my CV is lower than my LB.</p>",
      "rawMarkdown": "I have the opposite, my CV is lower than my LB.",
      "votes": null
    },
    {
      "id": "2106139",
      "postDate": "01/19/2023 01:50:29",
      "content": "<p>My CV vs LB is as follows.</p>\n<p>1 fold CV based on game_key. CV and LB are highly correlated.</p>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.702</td>\n<td>0.679</td>\n</tr>\n<tr>\n<td>0.704</td>\n<td>0.683</td>\n</tr>\n<tr>\n<td>0.709</td>\n<td>0.692</td>\n</tr>\n</tbody>\n</table>",
      "rawMarkdown": "My CV vs LB is as follows.\n\n1 fold CV based on game_key. CV and LB are highly correlated.\n\n| CV | LB |\n| --- | --- |\n| 0.702 | 0.679 |\n| 0.704 | 0.683 |\n| 0.709 | 0.692 |",
      "votes": null
    },
    {
      "id": "2106419",
      "postDate": "01/19/2023 06:16:46",
      "content": "<p>What are your scoring criteria?</p>",
      "rawMarkdown": "What are your scoring criteria?",
      "votes": null
    },
    {
      "id": "2106676",
      "postDate": "01/19/2023 09:46:39",
      "content": "<p>It is the Matthews Correlation Coefficient mentioned on <a href=\"https://www.kaggle.com/competitions/nfl-player-contact-detection/overview/evaluation\" target=\"_blank\">the Evaluation page</a>.</p>",
      "rawMarkdown": "It is the Matthews Correlation Coefficient mentioned on [the Evaluation page](https://www.kaggle.com/competitions/nfl-player-contact-detection/overview/evaluation).",
      "votes": null
    },
    {
      "id": "2106879",
      "postDate": "01/19/2023 12:09:26",
      "content": "<p>GroupKFold(5), group by game_key,  only fold-0 model.</p>\n<p>CV: 0.785 / LB: 0.764</p>",
      "rawMarkdown": "GroupKFold(5), group by game_key,  only fold-0 model.\n\nCV: 0.785 / LB: 0.764",
      "votes": null
    },
    {
      "id": "2107494",
      "postDate": "01/19/2023 20:19:17",
      "content": "<p>4-fold single model<br>\nCV:0.741, LB:0.732</p>",
      "rawMarkdown": "4-fold single model\nCV:0.741, LB:0.732",
      "votes": null
    },
    {
      "id": "2109689",
      "postDate": "01/21/2023 15:28:06",
      "content": "<p>3-fold single model</p>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.783</td>\n<td>0.771</td>\n</tr>\n<tr>\n<td>0.786</td>\n<td>0.774</td>\n</tr>\n<tr>\n<td>0.789</td>\n<td>0.779</td>\n</tr>\n<tr>\n<td>0.791</td>\n<td>0.780</td>\n</tr>\n<tr>\n<td>0.790</td>\n<td>0.776</td>\n</tr>\n<tr>\n<td>0.790</td>\n<td>0.782</td>\n</tr>\n</tbody>\n</table>",
      "rawMarkdown": "3-fold single model\n\n|CV|LB|\n|--|--|\n|0.783|0.771|\n|0.786|0.774|\n|0.789|0.779|\n|0.791|0.780|\n|0.790|0.776|\n|0.790|0.782|",
      "votes": null
    },
    {
      "id": "2110091",
      "postDate": "01/22/2023 00:06:41",
      "content": "<p>group k-fold, fold-0<br>\nCV:0.7212  LB:0.693<br>\nCV:0.7281  LB:0.704</p>",
      "rawMarkdown": "group k-fold, fold-0\nCV:0.7212  LB:0.693\nCV:0.7281  LB:0.704",
      "votes": null
    },
    {
      "id": "2110599",
      "postDate": "01/22/2023 09:52:52",
      "content": "<p>Use train or test data to score?</p>",
      "rawMarkdown": "Use train or test data to score?",
      "votes": null
    },
    {
      "id": "2110602",
      "postDate": "01/22/2023 09:57:44",
      "content": "<p>\"CV\" is the score from cross-validation on the training data and \"LB\" is the score on the (public) test data displayed on the leaderboard.</p>",
      "rawMarkdown": "\"CV\" is the score from cross-validation on the training data and \"LB\" is the score on the (public) test data displayed on the leaderboard.",
      "votes": null
    },
    {
      "id": "2110618",
      "postDate": "01/22/2023 10:12:05",
      "content": "<p>Thanks. I got it</p>",
      "rawMarkdown": "Thanks. I got it",
      "votes": null
    },
    {
      "id": "2112285",
      "postDate": "01/23/2023 14:12:40",
      "content": "<p>Very reliable CV approach. Good job!</p>",
      "rawMarkdown": "Very reliable CV approach. Good job!",
      "votes": null
    },
    {
      "id": "2113087",
      "postDate": "01/24/2023 04:35:38",
      "content": "<p>stratified group kfold by game_key<br>\ncv:0.7493 / LB: 0.731</p>",
      "rawMarkdown": "stratified group kfold by game_key\ncv:0.7493 / LB: 0.731",
      "votes": null
    },
    {
      "id": "2123996",
      "postDate": "01/31/2023 19:35:16",
      "content": "<p>On a subset of 20% data (2 folds) cv/lb = 723/711</p>\n<p>All data (3 folds) cv/lb = 781/771, <br>\nupdate cv/lb = 805/793</p>",
      "rawMarkdown": "On a subset of 20% data (2 folds) cv/lb = 723/711\n\nAll data (3 folds) cv/lb = 781/771, \nupdate cv/lb = 805/793",
      "votes": null
    },
    {
      "id": "2124960",
      "postDate": "02/01/2023 10:18:48",
      "content": "<p>Group k-fold</p>\n<p>Fold-0 results</p>\n<p>CV: 0.717  --  LB: 0.693<br>\nCV: 0.708  --  LB: 0.684</p>",
      "rawMarkdown": "Group k-fold\n\nFold-0 results\n\nCV: 0.717  --  LB: 0.693\nCV: 0.708  --  LB: 0.684",
      "votes": null
    },
    {
      "id": "2130141",
      "postDate": "02/05/2023 07:36:29",
      "content": "<p>group kfold by \"game_key\"<br>\n・CV: 0.664, LB: 0.690 with xgb (table only)<br>\n・CV: 0.729, LB: 0.718 with NN (table + image)</p>\n<p>[updata]<br>\n・CV: 0.7559, LB: 0.737</p>",
      "rawMarkdown": "group kfold by \"game_key\"\n・CV: 0.664, LB: 0.690 with xgb (table only)\n・CV: 0.729, LB: 0.718 with NN (table + image)\n\n[updata]\n・CV: 0.7559, LB: 0.737",
      "votes": null
    },
    {
      "id": "2147824",
      "postDate": "02/16/2023 22:39:36",
      "content": "<p>Single K-folds XGBoost model<br>\nCV: 0.728, LB: 0.719</p>",
      "rawMarkdown": "Single K-folds XGBoost model\nCV: 0.728, LB: 0.719",
      "votes": null
    },
    {
      "id": "2152683",
      "postDate": "02/20/2023 23:48:28",
      "content": "<p>StratifiedGroupKFold by \"game_key\"<br>\n5Folds XGBoost Model (table only)</p>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.721</td>\n<td>0.717</td>\n</tr>\n<tr>\n<td>0.726</td>\n<td>0.720</td>\n</tr>\n</tbody>\n</table>",
      "rawMarkdown": "StratifiedGroupKFold by \"game_key\"\n5Folds XGBoost Model (table only)\n\n| CV | LB |\n| --- | --- |\n|  0.721| 0.717 |\n|  0.726| 0.720 |",
      "votes": null
    },
    {
      "id": "2154917",
      "postDate": "02/22/2023 10:24:36",
      "content": "<p>group kfold, fold-0 result<br>\nCV: 0.767 / LB:0.750</p>",
      "rawMarkdown": "group kfold, fold-0 result\nCV: 0.767 / LB:0.750",
      "votes": null
    },
    {
      "id": "2154924",
      "postDate": "02/22/2023 10:33:42",
      "content": "<p>Single K-folds XGBoost model<br>\nCV: 0.745, LB: 0.736</p>",
      "rawMarkdown": "Single K-folds XGBoost model\nCV: 0.745, LB: 0.736",
      "votes": null
    },
    {
      "id": "2155657",
      "postDate": "02/22/2023 19:43:00",
      "content": "<p>Thanks for explain</p>",
      "rawMarkdown": "Thanks for explain",
      "votes": null
    },
    {
      "id": "2155667",
      "postDate": "02/22/2023 19:59:25",
      "content": "<p>Got 682cv, just 610 LB </p>",
      "rawMarkdown": "Got 682cv, just 610 LB",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2104758,
      "author_name": "carloshuertas",
      "author_url": "",
      "post_date": "01/18/2023 02:39:55",
      "content": "<p>I have the opposite, my CV is lower than my LB.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2106139,
      "author_name": "nomorevotch",
      "author_url": "",
      "post_date": "01/19/2023 01:50:29",
      "content": "<p>My CV vs LB is as follows.</p>\n<p>1 fold CV based on game_key. CV and LB are highly correlated.</p>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.702</td>\n<td>0.679</td>\n</tr>\n<tr>\n<td>0.704</td>\n<td>0.683</td>\n</tr>\n<tr>\n<td>0.709</td>\n<td>0.692</td>\n</tr>\n</tbody>\n</table>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2106419,
      "author_name": "tanyong666",
      "author_url": "",
      "post_date": "01/19/2023 06:16:46",
      "content": "<p>What are your scoring criteria?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2106676,
          "author_name": "ryotayoshinobu",
          "author_url": "",
          "post_date": "01/19/2023 09:46:39",
          "content": "<p>It is the Matthews Correlation Coefficient mentioned on <a href=\"https://www.kaggle.com/competitions/nfl-player-contact-detection/overview/evaluation\" target=\"_blank\">the Evaluation page</a>.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2110599,
              "author_name": "tanyong666",
              "author_url": "",
              "post_date": "01/22/2023 09:52:52",
              "content": "<p>Use train or test data to score?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2110602,
                  "author_name": "ryotayoshinobu",
                  "author_url": "",
                  "post_date": "01/22/2023 09:57:44",
                  "content": "<p>\"CV\" is the score from cross-validation on the training data and \"LB\" is the score on the (public) test data displayed on the leaderboard.</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2110618,
                      "author_name": "tanyong666",
                      "author_url": "",
                      "post_date": "01/22/2023 10:12:05",
                      "content": "<p>Thanks. I got it</p>",
                      "votes": null,
                      "replies": []
                    },
                    {
                      "id": 2155657,
                      "author_name": "",
                      "author_url": "",
                      "post_date": "02/22/2023 19:43:00",
                      "content": "<p>Thanks for explain</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2106879,
      "author_name": "kurupical",
      "author_url": "",
      "post_date": "01/19/2023 12:09:26",
      "content": "<p>GroupKFold(5), group by game_key,  only fold-0 model.</p>\n<p>CV: 0.785 / LB: 0.764</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2107494,
      "author_name": "kyoshioka47",
      "author_url": "",
      "post_date": "01/19/2023 20:19:17",
      "content": "<p>4-fold single model<br>\nCV:0.741, LB:0.732</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2109689,
      "author_name": "yururoi",
      "author_url": "",
      "post_date": "01/21/2023 15:28:06",
      "content": "<p>3-fold single model</p>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.783</td>\n<td>0.771</td>\n</tr>\n<tr>\n<td>0.786</td>\n<td>0.774</td>\n</tr>\n<tr>\n<td>0.789</td>\n<td>0.779</td>\n</tr>\n<tr>\n<td>0.791</td>\n<td>0.780</td>\n</tr>\n<tr>\n<td>0.790</td>\n<td>0.776</td>\n</tr>\n<tr>\n<td>0.790</td>\n<td>0.782</td>\n</tr>\n</tbody>\n</table>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2110091,
      "author_name": "yoichi7yamakawa",
      "author_url": "",
      "post_date": "01/22/2023 00:06:41",
      "content": "<p>group k-fold, fold-0<br>\nCV:0.7212  LB:0.693<br>\nCV:0.7281  LB:0.704</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2112285,
      "author_name": "toriglori",
      "author_url": "",
      "post_date": "01/23/2023 14:12:40",
      "content": "<p>Very reliable CV approach. Good job!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2113087,
      "author_name": "fuumin621",
      "author_url": "",
      "post_date": "01/24/2023 04:35:38",
      "content": "<p>stratified group kfold by game_key<br>\ncv:0.7493 / LB: 0.731</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2123996,
      "author_name": "nvnnghia",
      "author_url": "",
      "post_date": "01/31/2023 19:35:16",
      "content": "<p>On a subset of 20% data (2 folds) cv/lb = 723/711</p>\n<p>All data (3 folds) cv/lb = 781/771, <br>\nupdate cv/lb = 805/793</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2124960,
      "author_name": "samir95",
      "author_url": "",
      "post_date": "02/01/2023 10:18:48",
      "content": "<p>Group k-fold</p>\n<p>Fold-0 results</p>\n<p>CV: 0.717  --  LB: 0.693<br>\nCV: 0.708  --  LB: 0.684</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2130141,
      "author_name": "nori0724",
      "author_url": "",
      "post_date": "02/05/2023 07:36:29",
      "content": "<p>group kfold by \"game_key\"<br>\n・CV: 0.664, LB: 0.690 with xgb (table only)<br>\n・CV: 0.729, LB: 0.718 with NN (table + image)</p>\n<p>[updata]<br>\n・CV: 0.7559, LB: 0.737</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2147824,
      "author_name": "dwchen",
      "author_url": "",
      "post_date": "02/16/2023 22:39:36",
      "content": "<p>Single K-folds XGBoost model<br>\nCV: 0.728, LB: 0.719</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2152683,
      "author_name": "yuto0712",
      "author_url": "",
      "post_date": "02/20/2023 23:48:28",
      "content": "<p>StratifiedGroupKFold by \"game_key\"<br>\n5Folds XGBoost Model (table only)</p>\n<table>\n<thead>\n<tr>\n<th>CV</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.721</td>\n<td>0.717</td>\n</tr>\n<tr>\n<td>0.726</td>\n<td>0.720</td>\n</tr>\n</tbody>\n</table>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2154917,
      "author_name": "koheiumeki",
      "author_url": "",
      "post_date": "02/22/2023 10:24:36",
      "content": "<p>group kfold, fold-0 result<br>\nCV: 0.767 / LB:0.750</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2154924,
      "author_name": "dwchen",
      "author_url": "",
      "post_date": "02/22/2023 10:33:42",
      "content": "<p>Single K-folds XGBoost model<br>\nCV: 0.745, LB: 0.736</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2155667,
      "author_name": "",
      "author_url": "",
      "post_date": "02/22/2023 19:59:25",
      "content": "<p>Got 682cv, just 610 LB </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2104697": "Our team has lower LB than CV. How about you?\nCross-validation is a 5-fold GroupKFold by \"game_key\".\n\n(Edit: There was a bug in the LB 0.773 sub, so the scores in the table have been corrected.)\n\n| CV  | LB |\n| --- | --- |\n| 0.76933  | 0.768 |\n| 0.77354  | 0.771 |\n| ~~0.77794  ~~| ~~0.773 ~~|\n| 0.77949| 0.776 |",
    "2104758": "I have the opposite, my CV is lower than my LB.",
    "2106139": "My CV vs LB is as follows.\n\n1 fold CV based on game_key. CV and LB are highly correlated.\n\n| CV | LB |\n| --- | --- |\n| 0.702 | 0.679 |\n| 0.704 | 0.683 |\n| 0.709 | 0.692 |",
    "2106419": "What are your scoring criteria?",
    "2106676": "It is the Matthews Correlation Coefficient mentioned on [the Evaluation page](https://www.kaggle.com/competitions/nfl-player-contact-detection/overview/evaluation).",
    "2106879": "GroupKFold(5), group by game_key,  only fold-0 model.\n\nCV: 0.785 / LB: 0.764",
    "2107494": "4-fold single model\nCV:0.741, LB:0.732",
    "2109689": "3-fold single model\n\n|CV|LB|\n|--|--|\n|0.783|0.771|\n|0.786|0.774|\n|0.789|0.779|\n|0.791|0.780|\n|0.790|0.776|\n|0.790|0.782|",
    "2110091": "group k-fold, fold-0\nCV:0.7212  LB:0.693\nCV:0.7281  LB:0.704",
    "2110599": "Use train or test data to score?",
    "2110602": "\"CV\" is the score from cross-validation on the training data and \"LB\" is the score on the (public) test data displayed on the leaderboard.",
    "2110618": "Thanks. I got it",
    "2112285": "Very reliable CV approach. Good job!",
    "2113087": "stratified group kfold by game_key\ncv:0.7493 / LB: 0.731",
    "2123996": "On a subset of 20% data (2 folds) cv/lb = 723/711\n\nAll data (3 folds) cv/lb = 781/771, \nupdate cv/lb = 805/793",
    "2124960": "Group k-fold\n\nFold-0 results\n\nCV: 0.717  --  LB: 0.693\nCV: 0.708  --  LB: 0.684",
    "2130141": "group kfold by \"game_key\"\n・CV: 0.664, LB: 0.690 with xgb (table only)\n・CV: 0.729, LB: 0.718 with NN (table + image)\n\n[updata]\n・CV: 0.7559, LB: 0.737",
    "2147824": "Single K-folds XGBoost model\nCV: 0.728, LB: 0.719",
    "2152683": "StratifiedGroupKFold by \"game_key\"\n5Folds XGBoost Model (table only)\n\n| CV | LB |\n| --- | --- |\n|  0.721| 0.717 |\n|  0.726| 0.720 |",
    "2154917": "group kfold, fold-0 result\nCV: 0.767 / LB:0.750",
    "2154924": "Single K-folds XGBoost model\nCV: 0.745, LB: 0.736",
    "2155657": "Thanks for explain",
    "2155667": "Got 682cv, just 610 LB"
  },
  "source": "meta"
}