{
  "id": 535681,
  "title": "Handle your leaderboard scores with care!",
  "url": "/competitions/child-mind-institute-problematic-internet-use/discussion/535681",
  "author_name": "Ravi Ramakrishnan",
  "post_date": "2024-09-23T15:27:36.708000",
  "votes": 21,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hello all,</p>\n<p>I am now confident from my baseline experiments that the leaderboard score can fluctuate a lot with very small parameter tweaks in the models we have created insofar. As an example, I placed a good public kernel <a href=\"https://www.kaggle.com/code/abdmental01/cmi-single-lgbm\" target=\"_blank\">here</a> as a base and altered a few parameters and was surprised at the leaderboard score fluctuations! </p>\n<p>The table below illustrates the LB score fluctuations with a small tweak in the kernel, with the same features and model otherwise- <br></p>\n<table>\n<thead>\n<tr>\n<th>Tweak</th>\n<th>LB score</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Adjusted learning rate</td>\n<td>0.455</td>\n</tr>\n<tr>\n<td>Adjusted learning rate <br> Adjusted reg_lambda</td>\n<td>0.456</td>\n</tr>\n<tr>\n<td>Adjusted learning rate <br> Adjusted n_estimators</td>\n<td>0.457</td>\n</tr>\n<tr>\n<td>Adjusted reg_lambda  <br> Adjusted num_leaves</td>\n<td>0.456</td>\n</tr>\n</tbody>\n</table>\n<p><br>I encourage you too to focus on CV score development going ahead and focus on improving feature quality to thwart this observation. Using better features often helps in capturing better signals in the data and this will be evinced with a CV-LB improvement!</p>\n<p>Happy learning and regards.</p>",
  "messages": [
    {
      "id": 2996523,
      "postDate": "2024-09-23T15:27:36.710Z",
      "content": "<p>Hello all,</p>\n<p>I am now confident from my baseline experiments that the leaderboard score can fluctuate a lot with very small parameter tweaks in the models we have created insofar. As an example, I placed a good public kernel <a href=\"https://www.kaggle.com/code/abdmental01/cmi-single-lgbm\" target=\"_blank\">here</a> as a base and altered a few parameters and was surprised at the leaderboard score fluctuations! </p>\n<p>The table below illustrates the LB score fluctuations with a small tweak in the kernel, with the same features and model otherwise- <br></p>\n<table>\n<thead>\n<tr>\n<th>Tweak</th>\n<th>LB score</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Adjusted learning rate</td>\n<td>0.455</td>\n</tr>\n<tr>\n<td>Adjusted learning rate <br> Adjusted reg_lambda</td>\n<td>0.456</td>\n</tr>\n<tr>\n<td>Adjusted learning rate <br> Adjusted n_estimators</td>\n<td>0.457</td>\n</tr>\n<tr>\n<td>Adjusted reg_lambda  <br> Adjusted num_leaves</td>\n<td>0.456</td>\n</tr>\n</tbody>\n</table>\n<p><br>I encourage you too to focus on CV score development going ahead and focus on improving feature quality to thwart this observation. Using better features often helps in capturing better signals in the data and this will be evinced with a CV-LB improvement!</p>\n<p>Happy learning and regards.</p>",
      "rawMarkdown": "Hello all,\n\nI am now confident from my baseline experiments that the leaderboard score can fluctuate a lot with very small parameter tweaks in the models we have created insofar. As an example, I placed a good public kernel [here](https://www.kaggle.com/code/abdmental01/cmi-single-lgbm) as a base and altered a few parameters and was surprised at the leaderboard score fluctuations! \n\nThe table below illustrates the LB score fluctuations with a small tweak in the kernel, with the same features and model otherwise- <br>\n\n| Tweak | LB score |\n| --- | --- |\n| Adjusted learning rate                                                 | 0.455 |\n| Adjusted learning rate <br> Adjusted reg_lambda    | 0.456 |\n| Adjusted learning rate <br> Adjusted n_estimators  | 0.457 |\n| Adjusted reg_lambda  <br> Adjusted num_leaves    | 0.456  |\n\n<br>I encourage you too to focus on CV score development going ahead and focus on improving feature quality to thwart this observation. Using better features often helps in capturing better signals in the data and this will be evinced with a CV-LB improvement!\n\nHappy learning and regards.",
      "votes": 21
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2996523": "Hello all,\n\nI am now confident from my baseline experiments that the leaderboard score can fluctuate a lot with very small parameter tweaks in the models we have created insofar. As an example, I placed a good public kernel [here](https://www.kaggle.com/code/abdmental01/cmi-single-lgbm) as a base and altered a few parameters and was surprised at the leaderboard score fluctuations! \n\nThe table below illustrates the LB score fluctuations with a small tweak in the kernel, with the same features and model otherwise- <br>\n\n| Tweak | LB score |\n| --- | --- |\n| Adjusted learning rate                                                 | 0.455 |\n| Adjusted learning rate <br> Adjusted reg_lambda    | 0.456 |\n| Adjusted learning rate <br> Adjusted n_estimators  | 0.457 |\n| Adjusted reg_lambda  <br> Adjusted num_leaves    | 0.456  |\n\n<br>I encourage you too to focus on CV score development going ahead and focus on improving feature quality to thwart this observation. Using better features often helps in capturing better signals in the data and this will be evinced with a CV-LB improvement!\n\nHappy learning and regards."
  }
}