{
  "id": 555588,
  "title": "how was the public model trained? same params worse result?",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/555588",
  "author_name": "",
  "post_date": "2025-01-08T09:01:28.277335100Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I was using the almost the same params as the public model, but getting worse result. what could be the reason?<br>\n(getting 0.005)</p>\n<p>public model:<br>\n<a href=\"https://www.kaggle.com/code/hideyukizushi/js-nn-xgb-ridge-pub-mytrain-weightblend-lb-0-0079\" target=\"_blank\">https://www.kaggle.com/code/hideyukizushi/js-nn-xgb-ridge-pub-mytrain-weightblend-lb-0-0079</a></p>\n<p>I am using a 5 fold CV too:</p>\n<pre><code>kf = KFold(n_splits=, shuffle=, random_state=)\n fold, (train_index, valid_index)  (kf.split(X)):\n    ()\n    X_train, X_valid = X[train_index].to_numpy(), X[valid_index].to_numpy()\n    y_train, y_valid = y[train_index].to_numpy(), y[valid_index].to_numpy()\n    w_train, w_valid = w[train_index].to_numpy(), w[valid_index].to_numpy()\n</code></pre>\n<p>params like:</p>\n<pre><code>    XGB_Params = {\n        :          ,\n        :        ,\n        :          ,\n        : ,\n        :    ,\n        :           ,\n        :     ,\n        :              ,\n        :             ,\n        :      ,\n        :               ,\n        :              ,\n        :         seed,\n        :        ,\n        :             ,\n        :          r2_xgb,\n        :               \n    }\n</code></pre>\n<p>and adding online retrain (train a totally  new xgb model online), getting 0.0034…..<br>\nusing lag 1 responder as target, lag 2 and features as training data…. </p>\n<p>Been banging my head trying but can't figure out how to improve on these… Any suggestions would be much appriciated….</p>",
  "messages": [
    {
      "id": "3091278",
      "postDate": "01/08/2025 09:01:28",
      "content": "<p>I was using the almost the same params as the public model, but getting worse result. what could be the reason?<br>\n(getting 0.005)</p>\n<p>public model:<br>\n<a href=\"https://www.kaggle.com/code/hideyukizushi/js-nn-xgb-ridge-pub-mytrain-weightblend-lb-0-0079\" target=\"_blank\">https://www.kaggle.com/code/hideyukizushi/js-nn-xgb-ridge-pub-mytrain-weightblend-lb-0-0079</a></p>\n<p>I am using a 5 fold CV too:</p>\n<pre><code>kf = KFold(n_splits=, shuffle=, random_state=)\n fold, (train_index, valid_index)  (kf.split(X)):\n    ()\n    X_train, X_valid = X[train_index].to_numpy(), X[valid_index].to_numpy()\n    y_train, y_valid = y[train_index].to_numpy(), y[valid_index].to_numpy()\n    w_train, w_valid = w[train_index].to_numpy(), w[valid_index].to_numpy()\n</code></pre>\n<p>params like:</p>\n<pre><code>    XGB_Params = {\n        :          ,\n        :        ,\n        :          ,\n        : ,\n        :    ,\n        :           ,\n        :     ,\n        :              ,\n        :             ,\n        :      ,\n        :               ,\n        :              ,\n        :         seed,\n        :        ,\n        :             ,\n        :          r2_xgb,\n        :               \n    }\n</code></pre>\n<p>and adding online retrain (train a totally  new xgb model online), getting 0.0034…..<br>\nusing lag 1 responder as target, lag 2 and features as training data…. </p>\n<p>Been banging my head trying but can't figure out how to improve on these… Any suggestions would be much appriciated….</p>",
      "rawMarkdown": "I was using the almost the same params as the public model, but getting worse result. what could be the reason?\n(getting 0.005)\n\npublic model:\nhttps://www.kaggle.com/code/hideyukizushi/js-nn-xgb-ridge-pub-mytrain-weightblend-lb-0-0079\n\n\nI am using a 5 fold CV too:\n```python\nkf = KFold(n_splits=5, shuffle=True, random_state=42)\nfor fold, (train_index, valid_index) in enumerate(kf.split(X)):\n    print(f\"Start Training Fold {fold + 1}\")\n    X_train, X_valid = X[train_index].to_numpy(), X[valid_index].to_numpy()\n    y_train, y_valid = y[train_index].to_numpy(), y[valid_index].to_numpy()\n    w_train, w_valid = w[train_index].to_numpy(), w[valid_index].to_numpy()\n```\n\n\nparams like:\n```python\n    XGB_Params = {\n        'booster':          'gbtree',\n        'learning_rate':        0.05,\n        'n_estimators':          200,\n        'early_stopping_rounds': None,\n        'enable_categorical':    False,\n        'grow_policy':           None,\n        'max_cat_to_onehot':     None,\n        'max_depth':              6,\n        'subsample':             0.6,\n        'colsample_bytree':      0.8,\n        'reg_alpha':               2,\n        'reg_lambda':              8,\n        'random_state':         seed,\n        'tree_method':        'hist',\n        'device':             'cuda',\n        'eval_metric':          r2_xgb,\n        'verbosity':               1\n    }\n```\n\nand adding online retrain (train a totally  new xgb model online), getting 0.0034.....\nusing lag 1 responder as target, lag 2 and features as training data.... \n\nBeen banging my head trying but can't figure out how to improve on these... Any suggestions would be much appriciated....",
      "votes": null
    },
    {
      "id": "3091286",
      "postDate": "01/08/2025 09:15:16",
      "content": "<p>it might be related to the size of the data you train your model on (all data vs a subset of the 10 partitions).</p>",
      "rawMarkdown": "it might be related to the size of the data you train your model on (all data vs a subset of the 10 partitions).",
      "votes": null
    },
    {
      "id": "3091645",
      "postDate": "01/08/2025 15:53:34",
      "content": "<p>are you training on the same dataset?</p>",
      "rawMarkdown": "are you training on the same dataset?",
      "votes": null
    },
    {
      "id": "3096340",
      "postDate": "01/14/2025 09:24:48",
      "content": "<p>that make sense</p>",
      "rawMarkdown": "that make sense",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3091286,
      "author_name": "sritichaimae",
      "author_url": "",
      "post_date": "01/08/2025 09:15:16",
      "content": "<p>it might be related to the size of the data you train your model on (all data vs a subset of the 10 partitions).</p>",
      "votes": null,
      "replies": [
        {
          "id": 3096340,
          "author_name": "zoutain",
          "author_url": "",
          "post_date": "01/14/2025 09:24:48",
          "content": "<p>that make sense</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3091645,
      "author_name": "redfoongus",
      "author_url": "",
      "post_date": "01/08/2025 15:53:34",
      "content": "<p>are you training on the same dataset?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3091278": "I was using the almost the same params as the public model, but getting worse result. what could be the reason?\n(getting 0.005)\n\npublic model:\nhttps://www.kaggle.com/code/hideyukizushi/js-nn-xgb-ridge-pub-mytrain-weightblend-lb-0-0079\n\n\nI am using a 5 fold CV too:\n```python\nkf = KFold(n_splits=5, shuffle=True, random_state=42)\nfor fold, (train_index, valid_index) in enumerate(kf.split(X)):\n    print(f\"Start Training Fold {fold + 1}\")\n    X_train, X_valid = X[train_index].to_numpy(), X[valid_index].to_numpy()\n    y_train, y_valid = y[train_index].to_numpy(), y[valid_index].to_numpy()\n    w_train, w_valid = w[train_index].to_numpy(), w[valid_index].to_numpy()\n```\n\n\nparams like:\n```python\n    XGB_Params = {\n        'booster':          'gbtree',\n        'learning_rate':        0.05,\n        'n_estimators':          200,\n        'early_stopping_rounds': None,\n        'enable_categorical':    False,\n        'grow_policy':           None,\n        'max_cat_to_onehot':     None,\n        'max_depth':              6,\n        'subsample':             0.6,\n        'colsample_bytree':      0.8,\n        'reg_alpha':               2,\n        'reg_lambda':              8,\n        'random_state':         seed,\n        'tree_method':        'hist',\n        'device':             'cuda',\n        'eval_metric':          r2_xgb,\n        'verbosity':               1\n    }\n```\n\nand adding online retrain (train a totally  new xgb model online), getting 0.0034.....\nusing lag 1 responder as target, lag 2 and features as training data.... \n\nBeen banging my head trying but can't figure out how to improve on these... Any suggestions would be much appriciated....",
    "3091286": "it might be related to the size of the data you train your model on (all data vs a subset of the 10 partitions).",
    "3091645": "are you training on the same dataset?",
    "3096340": "that make sense"
  },
  "source": "meta"
}