{
  "id": 548550,
  "title": "Online learning",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/548550",
  "author_name": "Mihir Sutariya",
  "post_date": "2024-11-27T12:42:42.093000",
  "votes": 0,
  "comment_count": 3,
  "views": 0,
  "content": "<p>We have responder values for only day end in lag. I don't know how to apply online learning here. Can somebody help here :)</p>",
  "messages": [
    {
      "id": 3057942,
      "postDate": "2024-11-28T20:42:42.463Z",
      "content": "<p>Tried online learning with an XGBoost model. Pretty much always timed out due to the training latency.</p>\n<p>You have the features for the current date and responders for the previous date. You store the features in a history_cache and for the next date_id when the responder lags come in, those would be your corresponding responder values for the features you just stored in history_cache. So you can update your model weights with this new data.</p>\n<p>This allows you to update your model with the market dynamics.</p>",
      "rawMarkdown": "Tried online learning with an XGBoost model. Pretty much always timed out due to the training latency.\n\nYou have the features for the current date and responders for the previous date. You store the features in a history_cache and for the next date_id when the responder lags come in, those would be your corresponding responder values for the features you just stored in history_cache. So you can update your model weights with this new data.\n\nThis allows you to update your model with the market dynamics.",
      "votes": 1,
      "replies": [
        {
          "id": 3058306,
          "postDate": "2024-11-29T10:01:39.457Z",
          "content": "<p>Hi, <br>\nI think NN models are good for online learning. Can do 2-3 gradient descent in each inference or alternate. For XGB i guess we need to retrain whole model.</p>",
          "rawMarkdown": "Hi, \nI think NN models are good for online learning. Can do 2-3 gradient descent in each inference or alternate. For XGB i guess we need to retrain whole model."
        },
        {
          "id": 3064173,
          "postDate": "2024-12-05T10:21:21.550Z",
          "rawMarkdown": "",
          "votes": -1,
          "isDeleted": true
        }
      ]
    },
    {
      "id": 3056826,
      "postDate": "2024-11-27T12:42:42.093Z",
      "content": "<p>We have responder values for only day end in lag. I don't know how to apply online learning here. Can somebody help here :)</p>",
      "rawMarkdown": "We have responder values for only day end in lag. I don't know how to apply online learning here. Can somebody help here :)"
    }
  ],
  "comments": [
    {
      "id": 3057942,
      "author_name": "Devesh Shah",
      "author_url": "",
      "post_date": "2024-11-28T20:42:42.463000",
      "content": "<p>Tried online learning with an XGBoost model. Pretty much always timed out due to the training latency.</p>\n<p>You have the features for the current date and responders for the previous date. You store the features in a history_cache and for the next date_id when the responder lags come in, those would be your corresponding responder values for the features you just stored in history_cache. So you can update your model weights with this new data.</p>\n<p>This allows you to update your model with the market dynamics.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3058306,
          "author_name": "Mihir Sutariya",
          "author_url": "",
          "post_date": "2024-11-29T10:01:39.457000",
          "content": "<p>Hi, <br>\nI think NN models are good for online learning. Can do 2-3 gradient descent in each inference or alternate. For XGB i guess we need to retrain whole model.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3064173,
          "author_name": "",
          "author_url": "",
          "post_date": "2024-12-05T10:21:21.550000",
          "content": "",
          "votes": -1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3057942": "Tried online learning with an XGBoost model. Pretty much always timed out due to the training latency.\n\nYou have the features for the current date and responders for the previous date. You store the features in a history_cache and for the next date_id when the responder lags come in, those would be your corresponding responder values for the features you just stored in history_cache. So you can update your model weights with this new data.\n\nThis allows you to update your model with the market dynamics.",
    "3056826": "We have responder values for only day end in lag. I don't know how to apply online learning here. Can somebody help here :)"
  }
}