{
  "id": 542809,
  "title": "Question on time_id",
  "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/542809",
  "author_name": "",
  "post_date": "2024-10-27T00:38:08.411392Z",
  "votes": 5,
  "comment_count": 9,
  "views": 0,
  "content": "<p>Hi,</p>\n<p>As someone already pointed out in <a href=\"https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/542184\" target=\"_blank\">here</a>. I hope we can get more info for the \"time_id\" field.<br>\nFrom a quick check, I found there are 2 possible time_id ranges:</p>\n<ol>\n<li>0-967 -&gt; happens after day 677</li>\n<li>0-848 -&gt; before day 676</li>\n</ol>\n<p>Since the there is no exchange trading hours change recently, can I assume we got more time ids because we get data more frequently during a day?</p>\n<p>Thanks</p>",
  "messages": [
    {
      "id": "3029152",
      "postDate": "10/27/2024 00:38:08",
      "content": "<p>Hi,</p>\n<p>As someone already pointed out in <a href=\"https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/542184\" target=\"_blank\">here</a>. I hope we can get more info for the \"time_id\" field.<br>\nFrom a quick check, I found there are 2 possible time_id ranges:</p>\n<ol>\n<li>0-967 -&gt; happens after day 677</li>\n<li>0-848 -&gt; before day 676</li>\n</ol>\n<p>Since the there is no exchange trading hours change recently, can I assume we got more time ids because we get data more frequently during a day?</p>\n<p>Thanks</p>",
      "rawMarkdown": "Hi,\n\nAs someone already pointed out in [here](https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/542184). I hope we can get more info for the \"time_id\" field.\nFrom a quick check, I found there are 2 possible time_id ranges:\n1. 0-967 -> happens after day 677\n2. 0-848 -> before day 676\n\nSince the there is no exchange trading hours change recently, can I assume we got more time ids because we get data more frequently during a day?\n\nThanks",
      "votes": null
    },
    {
      "id": "3029279",
      "postDate": "10/27/2024 05:17:55",
      "content": "<p>Yes, I also think that way - this seems like a change in data collection strategy instead of change in trading hours <a href=\"https://www.kaggle.com/ruizhy\" target=\"_blank\">@ruizhy</a> </p>",
      "rawMarkdown": "Yes, I also think that way - this seems like a change in data collection strategy instead of change in trading hours @ruizhy",
      "votes": null
    },
    {
      "id": "3029416",
      "postDate": "10/27/2024 09:24:41",
      "content": "<p>Thanks for confirming.</p>",
      "rawMarkdown": "Thanks for confirming.",
      "votes": null
    },
    {
      "id": "3029583",
      "postDate": "10/27/2024 13:29:42",
      "content": "<p>Well. If only someone could confirm that it will remain 0-967 in the future that would be great.</p>",
      "rawMarkdown": "Well. If only someone could confirm that it will remain 0-967 in the future that would be great.",
      "votes": null
    },
    {
      "id": "3030858",
      "postDate": "10/29/2024 01:50:13",
      "content": "<p>Normalize the time_id for the same day (time_id/max_time_id), so that even if the collection strategy changes, it won't have much impact, right?</p>",
      "rawMarkdown": "Normalize the time_id for the same day (time_id/max_time_id), so that even if the collection strategy changes, it won't have much impact, right?",
      "votes": null
    },
    {
      "id": "3030861",
      "postDate": "10/29/2024 02:06:01",
      "content": "<p>It's not that easy. What if the new time_id correspond to pre-market trading? That would be data that we didn't have before.</p>",
      "rawMarkdown": "It's not that easy. What if the new time_id correspond to pre-market trading? That would be data that we didn't have before.",
      "votes": null
    },
    {
      "id": "3030940",
      "postDate": "10/29/2024 04:43:53",
      "content": "<p>I don't think that is possible, I am pretty sure they have seperate model for the pre-market period.</p>",
      "rawMarkdown": "I don't think that is possible, I am pretty sure they have seperate model for the pre-market period.",
      "votes": null
    },
    {
      "id": "3030960",
      "postDate": "10/29/2024 05:37:18",
      "content": "<p>At some point, they might added pre-market or after hours or even both. Host has to clarify this.</p>",
      "rawMarkdown": "At some point, they might added pre-market or after hours or even both. Host has to clarify this.",
      "votes": null
    },
    {
      "id": "3030981",
      "postDate": "10/29/2024 06:21:07",
      "content": "<p>I think there's an important point that is being missed in this discussion. There is no guarantee that <code>time_id</code>s are equally spaced out. It could already include trades outside market hours, it could not.</p>\n<p>The only information it provides is ordinal, i.e. for the same <code>date_id</code>, <code>time_id</code>=5 comes after <code>time_id</code>=4. There is no way to know apart from elaboration, whether this within standard market hours, a subsample of market hours, a 24-hour period, or even if it's being collected at the same time.</p>\n<p>I would have said it's event/tick time, but it would be odd for there to be exactly the same number of trades over a long time horizon.</p>",
      "rawMarkdown": "I think there's an important point that is being missed in this discussion. There is no guarantee that `time_id`s are equally spaced out. It could already include trades outside market hours, it could not.\n\nThe only information it provides is ordinal, i.e. for the same `date_id`, `time_id`=5 comes after `time_id`=4. There is no way to know apart from elaboration, whether this within standard market hours, a subsample of market hours, a 24-hour period, or even if it's being collected at the same time.\n\nI would have said it's event/tick time, but it would be odd for there to be exactly the same number of trades over a long time horizon.",
      "votes": null
    },
    {
      "id": "3037491",
      "postDate": "11/05/2024 18:44:58",
      "content": "<p>Although it doesn't answer exactly the question, the host has given some further information about time_id <a href=\"https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/540436#3036206\" target=\"_blank\">here</a>.</p>",
      "rawMarkdown": "Although it doesn't answer exactly the question, the host has given some further information about time_id [here](https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/540436#3036206).",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3029279,
      "author_name": "ravi20076",
      "author_url": "",
      "post_date": "10/27/2024 05:17:55",
      "content": "<p>Yes, I also think that way - this seems like a change in data collection strategy instead of change in trading hours <a href=\"https://www.kaggle.com/ruizhy\" target=\"_blank\">@ruizhy</a> </p>",
      "votes": null,
      "replies": [
        {
          "id": 3029416,
          "author_name": "ruizhy",
          "author_url": "",
          "post_date": "10/27/2024 09:24:41",
          "content": "<p>Thanks for confirming.</p>",
          "votes": null,
          "replies": [
            {
              "id": 3029583,
              "author_name": "verracodeguacas",
              "author_url": "",
              "post_date": "10/27/2024 13:29:42",
              "content": "<p>Well. If only someone could confirm that it will remain 0-967 in the future that would be great.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 3030858,
                  "author_name": "yunsuxiaozi",
                  "author_url": "",
                  "post_date": "10/29/2024 01:50:13",
                  "content": "<p>Normalize the time_id for the same day (time_id/max_time_id), so that even if the collection strategy changes, it won't have much impact, right?</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 3030861,
                      "author_name": "verracodeguacas",
                      "author_url": "",
                      "post_date": "10/29/2024 02:06:01",
                      "content": "<p>It's not that easy. What if the new time_id correspond to pre-market trading? That would be data that we didn't have before.</p>",
                      "votes": null,
                      "replies": [
                        {
                          "id": 3030940,
                          "author_name": "ruizhy",
                          "author_url": "",
                          "post_date": "10/29/2024 04:43:53",
                          "content": "<p>I don't think that is possible, I am pretty sure they have seperate model for the pre-market period.</p>",
                          "votes": null,
                          "replies": []
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "id": 3030981,
              "author_name": "abstrphil",
              "author_url": "",
              "post_date": "10/29/2024 06:21:07",
              "content": "<p>I think there's an important point that is being missed in this discussion. There is no guarantee that <code>time_id</code>s are equally spaced out. It could already include trades outside market hours, it could not.</p>\n<p>The only information it provides is ordinal, i.e. for the same <code>date_id</code>, <code>time_id</code>=5 comes after <code>time_id</code>=4. There is no way to know apart from elaboration, whether this within standard market hours, a subsample of market hours, a 24-hour period, or even if it's being collected at the same time.</p>\n<p>I would have said it's event/tick time, but it would be odd for there to be exactly the same number of trades over a long time horizon.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3030960,
      "author_name": "gunesevitan",
      "author_url": "",
      "post_date": "10/29/2024 05:37:18",
      "content": "<p>At some point, they might added pre-market or after hours or even both. Host has to clarify this.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3037491,
      "author_name": "aleixgim",
      "author_url": "",
      "post_date": "11/05/2024 18:44:58",
      "content": "<p>Although it doesn't answer exactly the question, the host has given some further information about time_id <a href=\"https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/540436#3036206\" target=\"_blank\">here</a>.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3029152": "Hi,\n\nAs someone already pointed out in [here](https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/542184). I hope we can get more info for the \"time_id\" field.\nFrom a quick check, I found there are 2 possible time_id ranges:\n1. 0-967 -> happens after day 677\n2. 0-848 -> before day 676\n\nSince the there is no exchange trading hours change recently, can I assume we got more time ids because we get data more frequently during a day?\n\nThanks",
    "3029279": "Yes, I also think that way - this seems like a change in data collection strategy instead of change in trading hours @ruizhy",
    "3029416": "Thanks for confirming.",
    "3029583": "Well. If only someone could confirm that it will remain 0-967 in the future that would be great.",
    "3030858": "Normalize the time_id for the same day (time_id/max_time_id), so that even if the collection strategy changes, it won't have much impact, right?",
    "3030861": "It's not that easy. What if the new time_id correspond to pre-market trading? That would be data that we didn't have before.",
    "3030940": "I don't think that is possible, I am pretty sure they have seperate model for the pre-market period.",
    "3030960": "At some point, they might added pre-market or after hours or even both. Host has to clarify this.",
    "3030981": "I think there's an important point that is being missed in this discussion. There is no guarantee that `time_id`s are equally spaced out. It could already include trades outside market hours, it could not.\n\nThe only information it provides is ordinal, i.e. for the same `date_id`, `time_id`=5 comes after `time_id`=4. There is no way to know apart from elaboration, whether this within standard market hours, a subsample of market hours, a 24-hour period, or even if it's being collected at the same time.\n\nI would have said it's event/tick time, but it would be odd for there to be exactly the same number of trades over a long time horizon.",
    "3037491": "Although it doesn't answer exactly the question, the host has given some further information about time_id [here](https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/540436#3036206)."
  },
  "source": "meta"
}