![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15357304%2F5c7f587487a8ac3ab92641b85d0ff90b%2Ffeature_01_day_1020.png?generation=1733044360416400&alt=media\")
When we have 968 time_id's per day, assuming they're equal intervals, that could roughly map to 16 hours x 60 minutes = 960 minutes. If we ignore the 8 extra minutes, it nicely aligns with 16 regular trading hours plus pre/post market. Anything can be traded during this timeframe (equities, futures, forex, crypto, etc)
\nOutside of this window, Futures trade 23 hours a day (23 x 60 = 1,380 / 968 = 1.43), Forex trade 24 hours a day (24 x 60 = 1,440 = 1.49). Regular Trading Hours is 6.5 hours (6.5 x 60 = 390 / 968 = 0.40) What's nice about 16 hours is that 16 x 60 = 960 / 968 = 0.99, which is pretty close to 1 (for 1-minute interval data). The other time ranges give odd partial ratios. And I don't think this would be tick or volume data as it's always the same number of time_id's (after it settles) per date_id with no missing values. So that suggests it's a clock-based sub-division.
\n| Trading Period | \nStart Time | \nEnd Time | \nHours | \nFraction | \nExpected IDs | \nActual IDs | \nID Range | \n
|---|---|---|---|---|---|---|---|
| Pre-Market | \n4:00 am ET | \n9:30 am ET | \n5.5 hrs | \n34.38% | \n330 | \n332 | \n0–331 | \n
| RTH | \n9:30 am ET | \n4:00 pm ET | \n6.5 hrs | \n40.62% | \n390 | \n393 | \n332–724 | \n
| After-Hours | \n4:00 pm ET | \n8:00 pm ET | \n4 hrs | \n25.0% | \n240 | \n243 | \n725–967 | \n
| Total | \n4:00 am ET | \n8:00 pm ET | \n16 hrs | \n100% | \n960 | \n968 | \n0–967 | \n
Here are plots of feature_01 (all symbols) over 5 days (1020 - 1024) with potential mappings to hours and pre-market, RTH, and after-hours. Not every feature shows cyclicality like this. But it's interesting how on some days, you can kind of see a change in character between pre-market and AH compared to RTH. I don't know if that's forcing a fit or not. It's not as obvious a change on every day, and not all features show a peak or valley at the potential boundaries.
\nWhat do you think?
\nUpdate: Here's a notebook with a function to make these plots: Plot Function with 16-hour Trading Periods. Also, as was pointed out below, the data description says \"the actual time intervals between time_id values may vary.\" That could be why there is a discrepancy, but it could also be that this rough mapping of 1 time-id to 1 minute is invalid.
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15357304%2F89f2b44799e79d0d75a7af582e8f1afc%2Ffeature_01_day_1021.png?generation=1733045608225375&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15357304%2Fea4392fa61f6c124701890594ca884d6%2Ffeature_01_day_1022.png?generation=1733045623595539&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15357304%2F5c652bc8716719b407477cc2b90d92c1%2Ffeature_01_day_1023.png?generation=1733045655593314&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15357304%2Facab5ad8aa8f73d4c594bb4a14209234%2Ffeature_01_day_1024.png?generation=1733045641540521&alt=media\")
When we have 968 time_id's per day, assuming they're equal intervals, that could roughly map to 16 hours x 60 minutes = 960 minutes. If we ignore the 8 extra minutes, it nicely aligns with 16 regular trading hours plus pre/post market. Anything can be traded during this timeframe (equities, futures, forex, crypto, etc)
\nOutside of this window, Futures trade 23 hours a day (23 x 60 = 1,380 / 968 = 1.43), Forex trade 24 hours a day (24 x 60 = 1,440 = 1.49). Regular Trading Hours is 6.5 hours (6.5 x 60 = 390 / 968 = 0.40) What's nice about 16 hours is that 16 x 60 = 960 / 968 = 0.99, which is pretty close to 1 (for 1-minute interval data). The other time ranges give odd partial ratios. And I don't think this would be tick or volume data as it's always the same number of time_id's (after it settles) per date_id with no missing values. So that suggests it's a clock-based sub-division.
\n| Trading Period | \nStart Time | \nEnd Time | \nHours | \nFraction | \nExpected IDs | \nActual IDs | \nID Range | \n
|---|---|---|---|---|---|---|---|
| Pre-Market | \n4:00 am ET | \n9:30 am ET | \n5.5 hrs | \n34.38% | \n330 | \n332 | \n0–331 | \n
| RTH | \n9:30 am ET | \n4:00 pm ET | \n6.5 hrs | \n40.62% | \n390 | \n393 | \n332–724 | \n
| After-Hours | \n4:00 pm ET | \n8:00 pm ET | \n4 hrs | \n25.0% | \n240 | \n243 | \n725–967 | \n
| Total | \n4:00 am ET | \n8:00 pm ET | \n16 hrs | \n100% | \n960 | \n968 | \n0–967 | \n
Here are plots of feature_01 (all symbols) over 5 days (1020 - 1024) with potential mappings to hours and pre-market, RTH, and after-hours. Not every feature shows cyclicality like this. But it's interesting how on some days, you can kind of see a change in character between pre-market and AH compared to RTH. I don't know if that's forcing a fit or not. It's not as obvious a change on every day, and not all features show a peak or valley at the potential boundaries.
\nWhat do you think?
\nUpdate: Here's a notebook with a function to make these plots: Plot Function with 16-hour Trading Periods. Also, as was pointed out below, the data description says \"the actual time intervals between time_id values may vary.\" That could be why there is a discrepancy, but it could also be that this rough mapping of 1 time-id to 1 minute is invalid.
\n
I think the approach is methodical in evaluating how 968 time IDs align with 16 hours of trading, with subdivisions into pre-market, RTH, and after-hours sessions.
\nThe 16-hour trading window described here likely represents the extended hours trading session for the U.S. stock market, which includes both pre-market and after-hours trading in addition to the regular trading hours (RTH).
so,when you combine these three trading periods, you get a total of 16 hours, from 4:00 am to 8:00 pm ET. During these 16 hours, the U.S. equities market is accessible for trading, though the liquidity and activity levels vary significantly between pre-market, regular hours, and after-hours sessions.
", "rawMarkdown": "I think the approach is methodical in evaluating how 968 time IDs align with 16 hours of trading, with subdivisions into pre-market, RTH, and after-hours sessions.\nThe 16-hour trading window described here likely represents the extended hours trading session for the U.S. stock market, which includes both pre-market and after-hours trading in addition to the regular trading hours (RTH). \n\n## Pre-Market: Generally opens at 4:00 am ET and goes until the start of the regular trading session at 9:30 am ET, providing 5.5 hours of trading.\n## Regular Trading Hours (RTH): From 9:30 am ET to 4:00 pm ET, totaling 6.5 hours of trading.\n## After-Hours: Starts at 4:00 pm ET and ends at 8:00 pm ET, providing another 4 hours.\n\nso,when you combine these three trading periods, you get a total of 16 hours, from 4:00 am to 8:00 pm ET. During these 16 hours, the U.S. equities market is accessible for trading, though the liquidity and activity levels vary significantly between pre-market, regular hours, and after-hours sessions.", "votes": 4, "replies": [ { "id": 3062374, "postDate": "2024-12-03T14:47:44.147Z", "content": "General liquidity by session, from least to greatest: after hours -> overnight -> regular. After hours is typically dominated by algorithmic trades, as they are \"free\" to do as they want, with a lack of non-algo traders present. They are less free during peak liquidity, but they are still very much active and influential over price movement.
", "rawMarkdown": "General liquidity by session, from least to greatest: after hours -> overnight -> regular. After hours is typically dominated by algorithmic trades, as they are \"free\" to do as they want, with a lack of non-algo traders present. They are less free during peak liquidity, but they are still very much active and influential over price movement. " } ] }, { "id": 3060174, "postDate": "2024-12-01T13:14:55.467Z", "content": "Which market is open for 16 hours? It might be useful knowing which market we are working on.
", "rawMarkdown": "Which market is open for 16 hours? It might be useful knowing which market we are working on.", "votes": 1, "replies": [ { "id": 3060414, "postDate": "2024-12-01T17:43:46.440Z", "content": "SGX , CME. considering after hours. mainly F&O is available for extented periods.
", "rawMarkdown": "SGX , CME. considering after hours. mainly F&O is available for extented periods." }, { "id": 3060423, "postDate": "2024-12-01T17:56:28.807Z", "content": "Some guesses from a domain expert https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/548636#3059674
", "rawMarkdown": "Some guesses from a domain expert https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/548636#3059674" }, { "id": 3060526, "postDate": "2024-12-01T20:23:46.930Z", "content": "How would this be useful?
", "rawMarkdown": "How would this be useful?" }, { "id": 3062391, "postDate": "2024-12-03T14:57:10.070Z", "content": "It is likely the futures markets, since JS are liquidity providers. This introduces some nuances:
\nContract rollovers: the processes of rolling a near expiration futures contract over to the next most liquid contract (also called the front-month). These rollovers can be quarterly or monthly, but this is not hard coded. For example, gold futures have slightly weird rollovers. This implies that if there is a signal in the data for these rollovers, it will differ for some symbol_ids.
Daily settlement: when the exchanges determine official closing price of a futures contract for any given trading day. It is used to calculate mark-to-market returns or losses and margin requirements for each trader. It varies per exchange and ticker, but for CME stock indices, it is from 5pm-6pm EST. Before this period, market participants execute last minute hedges, position adjustments, and order placements to meet margin requirements. Retail traders typically see increased slippage and larger bid-ask spreads during this time.
", "rawMarkdown": "It is likely the futures markets, since JS are liquidity providers. This introduces some nuances:\n\nContract rollovers: the processes of rolling a near expiration futures contract over to the next most liquid contract (also called the front-month). These rollovers can be quarterly or monthly, but this is not hard coded. For example, gold futures have slightly weird rollovers. This implies that if there is a signal in the data for these rollovers, it will differ for some `symbol_ids`. \n\nDaily settlement: when the exchanges determine official closing price of a futures contract for any given trading day. It is used to calculate mark-to-market returns or losses and margin requirements for each trader. It varies per exchange and ticker, but for CME stock indices, it is from 5pm-6pm EST. Before this period, market participants execute last minute hedges, position adjustments, and order placements to meet margin requirements. Retail traders typically see increased slippage and larger bid-ask spreads during this time. " } ] }, { "id": 3060773, "postDate": "2024-12-02T05:28:37.037Z", "content": "I did think about creating bins for time_id but opted against it (at least making it low priority) given we are explicitly told that there is non uniformity in the delta between time_ids. I also don't think we can assume that time_id=40 is the same time each day even if the intra-day deltas are not equal.
", "rawMarkdown": "I did think about creating bins for time_id but opted against it (at least making it low priority) given we are explicitly told that there is non uniformity in the delta between time_ids. I also don't think we can assume that time_id=40 is the same time each day even if the intra-day deltas are not equal.", "votes": 2 }, { "id": 3083233, "postDate": "2024-12-29T07:17:28.043Z", "content": "I did an analysis of count of timesteps per day for the entire period separately for each symbol_id (asset). I have listed the resulting data below and also the code to generate it. My observation is that most of the assets had initial timesteps of 849 per day, and later switched to 968. Few assets that started after this switch, do not have the data in this initial period (4,6,18,24,27,28,31,32,35 and 37).
\nWould we be able to include the initial period of data in our analysis? I am planning to use GPU and LSTM where, as I understand, time steps should be should be uniform.
\nHere is my notebook for this analysis: https://www.kaggle.com/code/nazir65/janestreettimestepanalysis
", "rawMarkdown": "I did an analysis of count of timesteps per day for the entire period separately for each symbol_id (asset). I have listed the resulting data below and also the code to generate it. My observation is that most of the assets had initial timesteps of 849 per day, and later switched to 968. Few assets that started after this switch, do not have the data in this initial period (4,6,18,24,27,28,31,32,35 and 37).\n\nWould we be able to include the initial period of data in our analysis? I am planning to use GPU and LSTM where, as I understand, time steps should be should be uniform. \n\nHere is my notebook for this analysis: https://www.kaggle.com/code/nazir65/janestreettimestepanalysis" }, { "id": 3060525, "postDate": "2024-12-01T20:23:20.287Z", "content": "\n\ndate_id and time_id - Integer values that are ordinally sorted, providing a chronological structure to the data, although the actual time intervals between time_id values may vary.
\n
this is from the dataset descriptions, they are said not to be equal intervals
", "rawMarkdown": ">date_id and time_id - Integer values that are ordinally sorted, providing a chronological structure to the data, although the actual time intervals between time_id values may vary.\n\nthis is from the dataset descriptions, they are said not to be equal intervals", "replies": [ { "id": 3061783, "postDate": "2024-12-03T01:40:17.347Z", "content": "yes you can see that in the data too…
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F522718%2Fd91189028e9d0a0e7c9c08eef1568803%2FScreenshot%202024-12-02%20175803.jpg?generation=1733180307879530&alt=media\")
Could I ask about how do you draw this graph? it seems great
", "rawMarkdown": "Could I ask about how do you draw this graph? it seems great", "votes": 1 }, { "id": 3061945, "postDate": "2024-12-03T06:29:38.880Z", "content": "I think he created it in Tableau, but I re-created it here: Plot Heatmap of Feature/Responder by Date and Time
", "rawMarkdown": "I think he created it in Tableau, but I re-created it here: [Plot Heatmap of Feature/Responder by Date and Time](https://www.kaggle.com/code/jimbeno/plot-heatmap-of-feature-responder-by-date-and-time)", "votes": 1 }, { "id": 3064146, "postDate": "2024-12-05T09:20:15.017Z", "content": "It is a clear dividing line. Can we move the block with time_id between 369~848 and date_id between 0~676 to the bottom, that is , there is a gap in the middle until date 677.
", "rawMarkdown": "It is a clear dividing line. Can we move the block with time_id between 369~848 and date_id between 0~676 to the bottom, that is , there is a gap in the middle until date 677." } ] } ] }, { "id": 3060670, "postDate": "2024-12-02T00:46:15.803Z", "content": "could you share the program
", "rawMarkdown": "could you share the program ", "isDeleted": true, "replies": [ { "id": 3060784, "postDate": "2024-12-02T05:36:17.470Z", "content": "Here's a notebook with a function to plot these charts. You can group by period or by symbol:
\nhttps://www.kaggle.com/code/jimbeno/plot-function-with-16-hour-trading-periods
", "rawMarkdown": "Here's a notebook with a function to plot these charts. You can group by period or by symbol:\n\nhttps://www.kaggle.com/code/jimbeno/plot-function-with-16-hour-trading-periods" } ] } ], "comments": [ { "id": 3060657, "author_name": "Haobin Mao", "author_url": "", "post_date": "2024-12-02T00:25:47.197000", "content": "I think the approach is methodical in evaluating how 968 time IDs align with 16 hours of trading, with subdivisions into pre-market, RTH, and after-hours sessions.
\nThe 16-hour trading window described here likely represents the extended hours trading session for the U.S. stock market, which includes both pre-market and after-hours trading in addition to the regular trading hours (RTH).
so,when you combine these three trading periods, you get a total of 16 hours, from 4:00 am to 8:00 pm ET. During these 16 hours, the U.S. equities market is accessible for trading, though the liquidity and activity levels vary significantly between pre-market, regular hours, and after-hours sessions.
", "votes": 4, "replies": [ { "id": 3062374, "author_name": "Tucker Arrants", "author_url": "", "post_date": "2024-12-03T14:47:44.147000", "content": "General liquidity by session, from least to greatest: after hours -> overnight -> regular. After hours is typically dominated by algorithmic trades, as they are \"free\" to do as they want, with a lack of non-algo traders present. They are less free during peak liquidity, but they are still very much active and influential over price movement.
", "votes": 0, "replies": [] } ] }, { "id": 3060174, "author_name": "yuanzhe zhou", "author_url": "", "post_date": "2024-12-01T13:14:55.467000", "content": "Which market is open for 16 hours? It might be useful knowing which market we are working on.
", "votes": 1, "replies": [ { "id": 3060414, "author_name": "Abhi", "author_url": "", "post_date": "2024-12-01T17:43:46.440000", "content": "SGX , CME. considering after hours. mainly F&O is available for extented periods.
", "votes": 0, "replies": [] }, { "id": 3060423, "author_name": "Lu Bin Liu", "author_url": "", "post_date": "2024-12-01T17:56:28.807000", "content": "Some guesses from a domain expert https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/548636#3059674
", "votes": 0, "replies": [] }, { "id": 3060526, "author_name": "Seqaeon", "author_url": "", "post_date": "2024-12-01T20:23:46.930000", "content": "How would this be useful?
", "votes": 0, "replies": [] }, { "id": 3062391, "author_name": "Tucker Arrants", "author_url": "", "post_date": "2024-12-03T14:57:10.070000", "content": "It is likely the futures markets, since JS are liquidity providers. This introduces some nuances:
\nContract rollovers: the processes of rolling a near expiration futures contract over to the next most liquid contract (also called the front-month). These rollovers can be quarterly or monthly, but this is not hard coded. For example, gold futures have slightly weird rollovers. This implies that if there is a signal in the data for these rollovers, it will differ for some symbol_ids.
Daily settlement: when the exchanges determine official closing price of a futures contract for any given trading day. It is used to calculate mark-to-market returns or losses and margin requirements for each trader. It varies per exchange and ticker, but for CME stock indices, it is from 5pm-6pm EST. Before this period, market participants execute last minute hedges, position adjustments, and order placements to meet margin requirements. Retail traders typically see increased slippage and larger bid-ask spreads during this time.
", "votes": 0, "replies": [] } ] }, { "id": 3060773, "author_name": "Michael Timbs", "author_url": "", "post_date": "2024-12-02T05:28:37.037000", "content": "I did think about creating bins for time_id but opted against it (at least making it low priority) given we are explicitly told that there is non uniformity in the delta between time_ids. I also don't think we can assume that time_id=40 is the same time each day even if the intra-day deltas are not equal.
", "votes": 2, "replies": [] }, { "id": 3083233, "author_name": "Mohammad Naziruddin", "author_url": "", "post_date": "2024-12-29T07:17:28.043000", "content": "I did an analysis of count of timesteps per day for the entire period separately for each symbol_id (asset). I have listed the resulting data below and also the code to generate it. My observation is that most of the assets had initial timesteps of 849 per day, and later switched to 968. Few assets that started after this switch, do not have the data in this initial period (4,6,18,24,27,28,31,32,35 and 37).
\nWould we be able to include the initial period of data in our analysis? I am planning to use GPU and LSTM where, as I understand, time steps should be should be uniform.
\nHere is my notebook for this analysis: https://www.kaggle.com/code/nazir65/janestreettimestepanalysis
", "votes": 0, "replies": [] }, { "id": 3060525, "author_name": "Seqaeon", "author_url": "", "post_date": "2024-12-01T20:23:20.287000", "content": "\n\ndate_id and time_id - Integer values that are ordinally sorted, providing a chronological structure to the data, although the actual time intervals between time_id values may vary.
\n
this is from the dataset descriptions, they are said not to be equal intervals
", "votes": 0, "replies": [ { "id": 3061783, "author_name": "David Dirring", "author_url": "", "post_date": "2024-12-03T01:40:17.347000", "content": "yes you can see that in the data too…
\n
Could I ask about how do you draw this graph? it seems great
", "votes": 1, "replies": [] }, { "id": 3061945, "author_name": "Jim Beno", "author_url": "", "post_date": "2024-12-03T06:29:38.880000", "content": "I think he created it in Tableau, but I re-created it here: Plot Heatmap of Feature/Responder by Date and Time
", "votes": 1, "replies": [] }, { "id": 3064146, "author_name": "Timothyzhang", "author_url": "", "post_date": "2024-12-05T09:20:15.017000", "content": "It is a clear dividing line. Can we move the block with time_id between 369~848 and date_id between 0~676 to the bottom, that is , there is a gap in the middle until date 677.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3060670, "author_name": "", "author_url": "", "post_date": "2024-12-02T00:46:15.803000", "content": "could you share the program
", "votes": 0, "replies": [ { "id": 3060784, "author_name": "Jim Beno", "author_url": "", "post_date": "2024-12-02T05:36:17.470000", "content": "Here's a notebook with a function to plot these charts. You can group by period or by symbol:
\nhttps://www.kaggle.com/code/jimbeno/plot-function-with-16-hour-trading-periods
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "3060017": "When we have 968 time_id's per day, assuming they're equal intervals, that could roughly map to 16 hours x 60 minutes = 960 minutes. If we ignore the 8 extra minutes, it nicely aligns with 16 regular trading hours plus pre/post market. Anything can be traded during this timeframe (equities, futures, forex, crypto, etc)\n\nOutside of this window, Futures trade 23 hours a day (23 x 60 = 1,380 / 968 = 1.43), Forex trade 24 hours a day (24 x 60 = 1,440 = 1.49). Regular Trading Hours is 6.5 hours (6.5 x 60 = 390 / 968 = 0.40) What's nice about 16 hours is that 16 x 60 = 960 / 968 = 0.99, which is pretty close to 1 (for 1-minute interval data). The other time ranges give odd partial ratios. And I don't think this would be tick or volume data as it's always the same number of time_id's (after it settles) per date_id with no missing values. So that suggests it's a clock-based sub-division.\n\n| **Trading Period** | **Start Time** | **End Time** | **Hours** | **Fraction** | **Expected IDs** | **Actual IDs** | **ID Range** |\n|--------------------------|----------------|----------------|-----------|--------------|------------------------|---------------------|--------------------|\n| Pre-Market | 4:00 am ET | 9:30 am ET | 5.5 hrs | 34.38% | 330 | 332 | 0–331 |\n| RTH | 9:30 am ET | 4:00 pm ET | 6.5 hrs | 40.62% | 390 | 393 | 332–724 |\n| After-Hours | 4:00 pm ET | 8:00 pm ET | 4 hrs | 25.0% | 240 | 243 | 725–967 |\n| **Total** | **4:00 am ET** | **8:00 pm ET** | **16 hrs** | **100%** | **960** | **968** | **0–967** |\n\n\nHere are plots of feature_01 (all symbols) over 5 days (1020 - 1024) with potential mappings to hours and pre-market, RTH, and after-hours. Not every feature shows cyclicality like this. But it's interesting how on some days, you can kind of see a change in character between pre-market and AH compared to RTH. I don't know if that's forcing a fit or not. It's not as obvious a change on every day, and not all features show a peak or valley at the potential boundaries.\n\nWhat do you think?\n\n**Update:** Here's a notebook with a function to make these plots: [Plot Function with 16-hour Trading Periods](https://www.kaggle.com/code/jimbeno/plot-function-with-16-hour-trading-periods). Also, as was pointed out below, the data description says \"the actual time intervals between time_id values may vary.\" That could be why there is a discrepancy, but it could also be that this rough mapping of 1 time-id to 1 minute is invalid.\n\n\n \n\n\n\n\n\n\n", "3060657": "I think the approach is methodical in evaluating how 968 time IDs align with 16 hours of trading, with subdivisions into pre-market, RTH, and after-hours sessions.\nThe 16-hour trading window described here likely represents the extended hours trading session for the U.S. stock market, which includes both pre-market and after-hours trading in addition to the regular trading hours (RTH). \n\n## Pre-Market: Generally opens at 4:00 am ET and goes until the start of the regular trading session at 9:30 am ET, providing 5.5 hours of trading.\n## Regular Trading Hours (RTH): From 9:30 am ET to 4:00 pm ET, totaling 6.5 hours of trading.\n## After-Hours: Starts at 4:00 pm ET and ends at 8:00 pm ET, providing another 4 hours.\n\nso,when you combine these three trading periods, you get a total of 16 hours, from 4:00 am to 8:00 pm ET. During these 16 hours, the U.S. equities market is accessible for trading, though the liquidity and activity levels vary significantly between pre-market, regular hours, and after-hours sessions.", "3060174": "Which market is open for 16 hours? It might be useful knowing which market we are working on.", "3060773": "I did think about creating bins for time_id but opted against it (at least making it low priority) given we are explicitly told that there is non uniformity in the delta between time_ids. I also don't think we can assume that time_id=40 is the same time each day even if the intra-day deltas are not equal.", "3083233": "I did an analysis of count of timesteps per day for the entire period separately for each symbol_id (asset). I have listed the resulting data below and also the code to generate it. My observation is that most of the assets had initial timesteps of 849 per day, and later switched to 968. Few assets that started after this switch, do not have the data in this initial period (4,6,18,24,27,28,31,32,35 and 37).\n\nWould we be able to include the initial period of data in our analysis? I am planning to use GPU and LSTM where, as I understand, time steps should be should be uniform. \n\nHere is my notebook for this analysis: https://www.kaggle.com/code/nazir65/janestreettimestepanalysis", "3060525": ">date_id and time_id - Integer values that are ordinally sorted, providing a chronological structure to the data, although the actual time intervals between time_id values may vary.\n\nthis is from the dataset descriptions, they are said not to be equal intervals", "3060670": "could you share the program " } }