![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F522718%2Fd91189028e9d0a0e7c9c08eef1568803%2FScreenshot%202024-12-02%20175803.jpg?generation=1733180307879530&alt=media\")
I like to use Tableau to get to know data. I was looking at the data for this competition and found that the responder 6 value changes (or resets?) for a large amount of the symbol ids at the exact same time every day for exactly 40% of the train data.
\nthe responder 6 value seems to change/reset at time_id 369 … but only for the first 678 days of train_data. Take a look at the screenshot below.
\nDoes anyone have any good theories?
\n
The first chunk is 369 time_ids (from 0 to 368), which if you think each time_id is a minute, then it is 6 hours 9 minutes.
\nThe second chunk is 480 time_ids (from 369 to 848), which is exactly 8 hours.
\nThe third chunk is 119 time_ids (from 849 to 967), which is 1 minute short of 2 hours.
I like to use Tableau to get to know data. I was looking at the data for this competition and found that the responder 6 value changes (or resets?) for a large amount of the symbol ids at the exact same time every day for exactly 40% of the train data.
\nthe responder 6 value seems to change/reset at time_id 369 … but only for the first 678 days of train_data. Take a look at the screenshot below.
\nDoes anyone have any good theories?
\n
The first chunk is 369 time_ids (from 0 to 368), which if you think each time_id is a minute, then it is 6 hours 9 minutes.
\nThe second chunk is 480 time_ids (from 369 to 848), which is exactly 8 hours.
\nThe third chunk is 119 time_ids (from 849 to 967), which is 1 minute short of 2 hours.
You will also see that something else changes at date_id=677: the time_id per symbol_id.
I believe this goes back to what I previously shared. The earlier data (before date_id=677) is from older algorithmic trading bots. Something changed, either due to regulations, changes in market conditions, or just an algorithmic adjustment, which is reflected in the data.
As for the resetting of responder_6, an explanation could be position clearing/inventory rebalancing. I would be more confident in that answer, if it did not cease after date_id=677. That is somewhat confusing and I cannot think of an answer, for now.
Also, as Jane Street specializes in liquidity providing, their algos are primarily active when liquidity is highest. This reset time at time_id=369 likely aligns with or before peak daily liquidity.
Thank you for sharing, this is very helpful.
", "rawMarkdown": "You will also see that something else changes at `date_id=677`: the `time_id` per `symbol_id`. \n\nI believe this goes back to what I previously shared. The earlier data (before `date_id=677`) is from older algorithmic trading bots. Something changed, either due to regulations, changes in market conditions, or just an algorithmic adjustment, which is reflected in the data. \n\nAs for the resetting of `responder_6`, an explanation could be position clearing/inventory rebalancing. I would be more confident in that answer, if it did not cease after `date_id=677`. That is somewhat confusing and I cannot think of an answer, for now. \n\nAlso, as Jane Street specializes in liquidity providing, their algos are primarily active when liquidity is highest. This reset time at `time_id=369` likely aligns with or before peak daily liquidity. \n\nThank you for sharing, this is very helpful.", "votes": null }, { "id": "3061949", "postDate": "12/03/2024 06:35:02", "content": "This is such a great visualization! I hope you don't mind, I don't have Tableau and wanted to explore this, so re-created it here: Plot Heatmap of Feature/Responder by Date and Time
\nThat split is very intriguing, lots of mysteries in this challenge!
", "rawMarkdown": "This is such a great visualization! I hope you don't mind, I don't have Tableau and wanted to explore this, so re-created it here: [Plot Heatmap of Feature/Responder by Date and Time](https://www.kaggle.com/code/jimbeno/plot-heatmap-of-feature-responder-by-date-and-time)\n\nThat split is very intriguing, lots of mysteries in this challenge!", "votes": null }, { "id": "3062767", "postDate": "12/03/2024 22:13:47", "content": "Starting to notice some interesting patterns in the responders leading up to the break points. There clearly seem to be 3 periods of data here. Two shorter periods that are concatenated in the earlier dates, and one long period in the later dates.
\nIt's interesting how some responders seem to \"trigger\" at different intervals before the period close: responder_0, responder_1, responder_2.
\nOthers seem to \"activate\" at the start and then decay over that trading period: responder_6, responder_7, responder_8.
\nAnd some seem to be a blend of both: responder_3, responder_4, responder_5.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15357304%2F9ff64317e2a1dda72b294331b895c76f%2Fsymbol_38_all_responders.png?generation=1733263683897344&alt=media\")
You will also see that something else changes at date_id=677: the time_id per symbol_id.
I believe this goes back to what I previously shared. The earlier data (before date_id=677) is from older algorithmic trading bots. Something changed, either due to regulations, changes in market conditions, or just an algorithmic adjustment, which is reflected in the data.
As for the resetting of responder_6, an explanation could be position clearing/inventory rebalancing. I would be more confident in that answer, if it did not cease after date_id=677. That is somewhat confusing and I cannot think of an answer, for now.
Also, as Jane Street specializes in liquidity providing, their algos are primarily active when liquidity is highest. This reset time at time_id=369 likely aligns with or before peak daily liquidity.
Thank you for sharing, this is very helpful.
", "votes": null, "replies": [] }, { "id": 3061949, "author_name": "jimbeno", "author_url": "", "post_date": "12/03/2024 06:35:02", "content": "This is such a great visualization! I hope you don't mind, I don't have Tableau and wanted to explore this, so re-created it here: Plot Heatmap of Feature/Responder by Date and Time
\nThat split is very intriguing, lots of mysteries in this challenge!
", "votes": null, "replies": [] }, { "id": 3062767, "author_name": "jimbeno", "author_url": "", "post_date": "12/03/2024 22:13:47", "content": "Starting to notice some interesting patterns in the responders leading up to the break points. There clearly seem to be 3 periods of data here. Two shorter periods that are concatenated in the earlier dates, and one long period in the later dates.
\nIt's interesting how some responders seem to \"trigger\" at different intervals before the period close: responder_0, responder_1, responder_2.
\nOthers seem to \"activate\" at the start and then decay over that trading period: responder_6, responder_7, responder_8.
\nAnd some seem to be a blend of both: responder_3, responder_4, responder_5.
\n