{ "id": 543047, "title": "Numeration of date_id: question to the competition Host", "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/543047", "author_name": "Victor Shlepov", "post_date": "2024-10-28T11:26:39.411000", "votes": 9, "comment_count": 10, "views": 0, "content": "

This is likely a couple questions to competition host @mdesaijs :

\n

1) could you please confirm if you will keep the consistent numeration of date_id through the test set? In other words, should we expect that it would start with some number which is larger than the last \"date_id\" of the train set and will increase by one on every other test \"date_id\"?

\n

2) Will there be any time gap between train and test sets? or the test set would start exactly on the next day after the last day of train set?

\n

It's not a big deal, but it somewhat affects the data preprocessing pipeline, so the answers are appreciated.

\n

Cheers,

\n

Victor

", "messages": [ { "id": 3030283, "postDate": "2024-10-28T11:26:39.413Z", "content": "

This is likely a couple questions to competition host @mdesaijs :

\n

1) could you please confirm if you will keep the consistent numeration of date_id through the test set? In other words, should we expect that it would start with some number which is larger than the last \"date_id\" of the train set and will increase by one on every other test \"date_id\"?

\n

2) Will there be any time gap between train and test sets? or the test set would start exactly on the next day after the last day of train set?

\n

It's not a big deal, but it somewhat affects the data preprocessing pipeline, so the answers are appreciated.

\n

Cheers,

\n

Victor

", "rawMarkdown": "This is likely a couple questions to competition host @mdesaijs :\n\n1) could you please confirm if you will keep the consistent numeration of date_id through the test set? In other words, should we expect that it would start with some number which is larger than the last \"date_id\" of the train set and will increase by one on every other test \"date_id\"?\n\n2) Will there be any time gap between train and test sets? or the test set would start exactly on the next day after the last day of train set?\n\nIt's not a big deal, but it somewhat affects the data preprocessing pipeline, so the answers are appreciated.\n\nCheers,\n\nVictor", "votes": 9 }, { "id": 3030287, "postDate": "2024-10-28T11:37:03.057Z", "content": "

If you are happy to waste a couple of submissions, you can answer these questions.

\n

Create a simple submission that always predicts 0. Throw an error if the first test date_id is not the day after the last training date_id.

", "rawMarkdown": "If you are happy to waste a couple of submissions, you can answer these questions.\n\nCreate a simple submission that always predicts 0. Throw an error if the first test date_id is not the day after the last training date_id.", "votes": 3, "replies": [ { "id": 3030311, "postDate": "2024-10-28T12:12:03.947Z", "content": "

You're right, it's not that I'm short on submissions now :)

", "rawMarkdown": "You're right, it's not that I'm short on submissions now :)", "votes": 2, "replies": [ { "id": 3060688, "postDate": "2024-12-02T01:21:00.783Z", "content": "

So may I know your experiment result please?

", "rawMarkdown": "So may I know your experiment result please?" } ] } ] }, { "id": 3030286, "postDate": "2024-10-28T11:30:18.263Z", "content": "

@victorshlepov time_id is periodic and starts with 0 on every date. I think date_id is likely to start with 0 in the test set @victorshlepov

", "rawMarkdown": "@victorshlepov time_id is periodic and starts with 0 on every date. I think date_id is likely to start with 0 in the test set @victorshlepov ", "votes": 1, "replies": [ { "id": 3030288, "postDate": "2024-10-28T11:38:03.167Z", "content": "

Sorry, just a typo - fixed it. Both questions are still there - would be nice to have feedback from folks…

", "rawMarkdown": "Sorry, just a typo - fixed it. Both questions are still there - would be nice to have feedback from folks...", "replies": [ { "id": 3030470, "postDate": "2024-10-28T14:48:32.677Z", "content": "

@victorshlepov date_id starts from 0. please refer here. And you can confirm in kaggle evaluation api code.

", "rawMarkdown": "@victorshlepov date_id starts from 0. please refer [here](https://www.kaggle.com/competitions/jane-street-real-time-market-data-forecasting/discussion/542022#3027301). And you can confirm in kaggle evaluation api code.", "votes": 4 }, { "id": 3030620, "postDate": "2024-10-28T17:52:33.050Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3031039, "postDate": "2024-10-29T08:31:42.973Z", "content": "

Thanks! This solves my first question, but the second question remains open. Currently the lagged responders from the public \"lags\" are the same as responders from date_id = 1698 (the last date_id of train data), in other words train and test (the only public test step) do not have any time gaps in between. It would good to know if this is the case with hidden test too - especially given the non-stationary nature of data.

", "rawMarkdown": "Thanks! This solves my first question, but the second question remains open. Currently the lagged responders from the public \"lags\" are the same as responders from date_id = 1698 (the last date_id of train data), in other words train and test (the only public test step) do not have any time gaps in between. It would good to know if this is the case with hidden test too - especially given the non-stationary nature of data.", "votes": 2 }, { "id": 3031041, "postDate": "2024-10-29T08:37:34.707Z", "content": "

@victorshlepov Based on the comment in the data tab, I believe that the train, public, and private data are all provided as connected datasets. This was also the case in past competitions.

\n
 help you author robust submissions, during the final weeks  the model training phase we will be extending the  test    data closer  the submission deadline. Predictions  this extended  will  be scored.\n\nAt the   the forecasting phase, the unscored  test  will be extended up  the final day  the model training phase  the private  updated roughly every two weeks. Submissions will be rescored at the    .\n\nDuring the forecasting phase, the evaluation API will serve test data  the beginning  the    the   the private . You must make predictions at every timestep, but,  this phase,  predictions  the private  are scored. (You may predict   the unscored segments,  you .)\n
", "rawMarkdown": "@victorshlepov Based on the comment in the data tab, I believe that the train, public, and private data are all provided as connected datasets. This was also the case in past competitions.\n\n~~~\nTo help you author robust submissions, during the final weeks of the model training phase we will be extending the public test set to include data closer to the submission deadline. Predictions on this extended set will not be scored.\n\nAt the start of the forecasting phase, the unscored public test set will be extended up to the final day of the model training phase and the private set updated roughly every two weeks. Submissions will be rescored at the time of each update.\n\nDuring the forecasting phase, the evaluation API will serve test data from the beginning of the public set to the end of the private set. You must make predictions at every timestep, but, in this phase, only predictions on the private set are scored. (You may predict 0.0 on the unscored segments, if you like.)\n~~~", "votes": 2 }, { "id": 3060747, "postDate": "2024-12-02T04:28:52.417Z", "content": "

If the first test has date_id all = 0, then what would the date_id in the first lags be? Would it also be 0, such that 'responder_lag_{i}_1' corresponds to the responders from date_id = -1?

", "rawMarkdown": "If the first test has date_id all = 0, then what would the date_id in the first lags be? Would it also be 0, such that 'responder_lag_{i}_1' corresponds to the responders from date_id = -1?" } ] } ] } ], "comments": [ { "id": 3030287, "author_name": "paddykb", "author_url": "", "post_date": "2024-10-28T11:37:03.057000", "content": "

If you are happy to waste a couple of submissions, you can answer these questions.

\n

Create a simple submission that always predicts 0. Throw an error if the first test date_id is not the day after the last training date_id.

", "votes": 3, "replies": [ { "id": 3030311, "author_name": "Victor Shlepov", "author_url": "", "post_date": "2024-10-28T12:12:03.947000", "content": "

You're right, it's not that I'm short on submissions now :)

", "votes": 2, "replies": [ { "id": 3060688, "author_name": "JinbyZ", "author_url": "", "post_date": "2024-12-02T01:21:00.783000", "content": "

So may I know your experiment result please?

", "votes": 0, "replies": [] } ] } ] }, { "id": 3030286, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-10-28T11:30:18.263000", "content": "

@victorshlepov time_id is periodic and starts with 0 on every date. I think date_id is likely to start with 0 in the test set @victorshlepov

", "votes": 1, "replies": [ { "id": 3030288, "author_name": "Victor Shlepov", "author_url": "", "post_date": "2024-10-28T11:38:03.167000", "content": "

Sorry, just a typo - fixed it. Both questions are still there - would be nice to have feedback from folks…

", "votes": 0, "replies": [ { "id": 3030470, "author_name": "chumajin", "author_url": "", "post_date": "2024-10-28T14:48:32.677000", "content": "

@victorshlepov date_id starts from 0. please refer here. And you can confirm in kaggle evaluation api code.

", "votes": 4, "replies": [] }, { "id": 3030620, "author_name": "", "author_url": "", "post_date": "2024-10-28T17:52:33.050000", "content": "", "votes": 0, "replies": [] }, { "id": 3031039, "author_name": "Victor Shlepov", "author_url": "", "post_date": "2024-10-29T08:31:42.973000", "content": "

Thanks! This solves my first question, but the second question remains open. Currently the lagged responders from the public \"lags\" are the same as responders from date_id = 1698 (the last date_id of train data), in other words train and test (the only public test step) do not have any time gaps in between. It would good to know if this is the case with hidden test too - especially given the non-stationary nature of data.

", "votes": 2, "replies": [] }, { "id": 3031041, "author_name": "chumajin", "author_url": "", "post_date": "2024-10-29T08:37:34.707000", "content": "

@victorshlepov Based on the comment in the data tab, I believe that the train, public, and private data are all provided as connected datasets. This was also the case in past competitions.

\n
 help you author robust submissions, during the final weeks  the model training phase we will be extending the  test    data closer  the submission deadline. Predictions  this extended  will  be scored.\n\nAt the   the forecasting phase, the unscored  test  will be extended up  the final day  the model training phase  the private  updated roughly every two weeks. Submissions will be rescored at the    .\n\nDuring the forecasting phase, the evaluation API will serve test data  the beginning  the    the   the private . You must make predictions at every timestep, but,  this phase,  predictions  the private  are scored. (You may predict   the unscored segments,  you .)\n
", "votes": 2, "replies": [] }, { "id": 3060747, "author_name": "Lu Bin Liu", "author_url": "", "post_date": "2024-12-02T04:28:52.417000", "content": "

If the first test has date_id all = 0, then what would the date_id in the first lags be? Would it also be 0, such that 'responder_lag_{i}_1' corresponds to the responders from date_id = -1?

", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3030283": "This is likely a couple questions to competition host @mdesaijs :\n\n1) could you please confirm if you will keep the consistent numeration of date_id through the test set? In other words, should we expect that it would start with some number which is larger than the last \"date_id\" of the train set and will increase by one on every other test \"date_id\"?\n\n2) Will there be any time gap between train and test sets? or the test set would start exactly on the next day after the last day of train set?\n\nIt's not a big deal, but it somewhat affects the data preprocessing pipeline, so the answers are appreciated.\n\nCheers,\n\nVictor", "3030287": "If you are happy to waste a couple of submissions, you can answer these questions.\n\nCreate a simple submission that always predicts 0. Throw an error if the first test date_id is not the day after the last training date_id.", "3030286": "@victorshlepov time_id is periodic and starts with 0 on every date. I think date_id is likely to start with 0 in the test set @victorshlepov " } }