{ "id": 556218, "title": "Test Phase", "url": "/competitions/jane-street-real-time-market-data-forecasting/discussion/556218", "author_name": "", "post_date": "2025-01-12T00:17:37.736568200Z", "votes": 1, "comment_count": 2, "views": 0, "content": "

During the test phase im assuming they pass the entire time series through at each point from start to finish?

\n

My online training doesn't persist state to the filesystem or anything. It just holds the weights in memory and updates each day without checkpointing.

\n

Not that im going to get close to a winning score but I just wanted to make sure this wont be an issue and they dont just fire up the server for our submissions and pass random batches of data in every month and then aggregate the results at the end

", "messages": [ { "id": "3094326", "postDate": "01/12/2025 00:17:37", "content": "

During the test phase im assuming they pass the entire time series through at each point from start to finish?

\n

My online training doesn't persist state to the filesystem or anything. It just holds the weights in memory and updates each day without checkpointing.

\n

Not that im going to get close to a winning score but I just wanted to make sure this wont be an issue and they dont just fire up the server for our submissions and pass random batches of data in every month and then aggregate the results at the end

", "rawMarkdown": "During the test phase im assuming they pass the entire time series through at each point from start to finish?\n\nMy online training doesn't persist state to the filesystem or anything. It just holds the weights in memory and updates each day without checkpointing. \n\nNot that im going to get close to a winning score but I just wanted to make sure this wont be an issue and they dont just fire up the server for our submissions and pass random batches of data in every month and then aggregate the results at the end", "votes": null }, { "id": "3094332", "postDate": "01/12/2025 00:28:18", "content": "
\n

At the start of the forecasting phase, the unscored public test set will be extended up to the final day of the model training phase and the private set updated roughly every two weeks. Submissions will be rescored at the time of each update.

\n

During the forecasting phase, the evaluation API will serve test data from the beginning of the public set to the end of the private set. You must make predictions at every timestep, but, in this phase, only predictions on the private set are scored. (You may predict 0.0 on the unscored segments, if you like.)

\n
\n

From the data card.

", "rawMarkdown": ">At the start of the forecasting phase, the unscored public test set will be extended up to the final day of the model training phase and the private set updated roughly every two weeks. Submissions will be rescored at the time of each update.\n\n>During the forecasting phase, the evaluation API will serve test data **from the beginning of the public set to the end of the private set.** You must make predictions at every timestep, but, in this phase, only predictions on the private set are scored. (You may predict 0.0 on the unscored segments, if you like.)\n\nFrom the data card.", "votes": null }, { "id": "3094340", "postDate": "01/12/2025 00:53:36", "content": "

So every time they add new data they reprocess from the start to the end.

\n

I thought so but just wanted to check

", "rawMarkdown": "So every time they add new data they reprocess from the start to the end.\n\nI thought so but just wanted to check", "votes": null } ], "comments": [ { "id": 3094332, "author_name": "shiyili", "author_url": "", "post_date": "01/12/2025 00:28:18", "content": "
\n

At the start of the forecasting phase, the unscored public test set will be extended up to the final day of the model training phase and the private set updated roughly every two weeks. Submissions will be rescored at the time of each update.

\n

During the forecasting phase, the evaluation API will serve test data from the beginning of the public set to the end of the private set. You must make predictions at every timestep, but, in this phase, only predictions on the private set are scored. (You may predict 0.0 on the unscored segments, if you like.)

\n
\n

From the data card.

", "votes": null, "replies": [ { "id": 3094340, "author_name": "michaeltimbs", "author_url": "", "post_date": "01/12/2025 00:53:36", "content": "

So every time they add new data they reprocess from the start to the end.

\n

I thought so but just wanted to check

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3094326": "During the test phase im assuming they pass the entire time series through at each point from start to finish?\n\nMy online training doesn't persist state to the filesystem or anything. It just holds the weights in memory and updates each day without checkpointing. \n\nNot that im going to get close to a winning score but I just wanted to make sure this wont be an issue and they dont just fire up the server for our submissions and pass random batches of data in every month and then aggregate the results at the end", "3094332": ">At the start of the forecasting phase, the unscored public test set will be extended up to the final day of the model training phase and the private set updated roughly every two weeks. Submissions will be rescored at the time of each update.\n\n>During the forecasting phase, the evaluation API will serve test data **from the beginning of the public set to the end of the private set.** You must make predictions at every timestep, but, in this phase, only predictions on the private set are scored. (You may predict 0.0 on the unscored segments, if you like.)\n\nFrom the data card.", "3094340": "So every time they add new data they reprocess from the start to the end.\n\nI thought so but just wanted to check" }, "source": "meta" }