{ "id": 376563, "title": "Question about final test set and evaluation", "url": "/competitions/otto-recommender-system/discussion/376563", "author_name": "", "post_date": "2023-01-07T05:16:58.018175700Z", "votes": 1, "comment_count": 2, "views": 0, "content": "

Hi, I’m new to kaggle and have some questions about the submission & evaluation process. On the competition github page it says the final test set will be published after the competition is finalized. A few questions I have about this:

\n
    \n
  1. On what date is the competition finalized & the final test set released? Is it 1/31, the final submission deadline as it says on the timeline page?
    2. \n
  2. For our submissions, what test set should we predict for? Is it the test.jsonl on the competition page or something else? If the final evaluation is done on the final test set, how will we make sure we submit predictions for this test set and not some other test set? Will we have time between when the final test set is published and the submission deadline?
    3. \n
  3. On the leaderboard page it says \"This leaderboard is calculated with approximately 33% of the test data. The final results will be based on the other 67%, so the final standings may be different\". Again is that test data the test.jsonl on the competition page? How does the competition prevent hardcoding of predictions based on the full test session ground truths for some known test set?
    4. \n
", "messages": [ { "id": "2090204", "postDate": "01/07/2023 05:16:58", "content": "

Hi, I’m new to kaggle and have some questions about the submission & evaluation process. On the competition github page it says the final test set will be published after the competition is finalized. A few questions I have about this:

\n
    \n
  1. On what date is the competition finalized & the final test set released? Is it 1/31, the final submission deadline as it says on the timeline page?
    2. \n
  2. For our submissions, what test set should we predict for? Is it the test.jsonl on the competition page or something else? If the final evaluation is done on the final test set, how will we make sure we submit predictions for this test set and not some other test set? Will we have time between when the final test set is published and the submission deadline?
    3. \n
  3. On the leaderboard page it says \"This leaderboard is calculated with approximately 33% of the test data. The final results will be based on the other 67%, so the final standings may be different\". Again is that test data the test.jsonl on the competition page? How does the competition prevent hardcoding of predictions based on the full test session ground truths for some known test set?
    4. \n
", "rawMarkdown": "Hi, I’m new to kaggle and have some questions about the submission & evaluation process. On the competition [github](https://github.com/otto-de/recsys-dataset) page it says the final test set will be published after the competition is finalized. A few questions I have about this:\n1. On what date is the competition finalized & the final test set released? Is it 1/31, the final submission deadline as it says on the [timeline page](https://www.kaggle.com/competitions/otto-recommender-system/overview/timeline)?\n2. For our submissions, what test set should we predict for? Is it the [test.jsonl](https://www.kaggle.com/competitions/otto-recommender-system/data?select=test.jsonl) on the competition page or something else? If the final evaluation is done on the final test set, how will we make sure we submit predictions for this test set and not some other test set? Will we have time between when the final test set is published and the submission deadline?\n3. On the leaderboard page it says \"This leaderboard is calculated with approximately 33% of the test data. The final results will be based on the other 67%, so the final standings may be different\". Again is that test data the [test.jsonl](https://www.kaggle.com/competitions/otto-recommender-system/data?select=test.jsonl) on the competition page? How does the competition prevent hardcoding of predictions based on the full test session ground truths for some known test set?", "votes": null }, { "id": "2090208", "postDate": "01/07/2023 05:25:37", "content": "

Just saw this thread here asking similar questions. If the comment there is correct, then I think my questions are addressed

", "rawMarkdown": "Just saw this thread [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/376547) asking similar questions. If the comment there is correct, then I think my questions are addressed", "votes": null }, { "id": "2090677", "postDate": "01/07/2023 15:29:22", "content": "

This is correct.

", "rawMarkdown": "This is correct.", "votes": null } ], "comments": [ { "id": 2090208, "author_name": "angihe", "author_url": "", "post_date": "01/07/2023 05:25:37", "content": "

Just saw this thread here asking similar questions. If the comment there is correct, then I think my questions are addressed

", "votes": null, "replies": [ { "id": 2090677, "author_name": "andreaswand", "author_url": "", "post_date": "01/07/2023 15:29:22", "content": "

This is correct.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2090204": "Hi, I’m new to kaggle and have some questions about the submission & evaluation process. On the competition [github](https://github.com/otto-de/recsys-dataset) page it says the final test set will be published after the competition is finalized. A few questions I have about this:\n1. On what date is the competition finalized & the final test set released? Is it 1/31, the final submission deadline as it says on the [timeline page](https://www.kaggle.com/competitions/otto-recommender-system/overview/timeline)?\n2. For our submissions, what test set should we predict for? Is it the [test.jsonl](https://www.kaggle.com/competitions/otto-recommender-system/data?select=test.jsonl) on the competition page or something else? If the final evaluation is done on the final test set, how will we make sure we submit predictions for this test set and not some other test set? Will we have time between when the final test set is published and the submission deadline?\n3. On the leaderboard page it says \"This leaderboard is calculated with approximately 33% of the test data. The final results will be based on the other 67%, so the final standings may be different\". Again is that test data the [test.jsonl](https://www.kaggle.com/competitions/otto-recommender-system/data?select=test.jsonl) on the competition page? How does the competition prevent hardcoding of predictions based on the full test session ground truths for some known test set?", "2090208": "Just saw this thread [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/376547) asking similar questions. If the comment there is correct, then I think my questions are addressed", "2090677": "This is correct." }, "source": "meta" }