For how long will the workspace of this competition be available after the submission deadline for this competition tomorrow, for those of us who want to continue working on this problem over a longer more open ended time frame? Say one of us makes a model in August (or even in 2026), will we still be able to test on the 40-ish new sequences just for our own benchmarking purposes, just not be eligible to win the competition? Or will there only be access to this for the notebooks that are actually participating?
", "messages": [ { "id": "3211769", "postDate": "05/28/2025 19:58:25", "content": "For how long will the workspace of this competition be available after the submission deadline for this competition tomorrow, for those of us who want to continue working on this problem over a longer more open ended time frame? Say one of us makes a model in August (or even in 2026), will we still be able to test on the 40-ish new sequences just for our own benchmarking purposes, just not be eligible to win the competition? Or will there only be access to this for the notebooks that are actually participating?
", "rawMarkdown": "For how long will the workspace of this competition be available after the submission deadline for this competition tomorrow, for those of us who want to continue working on this problem over a longer more open ended time frame? Say one of us makes a model in August (or even in 2026), will we still be able to test on the 40-ish new sequences just for our own benchmarking purposes, just not be eligible to win the competition? Or will there only be access to this for the notebooks that are actually participating?", "votes": null }, { "id": "3211795", "postDate": "05/28/2025 21:07:55", "content": "You can make submissions after the competition ends, you will get a score but this will not count for anything. The competition data will also be available but the test set (public and private) always remains hidden.
", "rawMarkdown": "You can make submissions after the competition ends, you will get a score but this will not count for anything. The competition data will also be available but the test set (public and private) always remains hidden.", "votes": null }, { "id": "3211956", "postDate": "05/29/2025 05:19:17", "content": "Isn't the public test set, by definition, public, i.e. disclosed (I understand that aside from the validation set, which was ALWAYS public, all the sequences added to the public dataset originated as a private test set in the previous phase--I mean AFTER the rollover/refresh of the leaderboard)? Or am I misunderstanding the definition of the word \"public\" here?
", "rawMarkdown": "Isn't the public test set, by definition, public, i.e. disclosed (I understand that aside from the validation set, which was ALWAYS public, all the sequences added to the public dataset originated as a private test set in the previous phase--I mean AFTER the rollover/refresh of the leaderboard)? Or am I misunderstanding the definition of the word \"public\" here?", "votes": null }, { "id": "3212184", "postDate": "05/29/2025 12:50:05", "content": "The leaderboard is composed of the test data, which is always hidden and split into two sets named public and private. The public test set is not public in the sense that you have access to it or can see it. This set is used to create a public leaderboard during the competition to get a sense on your ongoing model performance. The private test set on the other hand is also hidden and you do not get a score feedback until the end of the competition.
\nI have not seen barely any competition revealing the hidden test set after the competition's end. This would be pretty rare but I think there were some competitions that made the test set public after the end.
", "rawMarkdown": "The leaderboard is composed of the test data, which is always hidden and split into two sets named public and private. The public test set is not public in the sense that you have access to it or can see it. This set is used to create a public leaderboard during the competition to get a sense on your ongoing model performance. The private test set on the other hand is also hidden and you do not get a score feedback until the end of the competition.\n\nI have not seen barely any competition revealing the hidden test set after the competition's end. This would be pretty rare but I think there were some competitions that made the test set public after the end.", "votes": null } ], "comments": [ { "id": 3211795, "author_name": "alejopaullier", "author_url": "", "post_date": "05/28/2025 21:07:55", "content": "You can make submissions after the competition ends, you will get a score but this will not count for anything. The competition data will also be available but the test set (public and private) always remains hidden.
", "votes": null, "replies": [ { "id": 3211956, "author_name": "andrewrosko", "author_url": "", "post_date": "05/29/2025 05:19:17", "content": "Isn't the public test set, by definition, public, i.e. disclosed (I understand that aside from the validation set, which was ALWAYS public, all the sequences added to the public dataset originated as a private test set in the previous phase--I mean AFTER the rollover/refresh of the leaderboard)? Or am I misunderstanding the definition of the word \"public\" here?
", "votes": null, "replies": [ { "id": 3212184, "author_name": "alejopaullier", "author_url": "", "post_date": "05/29/2025 12:50:05", "content": "The leaderboard is composed of the test data, which is always hidden and split into two sets named public and private. The public test set is not public in the sense that you have access to it or can see it. This set is used to create a public leaderboard during the competition to get a sense on your ongoing model performance. The private test set on the other hand is also hidden and you do not get a score feedback until the end of the competition.
\nI have not seen barely any competition revealing the hidden test set after the competition's end. This would be pretty rare but I think there were some competitions that made the test set public after the end.
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3211769": "For how long will the workspace of this competition be available after the submission deadline for this competition tomorrow, for those of us who want to continue working on this problem over a longer more open ended time frame? Say one of us makes a model in August (or even in 2026), will we still be able to test on the 40-ish new sequences just for our own benchmarking purposes, just not be eligible to win the competition? Or will there only be access to this for the notebooks that are actually participating?", "3211795": "You can make submissions after the competition ends, you will get a score but this will not count for anything. The competition data will also be available but the test set (public and private) always remains hidden.", "3211956": "Isn't the public test set, by definition, public, i.e. disclosed (I understand that aside from the validation set, which was ALWAYS public, all the sequences added to the public dataset originated as a private test set in the previous phase--I mean AFTER the rollover/refresh of the leaderboard)? Or am I misunderstanding the definition of the word \"public\" here?", "3212184": "The leaderboard is composed of the test data, which is always hidden and split into two sets named public and private. The public test set is not public in the sense that you have access to it or can see it. This set is used to create a public leaderboard during the competition to get a sense on your ongoing model performance. The private test set on the other hand is also hidden and you do not get a score feedback until the end of the competition.\n\nI have not seen barely any competition revealing the hidden test set after the competition's end. This would be pretty rare but I think there were some competitions that made the test set public after the end." }, "source": "meta" }